GitHub (new tab)

Glyph

GradesMetMostly metPartly metNot metNot statedFailedNot decided

Petal, grade, what is missing first, what Met needs and what it has
IntegrityCan a fake reply pass the check?Partly metour reading
The limit is proved for the Lean verifier, not cp_verify.Met needsa proved limit of 2−128 for cp_verify, which we runHasq/2124 + 2−101 per proof, proved5 missing · publisher acts, Integrity
Missing

For Mostly met

  1. A limit for cp_verify
  2. A limit for each reply checked alone
  3. A record its condition holds for our files

For Met

  1. Every part of the limit at most 2−128
  2. The publisher's words for each point we read
Evidence

From the publisher's words and our reading

Publisher's words at 9e2bdd8 (15)“The Lean verifier accepts a proof of a false claim with probability at most about q/2^124 + 2^-101 after q hash queries (about 124 bits per query; not ‘128 bits’)” (new tab)“Lean's kernel has checked the proof.” (new tab)“The bound is for one commitment, one prompt, one output, one proof and one call of verify.” (new tab)“One proof. N proofs handed in at once under one commitment, or N members of a batch, cost N times that error, taken at the whole list's hash queries; for proofs checked one after another, or members checked in separate sessions, no bound is stated.” (new tab)“for separate sessions no theorem states it.” (new tab)“against a committer and a prover who may both cheat” (new tab)“so nothing is claimed against a quantum adversary” (new tab)“One proof format and model size: full size, not the small examples” (new tab)“Neither figure, 124 bits per query or 128 bits per round, is claimed for the C++ programs.” (new tab)“At all other files in scope nothing is known, so there the theorem may hold only because verify rejects everything.” (new tab)“The proofs from the runs on real weights were checked by cp_verify alone; none has been put to the Lean verifier” (new tab)“public files that are fixed before the hash function is sampled (a provider that tunes them to the real hash function is outside the statement)” (new tab)“No file shows that they were fixed before the provider could tune them to the hash function: that you grant.” (new tab)“The C++ verifier and the CUDA prover are written to follow the same protocol and are tested against it, not themselves proved.” (new tab)“The C++ programs' printed ‘bits’ are their own accounting, not a theorem” (new tab)
Our notes (2)

Classical adversaries only (THEOREM.md L394-L395), and the files-fixed-before-the-hash condition is counted under the conditions clause.

The row runs the full-size format at exact match. Whether its own files are in scope falls under the conditions clause, and the bound is stated for the Lean verifier and not claimed for cp_verify, which the row runs (the program clause).

The grade uses our reading that the limit holds against any cheating prover.

It does not use our reading that the limit is for the settings we run. Met needs the publisher's words there too.

Our Model, Prompt, Reply checks: passed ↓

Why this grade

Partly met: the limit is not shown to cover what we run (items 1 to 3 under Missing). Its exponents, as written, are 124 and 101.

Met needs, in full

A proved limit of 2−128 on a fake reply passing the verifier we run.

What it has

The publisher proves, and states that Lean has checked, a limit on the chance that a fake reply passes: at most q/2124 + 2−101 per proof. Here q counts the hash queries a cheater makes, each a run of the hash function while it searches for a proof that passes. The other part is fixed, but the first grows with q. It reaches 2−100 at 224 queries and 2−80 at 244 queries. Each reply has its own proof, and we make replies in batches of 24 (item 2 below).

The theorem is about the Lean verifier, a verifier the publisher wrote in Lean and proves things about. We run cp_verify, the publisher's C++ verifier (“Anyone can check the proof on an ordinary CPU in seconds with cp_verify, without the weights.” (new tab)). For cp_verify, the publisher states that it is tested, not proved, and claims no limit.

Our reading, on one point the grade uses (no single sentence of the publisher's says it): The publisher states the limit “against a committer and a prover who may both cheat” (the committer publishes the commitment, and the prover makes the proofs). Our rule asks for a limit against any cheating prover, one that runs its own machine as it likes, and the limit must not rely on trusted hardware or an honest outside party. We read the publisher's words as that limit. The publisher adds that nothing is claimed against a quantum computer, but our rule does not ask about one.

Each item today
  1. Today the theorem is about the Lean verifier only.
  2. We make replies in batches of 24 and check each reply's proof alone. The publisher's limit for a batch covers its 24 proofs only when they are checked together (24 times the limit for one proof, counting the hash queries made for the whole batch). For proofs checked one at a time it states no limit.
  3. The publisher states the condition as “public files that are fixed before the hash function is sampled (a provider that tunes them to the real hash function is outside the statement)”. In plain words, the theorem treats the hash function as drawn at random and covers only public files chosen before that draw, so it says nothing about files a provider tuned to the actual hash function. No file of ours shows when the files we run were fixed, and the publisher writes that the reader must grant it. The publisher also writes that at files other than its stored examples, the theorem may hold only because the verifier rejects every proof. A verifier that rejects every proof never accepts a fake, so there the theorem could hold and still say nothing about fake replies. Outside the grade, in our trial under Formally verified, the Lean verifier accepted 5 of 5 honest proofs at the files we run.
  4. Today the parts are q × 2−124 and 2−101.
  5. Today both are our reading.
Our records

In each of our Model, Prompt and Reply tests, we give the verifier a proof with the wrong model, prompt or reply. Each was refused on this run.

In the field's terms

Soundness. Bound: at most q/2124 + 2−101 per proof, after q hash queries. The first term is 2−100 at q = 224 and 2−80 at q = 244. Basis: proved. Stated for: the Lean verifier. We run: cp_verify (no bound claimed for it). Adversary: any prover (our reading). Unit: proof, one reply per proof. Batch: together. Ours is 24 replies, each verified in its own call. Parameters: row (our reading). Conditions shown for our files: no.

ConfidentialityDoes a proof keep the weights secret?Mostly metour reading
The theorem covers one request per proof, not our 24-reply batches.Met needsa zero-knowledge theorem for exactly what we runHasa zero-knowledge theorem for the method3 missing · publisher acts, Confidentiality
Missing

For Met

  1. The theorem for our 24-reply batches
  2. A statement that its programs keep the weights secret
  3. No visible item that depends on the weights (3 today)
Evidence

From the publisher's words and our reading

Publisher's words at 9e2bdd8 (6)“Zero knowledge, Lean's form of weight hiding, is proved in Lean for the protocol, at full size and at the exact-match policy, with fresh random masks.” (new tab)“Anyone can check the proof on an ordinary CPU in seconds with cp_verify, without the weights.” (new tab)“batches, which are what the programs serve, are not covered” (new tab)“we expect, rather than know, that the programs hide the weights at the default setting, in honest runs, if the provider keeps its duties (this does not reach any real model's weights, or the parameter file and scales fitted to them” (new tab)“Anything else is outside it, and so is any real model, the example's too, because it needs a parameter file fitted to it” (new tab)“At least eleven things are visible” (new tab)
Zero knowledge, as stated (7)
What stays visible (at least 11)

3 that depend on the weights

8 others, each with its kind (side channel: seen around a proof, such as time, lengths, failures or volume · shape: the architecture and sizes · the statement's input or output: the prompt, the reply, served or refused)

Our notes (1)

The publisher expects the programs to hide the weights, but says the expectation does not reach a real model's weights or the files fitted to them, and the row commits a real model's weights.

Our Verify without weights check: passed ↓

Why this grade

Mostly met: there is a zero-knowledge theorem, at least for the method. Met needs the 3 items under Missing.

Met needs, in full

A theorem that proofs of exactly what we run reveal nothing about the weights.

What it has

The publisher proves in Lean a zero-knowledge theorem: everything a verifier sees could have been produced without the weights. It is statistical, which means the two can differ only by a tiny amount that the publisher bounds, in figures given under the field's terms below. It also treats the hash function as perfectly random, an assumption called the random-oracle model. The theorem is about the method (the protocol as the publisher defines it in Lean), not about the programs we run, and it covers one request per proof. But we run batches of 24 replies, which it does not cover.

For its programs (the C++ and CUDA code we run), the publisher states an expectation, not a theorem, and says the expectation does not reach a real model's weights or the files fitted to them. We run a real model, so we count no statement for the programs we run. This is our reading.

The publisher lists at least 11 things that stay visible to clients or the public, in a proof or around it. By its own words, 3 of them depend on the weights:

Neither the publisher's list nor our records measure how much these reveal about the weights.

Each item today
  1. Today it covers one request per proof.
  2. Today, on our reading, we count none for the programs we run.
  3. Today 3 items do.
Our records

In our Verify without weights test, we check that every verdict came with no model file and no GPU. It passed on this run.

In the field's terms

Zero knowledge of the weights. Proved: statistical, random-oracle model, for the protocol, one request a proof. The publisher's bound: at or below 2−154 within 264 hash queries and 688,127 replies. Not for our 24-reply batches. Programs: not stated (our reading). Weights given to the verifier: no. Visible: at least 11, 3 tagged fitted to the weights.

FidelityDoes the proved model answer like the original?Partly met
Our test does not show an average above 48.0 tokens, which Mostly met needs.Met needsan average shown above 283.0 tokens, one request at a timeHas29.66 matched tokens (8-bit copy 65.71, 4-bit copy 30.29)3 missing · publisher 2, us 1, Fidelity
Missing

For Mostly met

  1. An average shown above 48.0 tokenspublisher

For Met

  1. An average shown above 283.0 tokenspublisher
  2. Our whole run one request at a time in llama.cppus
Why this grade
Met needs, in full

An average above 283.0 matched tokens, shown by our test, in a run of one request at a time.

What it has

We sent 360 prompts to FlashProve and to the original model (Llama 4 Maverick 17B-128E Instruct, 16-bit). On average, FlashProve's replies match the original's for their first 29.66 tokens, up to the first difference (512 at most).

For scale, we scored other runs on the same prompts: the original model run a second time scores 500.3, our 8-bit copy 65.71, our 4-bit copy 30.29, and a copy we damaged on purpose (3-bit) 11.15, the pass bar every system must beat.

FlashProve proves a copy with integer weights of 4.84 bits each on average, counting the weights each token uses (Llama 4 Maverick uses about 17B of its 400B weights for each token). Values between its layers are fixed-point numbers (integers with a fixed scale).

Each item today
  1. Today the average is 29.66 tokens, not shown above it. The number format of the copy FlashProve proves is the publisher's choice, so this item is the publisher's.
  2. Today the average is 29.66 tokens, not shown above it.
  3. Today the original and its second run got every prompt in one call, in vLLM, and our copies ran 16 at a time, in llama.cpp. The original run a second time, in vLLM, scores 500.3, not 512, and the 283.0 threshold is computed from that score.
Our records

Our fidelity run on Llama 4 Maverick 17B-128E Instruct, 360 prompts, each reply scored against the original's.

In the field's terms

Quantization error, in matched leading tokens (mean, n = 360). Above T_upper, 48.0 (Mostly met): not shown. Above T_green, 283.0 (Met): not shown. Test: paired sign-flip per prompt, alpha 0.05. Proved: 4.84-bit integer weights, fixed-point activations. Decoding: exact match (only the model's top token). Requests at a time (slots): no count for the original and its rerun (vLLM, one call) and 16 for the copies (llama.cpp).

Formally verifiedHas a computer checked that fake replies are refused?Mostly metour reading
The theorem stops at the Lean verifier, short of cp_verify.Met needsa rebuilt theorem reaching cp_verify, which we runHasa Lean theorem of the method, rebuilt by us3 missing · publisher 2, us 1, Formally verified
Missing

For Met

  1. A theorem reaching cp_verify, which we runpublisher
  2. The publisher's words for each point we readpublisher
  3. Our rebuild of that theoremus
Evidence

From the publisher's words, our rebuild, our reading (2 points)

Publisher's words at 9e2bdd8 (13)“Soundness is proved, in Lean, for the verifier written in Lean” (new tab)“At each position it is the greedy next token after the prompt and the output's own earlier tokens, as an integer program computes it, not the floating-point model.” (new tab)“That this interactive oracle proof is the protocol of the C++ programs is not proved” (new tab)“Scale files in scope can switch off any of the model's blocks, up to all of them (worked out by hand, not proved).” (new tab)“No theorem describes the five constant look-up tables in the Lean text (cosine, sine, the exponential, SiLU and the sigmoid).” (new tab)“the kernel checks proofs, not what a definition means” (new tab)“‘The model’ means the extracted weights, which no theorem says are the ones cp_commit put in” (new tab)“the system in the statement of the soundness theorem on verify has a constant committer (a committer that returns one constant whatever the weights, and so commits to nothing; the committer of the six lines is another: the one that compile makes (lean/Johnson/Face/Compiler.lean), which encodes the weights and hashes the key) where its honest committer goes, so of that system itself none of it can be stated with any content” (new tab)“The protocol's soundness is proved in Lean 4.” (new tab)“the pinned version, 4.29.1, is older than fixes of publicly known soundness bugs in it” (new tab)“leanprover/lean4:v4.29.1” (new tab)“Lean's kernel and three standard axioms: propext, Classical.choice, Quot.sound.” (new tab)“Five of the files that this function takes come from a parser outside Lean, of which nothing is proved” (new tab)
Our notes (2)

The theorem covers the Lean protocol and states the model as an integer program (THEOREM.md L231-233). The C++ verifier the row runs is tested, not proved (what-is-proved.md L7-8), so the reach stops short of the verifier code this row runs. The model statement is the publisher's Lean integer model, and its tables and its reach at a given scale file are not proved (THEOREM.md L229-L231, L242-L243).

Read 2026-11-15: the conclusion is about the publisher's integer model, a reference definition of the model's computation, at the weights the extractor reads. No theorem ties those weights to the ones cp_commit committed (THEOREM.md L227-L228), and at files other than the stored examples the theorem may hold only because verify rejects everything (L314-L315, under Integrity's conditions clause). Also, the system in the soundness theorem's statement has a constant committer, which commits to nothing, so of that system none of it can be stated with any content (L700-L704). So the conclusion is about a reference definition of the model, as a step above Partly met needs, but it is not tied to the weights this row commits.

Our rebuild: exit code 0 ↗ (new tab)

Our Model, Prompt, Reply checks: passed ↓

Why this grade

Mostly met, on our reading of the reach and the conclusion. It does not reach cp_verify.

Met needs, in full

A computer-checked theorem that the verifier we run refuses fake replies, rebuilt by us.

What it has

The publisher proves, in Lean, a theorem that the Lean verifier refuses fake replies, up to the chance limit under Integrity. Lean is a program that checks every step of a proof. The theorem's statement includes the model's computation (our reading, below).

We rebuilt it at the version we ran (9e2bdd8) with the Lean version the publisher pins, 4.29.1. It checked: the publisher's check command exited with code 0 after 68,494 s (about 19 hours). It uses only Lean's three standard axioms, the basic assumptions any Lean proof may use, so no step is skipped. Lean itself is trusted, and the limits the publisher states are listed below.

It does not reach cp_verify, the publisher's C++ verifier, which we run. The publisher states that cp_verify is tested, not proved. Input-reading code sits outside it: “Five of the files that this function takes come from a parser outside Lean, of which nothing is proved” (new tab), as the publisher states.

We hold no record of a review of the theorem's statement by an outside Lean expert.

Our reading, on two points the grade uses (no single sentence of the publisher's says it): (1) The theorem covers the whole method, the model's computation included. (2) Its conclusion is about the model: about the publisher's integer model (a written-out definition, in Lean, of what the model computes) at the weights read from the files. No theorem ties those weights to the ones cp_commit committed (the first limit below). But our rules accept a conclusion about such a definition as one about the model, and they do not ask for a theorem that ties its weights to the committed ones.

Each item today
  1. Today it stops at the Lean verifier, and input-reading code sits outside it.
  2. Today both are our reading.
Our records

Our rebuild: VerifierAndGoals.AtTheName.sound_of_verify_at ↗ (new tab), leanprover/lean4 v4.29.1, the publisher's command, started 2026-11-16, 68,494 s, exit code 0, axioms Classical.choice, Quot.sound, propext. Record ↗ (new tab)

Outside the grade: we ran the Lean verifier on 5 honest proofs from our runs, at the files we run (2 from counted sessions). It accepted 5 of 5. As a check that it can refuse, we also gave it one proof with a commitment that no run of ours made, and it refused that proof. The trial script's wall clock reads 1,237 to 1,530 s per proof (n = 5). Record ↗ (new tab)

Our Model, Prompt and Reply tests passed on this run.

Limits the publisher states, which the grade does not count
In the field's terms

Machine-checked soundness proof. Reach: the protocol, model included (our reading). Theorem: VerifierAndGoals.AtTheName.sound_of_verify_at ↗ (new tab). Tool: Lean 4, kernel 4.29.1. Trusted base: Lean's kernel, propext, Classical.choice, Quot.sound. Input-reading code outside the proof: yes. Conclusion: model (our reading). Stated for the Lean verifier, not cp_verify. No review of the statement by an outside Lean expert is recorded. Scope: the scope table further down this page ↓

Trust assumptionsDoes checking trust a setup or a party?Mostly metour reading
No single sentence of the publisher's says it.Met needsthe publisher's words: cryptography only, no setup secretHascryptography only, no setup secret, on our reading1 missing · publisher acts, Trust assumptions
Missing

For Met

  1. The publisher's own sentence saying so
Evidence

From the publisher's words and our reading

Publisher's words at 9e2bdd8 (7)“Six theorems are proved in Lean of one committer, prover and verifier, compile (flashProve setup), at every public setup setup (PublicSetup: three public files, what the reader returns at them, and FilesInUse).” (new tab)“Sound says that there is one extractor such that, whatever the committer and the prover, verify accepts a false claim with probability at most the bound at q = a + b.” (new tab)“public files that are fixed before the hash function is sampled (a provider that tunes them to the real hash function is outside the statement)” (new tab)“the soundness theorem on verify has no hypothesis beyond those in its statement, so nothing more is assumed by that road.” (new tab)“The random-oracle model for the hash, with a second, fixed hash function, built on BLAKE2b's compression function, assumed independent of it, against classical adversaries.” (new tab)“the random oracle and the second hash function belong to the statement, not to the proof's axioms.” (new tab)“Anyone can check the proof on an ordinary CPU in seconds with cp_verify, without the weights.” (new tab)
Our notes (1)

The publisher states soundness at every public setup of three public files, against any committer and prover, with the random-oracle model as its assumption. No sentence says in words that no setup produced a secret, so we read the petal as transparent and grade it Mostly met at most (GL-10, amended 2026-11-15).

No test of ours bears on it

Why this grade
Met needs, in full

The publisher's own sentence that checking needs no secret setup and no trusted party.

What it has

Anyone can check a proof, as the publisher states (“Anyone can check the proof on an ordinary CPU in seconds with cp_verify, without the weights.” (new tab)).

The publisher proves its theorems at every public setup (three public files it names), against any committer and prover. It assumes the random-oracle model, which treats the hash function as perfectly random. It also assumes that a second, fixed hash function, built on BLAKE2b's compression function, is independent of the first. The publisher's theorems are stated against classical adversaries, not quantum ones. These are idealizations of hash functions, not proved properties of them. On our reading, none of them needs a secret or a trusted party, and we count them among cryptographic assumptions. The theorem's condition that the public files are fixed before the hash function is drawn is counted under Integrity.

So, on our reading, soundness rests on cryptographic assumptions only, and no setup step produced a secret. But the publisher does not say this in one sentence.

Each item today
  1. Today this is our reading.
In the field's terms

Trusted setup or party. Rests on: transparent (no setup made a secret) (our reading). Publicly checkable: yes.

Statement

Verifiercp_verifyclaim card (new tab)
An accept meansthe committed model's own greedy reply to the prompt (stated)sources 1 (new tab) 2 (new tab)
From our commit recordthe commitment, 5 filessources 1 (new tab) 2 (new tab)
From the requestthe prompt (512 token ids), our 512-token replysource (new tab)
From the proof1 filesource (new tab)
Decodinggreedy, at every reply positionsource (new tab)
Not checked by any calllisted by the publishersource (new tab)
Architecturevisible: the architecture and every sizesource (new tab)
Claim card in full (7 quotes)

Erratum: the card's relation and coverage.tokens_covered describe the v0.1 range: r = T - 1, e = T, one checked token after a 1,024-token window. At Level 1: “For a prompt of P tokens and a reply of R, r = P − 1 and e = P + R − 1.” (new tab)

WhatGivenClaim card
An accept meansthe reply is the committed model's own greedy reply to the prompt, as the publisher states

Under exact match, a `YES` [is meant to show](../docs/limits.md#what-a-yes-covers) that each checked token is the committed integer model's top choice at its position source (new tab)

For a prompt of P tokens and a reply of R, r = P − 1 and e = P + R − 1. source (new tab)

From our commit recordthe commitment and the files commit wrote: policy.txt, KL.bin, layout.txt, key.bin, 0.bin

The commitment ID comes from "a party your user trusts, never from the server being checked or from files sent with the proof". source (new tab)

"The commitment ID is a hash, in 64 hex digits, of the configuration, the Merkle root and shape of the first weight table, a masked encoding of the weights, and the out-of-domain (OOD) seeds and answers. [...] the ID covers them by design, but the soundness theorem on `verify` is stated at given public files and does not itself say that one ID cannot verify under two sets of them. The ID does not cover the leaf files or the key file's proofs, which `cp_verify` checks on reading." source (new tab)

From the requestthe prompt, 512 token ids, and the 512-token reply from our record

"Only the named output positions are covered, and a reply stopped short verifies as a shorter one. [...] A proof names no client or time." source (new tab)

From the proof1 file: response.proof

"Some bytes of an honest proof, or of a commitment's files, can be altered and the proof still accepted, for the same true claim; non-malleability is not claimed, and an accept does not show that this exact file came from the prover." source (new tab)

Decoding rulegreedy, at every reply position

"the policy under which every checked token must be the integer model's top choice. It is `cp_verify`'s default." The published commitment's acceptance policy is exact match. source (new tab)

Checked by no call

"The client must trust what no call checks: the commitment ID, the expected policy and its numbers, the tokenizer and chat template, and `cp_verify` itself." source (new tab)

Architecturevisible: the architecture and every size

The declared list, "At least eleven things are visible": each reply, whether a request was served or failed, time, the architecture and every size, settings fitted to the weights, that `cp_commit`'s checks passed, that a batch failed or the service stopped, how much a provider has served and when, who shared a batch and how many, lengths, places in the batch and nonces, masked sums of the weights. "The list says what is visible by design, not that everything else is hidden." In Lean's theorem, one bit per request (served or refused) is revealed, "and it depends on the weights". source (new tab)

Checks

CheckStateRecord
Runpassedrun record (new tab)
Modelpassedcontrols (new tab)
Promptpassedcontrols (new tab)
Replypassedrun record (new tab)
Verify without weightspassedverify traces (new tab)
Fidelitypassedfidelity record (new tab)
Every record and trial (14 records)
Model

What ran proved with one weight changed: 3 trials, the same input each time, and in 3, the prover made no proof, as its publisher documents. An honest proof checked against the changed weights' commitment: 3 trials, the same input each time, and 3 refused by the verifier.

Outside the counted suite the verifier accepted 3 of 3 honest proofs (2 from counted sessions). Checked against another session's commitment of the same weights: 3 of 3 refused. So a refusal against the changed weights' commitment does not by itself show that the verifier tells weights apart, because it also refuses a second commitment of the same weights. trial record (new tab)

2026-11-11 · 1: 2 controls (new tab)2026-11-13 · 1: 1 control (new tab)
Prompt

What ran one prompt token changed: 3 trials, a new input each time, and 3 refused by the verifier.

2026-11-11 · 1: 1 control (new tab)
Reply

What ran the first reply token changed: 3 trials, the same input each time, and 3 refused by the verifier. The last reply token changed: 3 trials, the same input each time, and 3 refused by the verifier. One interior reply token changed: 3 trials, a new input each time, and 3 refused by the verifier.

Outside the counted suite the verifier accepted 1 of 1 honest proof. One reply token changed, at every reply position in turn: 512 of 512 refused. trial record (new tab)

2026-11-11 · 1: run record (new tab)2026-11-11 · 1: 1 control (new tab)2026-11-12 · 1: run record (new tab)2026-11-12 · 2: run record (new tab)

Cost

3 whole sessions of 120 replies · 1 × H200 · our runs only, not rerun by anyone else

tokens/s per H200, log scale
proved2.027tokens/s per H200
served2,669tokens/s per H200
served over proved is the cost vs serving
Cost vs serving, in two parts
commit and proving118.2×generation1,199×each over a served reply's GPU time, as the whole is
Verify time
17.83s/replyn = 3 · 17.66 to 18.07 s/replyverifier wall time
Proof size
18.78MB/replyn = 3 · 18.77 to 18.78 MB/replyevery proof file, as written
Cost
proved0.3751USD/replyserved0.0002848USD/replyH2005.345USD/GPU-hour
GPU time, hardware and serving

Serving Meta's FP8 release, vLLM 0.32.0, 4 × H200, full load at 512 concurrent requests · n = 3 timed batches, 1 run (2026-11-11 · 1 (new tab)) · the copy this row proves is under Fidelity

Proving 2 × H200 rented, 1 given to the job and priced, 3 whole sessions of 120 replies

Generation the publisher's integer model, one reply at a time “greedy decoding, one sequence at a time, one GPU” (new tab) · 0.4492GPU-s/reply token0.4457 to 0.4545 GPU-s/reply token · n = 3

Verifying cp_verify on 8 CPUs of an Intel(R) Xeon(R) Platinum 8488C, the prover's host, no GPU visible · 17.66 to 18.07 s/reply, the 3 sessions' means, n = 3 · Lean verifier: not timed in v1.0's sessions (a trial under Formally verified)

Proof size the bytes of the 2 files a proof is released as, as written: response.proof, response.y

Tokens per second 2.027tokens/s per GPU (2.007 to 2.044, n = 3 whole sessions, 1 × H200)

Evidence our runs only, not rerun by anyone else

Ran on Hugging Face Jobs, image flashprove@sha256:12611db7, harness 8512b94d (new tab), adapter aa5f31f9 (new tab)

Session GPU time252.6GPU-s/reply250.5 to 255.1 GPU-s/replycommit and proving22.67GPU-s/reply22.31 to 23.29 GPU-s/replygeneration230.0GPU-s/reply228.2 to 232.7 GPU-s/reply
Serving GPU time
0.1918GPU-s/reply
Commit GPU time
14.76GPU-s/reply14.36 to 15.44 GPU-s/reply
Proving GPU time
7.910GPU-s/reply7.856 to 7.950 GPU-s/reply
H200 price
5.345USD/GPU-houras of 2026-11-08

Serving ladder, 11 concurrencies

Concurrent requestsRate
10.05260replies/GPU-s
20.1034replies/GPU-s
40.2053replies/GPU-s
80.4036replies/GPU-s
160.7491replies/GPU-s
321.275replies/GPU-s
642.010replies/GPU-s
1282.928replies/GPU-s
2563.912replies/GPU-s
512 (full load)5.213replies/GPU-s
1,0244.268replies/GPU-s

Fidelity

top-token agreement, n = 360 prompts
FlashProve86.4%
3-bit copy71.9%
2-bit copy64.1%
Proved
4.84-bit integer weights, fixed-point activationsUD-Q4_K_XL
Against the proofs
11,034 of 11,034 positions357 of 357 where the two replies partwhere a proof states the model's token, FlashProve's code picked it
Fidelity check
passedFlashProve29.66tokenspass bar11.15tokensmatched leading tokens, mean
KL to the original weights
not measured
The copies, the tests and the sources

Proved UD-Q4_K_XL (new tab) (GGUF), 4.84 bits per weight, computed from its files (new tab). Integer weights, fixed-point activations, as the publisher states: “Model is Llama 4 Maverick's text transformer in integer arithmetic: integer weights, fixed-point activations with fitted power-of-two scales, and lookup tables for the non-linear functions” (new tab)

KL to the original weights: not measured (llama.cpp's KL base is the 16-bit file, which the level's llama.cpp pin cannot run on this rung)

RepliesKindTop-token agreementMatched leading tokensKL divergence
8-bit copy (Q8_0)honest96.2%median 96.0 %65.71tokens0.02669nats/token
FlashProvethis system86.4%median 85.4 %95 % interval 85.9 to 86.9 %29.66tokensn = 360 replies · above the bar: p < 5.0 × 10−6, the test's floor0.1829nats/token

FlashProve's top-token agreement and KL divergence: computed by the system's own code on the reference reply, not by the proved computation. What ran: the publisher's integer-model package (IntegerModel.extend), which the publisher says is "by design" the function the prover proves; that it equals cp_prove value by value at full size is not stated.

run-flashprove.json, teacher-forced-2.json (new tab)
4-bit copy (Q4_K_M)honest85.5%median 84.2 %30.29tokens0.2022nats/token
3-bit copy (Q3_K_S)damaged, the bar71.9%median 68.9 %11.15tokens0.6366nats/token
2-bit copy (Q2_K)damaged64.1%median 61.1 %5.083tokens0.9713nats/token

Original: unsloth/Llama-4-Maverick-17B-128E-Instruct at 86c5ecb, 16-bit. It is a copy on the Hub, not Meta's own repository. Fed its own reply, the original's engine picks that reply's token at 99.0 % of positions, which is the figure of the original itself.

Engines: the original and the controls run with it in vLLM, every prompt sent in one call, and the quantized controls in llama.cpp, 16 requests at a time.

Prompts: 512-token windows of 8 texts from www.gutenberg.org (public domain in the United States), with no chat template.

12 tests on top-token agreement, and they set no state

the original, run again
FlashProve above it: p = 1.0 · below it: p < 5.0 × 10−6, the test's floor
8-bit copy (Q8_0)
FlashProve above it: p = 1.0 · below it: p < 5.0 × 10−6, the test's floor
4-bit copy (Q4_K_M)
FlashProve above it: p < 5.0 × 10−6, the test's floor · below it: p = 1.0
3-bit copy (Q3_K_S)
FlashProve above it: p < 5.0 × 10−6, the test's floor · below it: p = 1.0
2-bit copy (Q2_K)
FlashProve above it: p < 5.0 × 10−6, the test's floor · below it: p = 1.0
a smaller model
FlashProve above it: p < 5.0 × 10−6, the test's floor · below it: p = 1.0

Sessions

3 counted · 5 release groups in each

SlotRunSessiongenerationSteady
12026-11-11 · 1 (new tab) · manifest (new tab) · log (new tab)252.3GPU-s/reply229.0GPU-s/reply236.3GPU-s/reply
22026-11-12 · 1 (new tab) · manifest (new tab) · log (new tab)250.5GPU-s/reply228.2GPU-s/replynoneuneven groups
32026-11-12 · 2 (new tab) · manifest (new tab) · log (new tab)255.1GPU-s/reply232.7GPU-s/replynoneuneven groups