Release v0.1, draft, mock
Provable inference, measured under one set of rules
The same models, the same reference hardware, the same timing rules, and a claim card saying what each proof establishes. The benchmark measures provers. It does not build, rank or certify them, and no table on this site has a combined score.
Systems
4
2 provers, 2 reference systems
Workloads
4
native division only in v0.1
Runs
20
18 completed, 1 failed, 1 superseded (mock)
Controls
41
over 4 configurations (mock)
What each proof establishes
Claim cards come before numbers on every page (RFC-0009 S1).
FlashProve
FlashProve@9e2bdd8One proof covers one request: a pass over a 1,024-token window, stating the next token (PRD §1).
- relation:
- The output token is the top choice of the integer model whose weights were committed under the given commitment ID, on the given 1,024-token window.
- security.notion:
- knowledge_soundness
- coverage:
- prefill, request
- residual:
- proves the integer model, which differs from the 16-bit model's top token at some positions
SASH zk-inference-prover
zk-inference-prover@eb95bfaOne proof in the benchmark's sense is a sequence's whole grid of segments, every prompt and continuation position (systems/sash/README.md).
- relation:
- not stated
- security.notion:
- not stated
- coverage:
- both, segment_grid
- residual:
- not stated
Configurations
Alphabetical by system, workload and profile.
| System | Workload | Profile | Tier | Runs |
|---|---|---|---|---|
| flashprove | fp-maverick-w1024-b1 | prover-h100 | tier reported | 3 of 3 |
| flashprove | fp-maverick-w1024-b24 | prover-h100 | tier reproduced | 3 of 3 |
| ref-recompute | sash-l31-8b-c4096 | prover-h100 | tier reported | 3 of 3 |
| ref-serve | fp-maverick-w1024-b24 | prover-h100 | tier reported | 0 of 0 |
| ref-serve | sash-l31-8b-c4096 | prover-h100 | tier reported | 3 of 3 |
| sash | sash-l31-8b-c1024 | prover-h100 | tier reported | 3 of 3 |
| sash | sash-l31-8b-c4096 | prover-h100 | tier peer-reviewed | 3 of 5 |
Evidence
Effective tier per configuration
- tier reported5 configurations
- tier peer-reviewed1 configuration
- tier reproduced1 configuration
Only reproduced numbers are called measured in prose (RFC-0007).
Negative control outcomes, all configurations
27 passed0 failed14 void, not applicable or error
Reproductions and reviews
flashprove on fp-maverick-w1024-b24
Mock runner B (unaffiliated), 2026-12-02: within tolerance
sash on sash-l31-8b-c4096
Mock reviewer C (another organisation), 2026-12-04: peer review recorded, rerun pending
sash on sash-l31-8b-c1024
Mock runner B (unaffiliated), 2026-12-05: failed: run aborted at commit, rerun scheduled
What the numbers do not show
- Native workloads differ in model, shape and claim. Rows from different native workloads are not a head-to-head comparison (RFC-0003).
- Passing byte-level controls shows integrity checking, not soundness (RFC-0008).
- The benchmark does not interpret security proofs. Security fields are copied from each system's documentation, or say not stated (RFC-0002).
- Cost prices GPU time only; CPU, host memory, disk and network are not priced (RFC-0006).