GitHub (new tab)

Scale

Cost vs serving

Cost vs serving
PositionSystem× serving, log scaleCost vs serving
400BLlama 4 Maverick 17B-128E InstructMixture of experts, 17B activeServing2,669tokens/s per H200
1, only system run on this modelFlashProvePassed1,317×
8.03BLlama 3.1 8B InstructDenseServing11,876tokens/s per H200
1, only system run on this modelSASHPassed62,160×
124MGPT-2DenseServingnot run yet
no system run yet

Proof size

Proof size
PositionSystemMB/reply, log scaleProof size
400BLlama 4 Maverick 17B-128E InstructMixture of experts, 17B active
1, only system run on this modelFlashProvePassed18.78MB/reply
8.03BLlama 3.1 8B InstructDense
1, only system run on this modelSASHPassed3,909MB/reply
124MGPT-2Dense
no system run yet

Verify time

Verify time
PositionSystems/reply, log scaleVerify time
400BLlama 4 Maverick 17B-128E InstructMixture of experts, 17B active
1, only system run on this modelFlashProvePassed17.83s/reply
8.03BLlama 3.1 8B InstructDense
1, only system run on this modelSASHPassed1,078s/reply
124MGPT-2Dense
no system run yet