tt-metal Qwen2.5-72B on QuietBox (8x Wormhole, TP=8): 15.4 tok/s/user, 492.8 tok/s @b32
15.4 tok/s per user (batch-32, aggregate 492.8 tok/s; ISL=128)
Fields
id
benchmarks/ttmetal-wormhole-n300-qwen25-72b-tp8-b32-tpsuMetric
tok_s_per_user
Value
15.4
Unit
tok/s per user (batch-32, aggregate 492.8 tok/s; ISL=128)
Model
Qwen2.5-72B
Methodology
VENDOR-MEASURED (Tenstorrent tt-metal README featured-models table). Model demos under models/tt_transformers and models/demos/llama3_70b_galaxy. ISL=128 for all LLM rows. T/S/U = tok/s per user (1/inter-token-latency); T/S = T/S/U x batch. TT-Metalium release pinned per row. Values are single-run vendor numbers on their own demo harness, not vLLM end-to-end (vendor notes results may differ under the vLLM TT plugin). Config: QuietBox Wormhole 8-chip node, TP=8, batch 32, TT-Metalium v0.62.0-rc25, vLLM-TT ced0161. TTFT 223 ms.
Measured by
vendor
notes
QuietBox = 8x n300-class Wormhole cards on a Warp100 mesh. Per-card aggregate = 492.8/8 = 61.6 tok/s at batch 32. Vendor 'target' T/S/U is 20 - the 15.4 is the achieved number, which the table itself flags as below target.
Notes
QuietBox = 8x n300-class Wormhole cards on a Warp100 mesh. Per-card aggregate = 492.8/8 = 61.6 tok/s at batch 32. Vendor 'target' T/S/U is 20 - the 15.4 is the achieved number, which the table itself flags as below target.