Tenstorrent Wormhole n150, tt-metal: Mistral-7B decode tok/s per-user + aggregate (vendor-measured b32)
28.7 tok/s per user at batch-32; 918 tok/s aggregate
Fields
id
benchmarks/comm-tt-wormhole-n150-mistral7b-decodeMetric
tok_s_per_user
Value
28.7
Unit
tok/s per user at batch-32; 918 tok/s aggregate
Model
Mistral-7B
Methodology
VENDOR-MEASURED in the tenstorrent/tt-metal model matrix (cited by research-exotic.md). Batch-32 decode, ISL=128. Mistral-7B on the n150: 28.7 t/s/u (tok/s per user) and 918 t/s aggregate across the 32 concurrent users. Not a single-stream number - per-user is the honest per-request figure.
Measured by
vendor
Reproducible
yes
notes
Gap-fill for the Oct-14 Phase-A ingest: research-exotic.md listed this row in the n150 measured table but only the Llama-3.1-8B record existed. Mistral-7B is marked 'Complete' support on Wormhole - one of the two officially green 7B-class models alongside Llama-3.1-8B.
Notes
Gap-fill for the Oct-14 Phase-A ingest: research-exotic.md listed this row in the n150 measured table but only the Llama-3.1-8B record existed. Mistral-7B is marked 'Complete' support on Wormhole - one of the two officially green 7B-class models alongside Llama-3.1-8B.