Tenstorrent Wormhole n150, tt-metal: Llama-3.1-8B decode tok/s per-user + aggregate (vendor-measured b32)
26.0 tok/s per user at batch-32; 832 tok/s aggregate
Fields
id
benchmarks/comm-tt-wormhole-n150-llama31-8b-decodeMetric
tok_s_per_user
Value
26
Unit
tok/s per user at batch-32; 832 tok/s aggregate
Model
Llama-3.1-8B
Methodology
VENDOR-MEASURED in the tenstorrent/tt-metal model matrix. Batch-32 decode, ISL=128 input sequence. Llama-3.1-8B on n150: 26.0 t/s/u (tok/s per user) and 832 t/s AGGREGATE across the 32 concurrent users. Single-stream reality check: ~26 t/s for an 8B is usable chat but roughly RTX-3090-Q4 territory - you pay ~$1k for the open Apache-2.0 stack + a learnable non-CUDA architecture, not for beating a used GPU on $/tok. Also measured on n150: Mistral-7B 28.7 t/s/u / 918 t/s, Falcon-7B 18.5, Llama-3.2-3B 46.6, Llama-3.2-1B 80.5.
Measured by
vendor
Reproducible
yes
notes
The only genuinely purchasable open-source-stack exotic accelerator family. n150s $999 / n150d $1,099 / n300d $1,449 NEW (in stock, ~2wk ship) - NOT $300-500 used, that is Grayskull (a different, legacy card). tt-metal/TT-Metalium is genuinely Apache-2.0, documented, has a bounty program. Llama-3.1-8B and Mistral-7B listed 'Complete' on n150 and n300; Qwen3-8B 'Functional'.
Notes
The only genuinely purchasable open-source-stack exotic accelerator family. n150s $999 / n150d $1,099 / n300d $1,449 NEW (in stock, ~2wk ship) - NOT $300-500 used, that is Grayskull (a different, legacy card). tt-metal/TT-Metalium is genuinely Apache-2.0, documented, has a bounty program. Llama-3.1-8B and Mistral-7B listed 'Complete' on n150 and n300; Qwen3-8B 'Functional'.