Tenstorrent Wormhole n150 Llama-3.1-8B token-gen (vendor b32)
26.0 tok/s per user at batch-32; 832 t/s agg
Fields
id
benchmarks/comm-tt-wormhole-n150-llama31-8b-tgMetric
tok_s_per_user
Value
26
Unit
tok/s per user at batch-32; 832 t/s agg
Model
Llama-3.1-8B
Methodology
Vendor tt-metal model matrix, batch-32 decode ISL=128
Measured by
vendor
Thermal state
unknown
notes
VENDOR-MEASURED. Mistral-7B 28.7 t/s/u; Falcon-7B 18.5; L3.2-3B 46.6; L3.2-1B 80.5. Single-user ~26 t/s on 8B is RTX-3090-Q4 territory.
Notes
VENDOR-MEASURED. Mistral-7B 28.7 t/s/u; Falcon-7B 18.5; L3.2-3B 46.6; L3.2-1B 80.5. Single-user ~26 t/s on 8B is RTX-3090-Q4 territory.