Tenstorrent Wormhole n300, tt-metal TP=2: Qwen2.5-7B decode tok/s per-user + aggregate (vendor-measured b32)
22.1 tok/s per user at batch-32; 707 tok/s aggregate
Fields
id
benchmarks/comm-tt-wormhole-n300-qwen25-7b-tp2-decodeMetric
tok_s_per_user
Value
22.1
Unit
tok/s per user at batch-32; 707 tok/s aggregate
Model
Qwen2.5-7B
Methodology
VENDOR-MEASURED in the tenstorrent/tt-metal model matrix (cited by research-exotic.md). Batch-32 decode, ISL=128, tensor-parallel across the n300's two chips (TP=2). Qwen2.5-7B on the n300: 22.1 t/s/u and 707 t/s aggregate. The n300 is a dual-Wormhole-chip card - the TP=2 split is why a 7B runs slower per-user than an 8B does on the single-chip n150 despite the bigger card.
Measured by
vendor
Reproducible
yes
notes
Gap-fill for the Oct-14 Phase-A ingest: research-exotic.md listed this row but the n300 only had a Llama-3.1-8B record. Useful as the concrete example that tensor-parallel overheads are real even on a vendor-tuned stack.
Notes
Gap-fill for the Oct-14 Phase-A ingest: research-exotic.md listed this row but the n300 only had a Llama-3.1-8B record. Useful as the concrete example that tensor-parallel overheads are real even on a vendor-tuned stack.