Tenstorrent Wormhole n150, tt-metal: Falcon-7B decode tok/s per-user (vendor-measured b32)
18.5 tok/s per user at batch-32
Fields
id
benchmarks/comm-tt-wormhole-n150-falcon7b-decodeMetric
tok_s_per_user
Value
18.5
Unit
tok/s per user at batch-32
Model
Falcon-7B
Methodology
VENDOR-MEASURED in the tenstorrent/tt-metal model matrix (cited by research-exotic.md). Batch-32 decode, ISL=128. Falcon-7B on the n150: 18.5 t/s/u. The Falcon-7B demo is ttnn_falcon7b / falcon7b_common and targets Wormhole/T3000/Galaxy - importantly, NOT Grayskull, which is why a used Grayskull card has no LLM path despite the same brand.
Measured by
vendor
Reproducible
yes
notes
Gap-fill for the Oct-14 Phase-A ingest. This row doubles as the evidence for the Grayskull paperweight gotcha: the Falcon-7B model lives in the Wormhole-class model tree, and the tt-metal model matrix has zero Grayskull rows.
Notes
Gap-fill for the Oct-14 Phase-A ingest. This row doubles as the evidence for the Grayskull paperweight gotcha: the Falcon-7B model lives in the Wormhole-class model tree, and the tt-metal model matrix has zero Grayskull rows.