Tenstorrent Wormhole N300, vLLM+TT-Metal: Llama-3.1-8B single-stream decode tok/s
30.0 tok/s (single-stream decode, batch 1, stabilized)
Fields
id
benchmarks/comm-tenstorrent-n300-llama31-8b-decodeMetric
decode_tok_s
Value
30
Unit
tok/s (single-stream decode, batch 1, stabilized)
Model
Llama-3.1-8B
Methodology
THIRD-PARTY measured (community video teardown, summarized in gist). vLLM + TT-Metal stack on a single N300 (dual-chip Wormhole, 24GB). Llama-3.1-8B decode warmed up from ~3.6 tok/s to a stabilized ~30 tok/s single stream. Prompt processing reported ~3700-4000 tok/s. 8 concurrent requests reached ~164 tok/s aggregate. Vendor-claimed reference for the same model is ~44 tok/s (different quantization).
Measured by
third_party
Power scope
device
Thermal state
steady
notes
Community measurement, lower confidence (video teardown via summary gist, not a controlled scoreboard). Stabilized single-stream value used; the 3.6 tok/s figure is warm-up. Vendor claims 44 tok/s at a different quant - status kept verified but confidence low. Power draw was NOT instrumented in the source (an earlier ~70 W figure was an estimate, removed rather than reported without a clock-lock disclosure).
Notes
Community measurement, lower confidence (video teardown via summary gist, not a controlled scoreboard). Stabilized single-stream value used; the 3.6 tok/s figure is warm-up. Vendor claims 44 tok/s at a different quant - status kept verified but confidence low. Power draw was NOT instrumented in the source (an earlier ~70 W figure was an estimate, removed rather than reported without a clock-lock disclosure).