tt-metal Llama-3.3-70B on Galaxy (TP=32): TTFT 53 ms (ISL=128, b32)
53.0 ms (time-to-first-token, ISL=128, batch-32)
Fields
id
benchmarks/ttmetal-wormhole-galaxy-llama33-70b-tp32-ttftMetric
ttft_ms
Value
53
Unit
ms (time-to-first-token, ISL=128, batch-32)
Model
Llama-3.3-70B
Methodology
VENDOR-MEASURED (Tenstorrent tt-metal README featured-models table). Model demos under models/tt_transformers and models/demos/llama3_70b_galaxy. ISL=128 for all LLM rows. T/S/U = tok/s per user (1/inter-token-latency); T/S = T/S/U x batch. TT-Metalium release pinned per row. Values are single-run vendor numbers on their own demo harness, not vLLM end-to-end (vendor notes results may differ under the vLLM TT plugin). Same Galaxy TP=32 row; TTFT column.
Measured by
vendor
notes
Very low TTFT because prefill is spread across 32 chips.
Notes
Very low TTFT because prefill is spread across 32 chips.