Wormhole n300 tt-metal: Llama-3.2-11B-Vision TP2 batch-16 TTFT
714.0 ms time-to-first-token, batch 16, TP=2
Fields
id
benchmarks/ttmetal-n300-llama32-11b-vision-tp2-b16-ttftMetric
ttft_ms
Value
714
Unit
ms time-to-first-token, batch 16, TP=2
Model
Llama-3.2-11B-Vision
Methodology
VENDOR demo, same row as the 18.1 tok/s/u figure. Llama 3.2 11B Vision, TP=2 on one n300 (Wormhole), batch 16, TT-Metalium v0.62.0-dev20251015, input sequence length 128. ttft 714 ms is the time to the first output token.
Measured by
vendor
Reproducible
yes
notes
Vendor demo numbers, not a vLLM serving trace. The README states t/s = t/s/u * batch and that vLLM-server performance may differ. LLM rows use input sequence length 128 except Mamba, which accepts any length. Starred Blackhole figures carry the vendor's own caveat that the software is still under active optimization.
Notes
Vendor demo numbers, not a vLLM serving trace. The README states t/s = t/s/u * batch and that vLLM-server performance may differ. LLM rows use input sequence length 128 except Mamba, which accepts any length. Starred Blackhole figures carry the vendor's own caveat that the software is still under active optimization.