Wormhole n300 tt-metal: Llama-3.2-11B-Vision TP2 batch-16 tok/s per user
18.1 tok/s per user at batch 16 across 2 chips (TP=2); 289.6 tok/s aggregate
Fields
id
benchmarks/ttmetal-n300-llama32-11b-vision-tp2-b16-tpsuMetric
tok_s_per_user
Value
18.1
Unit
tok/s per user at batch 16 across 2 chips (TP=2); 289.6 tok/s aggregate
Model
Llama-3.2-11B-Vision
Methodology
VENDOR demo in tenstorrent/tt-metal models/README.md (matrix last updated 2025-12-08). Llama 3.2 11B Vision, tensor-parallel 2 on one n300 (Wormhole, two chips), batch 16, TT-Metalium v0.62.0-dev20251015, input sequence length 128. t/s/u 18.1 is per-user 1/inter-token-latency; aggregate t/s is 289.6 = 18.1 * 16 concurrent requests, NOT a per-request rate and NOT a per-chip rate.
Measured by
vendor
Reproducible
yes
notes
Vendor demo numbers, not a vLLM serving trace. The README states t/s = t/s/u * batch and that vLLM-server performance may differ. LLM rows use input sequence length 128 except Mamba, which accepts any length. Starred Blackhole figures carry the vendor's own caveat that the software is still under active optimization.
Notes
Vendor demo numbers, not a vLLM serving trace. The README states t/s = t/s/u * batch and that vLLM-server performance may differ. LLM rows use input sequence length 128 except Mamba, which accepts any length. Starred Blackhole figures carry the vendor's own caveat that the software is still under active optimization.