NVIDIA Jetson page: Thor vLLM Llama 3.3 70B, concurrency 1, 4.7 tok/s
4.7 tok/s per request (max concurrency 1)
Fields
id
benchmarks/nv-thor-vllm-llama33-70b-c1-decodeMetric
decode_tok_s
Value
4.7
Unit
tok/s per request (max concurrency 1)
Model
Llama 3.3 70B
Methodology
NVIDIA Developer 'Jetson Benchmarks' page, Jetson AGX Thor Benchmarking Results table, read 2026-10-17. Jetson AGX Thor Developer Kit, JetPack 7.0, CUDA 13.0, TensorRT 10.13, vLLM (version not stated), input sequence length 2048, output sequence length 128. Weight dtype is not stated. The page gives no run count, no variance and no benchmark command, so confidence is below the MLPerf logs. Max concurrency 1 column, per-request rate.
Measured by
vendor
Reproducible
yes
notes
Vendor-measured. Weight quantization is not disclosed on the page, so format_id is left null rather than guessed. VLM rows from the same table were not filed.
Notes
Vendor-measured. Weight quantization is not disclosed on the page, so format_id is left null rather than guessed. VLM rows from the same table were not filed.