Jetson AGX Thor llama.cpp: Qwen2.5-1.5B Q4_K_M prefill 512 (Multimodalflow)
4298.3 tok/s (llama-bench pp, per-request)
Fields
id
benchmarks/thor-llamacpp-qwen25-15b-q4km-pp512Metric
prefill_tok_s
Value
4,298.3
Unit
tok/s (llama-bench pp, per-request)
Model
Qwen2.5-1.5B-Instruct Q4_K_M
Methodology
Multimodalflow live-hardware post dated 2026-06-01 on a Jetson AGX Thor Developer Kit. CUDA 13.0, compute capability 11.0, unified memory reported as 125,771 MiB (~123 GB), kernel 6.8.12-tegra. Not the MLPerf agentic-edge submission already in this repo. Same llama-bench invocation as the tg128 record. Prefill 512 tokens: 4298.3 +/- 158.5 t/s. Prefill 128 measured 3639.6 +/- 403.6 t/s and is noted, not stored. Per request.
Measured by
third_party
Reproducible
yes
notes
pp128 was 3639.6 tok/s with a large spread (+/- 403.6), so the longer 512-token prefill is the recorded figure.
Notes
pp128 was 3639.6 tok/s with a large spread (+/- 403.6), so the longer 512-token prefill is the recorded figure.