RTX 5090 llama.cpp: Qwen3 8B Q4_K_XL decode at 4k ctx (Hardware Corner)
185.91 tok/s (tg128, batch 1, per-request)
Fields
id
benchmarks/hc-rtx5090-qwen3-8b-q4kxl-4k-tg128Metric
decode_tok_s
Value
185.91
Unit
tok/s (tg128, batch 1, per-request)
Model
Qwen3 8B
Methodology
Hardware Corner (Allan Witt) llama-bench build 466c1911 on RTX 5090 32GB GDDR7 (EPYC 7642 host, Ubuntu 24.04). Qwen3 8B Q4_K_XL, -fa 1 -ngl 99, -p 1024/2048 -n 128 at context depth 4k. Per-request batch-1, NOT aggregate.
Measured by
third_party
Reproducible
yes
notes
PP at same ctx: 10406.47 t/s. Full ctx sweep 4k..131k in source.
Notes
PP at same ctx: 10406.47 t/s. Full ctx sweep 4k..131k in source.