Hardware Corner lab: RTX 5070 Qwen3-8B Q4_K_XL decode @ 32k ctx
43.6 tok/s (decode, per-request, batch 1)
Fields
id
benchmarks/hwcorner-rtx-5070-qwen3-8b-q4kxl-decode-32kMetric
decode_tok_s
Value
43.6
Unit
tok/s (decode, per-request, batch 1)
Model
Qwen3 8B
Methodology
THIRD-PARTY LAB MEASURED (Hardware Corner, 'Definitive GPU Ranking for LLMs', updated Dec 9 2025). llama.cpp llama-bench on Ubuntu 24.04, CUDA 12.8, Q4_K_XL quantization, single GPU. Decode tok/s for Qwen3 8B at 32k context length. Per-request batch-1 generation, not aggregate.
Measured by
third_party
Reproducible
yes
Thermal state
steady
notes
In-lab measured decode at 32k context. Part of a controlled cross-GPU sweep (same engine build + quant + harness on every card). Context-length scaling: decode falls as KV cache grows — 16K is the least-memory-pressured point. format_id gguf-q4-k-m used as nearest standard 4-bit GGUF record; the source specifies 'Q4_K_XL' (Unsloth XL dynamic quant) which has no dedicated quantization record.
Notes
In-lab measured decode at 32k context. Part of a controlled cross-GPU sweep (same engine build + quant + harness on every card). Context-length scaling: decode falls as KV cache grows — 16K is the least-memory-pressured point. format_id gguf-q4-k-m used as nearest standard 4-bit GGUF record; the source specifies 'Q4_K_XL' (Unsloth XL dynamic quant) which has no dedicated quantization record.