Used RTX 3090 24GB, llama.cpp CUDA: Qwen3-8B Q4_K decode tok/s @4K ctx (Hardware Corner measured)
115.3 decode tok/s (TG, single-stream, batch-1, 4K context)
Fields
id
benchmarks/comm-rtx3090-qwen3-8b-q4-decodeMetric
decode_tok_s
Value
115.3
Unit
decode tok/s (TG, single-stream, batch-1, 4K context)
Model
Qwen3-8B Q4_K_M (GGUF)
Methodology
THIRD-PARTY MEASURED (Hardware Corner llama.cpp llama-bench CUDA backend). Qwen3-8B Q4_K TG: 115.3 tok/s @4K ctx, 88 @16K, 68 @32K. Reference card the whole used-consumer market is still compared against. Same run: Qwen3-14B 70 @4K, Qwen3.5-27B Q4 ~33, Qwen3-32B Q4_K 35 @4K / 30 @16K, Qwen3.5-35B-A3B MXFP4 79 @128K (MoE-friendly).
Measured by
third_party
Reproducible
yes
notes
The used-3090 is no longer ~$550-650: Hardware Corner's tracker measured $750 Nov-2025 -> ~$950 Feb-2026, ~$1,000 avg market value now. At ~$900-1,000/card the value edge eroded but the stack is still best-in-class (CUDA, llama.cpp, vLLM, ExLlamaV2, FA2). Largest well-fit model: ~34B dense Q4 fully in VRAM; 70B needs offload/multi-GPU. 3090 Ti is ~+9% TG (123.7 vs 115.3 on 8B) for ~same money.
Notes
The used-3090 is no longer ~$550-650: Hardware Corner's tracker measured $750 Nov-2025 -> ~$950 Feb-2026, ~$1,000 avg market value now. At ~$900-1,000/card the value edge eroded but the stack is still best-in-class (CUDA, llama.cpp, vLLM, ExLlamaV2, FA2). Largest well-fit model: ~34B dense Q4 fully in VRAM; 70B needs offload/multi-GPU. 3090 Ti is ~+9% TG (123.7 vs 115.3 on 8B) for ~same money.