RTX 3090 Qwen3-32B Q4 token-gen
35.0 tok/s TG batch-1 @4K ctx
Fields
id
benchmarks/comm-rtx-3090-qwen3-32b-q4-tgMetric
decode_tok_s
Value
35
Unit
tok/s TG batch-1 @4K ctx
Model
Qwen3-32B
Methodology
llama.cpp llama-bench, Q4_K GGUF, CUDA, batch-1 decode; 30 t/s @16K (Hardware Corner measured table)
Measured by
third_party
Thermal state
unknown
notes
~34B dense Q4 is the largest well-fit single-card model.
Notes
~34B dense Q4 is the largest well-fit single-card model.