RTX PRO 6000 Blackwell Llama-3.3-70B Q4 token-gen @4K
32.0 tok/s TG batch-1 @4K ctx
Fields
id
benchmarks/comm-rtx-pro-6000-llama33-70b-q4-tg-4kMetric
decode_tok_s
Value
32
Unit
tok/s TG batch-1 @4K ctx
Model
Llama-3.3-70B
Methodology
llama.cpp llama-bench, Q4 GGUF, CUDA, batch-1 decode (Hardware Corner measured table)
Measured by
third_party
Thermal state
unknown
notes
16.6 @128K. gpt-oss-120B Q8 221 @4K / 123 @128K; Qwen3.5-122B Q4 98 @4K / 65.6 @256K; Mistral-123B Q4 19 @4K. The only single-card 120B+ dense answer.
Notes
16.6 @128K. gpt-oss-120B Q8 221 @4K / 123 @128K; Qwen3.5-122B Q4 98 @4K / 65.6 @256K; Mistral-123B Q4 19 @4K. The only single-card 120B+ dense answer.