Hardware Corner Mar 2026: RTX PRO 6000 Blackwell Gemma4 31B Q4_K decode @ 256k ctx
34.4 tok/s (decode, per-request, batch 1, KV depth 256k)
Fields
id
benchmarks/hc26-rtxpro6000-gemma4-31b-q4k-tg-256kMetric
decode_tok_s
Value
34.4
Unit
tok/s (decode, per-request, batch 1, KV depth 256k)
Model
Gemma4 31B Q4_K
Methodology
THIRD-PARTY LAB MEASURED (Hardware Corner per-GPU LLM benchmark page, Updated Benchmarks: March 2026). llama.cpp, single RTX PRO 6000 Blackwell (96 GB GDDR7). Context length is the KV depth the table column names. Token Generation column = batch-1 per-request decode tok/s, NOT aggregate across concurrent requests. Prompt Processing column = prefill tok/s at that context depth. Engine build hash, CUDA version and exact prompt/output token counts beyond the context-depth bucket are not published on the page except where a row is annotated llama.cpp (8288), which is noted per record.
Measured by
third_party
Reproducible
yes
Thermal state
unknown
notes
format_id is the nearest standard GGUF in this repo; the page labels the quant as shown in the model field. Single card, weights fully in the 96 GB GDDR7.
Notes
format_id is the nearest standard GGUF in this repo; the page labels the quant as shown in the model field. Single card, weights fully in the 96 GB GDDR7.