RTX PRO 6000 Blackwell Ollama 0.13.5: llama3.3:70b eval rate
31.96 tok/s (eval rate, per-request, batch 1)
Fields
id
benchmarks/dm-pro6000-ollama-llama33-70b-evalMetric
decode_tok_s
Value
31.96
Unit
tok/s (eval rate, per-request, batch 1)
Model
llama-3.3-70b
Methodology
Ollama 0.13.5 on one RTX PRO 6000 Blackwell 96 GB GDDR7 (compute capability 12.0). llama.cpp commit not printed. Quantization is the default Ollama pull, labelled 4-bit, exact GGUF quant not printed, format_id null. Prompt is the single question 'What is GPU?'; output length is model-determined. Eval rate is per-request decode, batch size 1, NOT aggregate. Not comparable to the Hardware Corner llama-bench rows for this card, which use fixed pp/tg lengths and a named GGUF quant. This row generated 579 tokens; model size 43 GB.
Measured by
third_party
Reproducible
yes
notes
GPU util 94%, GPU memory 41%.
Notes
GPU util 94%, GPU memory 41%.