RTX PRO 5000 Blackwell Ollama: Llama 3.3 70B eval rate (Database Mart)
21.42 tok/s (eval rate, per-request)
Fields
id
benchmarks/dm-pro5000-ollama-llama33-70b-evalMetric
decode_tok_s
Value
21.42
Unit
tok/s (eval rate, per-request)
Model
llama3.3:70b
Methodology
Database Mart hosted GPU-server benchmark of one NVIDIA RTX Pro 5000 Blackwell Server Edition (48 GB GDDR7 ECC, 300 W, PCIe 5.0 x16, driver 580.65.06, CUDA 13.0). Engine is Ollama 0.12.3 via the Ollama API (llama.cpp backend; Ollama build commit not printed). Quantization is the default Ollama pull for each tag, which the report labels INT4 and does not further specify (not asserted as Q4_K_M). Prompt is fixed at 2048 tokens (fixed-length text file, 50 repetitions); output length is not stated. The published column is eval rate (tokens/s), which is decode, per request, batch size 1, NOT an aggregate across concurrent requests. prompt eval rate is a separate column and is not recorded here. Card is the 48 GB Server Edition, not the 72 GB workstation SKU.
Measured by
third_party
Reproducible
yes
notes
Table row llama3.3:70b, 42 GB model, load 62.34 s, prompt eval 651.98 tok/s, eval 21.42 tok/s. Fits in the 48 GB card. Compare Hardware Corner llama.cpp Llama 3.3 70B Q4_K generation 24.0 tok/s at 4k on the same SKU: different engine, quant, and prompt length, so the gap is expected and not a contest.
Notes
Table row llama3.3:70b, 42 GB model, load 62.34 s, prompt eval 651.98 tok/s, eval 21.42 tok/s. Fits in the 48 GB card. Compare Hardware Corner llama.cpp Llama 3.3 70B Q4_K generation 24.0 tok/s at 4k on the same SKU: different engine, quant, and prompt length, so the gap is expected and not a contest.