RTX Pro 4000 Blackwell Ollama: Llama 3.1 8B eval rate (Computingforgeeks)
96.8 tok/s (eval rate, per-request)
Fields
id
benchmarks/cfgk-pro4000-ollama-llama31-8b-evalMetric
decode_tok_s
Value
96.8
Unit
tok/s (eval rate, per-request)
Model
llama3.1:8b
Methodology
Computingforgeeks single-card Ollama comparison, published 2026-09-11, run on Ubuntu 26.04 with NVIDIA driver 580 and Ollama 0.11. Each card tested alone. Prompt is a fixed ~2,000-token document summarization request, output capped at 500 tokens, three runs averaged. The reported eval rate is decode tokens/s, per request, batch size 1, not an aggregate. Quantization is whatever `ollama pull` ships for the tag (page does not name the GGUF quant). RTX Pro 5000 Blackwell here is the 48 GB GDDR7 ECC card (32 GB model footprint fits); the 72 GB SKU is a different accelerator and was not tested. Card: RTX Pro 4000 Blackwell, 24 GB GDDR7 ECC, single-slot 145 W.
Measured by
third_party
Reproducible
yes
notes
Table cell Pro 4000 / llama3.1:8b eval 96.8 tok/s, prompt eval 3,420 tok/s, VRAM 5.8 GB. First dense-model eval rate for this 24 GB workstation SKU beyond the existing llama.cpp 7B Q4_0 rows. Page is a vendor-adjacent review (affiliate links) so confidence is held at 0.55.
Notes
Table cell Pro 4000 / llama3.1:8b eval 96.8 tok/s, prompt eval 3,420 tok/s, VRAM 5.8 GB. First dense-model eval rate for this 24 GB workstation SKU beyond the existing llama.cpp 7B Q4_0 rows. Page is a vendor-adjacent review (affiliate links) so confidence is held at 0.55.