RTX A4000 Ollama 0.5.11: deepseek-coder-v2:16b per-request eval rate
22.89 tok/s (per-request decode, batch size 1)
Fields
id
benchmarks/dm-a4000-ollama-deepseek-coder-v2-16b-evalMetric
decode_tok_s
Value
22.89
Unit
tok/s (per-request decode, batch size 1)
Model
deepseek-coder-v2-16b
Methodology
Database Mart Ollama 0.5.11 on one RTX A4000 16 GB GDDR6 (Windows VPS, 24 cores, 32 GB RAM). Default Ollama pull, labelled 4-bit, exact GGUF quant not printed, so format_id is null. Prompt is a single short question and the output length is model-determined, so this is the eval-rate column only: per-request decode, batch size 1, NOT aggregate and NOT a prefill rate. The llama.cpp commit inside Ollama 0.5.11 is not printed. Ollama tag deepseek-coder-v2:16b. Reported GPU utilization 40%.
Measured by
third_party
Reproducible
yes
notes
Eval Rate (tokens/s) column. Per-request, not aggregate.
Notes
Eval Rate (tokens/s) column. Per-request, not aggregate.