RTX A4000 Ollama 0.5.11: qwen2.5:7b per-request eval rate
52.68 tok/s (per-request decode, batch size 1)
Fields
id
benchmarks/dm-a4000-ollama-qwen25-7b-evalMetric
decode_tok_s
Value
52.68
Unit
tok/s (per-request decode, batch size 1)
Model
qwen2.5-7b
Methodology
Database Mart Ollama 0.5.11 on one RTX A4000 16 GB GDDR6 (Windows VPS, 24 cores, 32 GB RAM). Default Ollama pull, labelled 4-bit, exact GGUF quant not printed, so format_id is null. Prompt is a single short question and the output length is model-determined, so this is the eval-rate column only: per-request decode, batch size 1, NOT aggregate and NOT a prefill rate. The llama.cpp commit inside Ollama 0.5.11 is not printed. Ollama tag qwen2.5:7b. Reported GPU utilization 12%.
Measured by
third_party
Reproducible
yes
notes
Eval Rate (tokens/s) column. Per-request, not aggregate.
Notes
Eval Rate (tokens/s) column. Per-request, not aggregate.