Geerling llama.cpp Vulkan on rtxa4000 (pc host): llama-32-3b Q4_K_M pp4096 4667.59 tok/s
4667.59 tok/s (pp4096)
Fields
id
benchmarks/geerling-vulkan-rtxa4000-pc-llama-32-3b-q4km-pp4096Metric
prefill_tok_s
Value
4,667.59
Unit
tok/s (pp4096)
Model
Llama-3.2-3B Q4_K_M GGUF (1.87 GiB, 3.21 B params)
Methodology
THIRD-PARTY MEASURED (Jeff Geerling, geerlingguy/ai-benchmarks). llama.cpp build not printed, backend Vulkan, default llama-bench flags (ngl 99). Host: Intel Core Ultra 265K, 64GB DDR5, Ubuntu 25.10. Model file size 1.87 GiB, 3.21 B params, Q4_K_M. prefill tok/s, prompt length 4096, per-request, batch 1. NOT aggregate. Whole-system wall power during the run 284.0 W (node scope, not device).
Measured by
third_party
Reproducible
yes
Mean power (W)
284
Power scope
node
Thermal state
unknown
notes
Source issue prints the llama-bench mean and the +/- spread; the stored value is the mean. The Pi-host rows are the same physical GPU on a PCIe-constrained host, so they are not a card ceiling.
Notes
Source issue prints the llama-bench mean and the +/- spread; the stored value is the mean. The Pi-host rows are the same physical GPU on a PCIe-constrained host, so they are not a card ceiling.