Geerling llama.cpp Vulkan on rtx4070ti (pi host): gpt-oss-20b Q4_K_M tg128 92.19 tok/s
92.19 tok/s (tg128)
Fields
id
benchmarks/geerling-vulkan-rtx4070ti-pi-gpt-oss-20b-q4km-tg128Metric
decode_tok_s
Value
92.19
Unit
tok/s (tg128)
Model
gpt-oss-20b-Q4_K_M.gguf (10.81 GiB, 20.91 B params)
Methodology
THIRD-PARTY MEASURED (Jeff Geerling, geerlingguy/ai-benchmarks). llama.cpp build 933414c0b (7293), backend Vulkan, default llama-bench flags (ngl 99). Host: Raspberry Pi CM5 16GB, Pi OS Trixie, PCIe-attached card. Model file size 10.81 GiB, 20.91 B params, Q4_K_M. tg128 mean decode tok/s, per-request, batch 1, generation length 128. NOT aggregate. Whole-system wall power during the run 102.2 W (node scope, not device).
Measured by
third_party
Reproducible
yes
Mean power (W)
102.2
Power scope
node
Thermal state
unknown
notes
Source issue prints the llama-bench mean and the +/- spread; the stored value is the mean. The Pi-host rows are the same physical GPU on a PCIe-constrained host, so they are not a card ceiling.
Notes
Source issue prints the llama-bench mean and the +/- spread; the stored value is the mean. The Pi-host rows are the same physical GPU on a PCIe-constrained host, so they are not a card ceiling.