inference/research
verifiedmedium · 0.75updated 2026-10-17

Geerling llama.cpp Vulkan on rtx4070ti (pi host): tinyllama-11b Q4_K_M pp4096+tg128 1609.44 tok/s

1609.44 tok/s (pp4096+tg128)

Fields

id
benchmarks/geerling-vulkan-rtx4070ti-pi-tinyllama-11b-q4km-pp4096-tg128
Metric
tps_aggregate
Value
1,609.44
Unit
tok/s (pp4096+tg128)
Model
TinyLlama-1.1B-Chat Q4_K_M GGUF (528.14 MiB, 1.10 B params)
Methodology
THIRD-PARTY MEASURED (Jeff Geerling, geerlingguy/ai-benchmarks). llama.cpp build 933414c0b (7293), backend Vulkan, default llama-bench flags (ngl 99). Host: Raspberry Pi CM5 16GB, Pi OS Trixie, PCIe-attached card. Model file size 528.14 MiB, 1.10 B params, Q4_K_M. llama-bench combined pp4096+tg128 figure: one prompt of 4096 tokens followed by 128 generated tokens, reported as a single tok/s. This is neither pure prefill nor pure decode; stored as tps_aggregate with concurrency 1. Whole-system wall power during the run 85.7 W (node scope, not device).
Measured by
third_party
Reproducible
yes
Mean power (W)
85.7
Power scope
node
Thermal state
unknown
notes
Source issue prints the llama-bench mean and the +/- spread; the stored value is the mean. The Pi-host rows are the same physical GPU on a PCIe-constrained host, so they are not a card ceiling.

Notes

Source issue prints the llama-bench mean and the +/- spread; the stored value is the mean. The Pi-host rows are the same physical GPU on a PCIe-constrained host, so they are not a card ceiling.

Cross-references