llama.cpp Discussion #15013 — Performance of llama.cpp on Nvidia CUDA (community scoreboard)

Curated maintainer-verified community scoreboard for llama.cpp CUDA backend: llama-2-7b.Q4_0, pp512/tg128, each row carries a build commit + named reporter. The canonical cross-GPU llama-bench comparison table. Re-read 2026-10-16: the FA-on table now also carries RTX PRO 4500 Blackwell tg128 (168.90, build becc481, @Hedede), RTX 4090 D (pp512 12506.97 / tg128 191.57, build 79c1160, @autonomous-ai-lab), RTX 5090 Laptop (pp512 7641.89 / tg128 158.14, build 29f1482, @rs38), Quadro RTX 6000 (pp512 2941.85 / tg128 93.11, build 1e74897, @Hedede) and Quadro RTX 8000 (pp512 2934.33 / tg128 91.88, build 79c1160, @Hedede). The 5090 Laptop row prints a 384-bit bus that the spec sheet does not support. Re-read 2026-10-17: the no-FA half of the table had never been filed, and the FA rows for the smaller cards (GTX 1660/1070 Ti/1060, Quadro T1000/P2000/P1000, Tesla P4/K80, P106-100, TITAN Xp, Quadro RTX 4000, RTX 2060 SUPER, RTX 4050 Laptop, Jetson AGX Orin, DGX Spark) were missing too. Both halves are now filed. The Quadro T1000 FA row prints the same 27.46 figure for pp512 and tg128, so only one record was stored. The Quadro T1000 memory cell says GDDR5; the spec sheet says GDDR6.