ahelpme.com — LLM inference benchmarks with llama.cpp on AMD EPYC 9554 (Genoa 64c, 12-channel DDR5-5600)

Single-socket EPYC 9554 64c Genoa, 12x DDR5-5600 (460.8 GB/s theoretical), Gentoo kernel 6.13.7, llama-bench build 51f311e0 (b4753), --numa distribute -t 64. Llama-3.3-70B Q4_K_M 7.12 t/s, DeepSeek-R1-Llama-70B 7.11, R1-8B 49.96, QwQ-32B 13.94, Llama-3.1-8B 49.64. Cache dropped + numa_balancing off before runs.