EPYC 9554 DeepSeek-R1-Distill-Llama-70B Q4_K_M tg2048 (ahelpme)
7.03 tok/s (tg decode, batch 1, per-request)
Fields
id
benchmarks/ah-epyc9554-deepseek-r1-llama-70b-q4km-tg2048Metric
decode_tok_s
Value
7.03
Unit
tok/s (tg decode, batch 1, per-request)
Model
DeepSeek-R1-Distill-Llama-70B
Methodology
ahelpme.com measured llama-bench, AMD EPYC 9554 Genoa 64C single-socket, 12ch DDR5-5600 (~460.8 GB/s), --numa distribute, -t 64, -p 0, -n 128..2048. BLAS+RPC backend, tg2048 decode tok/s batch-1, llama.cpp build 51f311e0 (b4753).
Measured by
third_party
Reproducible
yes
Thermal state
unknown
notes
Single-socket Genoa, all 12 DDR5 channels populated. CPU-only decode; memory-bandwidth-bound.
Notes
Single-socket Genoa, all 12 DDR5 channels populated. CPU-only decode; memory-bandwidth-bound.