EPYC Genoa, one 64GB DIMM, 34B Q4_0 decode (llama.cpp #3167)
1.5 tok/s, single-request llama.cpp CPU decode, one of twelve channels populated
Fields
id
benchmarks/llamacpp-epyc-genoa-one-dimm-34b-q4-decodeMetric
decode_tok_s
Value
1.5
Unit
tok/s, single-request llama.cpp CPU decode, one of twelve channels populated
Model
34B Q4_0 (family not named in the thread, ~34B dense)
Methodology
Community measurement reported in ggml-org/llama.cpp discussion #3167: a Genoa-class EPYC box with only ONE 64GB DIMM installed (a single memory channel of the twelve populated) running a 34B Q4_0 model under llama.cpp. Exact model family, prompt length, context length and llama.cpp commit are not stated in the captured thread, so this records the reported decode rate only. The number exists to quantify the channel-population effect, not as a hardware ceiling. Third-party forum measurement, not a vendor claim and not a controlled bench.
Measured by
third_party
Reproducible
no
Thermal state
unknown
notes
Pair with llamacpp-epyc-genoa-two-dimm-34b-q4-decode (4 tok/s once a second stick was added): one DIMM delivers ~37% of the two-DIMM rate and wastes ~92% of the socket's theoretical bandwidth. This is the measurement behind gotcha-epyc-must-populate-all-memory-channels. The accelerator id points at the 9554 record as the representative Genoa part; the thread does not name the exact SKU. Do not read 1.5 tok/s as what a populated Genoa does — the fully-populated 12-channel 9554 measures ~7 tok/s on 70B Q4_K_M (comm-epyc-9554-llama33-70b-q4-decode).
Notes
Pair with llamacpp-epyc-genoa-two-dimm-34b-q4-decode (4 tok/s once a second stick was added): one DIMM delivers ~37% of the two-DIMM rate and wastes ~92% of the socket's theoretical bandwidth. This is the measurement behind gotcha-epyc-must-populate-all-memory-channels. The accelerator id points at the 9554 record as the representative Genoa part; the thread does not name the exact SKU. Do not read 1.5 tok/s as what a populated Genoa does — the fully-populated 12-channel 9554 measures ~7 tok/s on 70B Q4_K_M (comm-epyc-9554-llama33-70b-q4-decode).