EPYC 9554 64c Llama-3.3-70B Q4_K_M token-gen
7.12 tok/s decode (--numa distribute)
Fields
id
benchmarks/comm-epyc-9554-llama33-70b-q4-tgMetric
decode_tok_s
Value
7.12
Unit
tok/s decode (--numa distribute)
Model
Llama-3.3-70B
Methodology
ahelpme.com llama-bench, single-socket 12x DDR5-5600 fully populated, --numa distribute
Measured by
third_party
Thermal state
unknown
notes
DeepSeek-R1-70B Q4_K_M 7.11; QwQ-32B Q4_K_M 13.9. ~7 t/s is what a FULLY POPULATED Genoa does - one DIMM alone drops to 1.5 t/s (wastes ~92% of bandwidth).
Notes
DeepSeek-R1-70B Q4_K_M 7.11; QwQ-32B Q4_K_M 13.9. ~7 t/s is what a FULLY POPULATED Genoa does - one DIMM alone drops to 1.5 t/s (wastes ~92% of bandwidth).