LeaseWeb: Llama-2-7B Q4_K_M on dual EPYC 9334, ~28 tok/s decode (24t)
28.0 tok/s (batch-1 decode, 24 threads)
Fields
id
benchmarks/leaseweb-epyc9334x2-llama2-7b-q4km-decodeMetric
decode_tok_s
Value
28
Unit
tok/s (batch-1 decode, 24 threads)
Model
Llama-2-7B Q4_K_M
Methodology
THIRD-PARTY MEASURED (LeaseWeb blog, Fran Baigorri Hauen, 2026-04-05). HPE ProLiant DL385 Gen11, 2x EPYC 9334 (Zen4, 32c/64t each, 2.7GHz, 210W, 128MB L3/socket), 64GB DDR5 total (12-channel/socket). llama-bench (llama.cpp) local run + OpenLLM/llmperf API-level check. Test config: 512-token prompt, 128 tokens generated, 24 CPU threads for LLM rows. Values read off published charts/prose; CPU util 20-30%. Q4_K_M gguf; 512+128 config; 24 threads.
Measured by
third_party
Reproducible
no
notes
TL;DR band '20-28 tok/s on Q4 7B-20B' - Llama-2-7B is the top of that band in the chart; recorded at the band's upper value with a note.
Notes
TL;DR band '20-28 tok/s on Q4 7B-20B' - Llama-2-7B is the top of that band in the chart; recorded at the band's upper value with a note.