verifiedmedium · 0.70updated 2026-10-15

EPYC 9554 Llama 3.1 8B Instruct (Q4_K_M) decode

49.64 tok/s (tg128)

Fields

benchmarks/comm-epyc-9554-llama31-8b-q4-decode

Metric

decode_tok_s

Value

49.64

Unit

tok/s (tg128)

Model

Llama 3.1 8B Instruct (Q4_K_M)

Methodology

ahelpme.com measured on AMD EPYC 9554 64c Genoa, ASUS K14PA-U24-T, 12x16GB DDR5-5600 (460.8 GB/s), Gentoo kernel 6.13.7, llama-bench build 51f311e0 (b4753), --numa distribute -t 64, cache dropped, numa_balancing off. Value is the tg128 column. Batch-1 per-request, BLAS backend. Third-party measured.

Measured by

third_party

Reproducible

yes

Cross-references

EPYC 9554 Llama 3.1 8B Instruct (Q4_K_M) decode

Fields

Cross-references

Accelerators (1)

Engine (1)

Format (1)