rocm-inference-bench: Llama-3-8B FP8 on MI300X, 68.7 tok/s (README table)
68.7 tok/s (batch-1, methodology undocumented)
Fields
id
benchmarks/kuhaku-mi300x-llama3-8b-fp8-decodeMetric
decode_tok_s
Value
68.7
Unit
tok/s (batch-1, methodology undocumented)
Model
meta-llama/Llama-3-8B
Methodology
COMMUNITY-REPORTED (Kuhaku413/rocm-inference-bench, single-commit README results table). PyTorch + ROCm 6.x on MI250X. No batch size, prompt length, or repeat count documented — reported 'Tokens/sec' treated as single-request end-to-end decode-class throughput. LOW CONFIDENCE: no methodology page, no commit-pinned framework versions.
Measured by
third_party
Reproducible
no
notes
MI300X 192GB HBM3 — first FP8 llama row in repo at this confidence tier; keep draft pending a corroborating run.
Notes
MI300X 192GB HBM3 — first FP8 llama row in repo at this confidence tier; keep draft pending a corroborating run.