rocm-inference-bench: Llama-3-8B BF16 on MI250X, 42.3 tok/s (README table)
42.3 tok/s (batch-1, methodology undocumented)
Fields
id
benchmarks/kuhaku-mi250x-llama3-8b-bf16-decodeMetric
decode_tok_s
Value
42.3
Unit
tok/s (batch-1, methodology undocumented)
Model
meta-llama/Llama-3-8B
Methodology
COMMUNITY-REPORTED (Kuhaku413/rocm-inference-bench, single-commit README results table). PyTorch + ROCm 6.x on MI250X. No batch size, prompt length, or repeat count documented — reported 'Tokens/sec' treated as single-request end-to-end decode-class throughput. LOW CONFIDENCE: no methodology page, no commit-pinned framework versions.
Measured by
third_party
Reproducible
no
notes
MI250X = dual-GCD CDNA2 OAM, 128GB HBM2e, 3.2 TB/s, gfx90a. 42.3 tok/s on 8B BF16 is plausible for the memory-bound regime but the source is a thin single-commit repo — marked draft/low-conf.
Notes
MI250X = dual-GCD CDNA2 OAM, 128GB HBM2e, 3.2 TB/s, gfx90a. 42.3 tok/s on 8B BF16 is plausible for the memory-bound regime but the source is a thin single-commit repo — marked draft/low-conf.