MLPerf v4.1 AMD 1x MI300X: llama2-70b-99.9 Offline aggregate tok/s
3062.72 tok/s (single MI300X 192GB)
Fields
id
benchmarks/mlperf-v4-1-amd-1xmi300x-llama2-70b-offline-tokensMetric
tps_aggregate
Value
3,062.72
Unit
tok/s (single MI300X 192GB)
Model
llama2-70b-99.9
Methodology
MLPerf Inference closed-division submission. v4.1, AMD reference system 1x MI300X 192GB + 2x EPYC 9374F, ROCm+vLLM. Offline: 3062.72 tok/s, VALID. SINGLE GPU. Values quoted verbatim from the submission's mlperf_log_summary.txt (performance/run_1). AGGREGATE throughput across all accelerators in the system, NOT per-request.
Measured by
vendor
Reproducible
yes