MLPerf v6.1 MiTAC 8x MI350X: llama2-70b-99.9 Offline aggregate tok/s
84016.4 tok/s AGGREGATE over 8x MI350X 288GB
Fields
id
benchmarks/mlperf-v6-1-mitac-8xmi350x-llama2-70b-offline-tokensMetric
tps_aggregate
Value
84,016.4
Unit
tok/s AGGREGATE over 8x MI350X 288GB
Model
llama2-70b-99.9
Methodology
MLPerf Inference closed-division submission. v6.1, MiTAC, 8x AMD Instinct MI350X 288GB HBM3e + 2x EPYC 9755. Offline: 'Tokens per second: 84016.4', VALID. Values quoted verbatim from the submission's mlperf_log_summary.txt (performance/run_1). AGGREGATE throughput across all accelerators in the system, NOT per-request.
Measured by
vendor
Reproducible
yes
notes
~10502 tok/s/GPU. MI350X = CDNA4, 288GB HBM3e.
Notes
~10502 tok/s/GPU. MI350X = CDNA4, 288GB HBM3e.