MLPerf v6.1 Dell XE7745 8x MI350P: llama2-70b-99.9 Offline aggregate tok/s
42286.2 tok/s AGGREGATE over 8x MI350P
Fields
id
benchmarks/mlperf-v6-1-dell-8xmi350p-llama2-70b-offline-tokensMetric
tps_aggregate
Value
42,286.2
Unit
tok/s AGGREGATE over 8x MI350P
Model
llama2-70b-99.9
Methodology
MLPerf Inference closed-division submission. v6.1, Dell XE7745, 8x AMD Instinct MI350P (PCIe, 144GB) + 2x EPYC 9455. Offline: 42286.2 tok/s, VALID. Values quoted verbatim from the submission's mlperf_log_summary.txt (performance/run_1). AGGREGATE throughput across all accelerators in the system, NOT per-request.
Measured by
vendor
Reproducible
yes
notes
~5286 tok/s/GPU. MI350P is the PCIe air-cooled CDNA4 sibling.
Notes
~5286 tok/s/GPU. MI350P is the PCIe air-cooled CDNA4 sibling.