MLPerf v5.0 Dell XE7745 8x H200 NVL 141GB: llama2-70b-99.9 Offline aggregate tok/s
31149.9 tok/s AGGREGATE over 8x H200 NVL
Fields
id
benchmarks/mlperf-v5-0-dell-xe7745-8xh200nvl-llama2-70b-offline-tokensMetric
tps_aggregate
Value
31,149.9
Unit
tok/s AGGREGATE over 8x H200 NVL
Model
llama2-70b-99.9
Methodology
MLPerf Inference closed-division submission. v5.0, Dell XE7745, 8x NVIDIA H200 NVL 141GB PCIe (NVLink bridge pairs), TensorRT-LLM. Offline: 'Tokens per second: 31149.9', VALID. Values quoted verbatim from the submission's mlperf_log_summary.txt (performance/run_1). AGGREGATE throughput across all accelerators in the system, NOT per-request.
Measured by
vendor
Reproducible
yes
notes
~3894 tok/s/GPU. H200 NVL = PCIe form-factor with 4-way NVLink bridge.
Notes
~3894 tok/s/GPU. H200 NVL = PCIe form-factor with 4-way NVLink bridge.