MLPerf v5.0 Dell XE7745 8x L40S: llama2-70b-99.9 Offline aggregate tokens/s
3481.53 tok/s AGGREGATE over 8x L40S 48GB
Fields
id
benchmarks/mlperf-v5-0-dell-xe7745-8xl40s-llama2-70b-offline-tokensMetric
tps_aggregate
Value
3,481.53
Unit
tok/s AGGREGATE over 8x L40S 48GB
Model
llama2-70b-99.9
Methodology
MLPerf Inference closed-division submission. v5.0, Dell PowerEdge XE7745, 8x NVIDIA L40S 48GB PCIe, TensorRT-LLM. Offline scenario: 'Tokens per second: 3481.53', VALID. Values quoted verbatim from the submission's mlperf_log_summary.txt (performance/run_1). AGGREGATE throughput across all accelerators in the system, NOT per-request.
Measured by
vendor
Reproducible
yes
notes
Per-GPU ~435 tok/s aggregate. See also Server scenario 3201.13 tok/s.
Notes
Per-GPU ~435 tok/s aggregate. See also Server scenario 3201.13 tok/s.