MLPerf v5.0 Dell XE7745 8x L40S: llama2-70b-99.9 Server aggregate tokens/s
3201.13 tok/s AGGREGATE over 8x L40S 48GB
Fields
id
benchmarks/mlperf-v5-0-dell-xe7745-8xl40s-llama2-70b-server-tokensMetric
tps_aggregate
Value
3,201.13
Unit
tok/s AGGREGATE over 8x L40S 48GB
Model
llama2-70b-99.9
Methodology
MLPerf Inference closed-division submission. v5.0, Dell XE7745, 8x L40S, TensorRT-LLM. Server scenario: 'Completed tokens per second: 3201.13', 11.05 completed samples/s, VALID. Values quoted verbatim from the submission's mlperf_log_summary.txt (performance/run_1). AGGREGATE throughput across all accelerators in the system, NOT per-request.
Measured by
vendor
Reproducible
yes