MLPerf Inference v5.0 HPE ProLiant DL380a Gen12 8x L40S llama2-70b-99.9 raw logs
Fields
sources/bench-mlperf-v5-0-hpe-dl380a-8xl40s-llama2-70bNotes
Closed-division datacenter submission, system id HPE_ProLiant_DL380a_L40S_PCIe_48GBx8_TRT_9.0. README system table: 8x NVIDIA L40S, 48 GB each, host interconnect listed as PCIe Gen5 Switch, accelerator frequency 1065 MHz, 2x Intel Xeon 6740E (96 cores), 1024 GB host memory, Ubuntu 24.04.1. Software table prints framework TensorRT 9.0 with CUDA 12.8 and driver 570.86.15, then sw_notes states the Llama2-70b workload used the MLPerf v4.1 TensorRT-LLM stack instead, so that is the engine recorded. The same README labels the memory configuration HBM2e, which is wrong for an L40S (GDDR6 ECC); treated as a submission error and not used. Raw log llama2-70b-99.9/Offline/performance/run_1/mlperf_log_summary.txt prints Tokens per second 3655.89, Result is VALID, samples_per_query 31680, target_qps 12, ttft_latency 2e9 ns, tpot_latency 2e8 ns. The Server run_1 log prints Completed tokens per second 3181.65, Completed samples per second 10.90, Result is VALID, samples_per_query 1, target_qps 11, and 99.00 percentile first token latency 525500719 ns. The 99 and 99.9 accuracy targets share these performance numbers in the README results table. Dell's 8x L40S submission in the same round scored 3481.53 Offline and 3201.13 Server on the same model.
Cross-references
This record references no other records.