MLPerf v5.0 Cisco X215 M8 2x L40S: gptj-99 Offline aggregate tokens/s
1754.97 tok/s AGGREGATE over 2x L40S 48GB, Offline
Fields
id
benchmarks/mlperf-v5-0-cisco-x215m8-2xl40s-gptj99-offline-tokensMetric
tps_aggregate
Value
1,754.97
Unit
tok/s AGGREGATE over 2x L40S 48GB, Offline
Model
gptj-99 (GPT-J 6B, MLPerf closed)
Methodology
MLPerf Inference v5.0 closed division, Cisco UCS X215 M8 (system id X215M8-L40Sx2_TRT), 2x NVIDIA L40S, TensorRT 10.8 / CUDA 12.8, driver 550.90, host 2x AMD EPYC 9224. gptj-99 Offline scenario, PerformanceOnly. Quoted verbatim from closed/Cisco/results/X215M8-L40Sx2_TRT/gptj-99/Offline/performance/run_1/mlperf_log_summary.txt: Tokens per second (inferred) 1754.97, Samples per second 25.4343, Result is VALID. Weights fp8 per the matching measurements JSON (weight_transformations: quantization, affine fusion); the log does not distinguish e4m3 from e5m2, so format_id is the repo fp8 bucket. AGGREGATE throughput of the 2-GPU system, NOT per-request and NOT per GPU. samples_per_query 16797, target_qps 25.45.
Measured by
vendor
Reproducible
yes
notes
Paired Server row is 1745.78 tok/s on the same system. The system JSON lists accelerator_memory_capacity 64 GB for the pair, which is less than 2x48 GB; the summary row still names this system and counts 2 accelerators. No gptj-99.9 submission exists for this system.
Notes
Paired Server row is 1745.78 tok/s on the same system. The system JSON lists accelerator_memory_capacity 64 GB for the pair, which is less than 2x48 GB; the summary row still names this system and counts 2 accelerators. No gptj-99.9 submission exists for this system.