MLPerf v5.0 Cisco X215 M8 2x L40S: gptj-99 Server aggregate tokens/s
1745.78 tok/s AGGREGATE over 2x L40S 48GB, Server
Fields
id
benchmarks/mlperf-v5-0-cisco-x215m8-2xl40s-gptj99-server-tokensMetric
tps_aggregate
Value
1,745.78
Unit
tok/s AGGREGATE over 2x L40S 48GB, Server
Model
gptj-99 (GPT-J 6B, MLPerf closed)
Methodology
MLPerf Inference v5.0 closed division, Cisco UCS X215 M8 (system id X215M8-L40Sx2_TRT), 2x NVIDIA L40S, TensorRT 10.8 / CUDA 12.8, driver 550.90, host 2x AMD EPYC 9224. gptj-99 Server scenario, PerformanceOnly. Quoted verbatim from closed/Cisco/results/X215M8-L40Sx2_TRT/gptj-99/Server/performance/run_1/mlperf_log_summary.txt: Completed tokens per second (inferred) 1745.78, Completed samples per second 25.30, Result is VALID, 99.90 percentile latency 17811939159 ns against a 20 s target. Weights fp8 per the matching measurements JSON. AGGREGATE throughput of the 2-GPU system under the Server latency constraint, NOT per-request. target_qps 25.35, samples_per_query 1, min_duration 600000 ms.
Measured by
vendor
Reproducible
yes
notes
Paired Offline row is 1754.97 tok/s on the same system. Server throughput sits within 1 percent of Offline because the submission is latency-bound near 25 queries/s, not compute-bound. Same memory-capacity caveat as the Offline record.
Notes
Paired Offline row is 1754.97 tok/s on the same system. Server throughput sits within 1 percent of Offline because the submission is latency-bound near 25 queries/s, not compute-bound. Same memory-capacity caveat as the Offline record.