verifiedhigh · 0.85updated 2026-10-14

MLPerf v5.1 IBM Cloud gx3 1x L40S vLLM: llama3.1-8b Offline aggregate tokens/s

1642.22 tok/s (single L40S 48GB, Offline)

Fields

benchmarks/mlperf-v5-1-ibm-1xl40s-vllm-llama31-8b-offline-tokens

Metric

tps_aggregate

Value

1,642.22

Unit

tok/s (single L40S 48GB, Offline)

Model

llama3.1-8b

Methodology

MLPerf Inference closed-division submission. v5.1, IBM Cloud gx3-48x240x2l40s (only 1 of 2 L40S in test), vLLM 0.10.0, RHEL 9.6, CUDA 12.9. Offline: 'Tokens per second: 1642.22'. Values quoted verbatim from the submission's mlperf_log_summary.txt (performance/run_1). AGGREGATE throughput across all accelerators in the system, NOT per-request.

Measured by

vendor

Reproducible

yes

Cross-references

MLPerf v5.1 IBM Cloud gx3 1x L40S vLLM: llama3.1-8b Offline aggregate tokens/s

Fields

Cross-references

Accelerators (1)

Engine (1)

Interconnect (1)