MLPerf v6.1 Cisco C240M8 5x RTX PRO 4500 32GB: llama3.1-8b Offline aggregate tok/s
12017.7 tok/s AGGREGATE over 5x RTX PRO 4500
Fields
id
benchmarks/mlperf-v6-1-cisco-5xrtxpro4500-llama31-8b-offline-tokensMetric
tps_aggregate
Value
12,017.7
Unit
tok/s AGGREGATE over 5x RTX PRO 4500
Model
llama3.1-8b
Methodology
MLPerf Inference closed-division submission. v6.1, Cisco UCS C240 M8, 5x RTX PRO 4500 Blackwell 32GB PCIe, TensorRT-LLM. Offline: 'Tokens per second: 12017.7', VALID. Values quoted verbatim from the submission's mlperf_log_summary.txt (performance/run_1). AGGREGATE throughput across all accelerators in the system, NOT per-request.
Measured by
vendor
Reproducible
yes
notes
~2403 tok/s per 4500. Same system also submitted Interactive+Server.
Notes
~2403 tok/s per 4500. Same system also submitted Interactive+Server.