MLPerf v6.1 Cisco C240 M8 5x RTX PRO 4500 32GB: llama3.1-8b Interactive 8494.68 tok/s
8494.68 tok/s aggregate Interactive
Fields
id
benchmarks/mlperf-v6-1-cisco-5xrtxpro4500-llama31-8b-interactive-tokensMetric
tps_aggregate
Value
8,494.68
Unit
tok/s aggregate Interactive
Model
Llama-3.1-8B
Methodology
MLPerf Inference v6.1 published result (Cisco UCS C240 M8, 5x RTX PRO 4500 32GB; software: TensorRT 10.14, CUDA 13.1). System-AGGREGATE output tokens/s for the named scenario, submitter-measured under MLCommons audit.
Measured by
vendor
Reproducible
yes
notes
ENGINE_ID NOTE: pinned to 'tensorrt-llm' as the closest existing engine record; the submission's actual stack is named in this methodology string.
Notes
ENGINE_ID NOTE: pinned to 'tensorrt-llm' as the closest existing engine record; the submission's actual stack is named in this methodology string.