MLPerf v6.1 Dell XE7745 8x RTX PRO 4500 32GB: llama2-70b-99.9 Offline aggregate tok/s
4374.06 tok/s AGGREGATE over 8x RTX PRO 4500
Fields
id
benchmarks/mlperf-v6-1-dell-8xrtxpro4500-llama2-70b-offline-tokensMetric
tps_aggregate
Value
4,374.06
Unit
tok/s AGGREGATE over 8x RTX PRO 4500
Model
llama2-70b-99.9
Methodology
MLPerf Inference closed-division submission. v6.1, Dell XE7745, 8x RTX PRO 4500 Blackwell 32GB, TensorRT-LLM. Offline: 'Tokens per second: 4374.06', VALID. Values quoted verbatim from the submission's mlperf_log_summary.txt (performance/run_1). AGGREGATE throughput across all accelerators in the system, NOT per-request.
Measured by
vendor
Reproducible
yes
notes
~547 tok/s/GPU on 70B. Server: 3012.33 tok/s.
Notes
~547 tok/s/GPU on 70B. Server: 3012.33 tok/s.