MLPerf v6.1 Oracle 8x GB300 (Blackwell Ultra): llama2-70b-99.9 Offline aggregate tok/s
132253 tok/s AGGREGATE over 8x GB300 288GB (NVL72 partition)
Fields
id
benchmarks/mlperf-v6-1-oracle-8xgb300-llama2-70b-offline-tokensMetric
tps_aggregate
Value
132,253
Unit
tok/s AGGREGATE over 8x GB300 288GB (NVL72 partition)
Model
llama2-70b-99.9
Methodology
MLPerf Inference closed-division submission. v6.1, Oracle GB300-NVL72 aarch64x8 partition (8x GB300 288GB + Grace), TensorRT-LLM. Offline: 'Tokens per second: 132253', VALID. Values quoted verbatim from the submission's mlperf_log_summary.txt (performance/run_1). AGGREGATE throughput across all accelerators in the system, NOT per-request.
Measured by
vendor
Reproducible
yes
notes
~16532 tok/s/GPU. Per-GPU record for the same part exists under nvidia-b300 submissions.
Notes
~16532 tok/s/GPU. Per-GPU record for the same part exists under nvidia-b300 submissions.