MLPerf v6.1 NVIDIA GB300 NVL72 slice, 8x GB300-288GB aarch64 across 2 nodes, llama2-70b-99.9 Offline run_1 harness summary log

Fields

id

sources/bench-mlperf-v6-1-nvidia-gb300-8gpu-llama2-offline-log

URL

https://raw.githubusercontent.com/mlcommons/inference_results_v6.1/main/closed/NVIDIA/results/GB300-NVL72_GB300-288GB_aarch64x8_TRT/llama2-70b-99.9/Offline/performance/run_1/mlperf_log_summary.txt

Publisher

MLCommons

Kind

benchmark

Accessed

2026-10-04

notes

Result is : VALID. Samples per second 481.461, Tokens per second 131464. THIS IS THE MOST IMPORTANT NUMBER IN THE v6.1 SET FOR ANYONE PLANNING A CLUSTER, and it is the opposite of the rack number: NVIDIA's own 8-GPU (2-node) slice of a GB300 NVL72 reaches 131464 tok/s, while CoreWeave's full 72-GPU rack reaches 1136100 tok/s. Nine times the GPUs for 8.6x the throughput - and per-GPU the 8-GPU slice is 16433 tok/s/GPU against 15779 for the rack. Buying a whole NVL72 for a throughput-bound Llama 2 70B workload is measurably worse per dollar than nine 8-GPU nodes, because the NVL72's scale-up fabric and its power/cooling envelope buy nothing at this model size.

Notes

Result is : VALID. Samples per second 481.461, Tokens per second 131464. THIS IS THE MOST IMPORTANT NUMBER IN THE v6.1 SET FOR ANYONE PLANNING A CLUSTER, and it is the opposite of the rack number: NVIDIA's own 8-GPU (2-node) slice of a GB300 NVL72 reaches 131464 tok/s, while CoreWeave's full 72-GPU rack reaches 1136100 tok/s. Nine times the GPUs for 8.6x the throughput - and per-GPU the 8-GPU slice is 16433 tok/s/GPU against 15779 for the rack. Buying a whole NVL72 for a throughput-bound Llama 2 70B workload is measurably worse per dollar than nine 8-GPU nodes, because the NVL72's scale-up fabric and its power/cooling envelope buy nothing at this model size.

Cross-references

This record references no other records.