NVIDIA GB200 NVL72 (per-accelerator)

AUDIT CORRECTION 2026-10-04, REVISED 2026-10-04 after adversarial re-verification. The FP64 'other' row was dense:false, which instructed every reader to halve a figure that carries no sparsity superscript at all. It is now dense:true. THE FLAG IS PROBABLY RIGHT BUT THE ORIGINAL EVIDENCE WAS FABRICATED: this note previously quoted a Blackwell datasheet footnote as 'All Tensor Core numbers except FP64 with sparsity.' No NVIDIA document contains that wording. What the Oct-2025 Blackwell datasheet's footnote 1 actually reads is the bare 'Specifications in sparse | dense', and the GB300 product page uses a third convention entirely - 'All Tensor Core specifications are with sparsity unless otherwise noted.' The defensible basis for dense:true is therefore the ABSENCE of any sparsity superscript on the FP64 row, not a quoted carve-out. Kept as dense:true because the absence is itself the evidence and no source supports halving it; the invented quotation is removed because a fabricated citation is worse than a missing one. This resolves the dense/sparse half of the contest; the 40-vs-37 NVIDIA/AMD disagreement that remains is a spec-table COLUMN difference (GB200 NVL72 prints 40, HGX B200 prints 37, AMD quotes the 37), not a dense/sparse dispute, and is documented on the FP64 row. The tensor rows on this record were independently re-verified against NVIDIA's Blackwell datasheet GB200 NVL72 column and are CORRECT: the column prints NVFP4 20 PFLOPS, FP8/FP6 10, INT8 10 POPS, FP16/BF16 5, TF32 2.5, FP32 80 under footnote 2, 'Specifications in sparse. Dense is one-half of the sparse spec shown', so dense NVFP4 10 / FP8 5 / INT8 5 / FP16-BF16 2.5 / TF32 1.25 PFLOPS per GPU are exactly what is stored. GB200 NVL72 is a RACK-SCALE system (72 Blackwell GPUs + 36 Grace CPUs, 2,592 Neoverse V2 cores), not a single accelerator; all flops here are PER-GPU, derived by dividing NVIDIA's system figures by 72 (1,440 NVFP4 / 72 = 20; 720 FP8 / 72 = 10; 360 FP16/BF16 / 72 = 5; 180 TF32 / 72 = 2.5). vram_gb 186 and memory_bandwidth 8 TB/s are the GB200 NVL72 per-GPU column of the same table and are correct for this record - do NOT carry them onto nvidia-b200, which is the 180 GB / 7.7 TB/s HGX B200 column. tdp_w 1200 is the per-GPU 'configurable up to 1,200W', NOT the rack total. The distinguishing spec here is INTERCONNECT, not FLOPS: 130 TB/s of NVLink across a single 72-GPU domain via NVLink Switch, i.e. ~1.8 TB/s per GPU - 2x H100 SXM's 900 GB/s - which is what lets a 72-way tensor-parallel model behave like one machine. Do not confuse this record with a whole GB200 superchip (the paired superchip is 20 sparse PFLOPS NVFP4 and 372 GB total).