NVIDIA Tesla P100 whitepaper (GP100, WP-08019-001 v01.1) - memory interface, HBM2 stacks, NVLink links, process

FIRST-PARTY NVIDIA whitepaper for GP100 / Tesla P100, WP-08019-001 v01.1, retrieved and read verbatim on 2026-10-04. This is the source that carries the memory-bus WIDTH, which neither the P100 datasheet nor any other reachable NVIDIA page publishes. VERBATIM, Table 1 'Tesla P100 Compared to Prior Generation Tesla products', K40 then M40 then P100: 'SMs 15 24 56', 'TPCs 15 24 28', 'FP32 CUDA Cores / SM 192 128 64', 'FP32 CUDA Cores / GPU 2880 3072 3584', 'FP64 CUDA Cores / SM 64 4 32', 'FP64 CUDA Cores / GPU 960 96 1792', 'Base Clock 745 MHz 948 MHz 1328 MHz', 'GPU Boost Clock 810/875 MHz 1114 MHz 1480 MHz', 'Peak FP32 GFLOPs[1] 5040 6840 10600', 'Peak FP64 GFLOPs[1] 1680 210 5300', 'Texture Units 240 192 224', 'Memory Interface 384-bit GDDR5 384-bit GDDR5 4096-bit HBM2', 'Memory Size Up to 12 GB Up to 24 GB 16 GB', 'L2 Cache Size 1536 KB 3072 KB 4096 KB', 'Register File Size / SM 256 KB 256 KB 256 KB', 'Register File Size / GPU 3840 KB 6144 KB 14336 KB', 'TDP 235 Watts 250 Watts 300 Watts', 'Transistors 7.1 billion 8 billion 15.3 billion', 'GPU Die Size 551 mm2 601 mm2 610 mm2', 'Manufacturing Process 28-nm 28-nm 16-nm FinFET'. Footnote 1 verbatim: 'The GFLOPS in this chart are based on GPU Boost Clocks.' VERBATIM, HBM2 chapter: 'Where HBM1 was limited to 125 GB/sec of bandwidth per stack, P100 supports 180GB/sec per stack with HBM2. As shown in the GP100 full-chip block diagram (Figure 7), the GP100 GPU connects to four HBM2 DRAM stacks. Two 512-bit memory controllers connect to each HBM2 stack for an effective 4096-bit-wide HBM2 memory interface. Initially, Tesla P100 accelerators will ship with four 4-die HBM2 stacks, for a total of 16 GB of HBM2 memory.' VERBATIM, GPU architecture chapter: 'A full GP100 consists of six GPCs, 60 Pascal SMs, 30 TPCs (each including two SMs), and eight 512-bit memory controllers (4096 bits total). ... The Tesla P100 accelerator uses 56 SM units. ... each HBM2 DRAM stack is controlled by a pair of memory controllers.' VERBATIM, NVLink chapter: 'A single Link supports up to 40 GB/sec of bidirectional bandwidth between the endpoints. Multiple Links can be combined to form Gangs for even higher-bandwidth connectivity between processors. The NVLink implementation in Tesla P100 supports up to four Links, enabling ganged configurations with aggregate maximum bidirectional bandwidth of 160 GB/sec.' NO SPARSITY FOOTNOTE OF ANY KIND APPEARS ON THIS DOCUMENT, which is why every P100 flops row in the P100 record is dense:true with no halving. This is the same evidence rule applied elsewhere in this repo: a figure is halved only when the row's own footnote says so. ARITHMETIC CHECK, and it is the check that makes the 4096-bit figure trustworthy rather than merely quoted: 4 stacks x 180 GB/s = 720 GB/s against the datasheet's published 732 GB/s aggregate. The whitepaper's per-stack figure and the datasheet's system figure agree to within the width of the ECC/dummy-bit accounting, and both independently require a 4096-bit bus (4096 x 1.875 Gbps effective / 8 = 960 GB/s theoretical before the HBM2 efficiency loss that lands it at 720-732).