A Domain-Specific Supercomputer for Training Deep Neural Networks (TPU v2/v3 architecture; ISCA 2017, CACM reprint 2020)

THE ONLY FIRST-PARTY SOURCE for TPU v2 per-chip figures, and the only first-party source for the v2/v3 per-chip ICI link rate - Google's cloud.google.com/tpu/docs/v2 page publishes NO specification table at all (it is a configurations page only; confirmed against both the live page and the 2024-04-10 Wayback capture), and cloud.google.com/tpu/docs/v3 publishes pod-scale and per-chip compute/memory but NO per-chip ICI bandwidth. Per-chip ICI for v2/v3 therefore has no docs-page home and this paper is the only place it is stated. Carries Table 3 'Key processor features' with TPUv1/TPUv2/TPUv3/Volta columns: peak TFLOPs per chip (v2 46 16-bit / 3 32-bit; v3 123 16-bit / 4 32-bit), network links x Gbits/s per chip (v2 4x496; v3 4x656), max chips per supercomputer (v2 256; v3 1024), peak PFLOPs per supercomputer (v2 11.8; v3 126), bisection Tbit/s per supercomputer (v2 15.9; v3 42.0), clock 700/940 MHz, TDP per chip 280/450 W, die size <611/<648 mm2, chip technology >12nm for both, memory size 32MiB on-chip + 16GiB (v2) / 32GiB (v3) off-chip, memory GB/s per chip 700 (v2) / 900 (v3), MXUs per core 1 (v2) / 2 (v3) at 128x128, cores per chip 2, chips per CPU host 4 (v2) / 8 (v3). URL CAVEAT: the canonical ACM Digital Library copy (dl.acm.org/doi/10.1145/3360307) is behind a Cloudflare bot challenge and was not directly reachable during the 2026-10-05 pass; this CACM Communications reprint is the same paper by the same authors and was read end-to-end. The Table 3 numbers were read from the article's figure image because the ACM HTML carries the table as a raster attachment, not as text.