AMD 'Introducing the AMD CDNA 2 Architecture' white paper - endnotes MI200-29 and MI200-43 (Infinity Fabric formula and per-card totals)

The single most useful source in the repo for this subject, because AMD states its own DERIVATION FORMULA rather than only its results. Endnote MI200-29 verbatim: 'Peak theoretical inter GDC to GDC data transport rate performance is calculated by Baud Rate * # lanes * # directions * # links / 8 = GB/s per card.' Endnote MI200-29 also gives per-card and per-link figures: 'AMD Instinct(TM) MI250/MI250X built on AMD CDNA(TM) 2 technology accelerators support AMD Infinity architecture with AMD Infinity Fabric(TM) technology providing up to 400 GB/s total aggregate theoretical inter GDC to GDC I/O data transport bandwidth per GPU', and 'AMD Instinct(TM) MI250 ... providing up to 100 GB/s peak total aggregate theoretical transport data GPU peer-to-peer (P2P) bandwidth per AMD Infinity Fabric link, and include up to eight links providing up to 800GB/s peak aggregate theoretical GPU (P2P) transport rate bandwidth performance per GPU OAM card'. Endnote MI200-43 gives the PCIe-generation parts AND their host figures, which is where the host attach number comes from: 'AMD Instinct(TM) MI210 ... support PCIe(R) Gen4 providing up to 64 GB/s peak theoretical data bandwidth from CPU to GPU per card. AMD Instinct MI210 CDNA 2 technology-based accelerators include three Infinity Fabric(TM) links providing up to 300 GB/s peak theoretical GPU to GPU or Peer-to-Peer (P2P) bandwidth performance per GPU card. Combined with PCIe Gen4 support, this provides an aggregate GPU card I/O peak bandwidth of up to 364 GB/s' - and the same endnote states the MI100 equivalent as 'up to 64 GB/s ... from CPU to GPU per card', 'three links providing up to 276 GB/s peak theoretical GPU P2P ... per GPU card', 'aggregate GPU card I/O peak bandwidth of up to 340 GB/s'. THE FORMULA IS THE CONVENTION: '# directions' as a multiplier, divided by 8 for bytes, is AMD confirming that every figure in this endnote family is BIDIRECTIONAL. Note the deliberate asymmetry in MI250X's 400 GB/s inter-GCD versus 800 GB/s P2P: the former is die-to-die inside one package, the latter is the package-to-package scale-up total, and they are different quantities that must not be substituted for each other.