AMD Infinity Fabric (CDNA 2 / MI250X OAM: 8 links, 100 GB/s bidirectional per link, 25 GT/sec, 16 lanes)

PER LINK, PER DIRECTION, 16-WIDE. AMD'S OWN DERIVATION FORMULA, CDNA 2 white paper endnote MI200-29: 'Peak theoretical inter GDC to GDC data transport rate performance is calculated by Baud Rate * # lanes * # directions * # links / 8 = GB/s per card.' The '# directions' multiplier is AMD confirming every figure in this endnote family is BIDIRECTIONAL. AMD states both halves: the ROCm MI250 microarchitecture page gives '25 GT/sec, which corresponds to a theoretical peak transfer rate of 50 GB/sec per link (or 100 GB/sec bidirectional peak transfer bandwidth)'; the MI250X board table prints 'Peak Infinity Fabric(TM) Link Bandwidth: 100 GB/s'. 25 x 16 / 8 = 50 per direction; 25 x 16 x 2 / 8 = 100 bidirectional. GPU-TO-GPU TOTAL: the CDNA 2 white paper endnote MI200-29 gives MI250/MI250X 'up to 800GB/s peak aggregate theoretical GPU (P2P) transport rate bandwidth performance per GPU OAM card' across 'up to eight links' - and 8 x 100 = 800 exactly, so the 8 links and the 100 GB/s per link are mutually consistent rather than independently rounded. *** DIE-TO-DIE IS A DIFFERENT NUMBER AND IS NOT THE SAME QUANTITY: *** the same white paper gives 'up to 400 GB/s total aggregate theoretical inter GDC to GDC I/O data transport bandwidth per GPU' for the four links INSIDE the package joining the two GCDs, and the ROCm page gives 'four AMD Infinity Fabric links running at a theoretical peak rate of 25 GT/sec, giving 200 GB/sec peak transfer bandwidth between the two GCDs of an OAM, or a bidirectional peak transfer bandwidth of 400 GB/sec for the same'. So 400 GB/s = die-to-die inside the package, 800 GB/s = package-to-package scale-up. Identical lane rate, identical width, different link count, different meaning. HOST PATH: the ROCm MI250 page gives PCIe Gen4 x16 per GCD - two per OAM - with the explicit warning 'Note that some platforms may offer an x8 interface to the GCDs, which reduces the available host-to-GPU bandwidth', so the host number is a WIDTH x COUNT matrix, not one figure (see notes).

TWO DISTINCT FABRICS SHARE ONE LINK BUDGET, and the ROCm page is explicit that the hardware multiplexes them: 'Both interfaces can drive four AMD Infinity Fabric links. One of the AMD Infinity Fabric links of the controller at the bottom can be configured as a PCIe link.' DIE-TO-DIE: four links between the OAM's two GCDs, so an MI250X presents as one device rather than two. GPU-TO-GPU: eight links per OAM, and the CDNA 2 white paper's Figure 2c shows an ML-optimised node where 'Red and Green links can create two bi-directional rings for All-reduce communication in machine learning networks' - i.e. AMD's own answer to collective bandwidth is a RING TOPOLOGY, not a full mesh, which is a materially worse bisection profile than the fully-connected 8-GPU domain of CDNA 3/4 and is the single biggest topology difference between this generation and the next. HOST PATH: PCIe Gen4 x16 per GCD, two per OAM, optionally through a PCIe switch.

None. Direct point-to-point links and rings only; there is no AMD fabric switch at this generation. The MI250X's one fabric privilege is CPU-GPU COHERENCY on the flagship HPC topology, where an optimized 3rd-gen EPYC lets the CPU cache GPU memory with invalidation from both sides - and AMD notes the coherent host interface's physical layer 'is implemented as a 16-lane Infinity Fabric link' that 'falls back to behave like a standard PCIe interface with non-coherent communication' against other x86 processors. So the SAME physical link is coherent or not depending on the host part.

WHY A NEW RECORD: no existing record carries the MI250X OAM's 8-link scale-up count, its 800 GB/s per-card P2P aggregate, or its 800 GB/s-versus-400 GB/s collision between the package-to-package and die-to-die totals. *** THE MATRIX AMD PUBLISHES INSTEAD OF A SINGLE NUMBER: *** for the host attach path AMD gives no single bandwidth on this generation. It publishes WIDTH against POSITION: x16 or x8 per GCD, one link per GCD, two per OAM, with or without an intermediate PCIe switch. That is four configurations - 2 x 64, 2 x 32, 1 x 64, 1 x 32 GB/s bidirectional - and AMD states which is possible without committing to one. Recording a single host bandwidth here would be inventing precision AMD explicitly declined to give. This is the honest shape of the data and is why bandwidth_basis carries the matrix rather than a number. Note also that on this generation AMD's per-link figure does NOT move with the part (100 GB/s on both MI210 and MI250X) while the link count does (3 versus 8) - the clearest available proof that per-link and total are independent quantities. INFERENCE RELEVANCE: 800 GB/s of P2P against 3.2 TB/s of HBM2e is ~4:1, workable for 8-way TP, but the ring topology means bisection, not the per-link rate, is what limits large collectives.