id
interconnect/amd-infinity-fabric-host-io-path
Version
Per generation: MI100 -> MI210/MI250X -> MI300X/MI325X -> MI350X/MI355X
Bandwidth (GB/s, per link, unidirectional)
128
Bandwidth basis
HOST-TO-GPU / I-O ATTACH, BIDIRECTIONAL, PCIe-class - DELIBERATELY NOT A GPU-TO-GPU NUMBER. This record exists because the repo kept conflating the two. THE CDNA 4 VALUE, 128 GB/s bidirectional: AMD's own board tables label it 'Scale-out (Peak) AMD Infinity Fabric(TM) Link Bandwidth: 128 GB/s' and 'Scale-out AMD Infinity Fabric(TM) Links (Per GPU): 1', and the CDNA 4 white paper says 'one Infinity Fabric link configured for PCIe(R) Gen 5 to connect to I/O devices such as storage and networking'. It is one PCIe Gen 5 x16: 32 GT/s x 16 lanes x 2 directions / 8 = 128 GB/s bidirectional, 64 GB/s per direction. INDEPENDENT PROOF IT IS A SUM-ADDRESSED SEPARATE PATH, NOT PART OF THE FABRIC TOTAL: the white paper's '1,203.2 total peak aggregate theoretical GPU transport rate bandwidth performance per GPU OAM module' equals 1,075.2 (GPU-to-GPU P2P) + 128 (this path) exactly, so AMD adds the two rather than folding one into the other. PER GENERATION, BIDIRECTIONAL, all first-party: MI100 - 'PCIe Gen 4 links at 16 GT/sec, which corresponds to a peak bidirectional link bandwidth of 32 GB/sec' (note the MI100 board table lists 'PCIe 4.0 x16 , PCIe 3.0 x16' and this sentence is the Gen4 rate). MI210 - 'up to 64 GB/s peak theoretical data bandwidth from CPU to GPU per card' (CDNA 2 white paper endnote MI200-43). MI250X - PCIe Gen 4 x16 PER GCD, two per OAM, with AMD's x8 caveat below. MI300X/MI325X - 'The MI300X OAMs attach to the host system via PCIe Gen 5 x16 links' (ROCm MI300 page), 128 GB/s bidirectional per OAM. MI350X/MI355X - 128 GB/s bidirectional, unchanged. *** WHY THIS IS A SEPARATE RECORD AND NOT A FIELD ON THE GENERATION RECORDS: *** the host path moves on the PCIe schedule, not the Infinity Fabric schedule. It was flat at 128 GB/s across the last two CDNA generations while the scale-up fabric rose 20%, and on CDNA 3 the host bidirectional figure happens to EQUAL the per-link bidirectional fabric figure (128 GB/s) - two different paths, same number, which is precisely why a single 'bandwidth' column per part is ambiguous.
Topology
CPU/GPU attach, not a fabric. One PCIe Gen 5 x16 per GPU OAM to the host processors and to I/O devices (storage, networking). On MI300X/MI325X the node figure shows the eight OAMs reaching optional PCIe switches via retimers and HGX connectors, so host reachability can be extended with a switch while GPU-to-GPU traffic stays on direct links. On CDNA 2 the MI250X host attach is PER GCD rather than per OAM, i.e. two x16 links per package. On CDNA 2 there is additionally a COHERENT host path: the CDNA 2 white paper describes a host interface 'implemented as a 16-lane Infinity Fabric link' which 'can behave like an Infinity Fabric interface when paired with an optimized 3rd Gen AMD EPYC(TM) processor, enabling the unique cache coherency capabilities', but 'falls back to behave like a standard PCIe interface with non-coherent communication' against other x86 processors. The physical link is one; whether it is coherent depends on the host part. CDNA 3 and CDNA 4 nodes are PCIe-attached and not coherent.
Switching
PCIe root complexes plus OPTIONAL PCIe switches on the host/IO side (MI300/MI350 node figures). AMD names a downstream PCIe 4.0 ESM link on CDNA 2 at 'up to 25Gbps' which 'is coupled to a PCIe root complex, which can drive I/O devices that are connected to the GPU' - a separate downstream root-complex path, distinct from the upstream host attach, and AMD does not publish its bandwidth.
notes
*** THE CONFLATION THIS RECORD EXISTS TO STOP. *** Three different numbers get called 'Infinity Fabric bandwidth' on AMD parts, and this repo had carried the wrong one twice. (1) GPU-to-GPU scale-up: 76.8 GB/s per direction per link on CDNA 4 - see amd-infinity-fabric-cdna4. (2) THIS RECORD's host/IO attach: 128 GB/s bidirectional on PCIe Gen 5, unchanged from CDNA 3. (3) The GPU-to-GPU total across the mesh: 1,075.2 GB/s per OAM on CDNA 4. A reader who wants to size a host-attached streaming or KV-transfer path wants (2); a reader sizing all-reduce wants (1) and (3). *** WHERE AMD PUBLISHES A MATRIX INSTEAD OF A NUMBER - and why the MI250X host figure is deliberately NOT recorded as a value: *** AMD never gives one unconditional host bandwidth for CDNA 2. The ROCm MI250 page states 'Note that some platforms may offer an x8 interface to the GCDs, which reduces the available host-to-GPU bandwidth' - so the published shape is WIDTH (x16 or x8) against LINK COUNT (one per GCD, two per OAM), optionally through a PCIe switch: 2 x 64, 2 x 32, 1 x 64 or 1 x 32 GB/s bidirectional. Four configurations, and AMD states which are possible rather than choosing one. That is the honest shape of this data and it is why this record's per-generation breakdown is prose rather than four typed numbers. *** WHAT AMD DOES NOT PUBLISH ANYWHERE, checked against first-party sources: *** (a) per-generation CPU-side Infinity Fabric link rates for Zen through Zen 5 - the EPYC 9755 page publishes 'PCI Express(R) Version: PCIe(R) 5.0 x128', 'Memory Channels: 12' and 'Per Socket Mem BW: 614 GB/s' and NO Infinity Fabric link count or per-link bandwidth, and the same is true of the EPYC 9004 and 9005 series pages and the AMD Infinity Architecture technology page. The 'Per Socket Mem BW: 614 GB/s' is DDR5 MEMORY bandwidth and substituting it for an IF figure is precisely the class of error this repo has been corrupted by; (b) the downstream PCIe ESM link's bandwidth (only its 25 Gbps signalling rate is given); (c) any measured host-bandwidth figure - every number here is theoretical peak; (d) whether the coherent CDNA 2 host path runs at the same rate as the non-coherent fallback, which the white paper's wording leaves open. INFERENCE RELEVANCE: at 128 GB/s bidirectional against 1,075.2 GB/s of scale-up, the host path is 8.4x narrower than the fabric, so tensor- and pipeline-parallel traffic belongs on Infinity Fabric and only streaming, KV transfer, and storage/IO belong on this path. The 8x-PCIe-Lanes-per-socket reality of EPYC (x128) is what forces multi-GPU hosts through switches or retimers, and that is also the mechanism by which some CDNA 3/4 deployments see host bandwidth below the 128 GB/s figure printed on the board.