inference/research
verifiedhigh · 0.92updated 2026-10-05

AMD Infinity Fabric link (CDNA 4 / MI350 class)

76.8 GB/s

Fields

id
interconnect/amd-infinity-fabric-cdna4
Kind
infinity-band
Version
CDNA 4 / AMD Instinct MI350 Series (MI350X, MI355X)
Bandwidth (GB/s, per link, unidirectional)
76.8
Bandwidth basis
PER LINK, PER DIRECTION - DIRECTION NOW STATED BY AMD ITSELF, which removes the ambiguity that forced amd-infinity-fabric-link to be contested for three audit passes. The CDNA 4 white paper (net2-amd-cdna4-whitepaper) says verbatim: 'The AMD CDNA 4 architecture comprises 8 AMD Infinity Fabric links that are 16-bits wide and fully bi-directional' and 'The Infinity Fabric links in the AMD CDNA 4 architecture run up to 20% faster generationally at 38.4Gbps for a total link bandwidth of 76.8GB/s IN EACH DIRECTION'. Endnote MI350-007 gives the matching aggregate: 'up to 153.6 GB/s peak total aggregate theoretical transport data GPU peer-to-peer (P2P) bandwidth per AMD Infinity Fabric link', and 'up to eight AMD Infinity Fabric links providing up to 1,075.2 GB/s peak aggregate peer-to-peer (P2P) and 1,203.2 total peak aggregate theoretical GPU transport rate bandwidth performance per GPU OAM module'. ARITHMETIC CROSS-CHECKS THAT ALL PASS, and they are what make this record trustworthy rather than a single unsourced quote: (a) 38.4 Gbps x 16 bits = 614.4 Gb/s = 76.8 GB/s per direction - the stated rate and the stated width multiply to the stated bandwidth; (b) 1,075.2 / 153.6 = exactly 7 links, matching the '7 AMD Infinity Fabric links' in the ROCm node description (net2-rocm-mi350-microarch), so the aggregate and the per-link figure are mutually consistent; (c) 1,075.2 + 128 = 1,203.2, reproducing AMD's stated total transport rate exactly, where the 128 is the separately-configured PCIe Gen 5 host link. GENERATIONAL MOVEMENT, per direction per link: CDNA 3 / MI300X at 64, CDNA 4 / MI350 at 76.8 - a 1.2x, which AMD itself describes as '20% faster generationally'. Per-direction aggregate across the 7-link fully-connected domain: 448 GB/s on MI300X, 537.6 GB/s on MI350. EXTENDED 2026-10-05 (w7a pass) - THE THREE FIGURES SEPARATED, WITH THE HOST PATH ISOLATED. This record previously established 76.8 GB/s per direction per link from the white paper alone. The 2026-10-05 pass adds the two facts that turn a per-link number into a usable fabric description. (1) THE HOST/IO PATH IS 128 GB/s AND IS NOT BUILT FROM THIS FIGURE: AMD's own MI350X and MI355X board tables carry a separate 'Scale-out (Peak) AMD Infinity Fabric Link Bandwidth: 128 GB/s' row and a 'Scale-out AMD Infinity Fabric Links (Per GPU): 1' row, beside 'Scale-up (Peak) AMD Infinity Fabric Link Bandwidth: 153 GB/s'. The 128 is one PCIe Gen 5 x16 (32 GT/s x 16 lanes x 2 directions / 8). Proof that it is a genuinely separate path rather than a restatement: the white paper's '1,203.2 total peak aggregate theoretical GPU transport rate bandwidth performance per GPU OAM module' equals 1,075.2 (this fabric's P2P total) + 128 exactly, so AMD SUMS the two. (2) THE GPU-TO-GPU TOTAL IS 1,075.2 GB/S PER OAM ACROSS 7 LINKS, and 1,075.2 / 153.6 = exactly 7, so the per-link and total figures corroborate each other rather than being independently rounded. WARNING THAT SURVIVES THE GENERATION BOUNDARY: 128 GB/s is ALSO the CDNA 3 per-link BIDIRECTIONAL figure, so the CDNA 4 host path must not be mistaken for an unchanged CDNA 3 GPU-to-GPU link - it is unchanged because PCIe Gen 5 did not change. The host path in its own right is recorded separately on amd-infinity-fabric-host-io-path.
Link count
7
Topology
Fully connected 8-GPU node, identical to the prior generation. AMD's MI350 microarchitecture reference (net2-rocm-mi350-microarch) states 'a fully connected 8-GPU node topology, identical to the prior generation' with each GPU reaching its 7 peers over 7 Infinity Fabric links at 38.4 Gbps, plus one PCIe Gen 5 x16 link to the host and 'one Infinity Fabric link configured for PCIe Gen 5 to connect to I/O devices such as storage and networking'. MI350X is an air-cooled 1000W OAM on a UBB8 baseboard that is drop-in compatible with the MI325X platform; MI355X is 1400W direct liquid-cooled, and AMD states up to 128 MI355X fit in a 200kW rack. THE PACKAGE-INTERIOR CHANGE IS THE REAL STORY: CDNA 4 consolidates the I/O dies from four in CDNA 3 to two larger ones, and 'the direct connection between the two IODs reduces latency and power consumption compared to the previous generation' - each IOD now carries four links. THE LINK BUDGET IS SPLIT 7 + 1, WHICH IS WHY AMD'S OWN SOURCES DISAGREE ON THE COUNT: seven is the scale-up mesh (the minimum for mutual connectivity among 8 nodes) and the eighth is 'one Infinity Fabric link configured for PCIe Gen 5 to connect to I/O devices such as storage and networking'. The MI350X/MI355X board tables print 7 because their scale-up row counts the mesh, whereas the CDNA 3 board tables print 8 because there the count includes the I/O link. Not a typo in either - a scope difference in what the row counts. Host path: one PCIe Gen 5 x16 per GPU to the host processors and to I/O devices, with optional PCIe switches on the host side only; there is still no switch part anywhere in the GPU-to-GPU path.
Scale-up
yes
Scale-out
no
Switching
No switch part. The scale-up domain is the 8-GPU node, built from direct point-to-point links, exactly as in CDNA 3. Scale-out is handled by external networking. The structural consequence is unchanged from amd-xgmi and amd-infinity-fabric-link: a direct-attached full mesh is non-blocking within its domain size, but there is no fabric to grow past 8, so a wider deployment must add nodes rather than GPUs. AMD's differentiation here is that its 8-way all-to-all domain needs no switch part at all, where NVIDIA reaches the same 8-way shape only by adding NVSwitch silicon (see nvswitch-4).
notes
NAMING, to avoid a collision that had to be fixed on creation: amd-infinity-fabric-link is AMD's '4th generation' link and covers CDNA 3 (MI300/MI325); this record is the NEXT generation, CDNA 4 (MI350). AMD's own marketing calls both 'Infinity Fabric' and numbers the CDNA generations, not the link generations, so the two are distinguished by CDNA generation rather than by a link generation number. Added 2026-10-03 because the Infinity Fabric record covered CDNA 3 only, and CDNA 4 is the generation that RESOLVED the direction ambiguity rather than deepening it - the single most useful thing this pass produced for AMD. WHY A SEPARATE RECORD RATHER THAN FOLDING INTO amd-infinity-fabric-link: that record is named '4th generation (MI300/MI325 class)' and its 64 GB/s is the CDNA 3 figure whose direction had to be inferred. Now that AMD states the convention outright, the per-generation values belong on separate records - CDNA 3 at 64 per direction, CDNA 4 at 76.8 - because they differ by 20% and a reader sizing a fabric needs the right one. WHAT AMD PUBLISHES FOR THIS GENERATION (white paper plus ROCm reference): 8 XCDs and 2 IODs on N3P/N6, 'communications with other GPUs through 7 AMD Infinity Fabric links operating at 38.4 Gbps, providing over 1 TB/s of aggregate communication bandwidth per GPU', 288 GB HBM3E at 8 TB/s, 256 MB Infinity Cache, one PCIe Gen 5 x16 host link. INFERENCE RELEVANCE, and it is a positive finding that reverses the usual read of this vendor: AMD's fabric has been closing the gap to NVIDIA's rather than lagging it. 537.6 GB/s per direction across the 7-link domain against NVLink 4's 450 GB/s per direction per GPU means an 8-GPU AMD node now has MORE per-GPU scale-up bandwidth than an 8-GPU Hopper node - 1.2x - while needing no switch silicon and offering a larger 8-GPU all-to-all domain than the 8-GPU direct-attach mesh NVIDIA could build without NVSwitch. The ratio against local memory is what actually decides TP viability, and there AMD is far behind: 537.6 GB/s per direction against MI355X's 8 TB/s of HBM3E is about 15x, versus NVLink 4's 450 against the H100's 3.35 TB/s at about 7.4x. So the honest summary is that AMD has the better fabric-to-GPU-count story and the worse fabric-to-memory-bandwidth story, and since decode is memory-bound at low batch, it is the fabric-to-memory ratio - not the fabric-to-fabric comparison - that sets the TP crossover (see docs/11-interconnect-decision.md). TWO SECONDARY POINTS. First, partitioning interacts with the fabric: AMD's NPS2 mode keeps all memory traffic inside one IOD and its associated XCDs, 'reducing the overhead of crossing the AMD Infinity Fabric network between the two IODs' - so on a 2-way partition you can avoid the fabric entirely for memory traffic, which is a lever that has no analogue on NVIDIA parts and is worth knowing before choosing TP degree on MI355X. Second, MI350P is the PCIe variant with 4 XCDs, one IOD, 144 GB and up to 8 cards per air-cooled server, and the ROCm page gives it no Infinity Fabric GPU-to-GPU path - so MI350P has NO scale-up fabric in the sources retrieved and should not be treated as an 8-GPU scale-up part. EXTENDED 2026-10-05 (w7a pass): this per-link figure is now anchored by per-generation sibling records, and the confusion it invites is documented as gotcha w7a-amd-infinity-fabric-scale-up-vs-host-io. PER-GENERATION CHAIN, per direction per link: 46 GB/s (MI100/CDNA, 23 GT/sec) -> 50 (CDNA 2, 25 GT/sec) -> 64 (CDNA 3, 32 Gbps) -> 76.8 (CDNA 4, 38.4 Gbps). GPU-to-GPU aggregate per OAM: 276 -> 300 -> 896 -> 1,075.2 GB/s. Host/IO path: 32 -> 64 -> 128 -> 128 GB/s bidirectional, i.e. FLAT across the last two CDNA generations while the scale-up fabric rose 20%. At 128 GB/s bidirectional against 1,075.2 GB/s of scale-up, the host path is now 8.4x narrower than the fabric. *** GENERATIONAL CORRECTION TO EARLIER SIBLING RECORDS, recorded here because the contradiction is in this family and this record is the one that carries the correct arithmetic: *** amd-infinity-fabric-link-cdna3 carries 'bandwidth_gbps: 128' described as 'PER DIRECTION', while amd-infinity-fabric-link and this record carry 64 GB/s per direction for CDNA 3. 64 IS CORRECT - AMD states the prior generation 'operated at 32Gbps' (CDNA 4 white paper) and 32 x 16 / 8 = 64 per direction, while 128 is the BIDIRECTIONAL figure the MI300X board table prints. amd-infinity-fabric-link-cdna4 similarly carries 153.6 described as 'per direction' where this record carries 76.8; by the same arithmetic 153.6 is the bidirectional per-link aggregate that AMD's own endnote MI350-007 labels 'peak total aggregate theoretical transport data GPU peer-to-peer (P2P) bandwidth per AMD Infinity Fabric link', and the per-direction figure is half of it. Those two siblings were left unmodified by this pass because correcting them is outside a per-generation pass's scope and they are referenced by accelerator and flops records; the correction is written down here so the next pass fixes them deliberately and visibly rather than silently. *** WHAT AMD DOES NOT PUBLISH FOR THIS GENERATION: *** the board tables never print the per-direction figure (153 is the bidirectional per-link number rounded to 3 significant figures); there is no board row for the on-package die-to-die path, so the IOD-to-IOD 'roughly 14% faster' figure and the '5.5 TB/s bisection' label inside Figure 5 exist only in prose and a figure label with no derivation published for either; there is no per-lane signalling-efficiency or encoding-overhead figure and no breakdown of the 1,203.2 beyond the 1,075.2 + 128 identity; and MI350P, the PCIe card of the same CDNA 4 family, publishes NO Infinity Fabric rows at all, so none of these numbers transfer to it. INFERENCE CORRECTION, added because the earlier note in this record drew the opposite conclusion: per-direction aggregate across the mesh is 537.6 GB/s against 8 TB/s of HBM3E, about 15:1, which is TIGHTER than CDNA 3's 448 GB/s against 5.3 TB/s at about 12:1 - the fabric improved 1.2x generationally while memory improved 1.5x, so the ratio that actually governs tensor-parallel headroom moved the WRONG way even though the headline link speed rose 20%. The 256 MB Infinity Cache, which absorbs a large share of reduction traffic before it reaches the fabric, is the reason this does not simply make 8-way TP worse.
way
each

Notes

NAMING, to avoid a collision that had to be fixed on creation: amd-infinity-fabric-link is AMD's '4th generation' link and covers CDNA 3 (MI300/MI325); this record is the NEXT generation, CDNA 4 (MI350). AMD's own marketing calls both 'Infinity Fabric' and numbers the CDNA generations, not the link generations, so the two are distinguished by CDNA generation rather than by a link generation number. Added 2026-10-03 because the Infinity Fabric record covered CDNA 3 only, and CDNA 4 is the generation that RESOLVED the direction ambiguity rather than deepening it - the single most useful thing this pass produced for AMD. WHY A SEPARATE RECORD RATHER THAN FOLDING INTO amd-infinity-fabric-link: that record is named '4th generation (MI300/MI325 class)' and its 64 GB/s is the CDNA 3 figure whose direction had to be inferred. Now that AMD states the convention outright, the per-generation values belong on separate records - CDNA 3 at 64 per direction, CDNA 4 at 76.8 - because they differ by 20% and a reader sizing a fabric needs the right one. WHAT AMD PUBLISHES FOR THIS GENERATION (white paper plus ROCm reference): 8 XCDs and 2 IODs on N3P/N6, 'communications with other GPUs through 7 AMD Infinity Fabric links operating at 38.4 Gbps, providing over 1 TB/s of aggregate communication bandwidth per GPU', 288 GB HBM3E at 8 TB/s, 256 MB Infinity Cache, one PCIe Gen 5 x16 host link. INFERENCE RELEVANCE, and it is a positive finding that reverses the usual read of this vendor: AMD's fabric has been closing the gap to NVIDIA's rather than lagging it. 537.6 GB/s per direction across the 7-link domain against NVLink 4's 450 GB/s per direction per GPU means an 8-GPU AMD node now has MORE per-GPU scale-up bandwidth than an 8-GPU Hopper node - 1.2x - while needing no switch silicon and offering a larger 8-GPU all-to-all domain than the 8-GPU direct-attach mesh NVIDIA could build without NVSwitch. The ratio against local memory is what actually decides TP viability, and there AMD is far behind: 537.6 GB/s per direction against MI355X's 8 TB/s of HBM3E is about 15x, versus NVLink 4's 450 against the H100's 3.35 TB/s at about 7.4x. So the honest summary is that AMD has the better fabric-to-GPU-count story and the worse fabric-to-memory-bandwidth story, and since decode is memory-bound at low batch, it is the fabric-to-memory ratio - not the fabric-to-fabric comparison - that sets the TP crossover (see docs/11-interconnect-decision.md). TWO SECONDARY POINTS. First, partitioning interacts with the fabric: AMD's NPS2 mode keeps all memory traffic inside one IOD and its associated XCDs, 'reducing the overhead of crossing the AMD Infinity Fabric network between the two IODs' - so on a 2-way partition you can avoid the fabric entirely for memory traffic, which is a lever that has no analogue on NVIDIA parts and is worth knowing before choosing TP degree on MI355X. Second, MI350P is the PCIe variant with 4 XCDs, one IOD, 144 GB and up to 8 cards per air-cooled server, and the ROCm page gives it no Infinity Fabric GPU-to-GPU path - so MI350P has NO scale-up fabric in the sources retrieved and should not be treated as an 8-GPU scale-up part. EXTENDED 2026-10-05 (w7a pass): this per-link figure is now anchored by per-generation sibling records, and the confusion it invites is documented as gotcha w7a-amd-infinity-fabric-scale-up-vs-host-io. PER-GENERATION CHAIN, per direction per link: 46 GB/s (MI100/CDNA, 23 GT/sec) -> 50 (CDNA 2, 25 GT/sec) -> 64 (CDNA 3, 32 Gbps) -> 76.8 (CDNA 4, 38.4 Gbps). GPU-to-GPU aggregate per OAM: 276 -> 300 -> 896 -> 1,075.2 GB/s. Host/IO path: 32 -> 64 -> 128 -> 128 GB/s bidirectional, i.e. FLAT across the last two CDNA generations while the scale-up fabric rose 20%. At 128 GB/s bidirectional against 1,075.2 GB/s of scale-up, the host path is now 8.4x narrower than the fabric. *** GENERATIONAL CORRECTION TO EARLIER SIBLING RECORDS, recorded here because the contradiction is in this family and this record is the one that carries the correct arithmetic: *** amd-infinity-fabric-link-cdna3 carries 'bandwidth_gbps: 128' described as 'PER DIRECTION', while amd-infinity-fabric-link and this record carry 64 GB/s per direction for CDNA 3. 64 IS CORRECT - AMD states the prior generation 'operated at 32Gbps' (CDNA 4 white paper) and 32 x 16 / 8 = 64 per direction, while 128 is the BIDIRECTIONAL figure the MI300X board table prints. amd-infinity-fabric-link-cdna4 similarly carries 153.6 described as 'per direction' where this record carries 76.8; by the same arithmetic 153.6 is the bidirectional per-link aggregate that AMD's own endnote MI350-007 labels 'peak total aggregate theoretical transport data GPU peer-to-peer (P2P) bandwidth per AMD Infinity Fabric link', and the per-direction figure is half of it. Those two siblings were left unmodified by this pass because correcting them is outside a per-generation pass's scope and they are referenced by accelerator and flops records; the correction is written down here so the next pass fixes them deliberately and visibly rather than silently. *** WHAT AMD DOES NOT PUBLISH FOR THIS GENERATION: *** the board tables never print the per-direction figure (153 is the bidirectional per-link number rounded to 3 significant figures); there is no board row for the on-package die-to-die path, so the IOD-to-IOD 'roughly 14% faster' figure and the '5.5 TB/s bisection' label inside Figure 5 exist only in prose and a figure label with no derivation published for either; there is no per-lane signalling-efficiency or encoding-overhead figure and no breakdown of the 1,203.2 beyond the 1,075.2 + 128 identity; and MI350P, the PCIe card of the same CDNA 4 family, publishes NO Infinity Fabric rows at all, so none of these numbers transfer to it. INFERENCE CORRECTION, added because the earlier note in this record drew the opposite conclusion: per-direction aggregate across the mesh is 537.6 GB/s against 8 TB/s of HBM3E, about 15:1, which is TIGHTER than CDNA 3's 448 GB/s against 5.3 TB/s at about 12:1 - the fabric improved 1.2x generationally while memory improved 1.5x, so the ratio that actually governs tensor-parallel headroom moved the WRONG way even though the headline link speed rose 20%. The 256 MB Infinity Cache, which absorbs a large share of reduction traffic before it reaches the fabric, is the reason this does not simply make 8-way TP worse.

Cross-references

This record references no other records — it is a leaf. It is cited by 2.