inference/research
verifiedhigh · 0.92majorupdated 2026-10-05

AMD 'Infinity Fabric bandwidth' means three different numbers depending on which table row you read (per-link scale-up vs GPU-to-GPU total vs host/IO path)

[major] hardware

Fields

id
gotchas/w7a-amd-infinity-fabric-scale-up-vs-host-io
Class
hardware
Severity
major
Concepts
tensor-parallelpipeline-parallelp2pallreduceinfinity-fabricxgmipciecdna3cdna4gfx942gfx950
Symptom
A part's 'Infinity Fabric bandwidth' is quoted as a single number and used to size the wrong path. The concrete case already seen in this repo: the MI355X record's '7 IF links @ 153 GB/s' was at risk of being read as a host or total figure when it is INFINITY FABRIC PER-LINK SCALE-UP bandwidth. Three distinct quantities share the name: (1) PER-LINK GPU-to-GPU - 76.8 GB/s per direction, 153.6 GB/s bidirectional per link on CDNA 4; (2) GPU-to-GPU TOTAL across the mesh - 1,075.2 GB/s per OAM on CDNA 4; (3) HOST/IO ATTACH - 128 GB/s bidirectional over PCIe Gen 5, which is not built from (1) at all. Lifting (1) and using it as (3) overstates the host path by 1.2x; lifting it as (2) understates the fabric by 7x; and on CDNA 3 the error is undetectable by inspection because (1) bidirectional and (3) are BOTH 128 GB/s.
Root cause
AMD reuses one product-page row label, 'Peak Infinity Fabric Link Bandwidth', across two physically different paths, and prints the row with NO direction stated and NO indication of whether it is per link or a total. Three separate mechanisms produce the confusion. (a) The row sits in the same board table as 'Bus Type: PCIe 5.0 x16', so it reads as a host figure at a glance - on MI355X 'Scale-out (Peak) AMD Infinity Fabric Link Bandwidth: 128 GB/s' is genuinely the host path and sits two rows below the scale-up figure. (b) DIRECTION IS UNSTATED on the board tables: AMD's 153 GB/s is bidirectional, so the per-direction figure is 76.8, and AMD says so only in the white paper prose ('76.8GB/s in each direction'), never in the table. (c) AMD publishes a link COUNT alongside a per-link bandwidth, and the count is not the same across sources or across form factors: MI300X/MI325X board tables say 8 links while the architecture page says 7, MI210 says 3 while MI250X says 8 on the same CDNA 2 generation, and MI350P publishes no fabric rows at all. A reader who multiplies '8 links x 128 GB/s' gets 1,024 and calls it the fabric total when AMD's actual P2P total is 896 (896/128 = 7 mesh links; the eighth is the PCIe-configured I/O link).
Workaround
Read the ROW LABEL, not the number. From the MI350 series onward AMD splits the table explicitly: 'Scale-up (Peak) AMD Infinity Fabric Link Bandwidth' is the per-link GPU-to-GPU figure and 'Scale-out (Peak) AMD Infinity Fabric Link Bandwidth' is the host/IO path. Before that split exists, take the per-link figure from the architecture or white paper and the host figure from the ROCm microarchitecture page, never from the product table alone. Confirm the direction with AMD's own formula from CDNA 2 white paper endnote MI200-29 - 'Baud Rate * # lanes * # directions * # links / 8 = GB/s per card' - whose '# directions' factor is AMD confirming that its published aggregates are BIDIRECTIONAL. Sanity-check any total against a stated aggregate: 896/128 = 7 on CDNA 3, and 1,075.2/153.6 = 7 on CDNA 4. Do not carry a GPU-to-GPU figure into the host path even when the numbers match - on CDNA 3 both are 128 GB/s bidirectional for entirely different reasons (32 Gbps x 16 lanes vs PCIe Gen 5 x16).
notes
THE FOOTNOTE CHAIN IS WHAT SETTLES IT, and reading it rather than lifting a headline number is the whole lesson here. For MI355X the chain is: board table prints '7' links and '153 GB/s'; the same table prints 'Scale-out AMD Infinity Fabric Links (Per GPU): 1' and 'Scale-out (Peak) AMD Infinity Fabric Link Bandwidth: 128 GB/s'; the CDNA 4 white paper states the per-link aggregate is '153.6 GB/s peak total aggregate theoretical transport data GPU peer-to-peer (P2P) bandwidth per AMD Infinity Fabric link' and separately that one link is 'configured for PCIe Gen 5 to connect to I/O devices'; and the white paper's third total, '1,203.2 total peak aggregate theoretical GPU transport rate', equals 1,075.2 + 128 exactly. Every step says the 153 is per-link and GPU-to-GPU. AMD ALSO LABELS THE GENERATION EXPLICITLY ONLY ON THE NEWEST PAGES: MI350X and MI355X say '4th Gen AMD Infinity Architecture', while MI100, MI210, MI250X, MI300X and MI325X say only 'AMD Infinity Architecture' - so keying records by CDNA generation is more honest than inventing a fabric generation number for the earlier parts. TWO MORE THINGS THAT ARE NOT INTERCHANGEABLE: the CDNA 4 IOD-to-IOD internal path is a separate figure ('roughly 14% faster than in the AMD CDNA 3 architecture', and a '5.5 TB/s bisection' label in Figure 5) from the 1,075.2 GPU-to-GPU total and from the 128 host path; and on CDNA 2 the 400 GB/s 'inter GDC to GDC' figure is DIE-TO-DIE inside one package while 800 GB/s is PACKAGE-TO-PACKAGE scale-up - same lane rate, same width, different link count, different meaning. A GENERATION-LEVEL CORRECTION this pass found: earlier records in this repo asserted CDNA 3 per-direction as 64 GB/s in one place and 128 GB/s in another. 64 is correct. AMD states CDNA 3 operated at 32 Gbps (CDNA 4 white paper, prior-generation sentence), and 32 x 16 / 8 = 64 per direction, with 128 being the bidirectional figure the board table prints. HONEST GAPS, left null rather than filled: AMD publishes no per-generation CPU-side Infinity Fabric link rate for Zen through Zen 5 (EPYC product pages give PCIe width and DDR5 memory bandwidth only - and 'Per Socket Mem BW: 614 GB/s' is memory bandwidth, not fabric), no lane-signalling-efficiency or encoding-overhead figures, no measured (as opposed to theoretical peak) host bandwidth, and no MI350P fabric rows at all.

Notes

THE FOOTNOTE CHAIN IS WHAT SETTLES IT, and reading it rather than lifting a headline number is the whole lesson here. For MI355X the chain is: board table prints '7' links and '153 GB/s'; the same table prints 'Scale-out AMD Infinity Fabric Links (Per GPU): 1' and 'Scale-out (Peak) AMD Infinity Fabric Link Bandwidth: 128 GB/s'; the CDNA 4 white paper states the per-link aggregate is '153.6 GB/s peak total aggregate theoretical transport data GPU peer-to-peer (P2P) bandwidth per AMD Infinity Fabric link' and separately that one link is 'configured for PCIe Gen 5 to connect to I/O devices'; and the white paper's third total, '1,203.2 total peak aggregate theoretical GPU transport rate', equals 1,075.2 + 128 exactly. Every step says the 153 is per-link and GPU-to-GPU. AMD ALSO LABELS THE GENERATION EXPLICITLY ONLY ON THE NEWEST PAGES: MI350X and MI355X say '4th Gen AMD Infinity Architecture', while MI100, MI210, MI250X, MI300X and MI325X say only 'AMD Infinity Architecture' - so keying records by CDNA generation is more honest than inventing a fabric generation number for the earlier parts. TWO MORE THINGS THAT ARE NOT INTERCHANGEABLE: the CDNA 4 IOD-to-IOD internal path is a separate figure ('roughly 14% faster than in the AMD CDNA 3 architecture', and a '5.5 TB/s bisection' label in Figure 5) from the 1,075.2 GPU-to-GPU total and from the 128 host path; and on CDNA 2 the 400 GB/s 'inter GDC to GDC' figure is DIE-TO-DIE inside one package while 800 GB/s is PACKAGE-TO-PACKAGE scale-up - same lane rate, same width, different link count, different meaning. A GENERATION-LEVEL CORRECTION this pass found: earlier records in this repo asserted CDNA 3 per-direction as 64 GB/s in one place and 128 GB/s in another. 64 is correct. AMD states CDNA 3 operated at 32 Gbps (CDNA 4 white paper, prior-generation sentence), and 32 x 16 / 8 = 64 per direction, with 128 being the bidirectional figure the board table prints. HONEST GAPS, left null rather than filled: AMD publishes no per-generation CPU-side Infinity Fabric link rate for Zen through Zen 5 (EPYC product pages give PCIe width and DDR5 memory bandwidth only - and 'Per Socket Mem BW: 614 GB/s' is memory bandwidth, not fabric), no lane-signalling-efficiency or encoding-overhead figures, no measured (as opposed to theoretical peak) host bandwidth, and no MI350P fabric rows at all.

Cross-references

Affects (12)