ROCm 'AMD Instinct MI250 microarchitecture' reference - 25 GT/sec links, die-to-die vs host PCIe, and the x8 caveat
Fields
sources/w7a-amd-mi250-microarch-if-25gt-and-per-gcd-host-pcieNotes
The CDNA 2 generation's own architecture reference, and the only AMD source that names all three of the quantities this repo needs separately. (1) PER LINK: 'Each of the AMD Infinity Fabric links between GPUs can run at up to 25 GT/sec, which correlates to a peak transfer bandwidth of 50 GB/sec for a 16-wide link ( two bytes per transaction)' and 'Each of these 16-wide links connects to one of the two GPU dies in the MI250 OAM and operates at 25 GT/sec, which corresponds to a theoretical peak transfer rate of 50 GB/sec per link (or 100 GB/sec bidirectional peak transfer bandwidth)'. (2) DIE-TO-DIE, a different quantity from GPU-to-GPU: 'The two GCDs in the package are connected via four AMD Infinity Fabric links running at a theoretical peak rate of 25 GT/sec, giving 200 GB/sec peak transfer bandwidth between the two GCDs of an OAM, or a bidirectional peak transfer bandwidth of 400 GB/sec for the same.' (3) HOST PATH: 'The MI250 OAMs attach to the host system via PCIe Gen 4 x16 links (yellow lines). Each GCD maintains its own PCIe x16 link to the host part of the system.' AND THE CAVEAT THAT INVALIDATES A SINGLE HOST NUMBER: 'Note that some platforms may offer an x8 interface to the GCDs, which reduces the available host-to-GPU bandwidth.' That x8 caveat is why the host attach bandwidth cannot be recorded as one unconditional figure for this generation - AMD publishes it as a matrix of width (x16 or x8) against link count (one per GCD, so two per OAM). Also: 'One of the AMD Infinity Fabric links of the controller at the bottom can be configured as a PCIe link', i.e. the link budget is re-configurable and the physical IF and PCIe paths share controllers.
Cross-references
This record references no other records.