ROCm 'AMD Instinct MI300 Series microarchitecture' reference - host PCIe Gen 5 x16 attach and the 7-link mesh
Fields
sources/w7a-amd-mi300-microarch-host-pcie-gen5Notes
States the CDNA 3 host path, which the CDNA 3 product pages give only as an undifferentiated 'Bus Type: PCIe 5.0 x16' with no bandwidth. Verbatim: 'The MI300X OAMs attach to the host system via PCIe Gen 5 x16 links (yellow lines). The GPUs are using seven high-bandwidth, low-latency AMD Infinity Fabric(TM) links (red lines) to form a fully connected 8-GPU system.' The caption for the node-level figure confirms '8 fully interconnected MI300X OAM modules connected to (optional) PCIe switches via retimers and HGX connectors'. Two things are established here that no other CDNA 3 source states: the host link is one PCIe Gen 5 x16 per OAM, and the scale-up mesh is exactly 7 links - which is the count that makes an 8-GPU system fully connected. This page publishes NO lane rate for the CDNA 3 fabric, which is why w7a-amd-cdna4-whitepaper-cdna3-lane-rate-32gbps is needed.
Cross-references
This record references no other records.