PCI Express 5.0

x16, PER DIRECTION, and the value is UNCHANGED in this audit - the earlier halving to 31.5 was itself the error. CONVENTION ESTABLISHED AGAINST PCI-SIG'S OWN FAQ, which settles the ambiguity that has corrupted this record three times: PCIe figures are quoted PER LANE PER DIRECTION. PCI-SIG's PCIe 5.0 FAQ states PCIe 5.0 'adds support for 32.0 Gigabits/second/Lane/DIRECTION of raw bandwidth' and that 'a PCIe 5.0 Link consisting of 32 Lanes and operating at a bit rate of 32.0 GT/s provides an effective raw bandwidth of 128.0 Gigabytes/second IN EACH DIRECTION SIMULTANEOUSLY' - and 32 lanes x 32 Gb/s / 8 = 128.0 GB/s checks out, confirming the figure is per-direction and NOT an aggregate across both directions. PCIe is dual-simplex: the TX and RX lane pairs are independent, so a figure quoted 'in each direction' is already the per-direction payload and must NOT be halved again. THE ERROR THIS RECORD HAS SUFFERED TWICE, recorded because it is the most repeated mistake in this file: each direction IS the unit for a PCIe lane count, so 16 GT/s x 128/130 / 8 x 16 lanes = 31.5 GB/s per direction with no further halving. Halving it a second time to 15.75 produces a value that breaks the strict x2-per-generation ladder against pcie-gen5. Arithmetic: 32 x 128/130 / 8 = 3.938 GB/s per lane per direction, x16 = 63.01 GB/s per direction; bidirectional aggregate 126.03 GB/s. PCI-SIG's own headline for this generation is '128.0 Gigabytes/second in each direction simultaneously' at x32, which is the raw per-direction rate and corroborates the 63.0 at x16 by the 2x lane-count ratio. THE 32 GT/s line rate is independently corroborated by AMD shipping MI300X OAM modules on a 'PCIe 5.0 x16' bus (link-amd-mi300x-specs). FULL CONVENTION CROSS-CHECK AGAINST THE WHOLE FAMILY, for the record that this repo corrupted most often: the payload per direction ladder at x16 for gens 1-7 is 4.0 / 8.0 / 15.75 / 31.5 / 63.0 / 121.0 / 242.0 (pcie-gen1 through pcie-gen7), every figure PER DIRECTION, every figure derived as GT/s x encoding_efficiency / 8 x 16 with no further halving. Note that this generation's payload ratio to the next is NOT 2x despite the GT/s rate doubling again: PCIe 6.0 introduces PAM4 with 242B/256B flit framing instead of NRZ with 128b/130b, so payload per direction goes 63.0 -> 121.0, a ratio of 1.92x. PCI-SIG's own wording for why, verbatim: 'NRZ uses two levels of signaling, while PAM4 uses four levels. It packs two bits of information into the same amount of time on a serial channel' (w7p-pcisig-pcie6-faq). A reader comparing Gen5 to Gen6 by GT/s alone would conclude exactly double and then find 1.92x in the payload and assume a record is wrong; both are right, and the framing overhead is the entire discrepancy. See pcie-gen6 and pcie-gen7.

Point-to-point host-to-device by default (root complex to endpoint), up to 16 lanes. Becomes a shared fabric only through PCIe switches or host bridges, in which case ordering and oversubscription apply.

Optional. A PCIe switch creates a shared domain, but it is store-and-forward with no in-network reduction, so collectives pay full hop latency per traversal and get no SHARP-equivalent.

PCIe sits between scale-up and scale-out and is the bottleneck people forget: it is the host attach path for PCIe accelerators, so an 8-GPU node on PCIe gives each GPU 63 GB/s per direction of fabric versus 450 GB/s per direction per GPU for NVLink 4 on Hopper - roughly 7x less per-direction fabric bandwidth for tensor parallel. Consequence: tensor-parallel sharding across PCIe is dramatically worse than across NVLink, because every TP boundary moves activations at the slower of the two rates. PCIe SKUs with an NVLink bridge (e.g. H100 NVL) are the vendor's way of recovering scale-up bandwidth while staying on a cheaper host attach - the accelerator still attaches at 63 GB/s per direction, but peers talk over NVLink. Treat PCIe as scale-up-capable but bandwidth-poor; do not size a TP plan off a PCIe host link. DIRECTION AUDIT NOTE: the version of this record that computed B* = 465 from 63 GB/s was correct and has been restored; an intervening pass that halved the value to 31.5 and recomputed B* = 232 was wrong, because 63 GB/s at x16 is already a per-direction figure per PCI-SIG's own convention.