NVIDIA ConnectX-7 NDR 400Gb/s InfiniBand HCA

Fields

id

interconnect/nic-connectx7-ndr-400gbps

Kind

infiniband

Version

NDR (4th generation InfiniBand), 100 Gb/s per lane

Bandwidth (GB/s, per link, unidirectional)

50

Bandwidth basis

PER LANE, PER DIRECTION. NDR is 400 Gb/s per port per direction = 50 GB/s per port per direction. The 400 Gb/s figure is the whole-port rate; NDR uses 4 lanes at 100 Gb/s each. This is the per-lane rate, consistent with the repo's convention of recording per-link rates.

Link count

4

Topology

Switched fat-tree or Dragonfly+ via Quantum-2 switches. ConnectX-7 is the adapter (HCA) endpoint.

Scale-up

no

Scale-out

yes

Switching

Quantum-2 switches provide 64 NDR 400Gb/s ports or 128 NDR200 200Gb/s ports per switch, 51.2 Tb/s bidirectional throughput, non-blocking. SHARPv3 in-network reduction.

notes

ConnectX-7 is the current shipping NDR 400Gb/s InfiniBand adapter. Key specs from datasheet: 400Gb/s total bandwidth, 1/2/4 ports, PCIe Gen5 x32, 330-370M messages/sec RDMA rate, 16M I/O channels, 256B-4KB MTU (2GB messages). In-Network Computing: collective operations offload, MPI tag matching, MPI AlltoAll acceleration, rendezvous protocol offload, in-network memory. GPUDirect RDMA and GPUDirect Storage. Adaptive routing, enhanced congestion control, burst buffer offload. Hardware-based reliable transport, XRC, DCT. Form factors: PCIe HHHL/FHHL, OCP3.0 TSFF/SFF. The ConnectX-7 is the adapter that connects a GPU node to the InfiniBand fabric - it is the NIC-level component that the interconnect records above (nvlink, etc.) do not cover.

way

each

Notes

ConnectX-7 is the current shipping NDR 400Gb/s InfiniBand adapter. Key specs from datasheet: 400Gb/s total bandwidth, 1/2/4 ports, PCIe Gen5 x32, 330-370M messages/sec RDMA rate, 16M I/O channels, 256B-4KB MTU (2GB messages). In-Network Computing: collective operations offload, MPI tag matching, MPI AlltoAll acceleration, rendezvous protocol offload, in-network memory. GPUDirect RDMA and GPUDirect Storage. Adaptive routing, enhanced congestion control, burst buffer offload. Hardware-based reliable transport, XRC, DCT. Form factors: PCIe HHHL/FHHL, OCP3.0 TSFF/SFF. The ConnectX-7 is the adapter that connects a GPU node to the InfiniBand fabric - it is the NIC-level component that the interconnect records above (nvlink, etc.) do not cover.

Cross-references

This record references no other records — it is a leaf. It is cited by 4.