inference/research
verifiedhigh · 0.85updated 2026-10-03

NVIDIA NVLink Switch 2

300 GB/s

Fields

id
interconnect/nvswitch-2
Kind
nvswitch
Version
NVLink Switch 2 (Ampere / A100)
Bandwidth (GB/s, per link, unidirectional)
300
Bandwidth basis
PER GPU, PER DIRECTION. NVIDIA's HGX A100 datasheet prints 'GPU-to-GPU Bandwidth 600 GB/s' for the 4-GPU, 8-GPU and 16-GPU configurations - a BIDIRECTIONAL AGGREGATE per GPU pair - so the recorded per-direction value is half, 300 GB/s. Cross-checked against the same table's 'NVIDIA NVLink 3rd generation' row and against the Ampere architecture white paper's '600 GB/sec total bandwidth' for A100, all of which agree on 600 GB/s aggregate. Total aggregate bandwidth for the domain is published alongside: 2.4 TB/s at 4 GPUs, 4.8 TB/s at 8, 9.6 TB/s at 16. NOTE THE NAMING TRAP, which is the reason this record is '2' rather than '3': NVIDIA's own HGX A100 datasheet calls the A100 switch 'Second-Generation NVIDIA NVSwitch', while the LINK is third-generation NVLink 3. Switch and link generations advance independently, so 'NVLink 3' and 'NVSwitch 2' describe the same A100 system. Anyone matching a switch record by the link generation number will pick the wrong record.
Link count
8
Topology
Switched, non-blocking within the domain. NVIDIA's HGX A100 table shows NVSwitch absent at 4 GPUs ('N/A') and present at 8 and 16, with the per-GPU rate unchanged at 600 GB/s aggregate across all three sizes - i.e. the switch changes the domain size and removes the mesh topology constraint without changing the per-GPU rate, exactly as it does at later generations.
Scale-up
yes
Scale-out
no
Switching
The switch is the point: NVSwitch 'powered by NVLink creates a unified networking fabric that allows the entire node to function as a single gigantic GPU'. Scale-out beyond the node is Mellanox/NVIDIA Magnum IO over InfiniBand or Ethernet, not NVSwitch.
notes
The Ampere-generation scale-up switch, covering the A100 family, and the reason the accelerator records reference a bare 'nvswitch' id that this record and nvswitch-4/5 together resolve by generation. PUBLISHED (NVIDIA HGX A100 datasheet, first-party): 'Third-Generation NVIDIA NVLink ... doubles the GPU-to-GPU direct bandwidth to 600 gigabytes per second (GB/s), almost 10X higher than PCIe Gen4', available in four-GPU and eight-GPU HGX A100 servers; 'Second-Generation NVIDIA NVSwitch powered by NVLink creates a unified networking fabric that allows the entire node to function as a single gigantic GPU'. Per-configuration table: 4-GPU = NVSwitch N/A / 600 GB/s / 2.4 TB/s aggregate; 8-GPU = NVSwitch 2nd generation / 600 GB/s / 4.8 TB/s; 16-GPU = NVSwitch 2nd generation / 600 GB/s / 9.6 TB/s. INFERENCE RELEVANCE: 300 GB/s per direction against a 1.55 TB/s memory interface makes this the generation where multi-GPU tensor parallelism became genuinely practical rather than aspirational - the fabric is no longer the bottleneck by a wide margin, so the sharding decision is driven by capacity and by collective efficiency rather than by whether the fabric can keep up. For a 70B-class model at 4-bit an 8-GPU A100 domain holds it comfortably, which is the workload class nvswitch-5 later generalises to 72 GPUs. GENERATION NAMING, stated plainly because it is an active trap in this repo's data: on A100 the LINK is NVLink 3 (see nvlink-3) and the SWITCH is NVSwitch 2 (this record). Do not assume link generation N implies switch generation N - on H100 the link is NVLink 4 and the switch is also NVSwitch 4, but on A100 they are 3 and 2 respectively. DIRECTION CONVENTION: NVIDIA states 600 GB/s as a bidirectional aggregate here, the same convention as nvswitch-4, nvswitch-5 and nvlink-3, so this record's typed value is halved to 300 GB/s per direction. Do not compare it against a vendor page's 600 GB/s without halving, and do not compare it against AMD's amd-infinity-fabric-link without checking that record's contested direction status.
way
each

Notes

The Ampere-generation scale-up switch, covering the A100 family, and the reason the accelerator records reference a bare 'nvswitch' id that this record and nvswitch-4/5 together resolve by generation. PUBLISHED (NVIDIA HGX A100 datasheet, first-party): 'Third-Generation NVIDIA NVLink ... doubles the GPU-to-GPU direct bandwidth to 600 gigabytes per second (GB/s), almost 10X higher than PCIe Gen4', available in four-GPU and eight-GPU HGX A100 servers; 'Second-Generation NVIDIA NVSwitch powered by NVLink creates a unified networking fabric that allows the entire node to function as a single gigantic GPU'. Per-configuration table: 4-GPU = NVSwitch N/A / 600 GB/s / 2.4 TB/s aggregate; 8-GPU = NVSwitch 2nd generation / 600 GB/s / 4.8 TB/s; 16-GPU = NVSwitch 2nd generation / 600 GB/s / 9.6 TB/s. INFERENCE RELEVANCE: 300 GB/s per direction against a 1.55 TB/s memory interface makes this the generation where multi-GPU tensor parallelism became genuinely practical rather than aspirational - the fabric is no longer the bottleneck by a wide margin, so the sharding decision is driven by capacity and by collective efficiency rather than by whether the fabric can keep up. For a 70B-class model at 4-bit an 8-GPU A100 domain holds it comfortably, which is the workload class nvswitch-5 later generalises to 72 GPUs. GENERATION NAMING, stated plainly because it is an active trap in this repo's data: on A100 the LINK is NVLink 3 (see nvlink-3) and the SWITCH is NVSwitch 2 (this record). Do not assume link generation N implies switch generation N - on H100 the link is NVLink 4 and the switch is also NVSwitch 4, but on A100 they are 3 and 2 respectively. DIRECTION CONVENTION: NVIDIA states 600 GB/s as a bidirectional aggregate here, the same convention as nvswitch-4, nvswitch-5 and nvlink-3, so this record's typed value is halved to 300 GB/s per direction. Do not compare it against a vendor page's 600 GB/s without halving, and do not compare it against AMD's amd-infinity-fabric-link without checking that record's contested direction status.

Cross-references

This record references no other records — it is a leaf. It is cited by 8.