NVIDIA technical blog: NVLink and NVSwitch supercharge large language model inference

Developer-blog statement of the inference consequence: NVSwitch provides non-blocking 900 GB/s GPU-to-GPU bandwidth on Hopper, eliminating the bandwidth degradation seen in point-to-point topologies; up to 1.5x higher throughput than point-to-point for Llama 3.1 70B on H200; fifth-generation NVLink at 1,800 GB/s per GPU with switch trays for 72-GPU domains.