Networking for AI

InfiniBand — Preferred for large-scale AI training:

• HDR: 200 Gb/s per port
• NDR: 400 Gb/s per port
• Sub-microsecond latency
• Native RDMA
• Managed by NVIDIA UFM (Unified Fabric Manager)

NVLink — GPU-to-GPU interconnect inside a node:

• NVLink 3.0 (A100): 600 GB/s total
• NVLink 4.0 (H100): 900 GB/s total
• Connected through NVSwitch for all-to-all GPU communication

RoCE (RDMA over Converged Ethernet):

• RDMA over standard Ethernet
• Requires lossless Ethernet with PFC (Priority Flow Control)
• Lower cost than InfiniBand

Key distinction: InfiniBand fabric → UFM. NVLink/NVSwitch → Fabric Manager. Do not mix these up.