What Is InfiniBand and Why Does It Matter for AI Clusters?
InfiniBand is a high-performance, low-latency network architecture originally developed for clustering supercomputers. In modern AI training infrastructure, InfiniBand has become the de facto standard because it delivers 200 Gbps (HDR) bandwidth with sub-microsecond latency—critical characteristics for distributed GPU training where thousands of GPUs must synchronize model parameters every training step.
Unlike Ethernet (even RDMA over Converged Ethernet / RoCE), InfiniBand uses a credit-based flow control mechanism that provides zero packet loss at the transport layer. This eliminates TCP retransmission overhead and makes InfiniBand the preferred choice for NVIDIA DGX systems, HGX platforms, and large-scale AI supercomputers.
Key InfiniBand Components in AI Data Centers
| Component | Model | Port Speed | Ports | Typical Role |
|---|---|---|---|---|
| Network Adapter | NVIDIA ConnectX-6 HX | HDR 200Gb/s | 1-2 | GPU server NIC |
| Network Adapter | NVIDIA ConnectX-7 | NDR 400Gb/s | 1-2 | Next-gen GPU server NIC |
| Network Switch | NVIDIA SN3700 | HDR 200Gb/s | 64 | Leaf/spine switch |
| Network Switch | NVIDIA SN3800 | NDR 400Gb/s | 8+16 | Core/aggregation switch |
Performance Comparison: InfiniBand vs Ethernet for AI
| Metric | InfiniBand HDR | 100GbE (RoCEv2) | 25GbE (TCP) |
|---|---|---|---|
| Raw Bandwidth | 200 Gbps | 100 Gbps | 25 Gbps |
| Effective Throughput | ~190 Gbps | ~85 Gbps | ~22 Gbps |
| Latency (RDMA write) | ~0.6 us | ~2-3 us | ~50-100 us |
| Packet Loss | Zero (credit-based) | PFC-based | TCP retransmission |
| NCCL AllReduce (8 GPUs) | ~35 GB/s | ~18 GB/s | ~4 GB/s |
Fabric tiers and what each one does
- Leaf switches sit in the rack and carry GPU-to-GPU traffic at line rate; spine switches interconnect the leaves
- A two-tier fat-tree is the normal shape for a single AI pod, with three tiers only when the pod outgrows one spine layer
- Oversubscription at the spine is the usual place a fabric loses its determinism
Sizing the fabric: ports, speed and node count
- Start from GPU count, not node count — each GPU contributes one NIC port to the leaf tier
- NDR 400G links with 2-way splitting give 200G per port when a fabric runs mixed generations
- Plan the uplink ratio so spine bandwidth does not become the ceiling on training throughput
Subnet management and the operational choices
- Managed Quantum-2 switches carry an on-board subnet manager, which keeps a small fabric self-contained
- Larger fabrics drive a central UFM instance for unified telemetry across the fabric
- Ask for the firmware level on any used switch you buy — it determines which management path is available
Related reading
- RoCE v2 vs InfiniBand — the InfiniBand vs Ethernet decision
- InfiniBand subnet manager HA — operational side of fabric management
- Spine-leaf architecture — topology detail
- Network switches — where the switch SKUs live
Frequently Asked Questions
How many InfiniBand switches does an AI cluster need?
Work from the GPU count. Each GPU adds one NIC port at the leaf tier, and leaf switches are sized to fill their cages — 32 or 64 NDR ports on the current Quantum-2 generation. A pod of 128 GPUs typically needs a small number of leaf switches plus one or two spine switches to interconnect them. Send us the GPU count and topology and we will size it with you.
What is the difference between a leaf and a spine switch in an AI fabric?
The leaf switch sits in the rack and connects GPU nodes at line rate, so all its ports face the workload. The spine switch connects the leaf switches and has far fewer, faster ports facing the fabric core. The design rule is that the spine tier must not become the bottleneck, so uplink oversubscription is the thing to get right.
Do I need UFM if my switch has an on-board subnet manager?
Not for a self-contained fabric. Managed Quantum-2 switches include an on-board subnet manager that handles fabrics up to roughly 2,000 nodes out of the box, which removes the need for a separate management host on day one. Larger fabrics, or ones where you want unified telemetry and centralized policy, are better served by a central UFM instance.
Can a Quantum-2 switch join an existing slower InfiniBand fabric?
Yes. Quantum-2 negotiates down to HDR 200G, EDR 100G and FDR, so a new switch joins a mixed-generation fabric and runs at the speed of the links it terminates. That makes staged upgrades practical rather than requiring a flag day.
Is InfiniBand or Ethernet better for AI training?
InfiniBand generally gives lower latency and a simpler path for lossless traffic without the tuning RoCEv2 needs. Ethernet with Spectrum-X has closed much of the gap and is often the better operational choice where the team already runs Ethernet fabrics. Our comparison article covers the decision in detail.
How do you verify a used InfiniBand switch?
We clean, re-flash and bench-test on our own test line, then confirm the port count, airflow direction and firmware level against the part number. Ask for photos of the actual unit and the firmware level before you commit, and check that the on-board subnet manager comes up on a fabric you can actually see.