NVIDIA ConnectX Network Adapters: Core of Data Center Connectivity
ConnectX adapters provide RDMA, RoCE, and InfiniBand capabilities for AI training clusters, distributed storage, and HPC. This guide compares ConnectX-5, ConnectX-6, and ConnectX-7.
Specifications Comparison
| Specification | ConnectX-5 Ex | ConnectX-6 Lx | ConnectX-6 HX | ConnectX-7 |
|---|---|---|---|---|
| Max Port Speed | 100Gb/s | 25/50/100GbE | HDR 200Gb/s | NDR 400Gb/s |
| Ports | 2 | 2 | 2 | 2 |
| PCIe | Gen3 x16 | Gen4 x8 | Gen4 x16 | Gen5 x16 |
| RDMA | IB+RoCEv2 | RoCEv2 | IB+RoCEv2 | IB+RoCEv2 |
| RDMA Latency | ~0.8 us | ~1.0 us | ~0.6 us | ~0.5 us |
| Cache | 4 MB | 8 MB | 16 MB | 32 MB |
| SecureBoot | No | No | Yes | Yes |
| Power | ~25W | ~15W | ~40W | ~60W |
ConnectX-5 Ex: The Proven 100Gb/s Platform
Widely deployed in DGX A100 clusters (4 per DGX), 100GbE storage networks, and HPC EDR fabrics. Best for: Existing 100Gb/s deployments needing cost-effective replacement.
ConnectX-6 Lx: The Cost-Effective Ethernet Accelerator
25/50/100GbE with RoCEv2, 15W TDP. Best for: 25-100GbE Ethernet deployments needing RDMA without InfiniBand.
ConnectX-6 HX: The Dual-Mode HDR 200Gb/s Accelerator
200Gb/s InfiniBand or Ethernet, dual-port dual-mode (IB+Eth simultaneously). Best for: AI training clusters and 200Gb/s storage fabrics.
ConnectX-7: The NDR 400Gb/s Next-Gen Adapter
400Gb/s for H100/H200, PCIe Gen5, OCP 3.0 form factor. Best for: H100/H200 GPU deployments and 400Gb/s infrastructure.
Choosing a speed class that matches your workload
- 25GbE suits storage and management traffic, and is the cheapest way to leave 1GbE behind
- 100GbE is the mainstream data center rate — most east-west traffic today terminates here
- 200GbE is for GPU training fabrics and storage arrays that can saturate 100G on both ends
Card form factors and the slots they need
- Single-port versus dual-port changes the number of PCIe lanes you spend and the switch ports you consume
- PCIe 3.0 x16 carries a 100GbE link comfortably; 200GbE wants PCIe 4.0 x16
- Check the physical bracket and cage height against your chassis before ordering
Media and connector choices that affect deployment
- QSFP28 covers 100GbE and QSFP56 covers 200GbE, so the optic type follows the speed class
- DAC and AEC cables work over short rack distances; optics take over beyond about 5 metres
- We confirm the card’s cage type and supported optic modes against the part number
Related reading
- MCX516A-CCAT dual-port 100GbE — the mainstream 100GbE option
- MCX653106A-ECAT dual-port — the newer dual-port generation
- ConnectX-7 vs ConnectX-8 — next generation
- Network adapters — category hub
Frequently Asked Questions
Which ConnectX adapter should I buy for a 100GbE server?
For PCIe 3.0 x16 servers, the MCX516A-CCAT is the common 100GbE choice. For PCIe 4.0 x16 servers, the MCX653105A-ECAT gives you a newer generation and more headroom. The deciding factors are your PCIe generation, how many switch ports you want to consume, and the media type you already have installed.
Is a dual-port ConnectX card better than two single-port cards?
Usually yes, because a dual-port card consumes one PCIe slot instead of two and gives you a single point of management. It also means one card to test and one SKU to stock. The trade-off is that a dual-port card uses more lanes, so check the slot width in your server.
Can I use a 25GbE adapter in a 100GbE switch?
Yes. A 100GbE switch with QSFP28 cages will break out to 25GbE on a compatible port, and a 25GbE card with the right cage will negotiate down to 25GbE. You will need the switch and the card to support the breakout or down-negotiation path — tell us both part numbers and we will confirm the combination.
Do I need 200GbE for a mid-sized AI cluster?
It depends on the model and the collective communication pattern. If your training runs heavy all-reduce traffic across many GPUs, 200GbE per GPU removes a real bottleneck. If the cluster is smaller or the workload is inference-weighted, 100GbE is usually sufficient and considerably cheaper. We can look at your GPU count and training pattern and give a direct recommendation.
What is the latency difference between ConnectX generations?
Generations differ less in latency than in bandwidth and in how much CPU they offload. The practical differences are the supported link speeds, the PCIe generation requirement and the offload features for storage and virtualization. For latency-sensitive small-message traffic the network design and the switch matter more than the adapter generation.
How do you test a used ConnectX adapter?
We clean, re-flash and bench-test on our own test line, then confirm the part number, port count, negotiated link rate and firmware level, and we read the error counters. A card that links but produces errors does not get listed. Ask for photos of the exact unit and its firmware level before you commit.