Quick answer: Direct Liquid Cooling (DLC) — cold plates on CPUs, GPUs, and increasingly on the NIC — reduces junction temperatures by 25–40 °C versus traditional hot-aisle air cooling. That thermal headroom lets you push ConnectX-7 / ConnectX-8 cards and Quantum-2 / Spectrum-4 switches to their full 400G/800G envelope with much lower failure rates. But the optics and the breakout DAC cables are still the weak link: anything that ends up outside the cold-plate loop must survive a hotter microenvironment, and immersion-cooled tanks need sealed, dielectric-rated optical modules and AOCs.
Why liquid cooling is no longer optional in modern AI training clusters
When NVIDIA published the reference design for the DGX H100 / H200 / B200 racks, the spec sheet quietly moved the heat budget from the CPU socket to the entire rack bus. A fully populated 8-way H200 rack now dumps 80–120 kW into ~60 cm of rack depth. Traditional Computer Room Air Handlers (CRAHs) cannot push that heat out of the hot aisle fast enough — the delta-T across the cold aisle collapses and the GPUs throttle. That is why hyperscalers (Meta, Microsoft, Oracle, ByteDance) all moved to direct-to-chip or full-immersion cooling for new AI buildouts in 2024–2026.
For datacenter operators building or refreshing a 1–10 MW AI training pod, the question has shifted from “should we adopt liquid cooling?” to “what does liquid cooling do to the network stack we just bought from Zhongxin Data?”
Three thermal zones inside one rack
| Zone | Typical ΔT vs ambient | Components | Cooling strategy |
|---|---|---|---|
| Zone A — Cold plate | +5 to +15 °C above coolant inlet | CPU, GPU, HBM stacks, increasingly the NIC SoC | Direct Liquid Cooling (DLC) cold plate |
| Zone B — Warm air pocket | +20 to +35 °C above cold-aisle air | DIMM DDR5 modules, VRMs, internal M.2 boot drives | High-velocity air / rear-door heat exchanger |
| Zone C — Patch panel & cabling | +15 to +25 °C above cold-aisle air | AOCs, DAC/ACC breakouts, QSFP-DD/OSFP faceplates, patch-panel transceivers | Sealed LC patches / routed coolant on the spine switch |
The networking gear never sits in Zone A in current-generation designs — even when the GPU cold plate gets the benefit of DLC, the NIC and the faceplate pluggable optics stay in Zone B/C. That asymmetry is the source of most optical-link failures we see in customer support tickets.
What changes for the NIC itself
ConnectX-7 / ConnectX-8 in DLC racks
Mellanox ConnectX-7 NDR (400G) and ConnectX-8 XDR (800G) cards are spec’d for a Tj of 105 °C and an ambient operating window of 0–55 °C. In a hot-aisle air-cooled rack, the NIC ASIC typically runs at 90–100 °C under sustained 400G load with traffic pushing P99. After you bolt a cold plate onto the card (which NVIDIA, Dell, and HPE all offer as a standard SKU), the junction drops to 60–70 °C. The win isn’t just lower temps — it is the headroom to:
- Hold sustained 400G line rate on RoCE without thermal throttling
- Run deeper SERDES pre-emphasis for longer-reach DAC and ACC breakouts
- Drop the ASIC failure rate meaningfully (every 10 °C roughly doubles silicon lifetime)
What about the optics?
Pluggable optics — the QSFP-DD800 and OSFP faceplates — are not cold-plated. They live in the patch panel, and they pull up to ~14 W each at 800G. In a fully populated rack that is 8–16 kW of optical heat *per rack* that has to be moved by air cooling alone. The realistic operating window for a sealed-top OSFP in this position is +5 to +70 °C ambient, with a long-term drift at the hot end.
| Ambient temp (faceplate inlet) | Predicted BER @ 800G PAM4 | Notes |
|---|---|---|
| 25 °C (cold aisle) | 1e-9 (clean) | Spec sheet baseline |
| 35 °C | 1e-8 | Marginal — bit-error-correction working hard |
| 45 °C (typical hot-aisle pocket) | 1e-7 to 1e-6 | FEC correcting 2–3 dB of margin loss |
| 55 °C + sustained load | FEC uncorrectable events begin | Replace with sealed OSFP, or relocate patch panel |
The cabling fork: DAC, ACC, AOCs, and optical line systems
As the speed per lane climbed from 25G (NRZ, 100G) to 50G (PAM4, 200/400G) to 100G (PAM4, 800G) and now to 200G-per-lane for 1.6T, the copper reach shrank dramatically. What used to be a 5 m passive DAC at 100G is now 1.5 m at 800G before the eye diagram collapses. That changes the cabling topology of a DLC rack in three ways:
- Inside the rack: — short ACC (Active Copper Cable) or near-active AECs for ≤ 2 m GPU↔leaf-switch runs.
- Across the row: — AOCs (Active Optical Cable) with sealed boots, because the coolant manifold cannot tolerate hot spots from passive copper at 800G.
- Between rows: — full line-system optical transceivers + single-mode patch, often routed through overhead trays.
| Cable type | Reach at 400G (PAM4 50G/lane) | Reach at 800G (PAM4 100G/lane) | Liquid-cooling compatibility |
|---|---|---|---|
| DAC (passive copper) | ≤ 3 m | ≤ 1.5 m | OK in Zone C (above cold plate) |
| ACC (linear-drive copper) | ≤ 5 m | ≤ 3 m | OK; check boot temp rating |
| AOC (active optical) | ≤ 30 m | ≤ 30 m | Sealed-rated required above coolant loop |
| SR4 / SR8 (parallel MMF) | ≤ 100 m (OM4) | ≤ 50 m (OM4, 800G) | Faceplate-only; never immersed |
| DR4 / DR8 (single-mode) | ≤ 500 m (WDM mux) | ≤ 500 m | Standard patch panel; OK |
| FR4 / 2×FR4 | ≤ 2 km | ≤ 2 km | Standard patch panel; OK |
The practical guidance from the field: at 800G, AOC starts at 3 m and replaces DAC almost everywhere outside of one-server-to-adjacent-switch topologies. We see DAC purchases drop by ~60% when a customer moves from air-cooled to DLC racks at the same speed per port.
Three procurement pitfalls to avoid on a first DLC deployment
We have walked through enough first-time DLC rollouts to flag the three mistakes that always come back to bite the team that spec’d the rack.
Pitfall #1 — buying optics off the same BOM as the air-cooled version. When the rack architecture changes, the optics SKU often needs to change with it. A standard QSFP-DD800 module rated for 0–70 °C is technically legal in a DLC rack, but the cooler racks live at 25–30 °C, which is well inside its spec, whereas pockets behind the patch panel can hit 55 °C on a hot day. Ask your supplier (us, in this case) for the “sealed-top” variant; it costs an amount quoted per part numbermore per module and pays back in the first summer.
Pitfall #2 — underprovisioning the CDU bypass loop. A 1 MW DLC pod needs 1.5 MW cooling capacity in steady state, but it also needs a fast bypass to absorb power spikes from GPUs going into and out of all-reduce rings. A CDU rated for exactly 1 MW will let coolant temperature spike 6–8 °C during a 90-second all-reduce event, which translates to a 0.5 °C drift on the NIC junction — enough to soft-throttle the ConnectX. We always recommend 1.5× peak sizing, which leaves thermal margin for one (1) failed pump in the secondary loop.
Pitfall #3 — ignoring dew-point and condensation during downtime. When you take a DLC rack offline for service, the cold plates stay cold. If you open the rack to swap a NIC in humid ambient air, condensation will form on the cold plate and on the NIC ASIC and on the optical faceplate. Worse, the moisture that wicks into the optical connector is invisible until the laser fails three weeks later. The fix is procedural: bring rack inlet air to < 30% humidity before opening, and never open a cold plate within 60 minutes of a power-down.
Software-side tuning that pays off on DLC
Hardware is only half the story. Three configuration changes on the NIC bring out the most of a DLC deployment:
- Set the thermal policy to “maximum performance” instead of “balanced.” The throttle ceiling jumps from 95 °C to 105 °C; you bought the headroom, use it.
- Disable the IR3 (interrupt moderation 3) preset when running all-reduce heavy workloads. DLC lowers the interrupt latency floor; you can move moderation from 20 µs to 5 µs without paying a CPU cost.
- Enable PFC + ECN on every lossless queue, even on single-tenant AI fabrics. Without it, a single congestion event drops a flow that needs 5+ retries to recover, and those retries propagate head-of-line blocking across the whole all-reduce ring.
These three settings, all in mlxconfig or the UFM/Spectrum-X manager, are the cheapest 5–8% per-iteration improvement you will find in 2026 — and they only became safe once DLC dropped the NIC junction out of the throttle band.
Migration playbook: 18-month timeline we recommend
For a customer running an existing 5 MW air-cooled AI cluster who wants to migrate to DLC without rebuilding from scratch, our typical playbook is:
- Months 0–3: Pilot on one rack. Replace one air-cooled compute rack with a DLC rack. Confirm the in-rack thermal map with rack-mounted sensors. Run shadow training traffic for 30 days before declaring success.
- Months 3–6: Roll out the leaf/spine network into the existing cold-aisle containment. You can mix DLC compute with air-cooled switches for the pilot — that’s the design we recommend.
- Months 6–12: Standardize the BOM across the fleet. Lock in the cold plate SKU per original-vendor server. Lock in the AOC cable BOM. Lock in the immersion-rated OSFP SKU for any immersion tiers.
- Months 12–18: Decommission the legacy CRAH units, switch the secondary loop to free-cooling, hit the < 1.2 PUE target.
This is roughly the path our customers at zhongxindata.hk have walked twice in the last 24 months. The articles above (in particular our Technical Guides series) document the failure modes and the recovery patterns in more depth.
Immersion vs cold-plate — different rules for optics
Single-phase immersion with dielectric fluorinert or mineral oil is enjoying a comeback for very-high-density AI training. Immersion gets the entire NIC, including the heat sinks, into the dielectric. It also gets the faceplate. Pluggable optics that are not immersion-rated will fail within weeks: the optical lubricant outgases, the laser wavelength drifts, and the housing plastic absorbs the fluid.
The fix is straightforward but procurement-specific: ask for immersion-rated OSFP/QSFP-DD modules. Mellanox/NVIDIA sells them with a sealed top (closed-cell foam gasket) and an inert laser package; the SKU is different. The bad news is that immersion-rated 800G optics currently carry ~25% price premium and longer lead times because the volumes are still niche. Many operators avoid the issue by using direct-to-chip DLC for the server side and air cooling for the spine switch — this is the dominant 2025/2026 design we ship to customers.
Switch-side thermal: Quantum-2 and Spectrum-4 in DLC pods
The leaf and spine switches — NVIDIA Quantum-2 QM9700 (64-port 400G NDR) and Spectrum-4 SN5600 (64-port 800G XDR) — pull ~1.5–2.2 kW per switch under full load with FEC on. Cold-plating an entire switch chassis is non-trivial (different thermal interface than a CPU), so the standard approach is to keep the switch on cold-aisle air but in a sealed cold-aisle containment with a higher ΔT allowance. The next-generation SN6000 series (2026) brings a integrated DLC option — direct manifold into the switch mid-plane — which addresses the heat density issue head-on.
For operators sizing a new AI pod in 2026, the rough budget per 1 MW of training compute is:
- ~16 racks of DLC compute (each ~60–65 kW)
- ~3 racks of network gear (spine + leaf), partly DLC on the SN6000
- Coolant distribution unit (CDU) sized at 1.5× peak heat (for transient spikes)
Closing note
Liquid cooling is now an assumption for any 2026 AI cluster buildout, not an option. The networking stack you put inside that rack has to be selected with the three thermal zones in mind — and the optics, surprisingly, end up being the deciding factor for whether the rack actually runs reliably. We’ve shipped more than 200 DLC-ready AI pods in the last 18 months; if you want help sizing your topology, talk to our solutions team.
The three thermal zones and what each one does to your hardware
- Compute sled, switch zone and the connection path have different thermal envelopes, and the part number has to match the zone it lands in
- An adapter that is correct for an air-cooled rack can be wrong for the same position in a cold-plate chassis
- We supply the adapter and switch families for these positions, and availability is confirmed per part number
Choosing media when the rack is wet
- Passive copper, active copper and optics each have different rules once a cold plate sits between the connector and the airflow
- The connection path — from the sled through the manifold to the rack boundary — usually needs rethinking before the optic choice does
- Confirm the reach, the port type and the thermal zone together, because they constrain each other
What changes in how we test the hardware for these racks
- Every unit is cleaned, re-flashed and bench-tested on our own test line before it is listed
- We confirm negotiated link rate, firmware revision and error counters against the part number you send
- Ask us for photos of the actual unit before you commit
Related reading
- liquid cooling and high-density deployment — the rack-level cooling context behind this component view
- rack-scale power and cooling for 100kW+ pods — the facility plan this retrofit follows
- data center cabling guide — media selection rules
- network adapters — the NIC families whose thermal envelope changes
Frequently Asked Questions
Do I need to re-spec every NIC and switch when migrating to liquid cooling?
Not every part, but the ones whose position moves into the cooled zone. The chassis design determines the thermal path, and the part number has to match that design. The parts sitting outside the cooled path usually carry over unchanged — but confirm per position rather than assuming, because a mixed chassis can put two different thermal envelopes in the same rack.
Are DAC cables still usable inside a DLC rack?
Yes, in the majority of designs. Passive DAC does not care about the cooling method as long as the connector stays within its temperature rating and the cable’s bend radius and routing are respected. What changes is the routing constraint — in a cold-plate chassis the cable path is narrower, so the bend radius and strain relief are worth re-checking against the new layout.
How does liquid cooling affect MTBF for optics?
It depends on whether the module’s inlet temperature rises. Transceivers are rated to a maximum ambient, and a module sitting in a warm recirculation path inside a liquid-cooled chassis can see a higher local temperature than the rack average. Where the design keeps the inlet within rating, the effect is small; where it does not, expect thermal throttling and error-counter drift rather than an outright failure.
Should I use immersion or cold-plate?
Cold-plate is the mainstream choice for AI training clusters because it keeps conventional server, switch and optic designs working. Immersion gives better cooling density but forces the connection path out of the fluid and changes the cabling rules substantially. Choose cold-plate unless you have a reason to accept the cabling rework.
What should I buy from Zhongxin Data to start a DLC retrofit?
We supply the switch and adapter families used in these racks, and availability is confirmed per part number. Send the chassis models, the positions you are replacing and the quantity, and we quote per part number with a condition note on the actual units. We do not publish stock counts.