By 2026, a single AI training rack is no longer a 10kW air-cooled box of 1U servers. NVIDIA HGX H200 pods routinely pull 80–100 kW, and the new GB200 NVL72 racks are specified to operate at 120–130 kW per rack, with hyperscale designs targeting up to 250 kW per rack using direct-to-chip liquid cooling and immersion-ready chassis. Air cooling alone cannot remove that heat at acceptable density. This guide walks through the engineering decisions you must make when planning a 100kW+ AI pod: busbar vs. whip distribution, coolant selection, manifold layouts, network fabric sizing, and how to choose the InfiniBand or Ethernet switches, adapters, and storage controllers that Zhongxin Data supplies for these workloads.
“Cooling is now a primary design constraint for AI infrastructure, not an afterthought. The rack, the room, and the grid must be co-designed.”
Why 100kW+ Racks Are the New Normal
Three forces are driving rack power density to unprecedented levels in 2026:
- Accelerator density: HGX H200 8-GPU baseboards draw ~10 kW each; a 42U rack with four baseboards plus switches easily exceeds 60 kW before networking.
- Blackwell-class systems: GB200 NVL72 compute trays integrate Grace CPUs with B200 GPUs in a single sled, pushing sled power past 5 kW each.
- GPU-to-GPU bandwidth: NVLink Switch fabric inside the rack multiplies interconnect power, while external ConnectX-8 / BlueField-3 SuperNICs add another 75–150 W per slot.
The result is that traditional 208V single-phase distribution and raised-floor airflow no longer suffice. New AI data centers are being built around 415V three-phase, 60kW+ per rack busbars, rear-door heat exchangers, and direct liquid cooling (DLC) loops with coolant distribution units (CDUs).
Rack Power Distribution Architecture
The table below compares the three dominant approaches for delivering 60–250 kW to a single rack. Use it as a starting checklist when sizing your busway, transformer, and UPS.
| Distribution method | Voltage / phase | Typical rack capacity | Best fit | Limitations |
|---|---|---|---|---|
| Traditional PDU + whip | 208V / 240V single-phase | Up to ~22 kW | Legacy air-cooled compute | Wire gauge, breaker count, and phase imbalance cap density |
| Three-phase PDU (415V) | 415V three-phase | 30–60 kW | Modern GPU servers, mixed CPU/GPU racks | Still air-cooling-bound for heat |
| Rack busbar + remote power panel | 415/480V three-phase | 80–250 kW | AI pods, immersion/DLC, hyperscale | Higher upfront cost, requires structured above-rack busway |
For an HGX H200 pod, expect four baseboards at ~10 kW each plus ~6 kW for switches and storage — that’s already 46 kW before overhead. For a GB200 NVL72 compute rack with nine compute trays, plan for 120+ kW of IT load alone, leaving little headroom for a 50/50 air/liquid split. Busbar distribution with a remote power panel (RPP) is the practical default for any 100kW+ AI rack deployed in 2026.
Liquid Cooling: Direct-to-Chip vs. Immersion vs. Rear-Door
Liquid cooling is no longer optional at 100 kW. The table below summarizes the three architectures and where each makes sense.
| Cooling architecture | Heat removal capacity per rack | Coolant | Serviceability | Best fit |
|---|---|---|---|---|
| Rear-door heat exchanger (RDHx) | 20–40 kW per rack | Facility water | High — no changes to servers | Brownfield retrofits, mixed CPU/GPU fleets |
| Direct-to-chip (DLC) cold plates | 60–150 kW per rack | PG25 / PG40 glycol-water | Medium — sealed loops per sled | New AI pods, HGX/GB200 baseboards |
| Single-phase immersion | 100–250 kW per rack | Engineered dielectric fluid (3M Novec, Shell S5) | Low — fluid handling, material compatibility | Dedicated greenfield AI data centers |
| Two-phase immersion | 200+ kW per rack | Low-boiling dielectric | Very low — specialized staff | Rare; research and a handful of hyperscalers |
Direct-to-chip with a closed-loop coolant distribution unit (CDU) is the dominant 2026 choice. It removes the bulk of the heat (CPUs, GPUs, often NIC retimers) via cold plates, while a smaller rear-door loop or air-handling unit (AHU) takes care of the rest. Plan the CDU at ~1.3× the steady-state IT load to leave headroom for transients and pump redundancy (N+1 pumps).
Coolant Selection and Water Quality
The single most common cause of premature cold-plate failure is poor water quality. Glycol-water mixtures (PG25 / PG40) buy you freeze protection and corrosion margin but reduce heat-transfer efficiency. Keep these targets in mind:
- Resistivity: > 1 MΩ·cm at 25 °C (deionized, monitored)
- pH: 8.5–9.5 for glycol-water loops; 7.0–9.0 for deionized water
- Particle count: < 100 particles/mL larger than 50 µm
- Microbial control:定期 conductivity sweeps + occasional biocide flush for PG loops
Inline filtration, deionizing cartridges, and a bypass conductivity probe are non-negotiable for 24/7 AI workloads. Plan a quarterly water analysis with your chemistry vendor.
Manifold, Pump, and CDU Sizing
The CDU is the heart of the liquid loop. For a 100 kW pod, a single 130 kW CDU with N+1 pumps is the typical baseline; for 250 kW+ designs, two CDUs in a lead/lag configuration with an auto-failover manifold. Use the table below to size secondary flow rates.
| IT load | Loop temperature target | PG25 flow rate (L/min) | Pipe diameter (primary) | Recommended pump head |
|---|---|---|---|---|
| 50 kW | Supply 35 °C / return 45 °C | ~72 | DN40 (1.5″) | 20 m |
| 100 kW | Supply 35 °C / return 45 °C | ~144 | DN50 (2″) | 25 m |
| 150 kW | Supply 32 °C / return 42 °C | ~215 | DN65 (2.5″) | 30 m |
| 250 kW | Supply 30 °C / return 40 °C | ~358 | DN80 (3″) | 35 m |
Lowering the loop temperature by even 3 °C increases cooling capacity at the cost of higher pumping power and chiller load. In warm climates, oversize the chiller plant to at least 1.4× peak IT load for free-cooling days.
Network Fabric for 100kW+ Pods
A high-density AI pod needs a network that can sustain all-to-all GPU gradients without starving the accelerators. That means NDR 400 Gb/s or NDR 800 Gb/s InfiniBand, or Spectrum-X 400/800 GbE, with at least one 400 Gb/s link per GPU sled. The table maps fabric choices to pod size and explains what to stock.
| Pod size (GPUs) | Recommended fabric | Per-GPU uplink | Spine switches | Zhongxin Data reference |
|---|---|---|---|---|
| 8–32 GPUs (single HGX) | NDR 400 InfiniBand or 400GbE | 1 × 400 Gb/s | Quantum-2 QM9700 | ConnectX-7 / ConnectX-8 Adapters |
| 64–256 GPUs (multi-HGX) | NDR 800 InfiniBand or Spectrum-X 800 | 1–2 × 800 Gb/s | Quantum-3 QM9700 / SN5600 | Spectrum SN5600 / SN5400 |
| 512+ GPUs (NVL72-scale) | NDR 800 InfiniBand + NVLink Switch | 2 × 800 Gb/s + NVLink | Quantum-3 + NVLink tray | NVIDIA Quantum-3 Switches |
For BlueField-3 SuperNICs or ConnectX-8 adapters, remember that each card adds 75–150 W. If your pod has 16 compute trays, that is an additional 1.2–2.4 kW per rack just for the NICs — include it in your cooling budget, not in the network-only BTU calculation.
Storage and RAID Considerations
AI training reads checkpoints and datasets at multi-GB/s rates. A MegaRAID 9560-16i or HBA 9500-16i at PCIe Gen4 x8 is the workhorse for NVMe hot-tier storage. For racks with 100+ kW of GPU compute, mirror hot-checkpoint NVMe drives (e.g. Kioxia CM7 or Samsung PM1743) at PCIe Gen5 where available. Use Broadcom MegaRAID adapters for RAID 6/10 protection on capacity-tier SAS or SATA SSD shelves.
Operations: Monitoring, Telemetry, and Failure Modes
Once a 100 kW pod is live, your operations team will live and die by three telemetry streams: inlet water temperature and pressure at each sled, coolant flow rate per cold plate, and rack-level power draw. The minimum viable monitoring stack is:
- Per-rack BMS gateway (Modbus TCP or Redfish) reporting coolant inlet/outlet temperatures, flow, and pressure.
- Per-PDU smart meter with 1-second sampling for spike detection.
- DCIM integration correlating thermal, power, and asset data — a 100kW+ rack that silently degrades from 32 °C inlet to 38 °C inlet is a fire risk, not just a performance issue.
- Coolant chemistry logging with weekly auto-export to your chemistry vendor.
Common failure modes to design around: pump cavitation after a power blip, biofilm accumulation in stagnant PG loops, scale build-up at heat-exchanger plates, and manifold O-ring dry-out. Each of these is preventable with a quarterly maintenance window — but only if your monitoring stack catches the early warning signs.
Site Planning: Whitespace, Slab Load, and Containment
Before a single rack arrives, three civil decisions dictate whether 100kW+ density is feasible:
- Raised floor vs. slab: New AI data centers are almost universally built on slab with overhead busway. Raised floors cap at ~1,500 kg/m² — a 250 kW DLC rack with manifolds can weigh 1,800+ kg, so reinforced slab (3,000–5,000 kg/m² load rating) is the safe default.
- Aisle containment: Cold-aisle containment (CAC) with blanking panels recovers 15–25% cooling efficiency versus open hot-aisle return. For DLC pods, hot-aisle containment (HAC) plus a sealed rear-door heat exchanger is often the cleaner thermal boundary.
- Ceiling height: Allow 3.6 m clear ceiling height above the finished slab to accommodate busbar drops, overhead cable trays, and rear-door heat-exchanger piping.
White-space planners should reserve a dedicated chiller plant yard sized for 1.3× peak IT load, and a CDU mechanical room adjacent to (but separate from) the white space for service access.
Total Cost of Ownership: Air vs. Liquid at 100 kW
A simplified TCO comparison for a 1 MW AI cluster running for 5 years. Assumes varies by configuration and volume/kWh blended electricity and a 12-year facility amortization.
| Design | Rack density | CapEx per MW (IT) | PUE | 5-year energy | 5-year TCO per MW |
|---|---|---|---|---|---|
| Air-cooled, 30 kW/rack | 30 kW | an amount quoted per part numberM | 1.35 | an amount quoted per part numberM | an amount quoted per part numberM |
| Hybrid DLC, 80 kW/rack | 80 kW | an amount quoted per part numberM | 1.15 | an amount quoted per part numberM | an amount quoted per part numberM |
| Full DLC + busbar, 120 kW/rack | 120 kW | an amount quoted per part numberM | 1.08 | an amount quoted per part numberM | an amount quoted per part numberM |
The higher-density designs recover their CapEx delta within 18–24 months purely on energy efficiency, and they leave far more headroom for the next GPU generation. Density is no longer a luxury — it is the cheapest path.
Build Your 100kW+ Pod with Zhongxin Data
Zhongxin Data supplies the networking, storage, and server-grade building blocks you need for AI pods — from Mellanox ConnectX-7 / ConnectX-8 NDR adapters and Quantum-3 / Spectrum-X switches, to Broadcom MegaRAID 9560 and HBA 9500 controllers, plus a full lineup of Dell PowerEdge, HPE ProLiant, and enterprise SSDs. Every used adapter is firmware-reflashed and soak-tested before shipping, so you can integrate them into your 100kW+ pod with confidence.
Need help sizing your busbar, CDU, or fabric for a new AI deployment? Contact our engineering team on WhatsApp +86 13068616914 or visit zhongxindata.hk for live inventory and custom configurations.
The network fabric decisions inside a 100kW pod
- Fabric choice at this density is driven by how much traffic the pod generates, and by whether the pod runs synchronous training or a mix of training and inference
- Switch and adapter variants have to match the pod’s thermal design and airflow direction, not just the port count
- We supply the switch families this pod profile uses, and availability is confirmed per part number
Storage and controller choices that scale with the pod
- The storage network inside the pod is sized separately from the training fabric, and checkpoint traffic should not share the collective path
- Controllers and adapters in the pod carry the same availability and testing standard as the rest of the estate
- Send the pod layout and the component list and we confirm availability per part number
What we verify on hardware that goes into a high-density pod
- Every unit is cleaned, re-flashed and bench-tested on our own test line before it is listed
- We confirm port count, negotiated link rate, firmware revision and error counters against the part number you send
- Ask us for photos of the actual unit before you commit
Related reading
- liquid cooling and high-density deployment — the cooling technology behind the pod design
- how liquid cooling affects NICs and optics — component-level thermal constraints
- spine-leaf architecture — the fabric inside the pod
- network switches — the switch families used at this density
Frequently Asked Questions
Can I retrofit an air-cooled data center to 100kW+ racks?
Rarely in place, and the constraint is usually the slab and the containment rather than the cooling kit. A retrofit to that density generally means new racks, a new power distribution path and a new coolant loop, and the existing electrical and structural work often costs more than the servers. For brownfield sites, raising the density of the racks that already exist is usually the cheaper route.
What is the realistic PUE for a liquid-cooled AI facility?
Well-designed direct-to-chip deployments typically land between 1.1 and 1.3 overall for the facility, though the figure is sensitive to where the boundary is drawn and how much of the IT load is cooling. Immersion generally pulls lower still. The number to plan against is the one for your design including the heat rejection path, not a generic benchmark.
How much water does a 100kW+ pod actually use?
The rack draws roughly 100kW of IT load, and at typical direct-to-chip supply and return temperatures the coolant loop moves a correspondingly large volume per second. Across a multi-megawatt pod this becomes a site water and heat rejection question, and the answer depends on the facility’s cooling tower or dry-cooler arrangement rather than on the network design.
What redundancy do I need for a single AI pod?
Redundancy has to cover the pod’s shared infrastructure — power distribution, coolant distribution units, and the fabric’s control plane — because a single failure point there takes out the whole pod rather than one rack. Network topology redundancy is necessary but not sufficient. Plan the failure domain of the pod’s shared services before the fabric.
Are used parts usable in a 100kW+ pod?
Yes, provided each unit has been tested and its condition is stated. Every unit we list is cleaned, re-flashed and bench-tested on our own test line before it is listed, and availability is confirmed per part number. In a pod this dense, we would rather see you confirm the specific units than rely on a general statement about condition.