Table of Contents

Introduction: Why Interconnect Choice Matters in the TOP500 Era

Every June and November, the TOP500 list ranks the world’s most powerful supercomputers by their LINPACK Rmax score. Headlines focus on exaflops, but the second-most-watched metric is the interconnect family: InfiniBand, Cray/HPE Slingshot, Intel Omni-Path, or standard Ethernet. Between June 2023 and June 2025, three shifts reshaped that fabric column:

  1. InfiniBand still rules the very top of the list (Frontier, Aurora, Fugaku, Leonardo, LUMI and most new exascale-class systems), but its share of total systems shrank.
  2. HPE Slingshot ate the middle of the list, taking a third of all TOP500 systems on the November 2025 snapshot.
  3. Ethernet (40/100/200 GbE) declined steadily, replaced by purpose-built AI fabrics — NVIDIA Spectrum-X on Ethernet, or RoCE fabrics tuned for HPC/AI — that don’t show up as “GigEthernet” in legacy monitoring columns.

If you operate or spec a cluster, the TOP500 interconnect column is a lagging indicator: the systems shipping today are already two generations ahead of what shows up in the public list. Read it as a <60-day-future signal, not a snapshot of the present.

This guide turns the TOP500 interconnect column into a practical decision frame for 2026 procurement: which fabric wins which workload, how bandwidth and topology trends project into your next refresh, and how to source the underlying switches, adapters and cables without paying new-list price.

What the TOP500 Actually Tells Us About Network Fabric

The TOP500 list has published an interconnect column since 2002. Over twenty-three years, the column has tracked three generations of fabric wars:

Table 1 — Three eras of TOP500 interconnect
Era Years Dominant fabric Peak per-port speed Defining design
Custom quadrics / Myrinet 2002–2010 Myrinet, Quadrics, custom torus 10 Gbps Proprietary NICs, host-based MPI
InfiniBand & 10GbE 2010–2020 InfiniBand FDR/EDR, then HDR 100 Gbps (EDR), 200 Gbps (HDR) RDMA, kernel bypass, SHARP in-network compute
HPC-AI convergence 2020–present InfiniBand NDR, Slingshot 200/400, Spectrum-X 400 Gbps (NDR), 800 Gbps (XDR / Spectrum-4) Co-packaged optics, adaptive routing, in-network collectives

What’s new in 2024–2026 is purpose-built AI Ethernet. Generic 100GbE/Ceph-style fabrics are getting replaced by AI-tuned fabrics — NVIDIA Spectrum-X, Ultra Ethernet Consortium (UEC) v1.0, and Broadcom Jericho3-AI. These are Ethernet at the link layer but with explicit RDMA, congestion control, and in-network compute that mirror what InfiniBand has done for 15 years.

2024-2025 TOP500 Interconnect Share

From the public November 2025 TOP500 snapshot (and the June 2025 sister list), the fabric distribution looks like this for systems on the list:

Table 2 — Approximate TOP500 fabric share, Nov 2025
Interconnect family Approx. share of TOP500 Major systems Hardware line
InfiniBand (HDR/NDR) ~31% Fugaku successor, Leonardo, MareNostrum 5 ACC, JUPITER Booster (partial) Quantum-2 (HDR 200G), Quantum-3 (NDR 400G)
HPE Slingshot ~33% Frontier, Aurora, El Capitan, LUMI, Perlmutter Slingshot 200/400 with Cassini/Aruba switches
Ethernet (AI-tuned) ~20% Spectrum-X, RoCE v2, UEC v1.0 early adopters Spectrum-4, Spectrum-3, Tomahawk 5, Jericho3-AI
Custom / Omni-Path ~10% Fugaku (Tofu-D), Aurora partial (Intel fabric) Fujitsu Tofu, Intel Omni-Path (legacy)
Standard Ethernet (10/25/40/100 GbE) ~6% Storage clusters, smaller university installs Generic top-of-rack switches

The “Ethernet” line in older TOP500 data conflates two very different things: commodity 100GbE storage fabric and AI-tuned 400G/800G Ethernet with RDMA. The latter category is what you should benchmark against — the former is a storage tier, not an HPC fabric.

The headline number to watch is the Slingshot vs InfiniBand ratio. In 2018 InfiniBand represented ~55% of TOP500; Slingshot was a rounding error. Seven years later, Slingshot has tied for top spot, and the first systems built on Spectrum-4 800G switches are now climbing the list. By the November 2026 list we expect Spectrum-X + UEC combined to overtake Slingshot in raw system count.

Bandwidth Growth: FDR to NDR 400G and XDR 800G

Track the per-port link rate over time, and you can predict which fabric your next refresh will land on:

Table 3 — Per-port bandwidth progression in TOP500-class systems
Year fabric shipped InfiniBand generation Slingshot generation Ethernet (AI-tuned) equiv. Per-port bidirectional Typical deployment
2014 FDR — 40 GbE ~56 Gbps Pre-AI HPC, MPI workloads
2016 EDR — 100 GbE ~100 Gbps Early DL training (single-GPU servers)
2019 HDR Slingshot 100 200 GbE (early) ~200 Gbps Multi-GPU nodes, SHARPv2
2022 NDR Slingshot 200 Spectrum-3 400G, UEC pre-spec ~400 Gbps LLM pre-training at 1000+ GPU scale
2024 XDR (announced) Slingshot 400 Spectrum-4 800G, Tomahawk 5 ~800 Gbps Frontier-scale HPL, Mixture-of-Experts training
2026 (projected) Generation 6 (codename pending) Slingshot 800 (rumored) UEC v1.0 1.6T, Spectrum-5 ~1.6 Tbps Million-GPU training fabrics, real-time agent inference

Three observations from this table that change sourcing decisions:

  • Per-port bandwidth doubled 3× in the last 6 years. A switch you bought in 2019 at HDR will be 4× slower than a Spectrum-4 800G switch in 2026. used that 2019-era kit for AI training only makes sense for fine-tuning or inference, not pre-training.
  • Ethernet has caught up in raw bandwidth — Spectrum-4 800G ties InfiniBand XDR for per-port speed. The remaining gap is in in-network compute (SHARP for InfiniBand; emerging UEC equivalents) and software maturity.
  • Optics refresh every 2 generations. HDR used 200G PAM4 with QSFP28. NDR moved to QSFP-DD for 400G. XDR uses OSFP with 800G PAM4. Sourcing optics across generations is the single biggest hidden cost of a multi-vendor fabric.

Five Workloads Driving Today’s Interconnect Choices

Pull the TOP500 interconnect column together with the workload tag each system runs, and a clear pattern emerges:

Table 4 — Workload-to-fabric mapping seen on recent TOP500 lists
Workload archetype Dominant fabric Why Representative systems
Foundation-model pre-training (LLMs, MLLMs) InfiniBand NDR + Spectrum-X SHARP/NCCL-tuned RoCE for collective-heavy comms JUPITER Booster, Leonardo, Grace Hopper clusters
Scientific HPC (CFD, climate, fusion) Slingshot / InfiniBand Low-latency MPI, mature RDMA, large bisection BW Frontier, LUMI, El Capitan
Mixed HPC + AI (digital twin, scientific LLM) Slingshot 200/400 One fabric serves both MPI and gradient all-reduce Aurora, Perlmutter-NextGen
AI inference at scale (real-time agents) Spectrum-X / UEC Ethernet Higher rack density, simpler tooling, lower TCO Hyperscaler internal pods (not public TOP500)
National research clouds (training-serving hybrids) UEC v1.0, Spectrum-X Multi-tenant, multi-vendor, cost-driven JSC Booster follow-on, CSCS Alps

Notice the workload split is not “InfiniBand vs Ethernet” — it’s collective-heavy HPC vs scale-out AI inference. The first still leans InfiniBand or Slingshot because SHARP and on-switch reductions save real GPU cycles. The second has been moving to Ethernet because the inference traffic pattern (small messages, burst arrivals, east-west from GPU pool to KV-cache pool) tolerates Ethernet congestion control as long as the switch latency is low enough.

How Ethernet Has Caught Up: Spectrum-X and Ultra Ethernet

The single biggest 2024-2026 shift in the TOP500 interconnect column is the rise of two AI-specific Ethernet stacks. Both target the same weak point: standard 100/200 GbE RoCE drops GPU utilization to 60-70% under heavy all-to-all because of head-of-line blocking and tail latency spikes.

Table 5 — AI-tuned Ethernet stacks compared
Stack Switch silicon Congestion control In-network compute Topology fit Bandwidth tier
NVIDIA Spectrum-X Spectrum-4 / Spectrum-3 Adaptive routing + per-flow load balancing SDK-level SHARP analogues Fat-tree, rail-optimized 400G / 800G
Ultra Ethernet (UEC) v1.0 Broadcom Tomahawk 5, Intel Silicon Photonics Packet spray + credit-based flow control None at v1.0 (planned for v2) Dragonfly+, fat-tree 200G / 400G / 800G
HPE Slingshot 400 (Ethernet family) Cassini / Aruba CX 10K Adaptive routing + virtual lanes None (software-managed collectives) Dragonfly 200G / 400G
Standard RoCE v2 Generic ToR PFC + ECN None Clos / leaf-spine 100G / 200G

The practical difference shows up in cluster-level benchmark numbers. On 1024-GPU AllReduce of a 1 GB gradient tensor, Spectrum-X reports sub-1.2× slowdown vs native NCCL SHARP, while standard RoCE v2 on the same hardware reports 1.8× to 2.4× slowdown. UEC v1.0 lands in the middle.

If you can’t get InfiniBand (supply, lead time, or power budget), the second-best fabric in 2026 is Spectrum-X on Spectrum-4 800G. The third-best is a Broadcom Tomahawk 5 cluster on UEC v1.0, which depends on the silicon vendor delivering in-network credit loop.

Wiring It Up: What Buyers Look for in 2026

Three signals from the last three TOP500 lists that should change your procurement checklist:

  1. HDR-to-NDR cuts the path to fine-tuning only. If you have Quantum-2 (HDR) fabric and your workload is inference or LoRA fine-tuning, you’re fine. For pre-training, move to NDR or Spectrum-4 within 18 months.
  2. Spectrum-4 is the new floor. New 800G Spectrum-4 systems are appearing in pilot form on the November 2025 list. By November 2026 they’re expected to dominate the top 10. If your next switch is 200/400G, plan a second refresh window 24–36 months out.
  3. Slingshot stays the best-of-breed for HPC, but its procurement is locked to HPE. If you’re not an HPE shop, the realistic alternatives are InfiniBand Quantum-3 or Spectrum-X.

And three that should not change your thinking:

  • Custom fabric is dead. Tofu-D, Aries, and Torus are now single-vendor, single-system curiosities. No new build goes custom.
  • FDR is e-waste. Anything older than 2018 is fine for development clusters but not for production training.
  • Generic 100GbE without RDMA is storage, not compute. It shows up in the TOP500 column because clusters are multi-purpose, but it doesn’t carry gradient comms in any modern AI training setup.

Practical Sourcing Notes from the Field

For the buyer who already owns, or is about to buy, InfiniBand or AI-tuned Ethernet, here is what we see across the channel right now:

  • Quantum-2 (HDR, 200G) — mature, plentiful on the used channel. Treat as the “good enough” tier for inference and fine-tuning; full retail on used ConnectX-6 and Quantum-2 switches is typically 35-50% of new list.
  • Quantum-3 (NDR, 400G) — available new, but lead time and minimum order quantity (MOQ) constraints make this a planned-procurement item. For smaller builds, used NDR with warranty terms are confirmed per model entry.
  • Spectrum-3 / Spectrum-4 switches — only, both. Spectrum-3 (400G Spectrum-X) is starting to appear on the secondary market; Spectrum-4 (800G) is not. Plan accordingly.
  • Optics — cross-generation optics are the single biggest procurement surprise. HDR-to-NDR cables are not interchangeable; budget 8-12% of switch CAPEX on optics.

For a buyer trying to match what you see on the TOP500 list without paying hyperscaler prices, the realistic path in 2026 is a mixed-generation fabric: Spectrum-3 or Quantum-3 for the training tier, used HDR for the inference tier, and at least one future-proofed 800G island for evaluation.

If you want help mapping your specific workloads to the right fabric generation, browse the models listed on zhongxindata.hk: InfiniBand switches, network adapters, and used switches, or message our solutions team on WhatsApp +86 13068616914 with your cluster size and workload profile. We can spec a fabric that mirrors the TOP500 tier your workload actually needs — not the one your vendor wants to clear.


Reading the interconnect share in the rankings

  • The ranking shows which platform operators chose, which is not the same as which performs better — adoption follows supply, support and familiarity
  • Look at the fabric generations appearing rather than the vendor split, because the generation tells you where the capability is
  • Both the InfiniBand and Ethernet switch families we list appear in this class of deployment, and availability is confirmed per part number

What buyers in this segment actually source

  • Lead time and firmware support path matter as much as the specification at this scale
  • A fabric is usually bought as a set — switches, adapters, optics — so availability per part number across the set matters
  • Send the fabric inventory and we confirm availability per part number and quote per part number

How we verify the fabric hardware we list

  • Every unit is cleaned, re-flashed and bench-tested on our own test line before it is listed
  • We confirm negotiated link rate, firmware revision and error counters against the part number you send
  • Ask us for photos of the actual unit before you commit


Related reading

Frequently Asked Questions

Does the TOP500 ranking actually reflect real-world AI workloads?

Partly. The systems are benchmarked on real HPC workloads, which are communication-heavy in a way that resembles large-scale training. But the rankings reflect what was available and affordable when each system was built, so a generation that was scarce at submission time appears rarer than it became afterwards. Read it as an adoption history rather than a quality scoreboard.

Is InfiniBand still worth buying given Ethernet advances?

For the workloads where the fabric is managed end-to-end and the collective traffic dominates the step time, the answer is usually yes. Ethernet has closed much of the gap and wins on cost per port. The deciding factors are usually the installed NIC population, the fabric management staffing, and whether the workload depends on native IB verbs semantics.

How long does a TOP500-class fabric remain competitive?

A fabric generation stays competitive for several years because the capital cost of a fabric is significant and the software stack has to be stable. The point at which it becomes a problem is usually software support and power efficiency rather than raw capability, and those are the two things to watch as a fabric ages.

Can I mix InfiniBand and Ethernet in one cluster?

Yes, as separate fabrics with a deliberate interconnect. Running both fabrics serves different purposes well — one fabric for the most demanding training job and another for storage, management and inference. Bridging them directly is where unexpected behaviour appears, so keep them separate and plan the interconnect explicitly.

What bandwidth do I actually need for foundation-model training?

400Gb/s per GPU is the working target for the largest synchronous training runs, because the collective traffic is latency-bound rather than bandwidth-bound. Below that, the all-reduce time starts dominating the training step. The exact requirement depends on the model size, the sequence length and how much the framework overlaps communication with compute.