AI Scaling Networks

Energy Limits Force AI Clusters to Operate Across Multiple Data Centers

Systems with over 100,000 accelerators require regional interconnection networks to bypass thermal and electrical bottlenecks in single facilities

• Updated 4 hours ago
Infraestrutura de supercomputação distribuída operando interconexões de alta largura de banda entre diferentes instalações
Infraestrutura de supercomputação distribuída operando interconexões de alta largura de banda entre diferentes instalações (Imagem conceitual editorial / NEWS by Box4Dev [Reinterpretação Visual])

Cluster Scale

  • Modern artificial intelligence systems already bring together over 100,000 accelerators and millions of processing nodes
  • Thermal bottlenecks and power supply limits make it unfeasible to concentrate these workloads in a single building
  • 'Scale-across' architectures connect multiple regional data centers to sustain the advancement of large models

The rapid expansion of foundation models has imposed a severe physical barrier on global processing centers. Contemporary clusters dedicated to training artificial intelligence exceed the mark of 100,000 dedicated accelerators and potentially aggregate millions of CPU endpoints. This massive concentration of hardware hits the ceiling of power availability and cooling capacity offered by individual facilities.

With the impossibility of keeping hundreds of thousands of graphics and tensor processing units under the same roof, hyperscalers are beginning the transition to distributed architectures. The strategy requires unified computational systems to operate geographically dispersed across multiple interconnected data centers.

The division between scale-up and scale-out

Traditionally, high-performance computing is structured around two essential interconnection domains. The internal expansion domain (scale-up) directly connects processors and accelerators in integrated systems, utilizing proprietary protocols and dedicated buses such as NVLink, AMD Infinity Fabric, Intel UPI, or the open UALink consortium.

Meanwhile, the external expansion domain (scale-out) is responsible for tying together autonomous servers and rack assemblies into large-scale networks. At this level, established technologies prevail, such as Ethernet, InfiniBand, HPE Slingshot, and, more recently, the emerging Ultra Ethernet standard, each designed with specific bandwidth profiles and congestion resilience.

The evolution of silicon exposes, however, a structural mismatch in this equation. The computational throughput of accelerators advances at a noticeably faster rate than the capacity of telecommunications networks, diminishing the proportional gain generated by faster processors if the data transport infrastructure presents latency or flow bottlenecks.

The emergence of 'scale-across' networks

To enable coordinated operation between distinct buildings and geographic zones, vendors and cloud operators are turning their attention to the concept of regionalized interconnection, termed scale-across. Companies like Google, Cisco, and Marvell are directing specific developments toward this intermediate layer of interregional traffic.

This approach seeks to mitigate local thermal and electrical restrictions while maintaining controlled latencies and data consistency between distinct industrial hubs. Consequently, training workloads that previously depended exclusively on immediate physical proximity are now orchestrated in a partitioned manner across optimized optical routes.

Practical Impact

From an architectural perspective, the transition to scale-across networks redefines the design of buses and optical switches, shifting the pressure for efficiency from the accelerator silicon to low-latency transmission subsystems and advanced buffer management techniques.

On the operational and economic front, the model alleviates the overload on power utilities in traditional hyperscale hubs, allowing operators to distribute new investments to regions with greater electrical availability and renewable energy matrices, without segmenting the clusters into isolated software silos.

Primary Source