When you ask an AI model a question, your request travels to a data center, is processed, and returns in fractions of a second. As AI shifts from training models to serving billions of real-time interactions, where that infrastructure sits—and how workloads are routed across it—is becoming the defining infrastructure challenge of the decade.

A crossover occurred in 2025. Training and inference traded places on the workload curve.

A training run is a one-time event. An inference workload is a permanent business with users on the other end. This fundamentally changes what gets built.

The practical implication of this shift is also significant—within two years, most of the electricity and capital deployed on AI will go towards ‘answering’ questions, not ‘learning’ answers. Deloitte predicts that inference will account for two-thirds of all AI compute by the end of 2026 alone.

Inference demands distribution

AI training wanted concentration. One data center campus, one connectivity fabric, one schedule, and the newest silicon chips at any price.

AI inference wants something almost opposite. It requires proximity to the user, runs on the most cost-efficient chip per useful token, and must meet a 100-millisecond round-trip budget, all while ensuring strict compliance with local regulations wherever the data resides.

Where training looked like a port, inference looks like a freight network. And when the inference workload map is viewed through that lens, the analogy holds true.

Real-time work (voice assistants, augmented reality, robotics) lives in the equivalent of a downtown depot. High rent, high density, and the user is across the street. For example, when you ask your voice assistant what the weather is like in your city, that query must be processed within milliseconds. The round-trip latency to a remote campus may make real-time work close to impossible.
Interactive work (customer service chatbots, AI agents, productivity assistants) sits further out in the equivalent of a suburban hub. A 100-millisecond responsewindow is acceptable; operating costs are lower, and the workload is predictable enough to plan around.
Batch work (overnight report generation, model fine-tuning, large-scale data processing jobs) runs when power is cheapest, in a manner similar to how warehouses sit beside highway interchanges in rural counties. There is no real-time user waiting on the other end; consequently, time is a trade-off in terms of cost.

Each inference tier has a different performance requirement, cost profile, and set of customers. They serve different parts of the same delivery economy.

A structural shift in the investment model

Distributed, latency-sensitive infrastructure requires a different investment model. Instead of anchoring capacity in a single mega-campus, operators must support workloads that move between regions, tenants, and sovereign zones as demand shifts.

Inference changes both infrastructure finance and silicon economics. Capacity must be deployed as a flexible network rather than a fixed campus, while compute is managed as a portfolio (or fleet) rather than a single hardware generation. This is a structurally different capital conversation. Enterprises that haven't yet realigned their investment strategy risk building the wrong asset for the next phase.

The newest chip accelerators handle the most demanding workloads, while depreciated hardware can continue to generate value at scale from high-volume workloads. Competitive advantage comes not from owning the latest and highest-spec silicon, but from routing each workload to the most efficient resource.

The sovereign dimension defining design

Sovereignty is becoming a defining constraint on AI infrastructure. Data residency laws, AI governance frameworks, and national security requirements increasingly require that workloads remain within specific jurisdictions.

This is making compute far less fungible across borders than in the training era. Just as an international shipment cannot skip customs, a data query from a Germany-based user may not always legally be processed on a server in Virginia, USA.

As sovereign AI initiatives accelerate across Europe, the Middle East, and Asia, infrastructure is being built inside a regulatory map, not on top of one. Enterprises that treat sovereignty as a compliance exercise, rather than a core design principle, risk being excluded from a growing share of global AI workloads.

Building the foundation for AI in the next decade

The enterprises that thrive in this shift to inference will be those already structured as a network. Multiple metros, multiple regions, multiple sovereign jurisdictions—all connected by a software layer that can move data capacity in milliseconds.

The dominant unit of AI infrastructure for the rest of the decade is not the gigawatt campus. It is the metro hub, the regional point of presence, and the sovereign node. A network interconnected by software-defined routing that treats compute as a dynamic, dispatchable resource rather than a fixed address.

The best freight hubs don't just route packages efficiently; they bring all the right partners under one roof, providing flexibility and ensuring seamless handoffs. This is what interconnection does for AI workloads.

Digital Realty has been building this for two decades. Our global data center platform, PlatformDIGITAL®, is designed to deliver scalable colocation and secure interconnection that help you deploy faster, reduce risk, and build your AI infrastructure into a competitive edge.

The first era of AI infrastructure answered one question: where do we put the biggest model? The next era answers a different one: how does a query reach the right chip, in the right place, at the right time, for the lowest cost?

Bentaus: Building a scalable foundation for AI Innovation with Digital Realty