AWS, Microsoft, Google, and Anthropic have independently converged on the same architectural shift: replacing request-level load balancing with session-aware execution for AI agents. Each platform treats the agent session as the fundamental unit of compute, but they differ in their isolation primitives — AWS uses dedicated Firecracker microVMs, Azure Foundry uses per-session VM-isolated sandboxes with Entra identities, Google Agent Engine uses a hybrid of request-scaled reasoning loops plus isolated code execution sandboxes, and Anthropic Managed Agents decouple the harness from a swappable sandbox layer. The shift is driven by two requirements: preserving conversational state across turns and enforcing strong security boundaries for untrusted, model-generated code. A real-world incident at Asana illustrates the risk when user-to-session bindings break even when platform-level isolation holds. The article argues that the agent session is becoming the operational unit of enterprise AI infrastructure, turning the dispatch layer into a stateful, lifecycle-aware control plane.

10m read timeFrom thenewstack.io
Post cover image
Table of contents
The traditional cloud scaling modelWhy agents break the modelWhy can the agent state not be pooled?Why the trust boundary forces isolationThe session as a unit of computeFour approaches to the same shiftThe binding the application still ownsHow this differs from sticky sessions
439 Impressions