--- title: Why AI Infrastructure must evolve for Agent Experience —... description: Modal CTO Akshat Bubna discusses how AI infrastructure must evolve from developer experience (DX) to agent experience (AX), following Modal's $355M Series C.... canonical: https://daily.dev/posts/why-ai-infrastructure-must-evolve-for-agent-experience-akshat-bubna-modal-cto-4emmjgd5z twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Why AI Infrastructure must evolve for Agent Experience — Akshat Bubna, Modal CTO | daily.dev og:description: Modal CTO Akshat Bubna discusses how AI infrastructure must evolve from developer experience (DX) to agent experience (AX), following Modal's $355M Series C.... og:url: https://daily.dev/posts/why-ai-infrastructure-must-evolve-for-agent-experience-akshat-bubna-modal-cto-4emmjgd5z og:image: https://api.daily.dev/og/posts/4EMmjGD5Z.png og:image:alt: Why AI Infrastructure must evolve for Agent Experience — Akshat Bubna, Modal CTO og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Why AI Infrastructure must evolve for Agent Experience — Akshat Bubna, Modal CTO **[Latent Space](https://daily.dev/sources/latentspace)** · 61 min read · 0 upvotes · 0 comments ## Summary Modal CTO Akshat Bubna discusses how AI infrastructure must evolve from developer experience (DX) to agent experience (AX), following Modal's $355M Series C. The conversation covers why Kubernetes was never designed for bursty AI workloads, Modal's shift to elastic inference for custom models, GPU snapshotting for faster cold starts, the DeFlash speculative decoding system, Auto Endpoints for optimized inference deployment, sandboxes for RL rollouts requiring up to 100,000 concurrent environments, networked containers with private IPv6 and RDMA for distributed training, and Modal's supercloud strategy spanning 17 cloud providers. Key insight: agents need tight feedback loops, fast iteration, and observability-first infrastructure rather than human-readable YAML configs. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### What is DeFlash and how does it improve LLM inference speed? DeFlash is an open-sourced block-based speculative decoding technique developed by Modal that predicts a block of tokens at once rather than one token at a time. Unlike kernel-level optimizations that yield only a few percentage points of improvement, increasing the accepted token length through better speculation multiplies inference speed, often by two to four times, with no drop in output quality since the larger model always verifies the draft tokens. _Engineers optimizing inference throughput can follow developments like this on daily.dev alongside other LLM serving techniques._ ### How many sandboxes might a reinforcement learning rollout require on Modal? Reinforcement learning rollouts can require up to 100,000 sandboxes running simultaneously, since RL workloads are described as insanely bursty compared to typical agent or inference traffic. Modal built its sandbox and elastic scaling infrastructure specifically to handle this kind of extreme, short-lived compute demand rather than steady-state web server traffic patterns. _Teams scaling RL infrastructure can track how compute providers handle bursty sandbox demand via daily.dev._ ### How many cloud providers does Modal use for its GPU capacity pool? Modal runs its capacity pool across 17 different cloud providers rather than owning its own data centers, relying on neo-clouds and metal providers worldwide. The company built its own reliability layer on top so that failures at any single provider, such as a GPU falling off the bus, do not affect user workloads, letting it tap far more capacity than a single-cloud user could access. _Developers comparing multi-cloud GPU strategies can follow this kind of infrastructure reporting on daily.dev._ ## Similar posts on daily.dev - [How Doppel eliminated ML infrastructure tax with Modal](https://daily.dev/posts/how-doppel-eliminated-ml-infrastructure-tax-with-modal-iak9tuoac) · Modal · 0 upvotes · 0 comments - [Cloud Native Is Becoming AI Native](https://daily.dev/posts/cloud-native-is-becoming-ai-native-g9b6ow133) · Cloud Native Now · 0 upvotes · 0 comments - [Multi-Modal AI Agents In Production 2026: Vision, Audio, Glue](https://daily.dev/posts/multi-modal-ai-agents-in-production-2026-vision-audio-glue-xmtiipzk9) · Alex CloudStar · 1 upvotes · 0 comments - [The infrastructure lock-in costing AI companies hundreds of millions](https://daily.dev/posts/the-infrastructure-lock-in-costing-ai-companies-hundreds-of-millions-wuyswiobz) · The New Stack · 0 upvotes · 0 comments - [Choosing the Right AI Abstraction Layer for modern AI systems](https://daily.dev/posts/choosing-the-right-ai-abstraction-layer-for-modern-ai-systems-vkvtqjitn) · DigitalOcean Community · 0 upvotes · 0 comments --- Tags: [#ai-agents](https://daily.dev/tags/ai-agents), [#serverless](https://daily.dev/tags/serverless), [#gpu](https://daily.dev/tags/gpu), [#ai-infrastructure](https://daily.dev/tags/ai-infrastructure) [View this post on daily.dev](https://daily.dev/posts/why-ai-infrastructure-must-evolve-for-agent-experience-akshat-bubna-modal-cto-4emmjgd5z) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Why AI Infrastructure must evolve for Agent Experience — Akshat Bubna, Modal CTO","url":"https://daily.dev/posts/why-ai-infrastructure-must-evolve-for-agent-experience-akshat-bubna-modal-cto-4emmjgd5z","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/why-ai-infrastructure-must-evolve-for-agent-experience-akshat-bubna-modal-cto-4emmjgd5z"},"datePublished":"2026-07-08T23:00:48.718Z","dateModified":"2026-09-14T08:39:26.600Z","description":"Modal CTO Akshat Bubna discusses how AI infrastructure must evolve from developer experience (DX) to agent experience (AX), following Modal's $355M Series C....","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/7f0119352fbcee34f6fe12316e71f521?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/7f0119352fbcee34f6fe12316e71f521?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Latent Space","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Latent Space","logo":"https://media.daily.dev/image/upload/s--DnJ9laFj--/f_auto,q_auto/v1780213196/logos/latentspace?_a=BAMAMiWQ0","url":"https://daily.dev/sources/latentspace"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/why-ai-infrastructure-must-evolve-for-agent-experience-akshat-bubna-modal-cto-4emmjgd5z","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"ai-agents,serverless,gpu,ai-infrastructure","timeRequired":"PT61M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Latent Space","item":"https://daily.dev/sources/latentspace"},{"@type":"ListItem","position":3,"name":"Why AI Infrastructure must evolve for Agent Experience — Akshat Bubna, Modal CTO"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/why-ai-infrastructure-must-evolve-for-agent-experience-akshat-bubna-modal-cto-4emmjgd5z#faq","mainEntity":[{"@type":"Question","name":"What is DeFlash and how does it improve LLM inference speed?","acceptedAnswer":{"@type":"Answer","text":"DeFlash is an open-sourced block-based speculative decoding technique developed by Modal that predicts a block of tokens at once rather than one token at a time. Unlike kernel-level optimizations that yield only a few percentage points of improvement, increasing the accepted token length through better speculation multiplies inference speed, often by two to four times, with no drop in output quality since the larger model always verifies the draft tokens. Engineers optimizing inference throughput can follow developments like this on daily.dev alongside other LLM serving techniques."}},{"@type":"Question","name":"How many sandboxes might a reinforcement learning rollout require on Modal?","acceptedAnswer":{"@type":"Answer","text":"Reinforcement learning rollouts can require up to 100,000 sandboxes running simultaneously, since RL workloads are described as insanely bursty compared to typical agent or inference traffic. Modal built its sandbox and elastic scaling infrastructure specifically to handle this kind of extreme, short-lived compute demand rather than steady-state web server traffic patterns. Teams scaling RL infrastructure can track how compute providers handle bursty sandbox demand via daily.dev."}},{"@type":"Question","name":"How many cloud providers does Modal use for its GPU capacity pool?","acceptedAnswer":{"@type":"Answer","text":"Modal runs its capacity pool across 17 different cloud providers rather than owning its own data centers, relying on neo-clouds and metal providers worldwide. The company built its own reliability layer on top so that failures at any single provider, such as a GPU falling off the bus, do not affect user workloads, letting it tap far more capacity than a single-cloud user could access. Developers comparing multi-cloud GPU strategies can follow this kind of infrastructure reporting on daily.dev."}}]} ```