Bringing serverless functions to the speed of wire

Modal has redesigned its I/O plane for serverless Function Calls, reducing end-to-end latency by ~80ms at p50 and adding geographic distribution across 4+ regions. The new system lets users specify a routing_region to minimize network overhead between clients and containers. Key engineering changes include a full rewrite of the server layer from Python to Go (for better concurrency under gRPC load), moving non-critical work to async background tasks, minimizing hot-path Redis interactions, client-side metadata caching, and JWT-based auth refresh. The post also covers Redis 7.1 vs 7.2/Valkey tradeoffs for input queue performance, and offers user-facing tips: choose a nearby routing_region, keep payloads under 2MiB, and batch small inputs to reduce network hops.