A deep dive into optimizing AI cold starts on Google Cloud Run with GPUs. Covers the four phases of an AI cold start (infrastructure provisioning, image streaming, engine initialization, and model/VRAM loading), and provides actionable strategies for each. Key recommendations include using concurrent Cloud Storage downloads for model weights, 4-bit quantization, GGUF/Safetensors formats, Startup CPU Boost, Direct VPC Egress with Private Google Access, and proper concurrency tuning. Also covers scaling strategies like single-region always-on deployments, proactive wake-up calls using non-inference endpoints, and startup probe configuration. Includes production lessons from Elastic, which serves millions of daily requests across 17+ model variants on Cloud Run using eager execution, standalone LoRA checkpoints, and per-workload service isolation.
