Deploy vLLM on Kubernetes with NFS shared storage to eliminate redundant model downloads. Instead of each pod downloading multi-gigabyte LLM models from HuggingFace at startup, download once to a managed NFS share and let all pods load directly from there. This approach reduces startup time, removes external runtime dependencies, and enables instant scaling across GPU nodes. The guide walks through setting up DigitalOcean Kubernetes with H100 GPUs, configuring NFS persistent volumes, running a one-time model download job, and deploying vLLM with shared model access.
Table of contents
Key TakeawaysThe Problem: Why “Download Every Time” HurtsThe Solution: Download Once, Infer EverywhereArchitecture OverviewPrerequisitesStep 1: Set Up Your InfrastructureStep 2: Create Your Managed NFS ShareStep 3: Connect Kubernetes to NFSStep 4: Download the Model (Once!)Step 5: Deploy vLLMStep 6: Test Your Inference EndpointThe Scaling Story: What Happens When You Add More GPUs?CleanupFrequently Asked QuestionsConclusionNext Steps634 Impressions