Securely deploy LLMs inside VPCs with Hugging Face and Amazon SageMaker

A step-by-step guide for deploying open-source LLMs to Amazon SageMaker inside a VPC without internet access, using the Hugging Face LLM Inference DLC and loading model weights from Amazon S3. The tutorial covers packaging model weights as a model.tar.gz archive with safetensors format, uploading to S3, retrieving the Hugging Face LLM container image URI, deploying the starchat-beta model on a g5.12xlarge GPU instance with TGI auto-sharding, and running inference — enabling organizations with strict security requirements to serve LLMs in an air-gapped, internet-free environment.