A home lab enthusiast built a fully local, cloud-free smart home setup by running quantized LLMs (Gemma-4-26B and GPT-OSS-20B) on a Proxmox LXC with a decade-old gaming GPU via llama.cpp. Home Assistant is connected to the LLMs through the Home Agent HACS integration, with Whisper for speech-to-text and Piper for text-to-speech. An old Android tablet acts as a voice satellite for wake word detection. The setup achieves ~15-second response times and extends to other self-hosted services like Nextcloud and TrueNAS via MCP servers, eliminating reliance on any cloud AI service.
Table of contents
I use a Proxmox LXC to host my llama.cpp LLMsHome Agent lets me pair my local LLMs with Home AssistantSTT and TTS models provide the voice control functionality1.9K Impressions