--- title: "3 Agents. 3 LLMs. 1 Aging GPU: Engineering Parallel Inference on Bare Metal" url: https://daily.dev/posts/3-agents-3-llms-1-aging-gpu-engineering-parallel-inference-on-bare-metal-shca3jeyp source_url: https://towardsdatascience.com/3-agents-3-llms-1-aging-gpu-engineering-parallel-inference-on-bare-metal type: article source: "Towards Data Science" published: 2026-06-25T15:39:02.600Z updated: 2026-06-25T18:20:45.654Z tags: ["ai-agents", "cuda", "ai-inference", "llama-cpp"] reading_time: 22 upvotes: 1 comments: 0 language: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # 3 Agents. 3 LLMs. 1 Aging GPU: Engineering Parallel Inference on Bare Metal **[Towards Data Science](https://daily.dev/sources/tds)** · 22 min read · 1 upvotes · 0 comments ## Summary Running three different LLMs simultaneously on a single 8GB GPU fails because llama.cpp pre-allocates the full KV cache upfront, causing OOM errors for the second and third processes. The solution is a C++ daemon called lmxd that implements Connection Admission Control (borrowed from 5G/telecom) as a VRAM ledger: it tracks allocated bytes, enforces a 90% cap, and refuses new agent registrations before any GPU allocation is attempted. The daemon also handles KV-cache swapping to host RAM between agent switches, enabling multiple agents to share one GPU context slot. Additionally, a layer streaming technique using two CUDA streams overlaps compute and weight transfer, achieving ~22-32% wall-clock savings on a GTX 1080. The repo ships the admission control daemon and the streaming primitive as separate, composable components. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Similar posts on daily.dev - [I built a free local LLM workflow with my 10-year-old-GPU, and it's reliable enough to replace the cloud](https://daily.dev/posts/i-built-a-free-local-llm-workflow-with-my-10-year-old-gpu-and-it-s-reliable-enough-to-replace-the-c-vlzohikq1) · XDA Developers · 1 upvotes · 0 comments - [Multiple Local LLMs 2026](https://daily.dev/posts/multiple-local-llms-2026-sm70tskwz) · SitePoint · 0 upvotes · 0 comments --- Tags: [#ai-agents](https://daily.dev/tags/ai-agents), [#cuda](https://daily.dev/tags/cuda), [#ai-inference](https://daily.dev/tags/ai-inference), [#llama-cpp](https://daily.dev/tags/llama-cpp) [View this post on daily.dev](https://daily.dev/posts/3-agents-3-llms-1-aging-gpu-engineering-parallel-inference-on-bare-metal-shca3jeyp)