Mesh LLM pools GPU resources across multiple machines into a single OpenAI-compatible API endpoint, built on top of iroh — a peer-to-peer networking library. Each node boots an iroh endpoint (identified by public key) that handles NAT traversal and authenticated QUIC connections without a central server. Models can run locally, be routed to a peer that has them loaded, or be split across multiple machines in a pipeline (called 'Skippy') for models too large for any single GPU. The protocol uses QUIC ALPN negotiation with three distinct protocols for mesh communication, control plane, and latency-sensitive activation transport. The system exposes itself as localhost:9337/v1 to any standard OpenAI client, hiding all the distributed complexity. It ships with 40+ models ranging from sub-billion parameter models to 235B MoE giants.
