Most people use Ollama or llama.cpp for local LLMs, but these are the tools I switch to when it gets serious
This title could be clearer and more informative.Try out Clickbait Shieldfor free (5 uses left this month).
Ollama and llama.cpp are great starting points for local LLMs, but once models become part of a real workflow, more specialized runtimes become necessary. vLLM offers high-throughput inference with PagedAttention, continuous batching, and OpenAI-compatible APIs for serving models as backend infrastructure. SGLang targets structured generation and agent-style workloads with prefix caching and constrained outputs. For Apple Silicon, vMLX provides a Mac-native serving stack built on Apple's MLX framework. MLC-LLM enables deployment to browsers (WebGPU), iOS, and Android, while ExLlamaV3 squeezes performance from consumer GPUs with its EXL3 quantization format. Other notable tools include llama-swap for routing, TensorRT-LLM for Nvidia, LMDeploy, Lemonade for AMD, KTransformers for heterogeneous CPU/GPU inference, and LocalAI.
