llama.cpp vs. vLLM: Choosing the right local LLM inference engine

A practical comparison of llama.cpp and vLLM for running open-weight LLMs locally. llama.cpp excels on consumer hardware through quantization (GGUF format) and CPU-first design, powering tools like Ollama and LM Studio. vLLM targets production deployments with GPU accelerators, using PagedAttention for KV cache management and continuous batching for high concurrency. Benchmarks on Llama 3.1 8B with an NVIDIA H200 show both engines perform similarly at single-user concurrency, but vLLM scales significantly better as concurrent users increase. The recommended path: use llama.cpp for local prototyping and offline use cases, switch to vLLM when serving multiple users in production with data center GPUs.