--- title: "Why I Switched from Ollama to llama.cpp — and What I Learned" url: https://daily.dev/posts/why-i-switched-from-ollama-to-llama-cpp-and-what-i-learned-o7xa5cpjq source_url: https://blog.devgenius.io/why-i-switched-from-ollama-to-llama-cpp-and-what-i-learned-0ec71bf2fc95 type: article source: "Dev Genius" published: 2026-08-21T14:28:58.812Z updated: 2026-08-21T15:19:34.744Z tags: ["gpu", "local-ai", "ollama", "qwen", "llama-cpp"] reading_time: 9 upvotes: 2 comments: 0 language: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Why I Switched from Ollama to llama.cpp — and What I Learned **[Dev Genius](https://daily.dev/sources/devgenius)** · 9 min read · 2 upvotes · 0 comments ## Summary A hands-on account of switching a local LLM setup from Ollama to llama.cpp on a dual-GPU AMD APU handheld (Ryzen 7840U with integrated Radeon 780M and discrete RX 7600M XT). Covers determining available VRAM via free/glxinfo, picking the right GPU device with --list-devices and visibility env vars, symlinking Ollama-downloaded GGUF blobs for use with llama-server, tuning -ngl layer offload and context size, wiring llama-server's OpenAI-compatible endpoint into Obsidian plugins, and reading server logs to diagnose CPU-offload bottlenecks. Ends with a two-week verdict: llama.cpp trades convenience for granular hardware control versus Ollama's simplicity. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How do I calculate the right -ngl value for llama-server on a GPU with limited VRAM? Calculate the total quantized model weight size, divide by the number of layers to get per-layer size, then multiply by the number of layers you want on GPU and add the KV cache size for your context length. For example, with Qwen2.5-14B Q4_K_M (~9.04GB total, 48 layers), offloading 32 layers to GPU uses about 6.0GB of weights plus 1.1GB KV cache at -c 8192, fitting under a 7079MB VRAM budget. _Track hardware-tuning tips like this alongside other llama.cpp workflows on daily.dev._ ### Why does llama.cpp pick the wrong GPU on a laptop with both an integrated and a discrete GPU? llama.cpp defaults to the first device the backend enumerates, and on Vulkan the integrated GPU is often listed first, leaving the discrete GPU idle even though it would run models several times faster. Use --list-devices and --device to check and override the choice, or set GGML_VK_VISIBLE_DEVICES (Vulkan), HIP_VISIBLE_DEVICES (ROCm), or CUDA_VISIBLE_DEVICES (CUDA) to force the correct GPU. _Developers juggling multi-GPU inference setups can follow tuning details like these on daily.dev._ ### Is it safe to run llama-server with CORS allowing all origins and no API key? It is acceptable only because llama-server binds to 127.0.0.1 by default, making it reachable solely from the local machine despite the warning about cross-origin risk. If the server is ever exposed with --host 0.0.0.0, an --api-key should be set first to avoid the security risk the CORS warning flags. _Keep local-LLM security defaults like this in view by following llama.cpp coverage on daily.dev._ ## Similar posts on daily.dev - [I Switched From Ollama And LM Studio To llama.cpp And Absolutely Loving It](https://daily.dev/posts/i-switched-from-ollama-and-lm-studio-to-llama-cpp-and-absolutely-loving-it-ftqjoxmri) · It's Foss · 6 upvotes · 0 comments - [I built a free local LLM workflow with my 10-year-old-GPU, and it's reliable enough to replace the cloud](https://daily.dev/posts/i-built-a-free-local-llm-workflow-with-my-10-year-old-gpu-and-it-s-reliable-enough-to-replace-the-c-vlzohikq1) · XDA Developers · 1 upvotes · 0 comments - [I finally ditched Ollama after using llama.cpp's WebUI, and I'm not going back anytime soon](https://daily.dev/posts/i-finally-ditched-ollama-after-using-llama-cpp-s-webui-and-i-m-not-going-back-anytime-soon-a2dcksws3) · XDA Developers · 1 upvotes · 0 comments --- Tags: [#gpu](https://daily.dev/tags/gpu), [#local-ai](https://daily.dev/tags/local-ai), [#ollama](https://daily.dev/tags/ollama), [#qwen](https://daily.dev/tags/qwen), [#llama-cpp](https://daily.dev/tags/llama-cpp) [View this post on daily.dev](https://daily.dev/posts/why-i-switched-from-ollama-to-llama-cpp-and-what-i-learned-o7xa5cpjq)