--- title: I ditched Ollama for Docker, and my local LLM setup... description: A developer shares their experience switching from Ollama to Docker Model Runner for running local LLMs. The main motivation is convenience — Docker is already... canonical: https://daily.dev/posts/i-ditched-ollama-for-docker-and-my-local-llm-setup-finally-stopped-being-a-hassle-8b7ururzn twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: I ditched Ollama for Docker, and my local LLM setup finally stopped being a hassle | daily.dev og:description: A developer shares their experience switching from Ollama to Docker Model Runner for running local LLMs. The main motivation is convenience — Docker is already... og:url: https://daily.dev/posts/i-ditched-ollama-for-docker-and-my-local-llm-setup-finally-stopped-being-a-hassle-8b7ururzn og:image: https://api.daily.dev/og/posts/8B7uruRZn.png og:image:alt: I ditched Ollama for Docker, and my local LLM setup finally stopped being a hassle og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # I ditched Ollama for Docker, and my local LLM setup finally stopped being a hassle **[XDA Developers](https://daily.dev/sources/xda-developers)** · 4 min read · 1 upvotes · 0 comments ## Summary A developer shares their experience switching from Ollama to Docker Model Runner for running local LLMs. The main motivation is convenience — Docker is already used for other services, and Model Runner integrates naturally via Docker Compose. Migration was straightforward: enable Model Runner in Docker Desktop, pull models again, and update the API base URL from port 11434 to 12434. Docker Model Runner supports OpenAI and Ollama-compatible APIs, uses llama.cpp with Metal acceleration on Apple Silicon, and also supports vLLM for NVIDIA GPU workloads. Models can be versioned and pushed to Docker Hub or OCI registries, making team sharing easier. The author notes Ollama remains a solid choice for those not already using Docker. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### What port does Docker Model Runner use for its API compared to Ollama? Docker Model Runner serves its API on port 12434, while Ollama uses port 11434. Since Model Runner supports OpenAI and Ollama-compatible API formats, most applications only need their base URL endpoint updated to switch between the two, without other configuration changes. _Track infrastructure changes like this port difference before migrating your local LLM stack, curated on daily.dev._ ### Does Docker Model Runner support running models on NVIDIA GPUs with vLLM? Yes, Docker Model Runner supports vLLM for models stored in the Safetensors format, which focuses on serving multiple requests efficiently through batching for higher throughput. It requires an NVIDIA GPU and currently runs on Linux x86-64 or Windows via WSL2, making it unavailable on Mac hardware. _Developers weighing GPU-backed inference options can follow vLLM and Docker updates on daily.dev._ ### How do I migrate my local LLM setup from Ollama to Docker Model Runner? Migration involves enabling Model Runner in Docker Desktop settings, pulling models again using the docker model pull command since Docker stores models separately from Ollama's files, and updating the API endpoint in connected applications to Docker's port 12434. Existing applications generally require no other changes beyond the base URL. _Anyone planning this kind of local LLM migration can follow Docker and Ollama developments on daily.dev._ ## Similar posts on daily.dev - [Docker Model Runner does everything Ollama does, but I'm still not switching](https://daily.dev/posts/docker-model-runner-does-everything-ollama-does-but-i-m-still-not-switching-yw9thvdhg) · XDA Developers · 1 upvotes · 1 comments - [Docker turned my AI setup into something I can rebuild in four minutes](https://daily.dev/posts/docker-turned-my-ai-setup-into-something-i-can-rebuild-in-four-minutes-rcpqm6blv) · XDA Developers · 0 upvotes · 0 comments --- Tags: [#docker](https://daily.dev/tags/docker), [#local-ai](https://daily.dev/tags/local-ai), [#ollama](https://daily.dev/tags/ollama), [#docker-compose](https://daily.dev/tags/docker-compose), [#llama-cpp](https://daily.dev/tags/llama-cpp) [View this post on daily.dev](https://daily.dev/posts/i-ditched-ollama-for-docker-and-my-local-llm-setup-finally-stopped-being-a-hassle-8b7ururzn) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"I ditched Ollama for Docker, and my local LLM setup finally stopped being a hassle","url":"https://daily.dev/posts/i-ditched-ollama-for-docker-and-my-local-llm-setup-finally-stopped-being-a-hassle-8b7ururzn","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/i-ditched-ollama-for-docker-and-my-local-llm-setup-finally-stopped-being-a-hassle-8b7ururzn"},"datePublished":"2026-07-28T11:31:22.784Z","dateModified":"2026-09-14T06:02:25.935Z","description":"A developer shares their experience switching from Ollama to Docker Model Runner for running local LLMs. The main motivation is convenience — Docker is already...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/d0dca241d883bbb80d592b89a5de155f?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/d0dca241d883bbb80d592b89a5de155f?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"XDA Developers","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"XDA Developers","logo":"https://media.daily.dev/image/upload/s--kCg6yyAP--/f_auto,q_auto/v1774964407/logos/xda-developers?_a=BAMAMiWQ0","url":"https://daily.dev/sources/xda-developers"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/i-ditched-ollama-for-docker-and-my-local-llm-setup-finally-stopped-being-a-hassle-8b7ururzn","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"docker,local-ai,ollama,docker-compose,llama-cpp","timeRequired":"PT4M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"XDA Developers","item":"https://daily.dev/sources/xda-developers"},{"@type":"ListItem","position":3,"name":"I ditched Ollama for Docker, and my local LLM setup finally stopped being a hassle"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/i-ditched-ollama-for-docker-and-my-local-llm-setup-finally-stopped-being-a-hassle-8b7ururzn#faq","mainEntity":[{"@type":"Question","name":"What port does Docker Model Runner use for its API compared to Ollama?","acceptedAnswer":{"@type":"Answer","text":"Docker Model Runner serves its API on port 12434, while Ollama uses port 11434. Since Model Runner supports OpenAI and Ollama-compatible API formats, most applications only need their base URL endpoint updated to switch between the two, without other configuration changes. Track infrastructure changes like this port difference before migrating your local LLM stack, curated on daily.dev."}},{"@type":"Question","name":"Does Docker Model Runner support running models on NVIDIA GPUs with vLLM?","acceptedAnswer":{"@type":"Answer","text":"Yes, Docker Model Runner supports vLLM for models stored in the Safetensors format, which focuses on serving multiple requests efficiently through batching for higher throughput. It requires an NVIDIA GPU and currently runs on Linux x86-64 or Windows via WSL2, making it unavailable on Mac hardware. Developers weighing GPU-backed inference options can follow vLLM and Docker updates on daily.dev."}},{"@type":"Question","name":"How do I migrate my local LLM setup from Ollama to Docker Model Runner?","acceptedAnswer":{"@type":"Answer","text":"Migration involves enabling Model Runner in Docker Desktop settings, pulling models again using the docker model pull command since Docker stores models separately from Ollama's files, and updating the API endpoint in connected applications to Docker's port 12434. Existing applications generally require no other changes beyond the base URL. Anyone planning this kind of local LLM migration can follow Docker and Ollama developments on daily.dev."}}]} ```