--- title: Docker Model Runner + vLLM: High-Throughput Inference description: Docker Model Runner now integrates vLLM inference engine and safetensors models, enabling high-throughput AI inference on NVIDIA GPUs. The integration... canonical: https://daily.dev/posts/docker-model-runner-vllm-high-throughput-inference-0lq7wedak twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Docker Model Runner + vLLM: High-Throughput Inference | daily.dev og:description: Docker Model Runner now integrates vLLM inference engine and safetensors models, enabling high-throughput AI inference on NVIDIA GPUs. The integration... og:url: https://daily.dev/posts/docker-model-runner-vllm-high-throughput-inference-0lq7wedak og:image: https://api.daily.dev/og/posts/0LQ7WedAK.png og:image:alt: Docker Model Runner + vLLM: High-Throughput Inference og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Docker Model Runner + vLLM: High-Throughput Inference **[Docker](https://daily.dev/sources/docker)** · 6 min read · 0 upvotes · 0 comments ## Summary Docker Model Runner now integrates vLLM inference engine and safetensors models, enabling high-throughput AI inference on NVIDIA GPUs. The integration automatically routes requests between llama.cpp and vLLM based on model format (GGUF vs safetensors), allowing developers to prototype locally and scale to production using the same Docker commands. Currently available for x86_64 with NVIDIA GPUs, with WSL2/Docker Desktop and DGX compatibility planned. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Similar posts on daily.dev - [Docker Model Runner Adds vLLM Support on Windows](https://daily.dev/posts/docker-model-runner-adds-vllm-support-on-windows-nlfijbyki) · Docker · 0 upvotes · 0 comments - [Docker Model Runner Adds vLLM Support on macOS](https://daily.dev/posts/docker-model-runner-adds-vllm-support-on-macos-1lss5kmxo) · Docker · 1 upvotes · 0 comments - [Docker Model Runner does everything Ollama does, but I'm still not switching](https://daily.dev/posts/docker-model-runner-does-everything-ollama-does-but-i-m-still-not-switching-yw9thvdhg) · XDA Developers · 1 upvotes · 1 comments - [Why vLLM is the best choice for AI inference today](https://daily.dev/posts/why-vllm-is-the-best-choice-for-ai-inference-today-7zxdzzpx7) · Red Hat Developer · 0 upvotes · 0 comments --- Tags: [#docker](https://daily.dev/tags/docker), [#llm](https://daily.dev/tags/llm), [#machine-learning](https://daily.dev/tags/machine-learning), [#nvidia](https://daily.dev/tags/nvidia), [#vllm](https://daily.dev/tags/vllm) [View this post on daily.dev](https://daily.dev/posts/docker-model-runner-vllm-high-throughput-inference-0lq7wedak) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Docker Model Runner + vLLM: High-Throughput Inference","url":"https://daily.dev/posts/docker-model-runner-vllm-high-throughput-inference-0lq7wedak","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/docker-model-runner-vllm-high-throughput-inference-0lq7wedak"},"datePublished":"2025-11-20T13:48:17.862Z","dateModified":"2026-10-14T02:29:37.701Z","description":"Docker Model Runner now integrates vLLM inference engine and safetensors models, enabling high-throughput AI inference on NVIDIA GPUs. The integration...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/7be8a0db8ab91f7126d8d6f3da0e7f7f?_a=AQAEulh","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/7be8a0db8ab91f7126d8d6f3da0e7f7f?_a=AQAEulh","isAccessibleForFree":true,"articleSection":"Docker","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Docker","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/8a1022bebfc04a7d824c309bd3686787","url":"https://daily.dev/sources/docker"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/docker-model-runner-vllm-high-throughput-inference-0lq7wedak","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"docker,llm,machine-learning,nvidia,vllm","timeRequired":"PT6M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Docker","item":"https://daily.dev/sources/docker"},{"@type":"ListItem","position":3,"name":"Docker Model Runner + vLLM: High-Throughput Inference"}]} ```