--- title: How to run an LLM locally | daily.dev description: Running an LLM locally means downloading an open-weight model and executing it on your own machine with Ollama, LM Studio, or llama.cpp. Ollama gives a... canonical: https://daily.dev/posts/how-to-run-an-llm-locally-zpiau666c twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: How to run an LLM locally | daily.dev og:description: Running an LLM locally means downloading an open-weight model and executing it on your own machine with Ollama, LM Studio, or llama.cpp. Ollama gives a... og:url: https://daily.dev/posts/how-to-run-an-llm-locally-zpiau666c og:image: https://api.daily.dev/og/posts/ZpIau666C.png og:image:alt: How to run an LLM locally og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # How to run an LLM locally **[Flavio Copes](https://daily.dev/sources/flaviocopes)** · 6 min read · 1 upvotes · 0 comments ## Summary Running an LLM locally means downloading an open-weight model and executing it on your own machine with Ollama, LM Studio, or llama.cpp. Ollama gives a terminal chat and an HTTP API at localhost:11434 with no API key. Weights need roughly parameters in billions times bits divided by 8 in GB, so an 8B model at 4-bit quantization is about 4 GB, or around 5.45 GB with 8k context. LM Studio serves the same kind of model at localhost:1234 unless the port is changed, with OpenAI-compatible routes under /v1, while llama.cpp is the underlying C++ engine both apps build on. The Ollama chat endpoint accepts a model name and messages, and the server does not retain conversation history between calls, so earlier messages must be resent each turn. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How much memory do I need to run an 8 billion parameter LLM locally at 4-bit quantization? An 8 billion parameter model at 4-bit quantization takes about 4 GB just for the weights. Once you add the generation cache and context, a worked example with 8k of context puts total memory needs around 5.45 GB. The formula is parameters in billions times bits divided by 8 to estimate weight size. _Sizing hardware for a local model is easier when you track real benchmarks on daily.dev before committing memory._ ### What port does Ollama's local API run on and how do I call it without an API key? Ollama's HTTP API runs on localhost:11434 with no API key required. You send a POST request to /api/chat with a model name and a messages array; setting stream to false returns one JSON object instead of a stream, and the reply text is found in message.content. _Developers wiring local models into apps follow local-first AI tooling changes on daily.dev._ ### Does Ollama remember conversation history between API calls? No, the Ollama server does not retain the previous HTTP call's context. Each new turn requires resending all earlier messages in the messages array to maintain conversation history, since the server treats every request independently. _Anyone building chat features on local LLMs tracks these API quirks on daily.dev before shipping._ ## Similar posts on daily.dev - [How to Run LLMs Locally: A Practical Guide for Developers](https://daily.dev/posts/how-to-run-llms-locally-a-practical-guide-for-developers-tijurupob) · BigData Boutique blog · 2 upvotes · 0 comments - [Run a Local AI Model with Ollama in 15 Minutes](https://daily.dev/posts/run-a-local-ai-model-with-ollama-in-15-minutes-5wx17jt3x) · Machine Learning Mastery · 1 upvotes · 0 comments - [Qwen3.5 - How to Run Locally Guide](https://daily.dev/posts/qwen3-5---how-to-run-locally-guide-t89pkcicv) · Hacker News · 2 upvotes · 0 comments --- Tags: [#self-hosting](https://daily.dev/tags/self-hosting), [#ai-inference](https://daily.dev/tags/ai-inference), [#ollama](https://daily.dev/tags/ollama), [#llama-cpp](https://daily.dev/tags/llama-cpp), [#lm-studio](https://daily.dev/tags/lm-studio) [View this post on daily.dev](https://daily.dev/posts/how-to-run-an-llm-locally-zpiau666c) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"How to run an LLM locally","url":"https://daily.dev/posts/how-to-run-an-llm-locally-zpiau666c","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/how-to-run-an-llm-locally-zpiau666c"},"datePublished":"2026-10-31T18:11:31.081Z","dateModified":"2026-10-31T18:11:59.231Z","description":"Running an LLM locally means downloading an open-weight model and executing it on your own machine with Ollama, LM Studio, or llama.cpp. Ollama gives a...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/a86ea77d58f1e2420f25aad9ad346c2e?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/a86ea77d58f1e2420f25aad9ad346c2e?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Flavio Copes","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Flavio Copes","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/1f2dbd275797411bbd37bfdaaf0951a9","url":"https://daily.dev/sources/flaviocopes"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/how-to-run-an-llm-locally-zpiau666c","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"self-hosting,ai-inference,ollama,llama-cpp,lm-studio","timeRequired":"PT6M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Flavio Copes","item":"https://daily.dev/sources/flaviocopes"},{"@type":"ListItem","position":3,"name":"How to run an LLM locally"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/how-to-run-an-llm-locally-zpiau666c#faq","mainEntity":[{"@type":"Question","name":"How much memory do I need to run an 8 billion parameter LLM locally at 4-bit quantization?","acceptedAnswer":{"@type":"Answer","text":"An 8 billion parameter model at 4-bit quantization takes about 4 GB just for the weights. Once you add the generation cache and context, a worked example with 8k of context puts total memory needs around 5.45 GB. The formula is parameters in billions times bits divided by 8 to estimate weight size. Sizing hardware for a local model is easier when you track real benchmarks on daily.dev before committing memory."}},{"@type":"Question","name":"What port does Ollama's local API run on and how do I call it without an API key?","acceptedAnswer":{"@type":"Answer","text":"Ollama's HTTP API runs on localhost:11434 with no API key required. You send a POST request to /api/chat with a model name and a messages array; setting stream to false returns one JSON object instead of a stream, and the reply text is found in message.content. Developers wiring local models into apps follow local-first AI tooling changes on daily.dev."}},{"@type":"Question","name":"Does Ollama remember conversation history between API calls?","acceptedAnswer":{"@type":"Answer","text":"No, the Ollama server does not retain the previous HTTP call's context. Each new turn requires resending all earlier messages in the messages array to maintain conversation history, since the server treats every request independently. Anyone building chat features on local LLMs tracks these API quirks on daily.dev before shipping."}}]} ```