--- title: Can I run models myself? — The Agentic AI Hub | daily.dev description: Open weights, the local stack, serving, and the hardware math for what fits on your machines. canonical: https://daily.dev/agentic-ai-hub/paths/run-models-yourself/ og:type: website og:url: https://daily.dev/agentic-ai-hub/paths/run-models-yourself/ og:title: Can I run models myself? — The Agentic AI Hub | daily.dev og:description: Open weights, the local stack, serving, and the hardware math for what fits on your machines. og:image: https://daily.dev/assets/og/agentic-ai-hub.png?v=40e6ff69 og:site_name: daily.dev og:locale: en_US twitter:card: summary_large_image twitter:site: @dailydotdev twitter:creator: @dailydotdev twitter:title: Can I run models myself? — The Agentic AI Hub | daily.dev twitter:description: Open weights, the local stack, serving, and the hardware math for what fits on your machines. twitter:image: https://daily.dev/assets/og/agentic-ai-hub.png?v=40e6ff69 --- [Skip to main content ](#main-content) # Can I run models myself? Open weights, the local stack, serving, and the hardware math for what fits on your machines. [Start reading → ](/agentic-ai-hub/open-weight-models/?path=run-models-yourself&i=0) [Check yourself first ↓ ](#quiz) 1. [ 1 Open-Weight Models The models you can download, self-host, and fine-tune. As of July 2026\. Two things to know: (1) "open weight" ≠ open source. › ](/agentic-ai-hub/open-weight-models/?path=run-models-yourself&i=0) 2. [ 2 Local & Self-Hosting Stack Running models on your own machine or servers. › ](/agentic-ai-hub/local-self-hosting-stack/?path=run-models-yourself&i=1) 3. [ 3 Inference, Serving & Optimization TTFT (Time To First Token). Latency until the first token appears. Dominated by the prefill (prompt-processing) phase and prompt length. › ](/agentic-ai-hub/inference-serving-optimization/?path=run-models-yourself&i=2) 4. [ 4 AI Hardware & Accelerators The 2026 picture is that NVIDIA still dominates training and general inference. › ](/agentic-ai-hub/ai-hardware-accelerators/?path=run-models-yourself&i=3) ## Check yourself. One question per step. Take it cold to find where to start, or after reading to see what stuck. Nobody's grading you. Best so far: Question 1 of 4 1. Step 1 A MoE model computes only a fraction of its parameters per token. What does that save you at inference? A Compute and GPU memory both B GPU memory, but not compute C Compute, but not GPU memory D Neither, since MoE only speeds up training MoE models load all experts into memory but only run the active set, so they're fast but still memory-hungry. VRAM needs follow total parameters, roughly params × bytes-per-param. [Covered in step 1 · Open-Weight Models → ](/agentic-ai-hub/open-weight-models/?path=run-models-yourself&i=0) Next question → 2. Step 2 When does self-hosting models lose on cost? A At sustained high volume, where API pricing wins B At low or spiky volume, since an idle GPU still costs money C Whenever you run quantized instead of full-precision models D When you need offline operation or strict data control Self-hosting wins at high volume thanks to zero marginal cost, but loses cost-efficiency when traffic is low or spiky, because you pay for the GPU whether or not it's busy. [Covered in step 2 · Local & Self-Hosting Stack → ](/agentic-ai-hub/local-self-hosting-stack/?path=run-models-yourself&i=1) Next question → 3. Step 3 You're serving an open model on your own GPUs. What's the single biggest throughput win in modern serving? A Speculative decoding with a small draft model B Quantizing weights down to INT4 C Continuous batching that adds requests every step D Compiling the model with TensorRT-LLM Continuous (in-flight) batching adds and removes requests every step as sequences finish, keeping the GPU saturated. The chapter calls it the single biggest throughput win in modern serving. [Covered in step 3 · Inference, Serving & Optimization → ](/agentic-ai-hub/inference-serving-optimization/?path=run-models-yourself&i=2) Next question → 4. Step 4 How should you compare GPUs when buying or renting for inference? A Dollars per token, since throughput matters more than sticker price B The lowest hourly rental price you can find C The most VRAM you can get per dollar D The newest architecture generation on the market Cost per token is set by throughput, not sticker price: a pricier GPU that serves 3× the tokens comes out cheaper per token, which is why quantization, batching, and the serving engine move the economics most. [Covered in step 4 · AI Hardware & Accelerators → ](/agentic-ai-hub/ai-hardware-accelerators/?path=run-models-yourself&i=3) See results → of 4 * [ 1 Open-Weight Models → ](/agentic-ai-hub/open-weight-models/?path=run-models-yourself&i=0) * [ 2 Local & Self-Hosting Stack → ](/agentic-ai-hub/local-self-hosting-stack/?path=run-models-yourself&i=1) * [ 3 Inference, Serving & Optimization → ](/agentic-ai-hub/inference-serving-optimization/?path=run-models-yourself&i=2) * [ 4 AI Hardware & Accelerators → ](/agentic-ai-hub/ai-hardware-accelerators/?path=run-models-yourself&i=3) Try again [Where to next → ](#path-next-title) ## Where to next. More guided paths through the handbook. Pick the next question worth answering. [ How do I build my own agent? → ](/agentic-ai-hub/paths/build-your-own-agent/)[ Can I trust what agents produce? → ](/agentic-ai-hub/paths/trust-agent-output/)[ How do I ground AI in my company's data? → ](/agentic-ai-hub/paths/ground-in-your-data/) [All reading paths → ](/agentic-ai-hub/#paths-start-here) [Browse every chapter ](/agentic-ai-hub/#pages) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/og-image.png?v=a830cdf1","width":1200,"height":630},"sameAs":["https://twitter.com/dailydotdev","https://www.linkedin.com/company/dailydotdev","https://github.com/dailydotdev","https://www.instagram.com/dailydotdev"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","description":"Free, personalized developer news aggregator. Stay on top of software development news, AI coding tools, and web dev - curated daily from trusted sources.","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}},{"@type":"WebPage","@id":"https://daily.dev/agentic-ai-hub/paths/run-models-yourself/","url":"https://daily.dev/agentic-ai-hub/paths/run-models-yourself/","name":"Can I run models myself? | The Agentic AI Hub","description":"Open weights, the local stack, serving, and the hardware math for what fits on your machines.","inLanguage":"en-US","isPartOf":{"@id":"https://daily.dev/#website"}},{"@type":"ItemList","numberOfItems":4,"itemListElement":[{"@type":"ListItem","position":1,"url":"https://daily.dev/agentic-ai-hub/open-weight-models/","name":"Open-Weight Models"},{"@type":"ListItem","position":2,"url":"https://daily.dev/agentic-ai-hub/local-self-hosting-stack/","name":"Local & Self-Hosting Stack"},{"@type":"ListItem","position":3,"url":"https://daily.dev/agentic-ai-hub/inference-serving-optimization/","name":"Inference, Serving & Optimization"},{"@type":"ListItem","position":4,"url":"https://daily.dev/agentic-ai-hub/ai-hardware-accelerators/","name":"AI Hardware & Accelerators"}],"@id":"https://daily.dev/agentic-ai-hub/paths/run-models-yourself/#steps"},{"@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev/"},{"@type":"ListItem","position":2,"name":"The Agentic AI Hub","item":"https://daily.dev/agentic-ai-hub/"},{"@type":"ListItem","position":3,"name":"Build with AI","item":"https://daily.dev/agentic-ai-hub/#paths-build"},{"@type":"ListItem","position":4,"name":"Can I run models myself?","item":"https://daily.dev/agentic-ai-hub/paths/run-models-yourself/"}]}]} ```