--- title: What RAG Actually Costs to Run in Production: A Full... description: A detailed cost breakdown for running a retrieval-augmented generation (RAG) pipeline over a 100,000-document corpus on DigitalOcean shows that embedding and... canonical: https://daily.dev/posts/what-rag-actually-costs-to-run-in-production-a-full-cost-breakdown-26v3uoonx twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: What RAG Actually Costs to Run in Production: A Full Cost Breakdown | daily.dev og:description: A detailed cost breakdown for running a retrieval-augmented generation (RAG) pipeline over a 100,000-document corpus on DigitalOcean shows that embedding and... og:url: https://daily.dev/posts/what-rag-actually-costs-to-run-in-production-a-full-cost-breakdown-26v3uoonx og:image: https://api.daily.dev/og/posts/26v3uooNx.png og:image:alt: What RAG Actually Costs to Run in Production: A Full Cost Breakdown og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # What RAG Actually Costs to Run in Production: A Full Cost Breakdown **[DigitalOcean Community](https://daily.dev/sources/do_community)** · 20 min read · 0 upvotes · 0 comments ## Summary A detailed cost breakdown for running a retrieval-augmented generation (RAG) pipeline over a 100,000-document corpus on DigitalOcean shows that embedding and vector storage are nearly free — about $6.86 one-time and roughly $1/month respectively — while reranking and answer generation account for over 99% of ongoing model spend, scaling linearly with query traffic. At 1,000 queries/day the whole system costs about $98/month, rising to roughly $4,000/month at 100,000 queries/day. The three highest-leverage cost levers identified are the choice of generation model (a 4x price swing between gpt-oss-120b and Llama 3.3 70B), the number of chunks reranked per query, and the crossover point (~$3,200/month) where switching from serverless to dedicated GPU inference becomes cheaper. The piece also walks through building the full pipeline: chunking, embeddings via Qwen3 Embedding 0.6B, storage in Managed PostgreSQL with pgvector and HNSW indexing, LLM-based reranking with DeepSeek V4 Flash, and answer generation with gpt-oss-120b, including optional batch-inference chunk enrichment. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### What does it cost to run a RAG pipeline over 100,000 documents in production? Roughly $98 per month at 1,000 queries per day, rising to about $496 at 10,000 queries per day and $4,000 at 100,000 queries per day, using Qwen3 Embedding 0.6B, DeepSeek V4 Flash for reranking, and gpt-oss-120b for answer generation on DigitalOcean. Embedding the full 100,000-document corpus is a one-time $6.86, and vector storage adds about $1 per month; reranking and generation make up over 99% of ongoing model spend and scale with query volume. _daily.dev surfaces cost-breakdown deep dives like this for teams budgeting a RAG deployment._ ### Why is the reranking step the most expensive part of a RAG pipeline at high query volume? The reranker reads all 20 retrieved candidate chunks per query, about 10,400 input tokens, roughly four times more tokens than the answer-generation model sees over its top 5 chunks. At high traffic this token volume makes reranking the single largest line item, even though model price per token is relatively low; reducing candidates to 10, reranking only ambiguous queries, or using a cheaper dedicated reranker (like BGE Reranker v2 m3) can cut this cost significantly. _Track token-cost tradeoffs like this reranking bottleneck on daily.dev before scaling a RAG service._ ### When does dedicated GPU inference become cheaper than serverless inference for a RAG pipeline? A single NVIDIA H100 dedicated instance costs a flat $4.41 per hour, about $3,220 per month, regardless of query volume, while serverless inference pricing scales linearly with traffic. Once monthly serverless spend on generation and reranking approaches that flat rate, roughly around the 100,000-queries-per-day tier in this cost model, dedicated capacity for an open-weight model starts winning on both price and latency consistency. _Developers comparing serverless versus dedicated inference costs can find breakdowns like this on daily.dev._ ## Similar posts on daily.dev - [Designing Production-Scale RAG Systems](https://daily.dev/posts/designing-production-scale-rag-systems-jlubzbbji) · Medium · 1 upvotes · 0 comments - [What Does a RAG System Actually Cost in Nigeria? \(Naira Pricing Breakdown, 2026\)](https://daily.dev/posts/what-does-a-rag-system-actually-cost-in-nigeria-naira-pricing-breakdown-2026--hftxht4ko) · Medium · 0 upvotes · 0 comments - [RAG Is Burning Money — I Built a Cost Control Layer to Fix It](https://daily.dev/posts/rag-is-burning-money-i-built-a-cost-control-layer-to-fix-it-95uruwovy) · Towards Data Science · 1 upvotes · 0 comments - [RAG Cost Control for AI Agents: How to Prevent AI Spend Drifts](https://daily.dev/posts/rag-cost-control-for-ai-agents-how-to-prevent-ai-spend-drifts-0poobdqqj) · WunderGraph · 0 upvotes · 0 comments - [How to Build a Production RAG System with Cloudflare Workers – a Handbook for Devs](https://daily.dev/posts/how-to-build-a-production-rag-system-with-cloudflare-workers-a-handbook-for-devs-pbdwygvpp) · freeCodeCamp · 1 upvotes · 0 comments --- Tags: [#rag](https://daily.dev/tags/rag), [#vector-search](https://daily.dev/tags/vector-search), [#digitalocean](https://daily.dev/tags/digitalocean), [#ai-inference](https://daily.dev/tags/ai-inference), [#pgvector](https://daily.dev/tags/pgvector) [View this post on daily.dev](https://daily.dev/posts/what-rag-actually-costs-to-run-in-production-a-full-cost-breakdown-26v3uoonx) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"What RAG Actually Costs to Run in Production: A Full Cost Breakdown","url":"https://daily.dev/posts/what-rag-actually-costs-to-run-in-production-a-full-cost-breakdown-26v3uoonx","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/what-rag-actually-costs-to-run-in-production-a-full-cost-breakdown-26v3uoonx"},"datePublished":"2026-08-14T15:48:13.816Z","dateModified":"2026-08-14T15:48:44.796Z","description":"A detailed cost breakdown for running a retrieval-augmented generation (RAG) pipeline over a 100,000-document corpus on DigitalOcean shows that embedding and...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/8f2172bba436e9bce87684915dfd91c6?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/8f2172bba436e9bce87684915dfd91c6?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"DigitalOcean Community","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"DigitalOcean Community","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/c1b9d07730e34ea388c39a498a753d6c","url":"https://daily.dev/sources/do_community"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/what-rag-actually-costs-to-run-in-production-a-full-cost-breakdown-26v3uoonx","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"rag,vector-search,digitalocean,ai-inference,pgvector","timeRequired":"PT20M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"DigitalOcean Community","item":"https://daily.dev/sources/do_community"},{"@type":"ListItem","position":3,"name":"What RAG Actually Costs to Run in Production: A Full Cost Breakdown"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/what-rag-actually-costs-to-run-in-production-a-full-cost-breakdown-26v3uoonx#faq","mainEntity":[{"@type":"Question","name":"What does it cost to run a RAG pipeline over 100,000 documents in production?","acceptedAnswer":{"@type":"Answer","text":"Roughly $98 per month at 1,000 queries per day, rising to about $496 at 10,000 queries per day and $4,000 at 100,000 queries per day, using Qwen3 Embedding 0.6B, DeepSeek V4 Flash for reranking, and gpt-oss-120b for answer generation on DigitalOcean. Embedding the full 100,000-document corpus is a one-time $6.86, and vector storage adds about $1 per month; reranking and generation make up over 99% of ongoing model spend and scale with query volume. daily.dev surfaces cost-breakdown deep dives like this for teams budgeting a RAG deployment."}},{"@type":"Question","name":"Why is the reranking step the most expensive part of a RAG pipeline at high query volume?","acceptedAnswer":{"@type":"Answer","text":"The reranker reads all 20 retrieved candidate chunks per query, about 10,400 input tokens, roughly four times more tokens than the answer-generation model sees over its top 5 chunks. At high traffic this token volume makes reranking the single largest line item, even though model price per token is relatively low; reducing candidates to 10, reranking only ambiguous queries, or using a cheaper dedicated reranker (like BGE Reranker v2 m3) can cut this cost significantly. Track token-cost tradeoffs like this reranking bottleneck on daily.dev before scaling a RAG service."}},{"@type":"Question","name":"When does dedicated GPU inference become cheaper than serverless inference for a RAG pipeline?","acceptedAnswer":{"@type":"Answer","text":"A single NVIDIA H100 dedicated instance costs a flat $4.41 per hour, about $3,220 per month, regardless of query volume, while serverless inference pricing scales linearly with traffic. Once monthly serverless spend on generation and reranking approaches that flat rate, roughly around the 100,000-queries-per-day tier in this cost model, dedicated capacity for an open-weight model starts winning on both price and latency consistency. Developers comparing serverless versus dedicated inference costs can find breakdowns like this on daily.dev."}}]} ```