--- title: Building a Production RAG Assistant on DigitalOcean description: A step-by-step recap of a DigitalOcean webinar on building HelpBot, a production RAG customer support assistant, using the DigitalOcean AI Platform. Covers... canonical: https://daily.dev/posts/building-a-production-rag-assistant-on-digitalocean-nphu6g3iz twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Building a Production RAG Assistant on DigitalOcean | daily.dev og:description: A step-by-step recap of a DigitalOcean webinar on building HelpBot, a production RAG customer support assistant, using the DigitalOcean AI Platform. Covers... og:url: https://daily.dev/posts/building-a-production-rag-assistant-on-digitalocean-nphu6g3iz og:image: https://api.daily.dev/og/posts/nPHu6g3IZ.png og:image:alt: Building a Production RAG Assistant on DigitalOcean og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Building a Production RAG Assistant on DigitalOcean **[DigitalOcean Community](https://daily.dev/sources/do_community)** · 11 min read · 0 upvotes · 0 comments ## Summary A step-by-step recap of a DigitalOcean webinar on building HelpBot, a production RAG customer support assistant, using the DigitalOcean AI Platform. Covers making raw inference calls, building a knowledge base with chunking and embeddings for retrieval, setting up an inference router with model pools and fallback, attaching guardrails (jailbreak, content moderation, sensitive data), running LLM-as-a-Judge evaluations with metrics and thresholds, and finally calling the shipped agent's own API endpoint that ties retrieval, routing, guardrails, and evals together. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How much can model affinity or session pinning reduce token costs in a multi-turn chat with an inference router? Session pinning a model after the first routing decision in a multi-turn conversation can lower input-token costs by 45-80%, since the same model is reused and its KV cache stays warm instead of re-evaluating routing on every turn. This is done by sending the same X-Model-Affinity header on subsequent calls, which causes the response to include a pinned: true flag. _Developers optimizing multi-turn LLM costs can track routing and caching techniques like this on daily.dev._ ### What file format and size limits does DigitalOcean's LLM evaluation feature require for a dataset? Datasets must be CSV or JSONL, under 1GB in size, and contain fewer than 1,000 rows. A CSV file needs a bare, unquoted header row of query,expected_response, UTF-8 encoding, and LF line endings. The evaluation then runs the candidate model or router against every row and scores answers with an LLM judge on chosen metrics. _Anyone setting up automated LLM evaluation pipelines can follow practical setup details like these on daily.dev._ ### What are the three built-in guardrails available for an AI agent on DigitalOcean's AI Platform? The three built-in guardrails are Jailbreak, Content Moderation, and Sensitive Data, which screen both incoming prompts and generated responses and override matches with a safe, predefined message. Sensitive Data guardrails can be customized by duplicating the built-in original and rewriting the default response, though built-in originals can only be detached, not deleted, and guardrails aren't available on agents built with the Agent Development Kit. _Teams adding safety layers to production LLM agents can find hands-on guardrail patterns on daily.dev._ ## Similar posts on daily.dev - [Building a Robust Documentation Agent with DigitalOcean Gradient AI Platform](https://daily.dev/posts/building-a-robust-documentation-agent-with-digitalocean-gradient-ai-platform-goz2oer8t) · DigitalOcean · 0 upvotes · 0 comments - [Building a RAG Investment Assistant](https://daily.dev/posts/building-a-rag-investment-assistant-7trsiwger) · Serokell · 0 upvotes · 0 comments - [How to Add RAG to Hermes Agent with DigitalOcean Knowledge Bases](https://daily.dev/posts/how-to-add-rag-to-hermes-agent-with-digitalocean-knowledge-bases-bpkknul1p) · DigitalOcean Community · 0 upvotes · 0 comments - [Debugging Our Docs RAG, Part 1: Evaluating a Production RAG System](https://daily.dev/posts/debugging-our-docs-rag-part-1-evaluating-a-production-rag-system-nspolfjkp) · dltHub · 1 upvotes · 0 comments - [Deploy an enterprise RAG chatbot with Red Hat OpenShift AI](https://daily.dev/posts/deploy-an-enterprise-rag-chatbot-with-red-hat-openshift-ai-2d2emq6qn) · Red Hat Developer · 1 upvotes · 0 comments --- Tags: [#llm](https://daily.dev/tags/llm), [#rag](https://daily.dev/tags/rag), [#vector-search](https://daily.dev/tags/vector-search), [#digitalocean](https://daily.dev/tags/digitalocean), [#llmops](https://daily.dev/tags/llmops) [View this post on daily.dev](https://daily.dev/posts/building-a-production-rag-assistant-on-digitalocean-nphu6g3iz) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Building a Production RAG Assistant on DigitalOcean","url":"https://daily.dev/posts/building-a-production-rag-assistant-on-digitalocean-nphu6g3iz","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/building-a-production-rag-assistant-on-digitalocean-nphu6g3iz"},"datePublished":"2026-08-13T15:35:23.442Z","dateModified":"2026-08-13T15:35:53.383Z","description":"A step-by-step recap of a DigitalOcean webinar on building HelpBot, a production RAG customer support assistant, using the DigitalOcean AI Platform. Covers...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/3a559c83e8123ba4dd4137af5754378a?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/3a559c83e8123ba4dd4137af5754378a?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"DigitalOcean Community","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"DigitalOcean Community","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/c1b9d07730e34ea388c39a498a753d6c","url":"https://daily.dev/sources/do_community"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/building-a-production-rag-assistant-on-digitalocean-nphu6g3iz","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"llm,rag,vector-search,digitalocean,llmops","timeRequired":"PT11M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"DigitalOcean Community","item":"https://daily.dev/sources/do_community"},{"@type":"ListItem","position":3,"name":"Building a Production RAG Assistant on DigitalOcean"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/building-a-production-rag-assistant-on-digitalocean-nphu6g3iz#faq","mainEntity":[{"@type":"Question","name":"How much can model affinity or session pinning reduce token costs in a multi-turn chat with an inference router?","acceptedAnswer":{"@type":"Answer","text":"Session pinning a model after the first routing decision in a multi-turn conversation can lower input-token costs by 45-80%, since the same model is reused and its KV cache stays warm instead of re-evaluating routing on every turn. This is done by sending the same X-Model-Affinity header on subsequent calls, which causes the response to include a pinned: true flag. Developers optimizing multi-turn LLM costs can track routing and caching techniques like this on daily.dev."}},{"@type":"Question","name":"What file format and size limits does DigitalOcean's LLM evaluation feature require for a dataset?","acceptedAnswer":{"@type":"Answer","text":"Datasets must be CSV or JSONL, under 1GB in size, and contain fewer than 1,000 rows. A CSV file needs a bare, unquoted header row of query,expected_response, UTF-8 encoding, and LF line endings. The evaluation then runs the candidate model or router against every row and scores answers with an LLM judge on chosen metrics. Anyone setting up automated LLM evaluation pipelines can follow practical setup details like these on daily.dev."}},{"@type":"Question","name":"What are the three built-in guardrails available for an AI agent on DigitalOcean's AI Platform?","acceptedAnswer":{"@type":"Answer","text":"The three built-in guardrails are Jailbreak, Content Moderation, and Sensitive Data, which screen both incoming prompts and generated responses and override matches with a safe, predefined message. Sensitive Data guardrails can be customized by duplicating the built-in original and rewriting the default response, though built-in originals can only be detached, not deleted, and guardrails aren't available on agents built with the Agent Development Kit. Teams adding safety layers to production LLM agents can find hands-on guardrail patterns on daily.dev."}}]} ```