A step-by-step recap of a DigitalOcean webinar on building HelpBot, a production RAG customer support assistant, using the DigitalOcean AI Platform. Covers making raw inference calls, building a knowledge base with chunking and embeddings for retrieval, setting up an inference router with model pools and fallback, attaching guardrails (jailbreak, content moderation, sensitive data), running LLM-as-a-Judge evaluations with metrics and thresholds, and finally calling the shipped agent's own API endpoint that ties retrieval, routing, guardrails, and evals together.
Table of contents
Watch the webinar recordingBefore you startStep 1: Orientation & your first serverless callStep 2: Grounding with RAG & Knowledge BasesStep 3: The Inference Router: resilience, cost, and latencyStep 4: Guardrails: PII, jailbreaks, and content moderationStep 5: Evaluations: proving quality with LLM-as-a-JudgeStep 6: Putting it together & productionizingBuild your own production AI agent with DigitalOcean AI PlatformQuestions this post answers
How much can model affinity or session pinning reduce token costs in a multi-turn chat with an inference router?
Session pinning a model after the first routing decision in a multi-turn conversation can lower input-token costs by 45-80%, since the same model is reused and its KV cache stays warm instead of re-evaluating routing on every turn. This is done by sending the same X-Model-Affinity header on subsequent calls, which causes the response to include a pinned: true flag. Developers optimizing multi-turn LLM costs can track routing and caching techniques like this on daily.dev.
What file format and size limits does DigitalOcean's LLM evaluation feature require for a dataset?
Datasets must be CSV or JSONL, under 1GB in size, and contain fewer than 1,000 rows. A CSV file needs a bare, unquoted header row of query,expected_response, UTF-8 encoding, and LF line endings. The evaluation then runs the candidate model or router against every row and scores answers with an LLM judge on chosen metrics. Anyone setting up automated LLM evaluation pipelines can follow practical setup details like these on daily.dev.
What are the three built-in guardrails available for an AI agent on DigitalOcean's AI Platform?
The three built-in guardrails are Jailbreak, Content Moderation, and Sensitive Data, which screen both incoming prompts and generated responses and override matches with a safe, predefined message. Sensitive Data guardrails can be customized by duplicating the built-in original and rewriting the default response, though built-in originals can only be detached, not deleted, and guardrails aren't available on agents built with the Agent Development Kit. Teams adding safety layers to production LLM agents can find hands-on guardrail patterns on daily.dev.