--- title: How do I ship an AI feature to production? — The Agentic AI Hub | daily.dev description: Model choice, APIs and gateways, serving, and the ops that keep it honest: evals, observability, guardrails. canonical: https://daily.dev/agentic-ai-hub/paths/ship-to-production/ og:type: website og:url: https://daily.dev/agentic-ai-hub/paths/ship-to-production/ og:title: How do I ship an AI feature to production? — The Agentic AI Hub | daily.dev og:description: Model choice, APIs and gateways, serving, and the ops that keep it honest: evals, observability, guardrails. og:image: https://daily.dev/assets/og/agentic-ai-hub.png?v=40e6ff69 og:site_name: daily.dev og:locale: en_US twitter:card: summary_large_image twitter:site: @dailydotdev twitter:creator: @dailydotdev twitter:title: How do I ship an AI feature to production? — The Agentic AI Hub | daily.dev twitter:description: Model choice, APIs and gateways, serving, and the ops that keep it honest: evals, observability, guardrails. twitter:image: https://daily.dev/assets/og/agentic-ai-hub.png?v=40e6ff69 --- [Skip to main content ](#main-content) # How do I ship an AI feature to production? Model choice, APIs and gateways, serving, and the ops that keep it honest: evals, observability, guardrails. [Start reading → ](/agentic-ai-hub/how-to-choose-a-model/?path=ship-to-production&i=0) [Check yourself first ↓ ](#quiz) 1. [ 1 How to Choose a Model The honest answer is to shortlist from benchmarks and reputation, then run a small eval on your actual task. › ](/agentic-ai-hub/how-to-choose-a-model/?path=ship-to-production&i=0) 2. [ 2 APIs, Inference Platforms & Gateways Four rough tiers, each with different trade-offs on model access, speed, price, and control. › ](/agentic-ai-hub/apis-inference-platforms-gateways/?path=ship-to-production&i=1) 3. [ 3 Inference, Serving & Optimization TTFT (Time To First Token). Latency until the first token appears. Dominated by the prefill (prompt-processing) phase and prompt length. › ](/agentic-ai-hub/inference-serving-optimization/?path=ship-to-production&i=2) 4. [ 4 LLMOps: Evals, Observability & Guardrails Shipping LLM features without evals and observability is flying blind: outputs are non-deterministic, quality is subjective, and regressions are silent. › ](/agentic-ai-hub/llmops-evals-observability-guardrails/?path=ship-to-production&i=3) 5. [ 5 Pricing & Cost Reference Pricing is a Frontier Model Comparison snapshot. This section is the durable part: how to reason about cost so you're not surprised by a bill. › ](/agentic-ai-hub/pricing-cost-reference/?path=ship-to-production&i=4) ## Check yourself. One question per step. Take it cold to find where to start, or after reading to see what stuck. Nobody's grading you. Best so far: Question 1 of 5 1. Step 1 What's the most reliable way to choose a model for your use case? A Pick whichever model tops the current leaderboards B Standardize on one lab's flagship and stick with it C Shortlist candidates, then run a small eval on your actual task D Start with the cheapest tier and upgrade only if it fails Model rankings flip depending on the workload, so benchmarks and reputation are only good for a shortlist. A small eval on your actual task makes the final call. [Covered in step 1 · How to Choose a Model → ](/agentic-ai-hub/how-to-choose-a-model/?path=ship-to-production&i=0) Next question → 2. Step 2 Your LLM feature just moved past prototype and has real traffic. What's the recommended next infrastructure step? A Put a gateway like LiteLLM or Portkey in front of your providers B Migrate everything to Bedrock or Vertex for compliance C Move all traffic to a fast open-model host like Groq D Lock in an enterprise contract with a single lab Most teams past prototype should run a gateway so no application code is welded to one vendor. It centralizes key management, fallback, rate limiting, and cost tracking. [Covered in step 2 · APIs, Inference Platforms & Gateways → ](/agentic-ai-hub/apis-inference-platforms-gateways/?path=ship-to-production&i=1) Next question → 3. Step 3 Users complain your chat feature feels slow to start responding. What most likely dominates time to first token? A Slow decode speed on the serving GPU B The length of the model's answer C Network latency between you and the provider D Prompt length and the prefill phase TTFT is dominated by prefill, the compute-bound phase that processes the whole prompt, so long prompts delay the first token. Decode speed only affects how fast text streams afterward. [Covered in step 3 · Inference, Serving & Optimization → ](/agentic-ai-hub/inference-serving-optimization/?path=ship-to-production&i=2) Next question → 4. Step 4 What's the highest-ROI evaluation investment for an LLM feature in production? A Running LLM-as-judge scoring on all live traffic B A golden dataset of 50 to 100 representative examples C Tracking scores on public capability benchmarks D A large-scale human annotation program A golden dataset is your regression suite: run it on every prompt or model change and diff the scores. Even 50 to 100 examples catch most regressions. [Covered in step 4 · LLMOps: Evals, Observability & Guardrails → ](/agentic-ai-hub/llmops-evals-observability-guardrails/?path=ship-to-production&i=3) Next question → 5. Step 5 Your production agent resends the same big system prompt on every request. What's the single biggest cost lever? A Move the traffic to a batch API B Cap the model's reasoning budget C Trim the system prompt down D Prompt caching, with the stable prefix first Cached repeated prefixes are discounted \~90% on input (sometimes \~99% on hits). It is the single biggest win for agents and RAG that resend the same context, which is why the stable part of a prompt should come first. [Covered in step 5 · Pricing & Cost Reference → ](/agentic-ai-hub/pricing-cost-reference/?path=ship-to-production&i=4) See results → of 5 * [ 1 How to Choose a Model → ](/agentic-ai-hub/how-to-choose-a-model/?path=ship-to-production&i=0) * [ 2 APIs, Inference Platforms & Gateways → ](/agentic-ai-hub/apis-inference-platforms-gateways/?path=ship-to-production&i=1) * [ 3 Inference, Serving & Optimization → ](/agentic-ai-hub/inference-serving-optimization/?path=ship-to-production&i=2) * [ 4 LLMOps: Evals, Observability & Guardrails → ](/agentic-ai-hub/llmops-evals-observability-guardrails/?path=ship-to-production&i=3) * [ 5 Pricing & Cost Reference → ](/agentic-ai-hub/pricing-cost-reference/?path=ship-to-production&i=4) Try again [Where to next → ](#path-next-title) ## Where to next. More guided paths through the handbook. Pick the next question worth answering. [ How do I build my own agent? → ](/agentic-ai-hub/paths/build-your-own-agent/)[ Can I trust what agents produce? → ](/agentic-ai-hub/paths/trust-agent-output/)[ How do I ground AI in my company's data? → ](/agentic-ai-hub/paths/ground-in-your-data/) [All reading paths → ](/agentic-ai-hub/#paths-start-here) [Browse every chapter ](/agentic-ai-hub/#pages) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/og-image.png?v=a830cdf1","width":1200,"height":630},"sameAs":["https://twitter.com/dailydotdev","https://www.linkedin.com/company/dailydotdev","https://github.com/dailydotdev","https://www.instagram.com/dailydotdev"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","description":"Free, personalized developer news aggregator. Stay on top of software development news, AI coding tools, and web dev - curated daily from trusted sources.","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}},{"@type":"WebPage","@id":"https://daily.dev/agentic-ai-hub/paths/ship-to-production/","url":"https://daily.dev/agentic-ai-hub/paths/ship-to-production/","name":"How do I ship an AI feature to production? | The Agentic AI Hub","description":"Model choice, APIs and gateways, serving, and the ops that keep it honest: evals, observability, guardrails.","inLanguage":"en-US","isPartOf":{"@id":"https://daily.dev/#website"}},{"@type":"ItemList","numberOfItems":5,"itemListElement":[{"@type":"ListItem","position":1,"url":"https://daily.dev/agentic-ai-hub/how-to-choose-a-model/","name":"How to Choose a Model"},{"@type":"ListItem","position":2,"url":"https://daily.dev/agentic-ai-hub/apis-inference-platforms-gateways/","name":"APIs, Inference Platforms & Gateways"},{"@type":"ListItem","position":3,"url":"https://daily.dev/agentic-ai-hub/inference-serving-optimization/","name":"Inference, Serving & Optimization"},{"@type":"ListItem","position":4,"url":"https://daily.dev/agentic-ai-hub/llmops-evals-observability-guardrails/","name":"LLMOps: Evals, Observability & Guardrails"},{"@type":"ListItem","position":5,"url":"https://daily.dev/agentic-ai-hub/pricing-cost-reference/","name":"Pricing & Cost Reference"}],"@id":"https://daily.dev/agentic-ai-hub/paths/ship-to-production/#steps"},{"@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev/"},{"@type":"ListItem","position":2,"name":"The Agentic AI Hub","item":"https://daily.dev/agentic-ai-hub/"},{"@type":"ListItem","position":3,"name":"Build with AI","item":"https://daily.dev/agentic-ai-hub/#paths-build"},{"@type":"ListItem","position":4,"name":"How do I ship an AI feature to production?","item":"https://daily.dev/agentic-ai-hub/paths/ship-to-production/"}]}]} ```