--- title: How ngrok helps Sam develop and test LLM-powered RAG... description: A guest post from HeySam's CEO describes how the AI Sales Engineer platform, Sam, uses ngrok to develop and test its distributed, LLM-powered RAG architecture.... canonical: https://daily.dev/posts/how-ngrok-helps-sam-develop-and-test-llm-powered-rag-applications-8zyoch3of twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: How ngrok helps Sam develop and test LLM-powered RAG applications | daily.dev og:description: A guest post from HeySam's CEO describes how the AI Sales Engineer platform, Sam, uses ngrok to develop and test its distributed, LLM-powered RAG architecture.... og:url: https://daily.dev/posts/how-ngrok-helps-sam-develop-and-test-llm-powered-rag-applications-8zyoch3of og:image: https://api.daily.dev/og/posts/8zyoCH3OF.png og:image:alt: How ngrok helps Sam develop and test LLM-powered RAG applications og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # How ngrok helps Sam develop and test LLM-powered RAG applications **[ngrok Blog](https://daily.dev/sources/ngrok)** · 3 min read · 0 upvotes · 0 comments ## Summary A guest post from HeySam's CEO describes how the AI Sales Engineer platform, Sam, uses ngrok to develop and test its distributed, LLM-powered RAG architecture. It explains how engineers use ngrok webhooks to connect local services to staging bots for real-time transcription and question detection, and how production traffic can be shadowed to local endpoints to safely test new LLM prompts against a comparison service before deploying changes. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How can I safely test a new LLM prompt against real production traffic without affecting users? Register an ngrok endpoint with the production question-answering service and use a feature flagging service to forward a percentage of real production questions to that endpoint. The new prompt generates an answer locally, saved alongside the original question ID, then a comparison service evaluates the quality of the shadowed answer against the production answer. _daily.dev surfaces practical patterns like this for engineers shipping safer LLM prompt iterations._ ### How do teams develop against a distributed RAG pipeline without running every service locally? Engineers use ngrok webhooks to connect individual services running locally to shared staging infrastructure, rather than running the entire pipeline on their machine. For example, a developer can register an ngrok webhook with a staging bot server so a live Zoom call's audio streams to their locally running transcription service for testing. _Developers wiring up distributed AI pipelines can find similar workflow patterns on daily.dev._ --- Tags: [#llm](https://daily.dev/tags/llm), [#devtools](https://daily.dev/tags/devtools), [#distributed-systems](https://daily.dev/tags/distributed-systems), [#rag](https://daily.dev/tags/rag) [View this post on daily.dev](https://daily.dev/posts/how-ngrok-helps-sam-develop-and-test-llm-powered-rag-applications-8zyoch3of) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"How ngrok helps Sam develop and test LLM-powered RAG applications","url":"https://daily.dev/posts/how-ngrok-helps-sam-develop-and-test-llm-powered-rag-applications-8zyoch3of","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/how-ngrok-helps-sam-develop-and-test-llm-powered-rag-applications-8zyoch3of"},"datePublished":"2026-08-23T07:21:10.550Z","dateModified":"2026-08-23T07:23:24.197Z","description":"A guest post from HeySam's CEO describes how the AI Sales Engineer platform, Sam, uses ngrok to develop and test its distributed, LLM-powered RAG architecture....","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/283f87fedd358887a8064abd47a6f3fe?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/283f87fedd358887a8064abd47a6f3fe?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"ngrok Blog","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"ngrok Blog","logo":"https://media.daily.dev/image/upload/s--718XXxvE--/f_auto,q_auto/v1787469610/logos/ngrok?_a=BAMAMicg0","url":"https://daily.dev/sources/ngrok"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/how-ngrok-helps-sam-develop-and-test-llm-powered-rag-applications-8zyoch3of","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"llm,devtools,distributed-systems,rag","timeRequired":"PT3M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"ngrok Blog","item":"https://daily.dev/sources/ngrok"},{"@type":"ListItem","position":3,"name":"How ngrok helps Sam develop and test LLM-powered RAG applications"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/how-ngrok-helps-sam-develop-and-test-llm-powered-rag-applications-8zyoch3of#faq","mainEntity":[{"@type":"Question","name":"How can I safely test a new LLM prompt against real production traffic without affecting users?","acceptedAnswer":{"@type":"Answer","text":"Register an ngrok endpoint with the production question-answering service and use a feature flagging service to forward a percentage of real production questions to that endpoint. The new prompt generates an answer locally, saved alongside the original question ID, then a comparison service evaluates the quality of the shadowed answer against the production answer. daily.dev surfaces practical patterns like this for engineers shipping safer LLM prompt iterations."}},{"@type":"Question","name":"How do teams develop against a distributed RAG pipeline without running every service locally?","acceptedAnswer":{"@type":"Answer","text":"Engineers use ngrok webhooks to connect individual services running locally to shared staging infrastructure, rather than running the entire pipeline on their machine. For example, a developer can register an ngrok webhook with a staging bot server so a live Zoom call's audio streams to their locally running transcription service for testing. Developers wiring up distributed AI pipelines can find similar workflow patterns on daily.dev."}}]} ```