--- title: AI Chatbot Evaluation and Observability with Promptfoo... description: AI chatbots that pass demos often fail in production due to non-deterministic outputs, retrieval issues, and prompt sensitivity. Two tools address different... canonical: https://daily.dev/posts/ai-chatbot-evaluation-and-observability-with-promptfoo-and-langfuse-rccwse2wk twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: AI Chatbot Evaluation and Observability with Promptfoo and Langfuse | daily.dev og:description: AI chatbots that pass demos often fail in production due to non-deterministic outputs, retrieval issues, and prompt sensitivity. Two tools address different... og:url: https://daily.dev/posts/ai-chatbot-evaluation-and-observability-with-promptfoo-and-langfuse-rccwse2wk og:image: https://api.daily.dev/og/posts/rcCWSe2Wk.png og:image:alt: AI Chatbot Evaluation and Observability with Promptfoo and Langfuse og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # AI Chatbot Evaluation and Observability with Promptfoo and Langfuse **[Netguru](https://daily.dev/sources/netguru)** · 8 min read · 0 upvotes · 0 comments ## Summary AI chatbots that pass demos often fail in production due to non-deterministic outputs, retrieval issues, and prompt sensitivity. Two tools address different stages of this problem: Promptfoo handles pre-deployment evaluation through prompt testing, regression testing, model comparison, and adversarial testing; Langfuse handles post-deployment observability through tracing, latency monitoring, cost tracking, and feedback loops. The recommended workflow combines both: run Promptfoo evaluations before deploying prompt or model changes, trace production conversations with Langfuse, then convert real production failures into new Promptfoo regression tests. Key evaluation dimensions include factual accuracy, groundedness, retrieval quality, relevance, safety, latency, task success, and user satisfaction. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Similar posts on daily.dev - [Testing AI prompts and comparing models with promptfoo](https://daily.dev/posts/testing-ai-prompts-and-comparing-models-with-promptfoo-mxhomphpc) · Tim Deschryver · 1 upvotes · 1 comments - [Core Bottleneck In AI Engineering Isn't Writing Code. It's Trusting What Code Produces](https://daily.dev/posts/core-bottleneck-in-ai-engineering-isn-t-writing-code-it-s-trusting-what-code-produces-sslmnacfy) · Netguru · 0 upvotes · 0 comments --- Tags: [#llm](https://daily.dev/tags/llm), [#observability](https://daily.dev/tags/observability), [#rag](https://daily.dev/tags/rag) [View this post on daily.dev](https://daily.dev/posts/ai-chatbot-evaluation-and-observability-with-promptfoo-and-langfuse-rccwse2wk) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"AI Chatbot Evaluation and Observability with Promptfoo and Langfuse","url":"https://daily.dev/posts/ai-chatbot-evaluation-and-observability-with-promptfoo-and-langfuse-rccwse2wk","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/ai-chatbot-evaluation-and-observability-with-promptfoo-and-langfuse-rccwse2wk"},"datePublished":"2026-07-09T07:58:57.717Z","dateModified":"2026-07-09T07:59:28.299Z","description":"AI chatbots that pass demos often fail in production due to non-deterministic outputs, retrieval issues, and prompt sensitivity. Two tools address different...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/83f5f99dad1cc2f85794ffd52f15730d?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/83f5f99dad1cc2f85794ffd52f15730d?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Netguru","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Netguru","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/201e234294cb4132ab31418b7627dbab","url":"https://daily.dev/sources/netguru"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/ai-chatbot-evaluation-and-observability-with-promptfoo-and-langfuse-rccwse2wk","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"llm,observability,rag","timeRequired":"PT8M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Netguru","item":"https://daily.dev/sources/netguru"},{"@type":"ListItem","position":3,"name":"AI Chatbot Evaluation and Observability with Promptfoo and Langfuse"}]} ```