--- title: How we built integration testing for fast-moving AI backend description: A Red Hat OpenShift AI team shares how they replaced mocked unit tests with real integration tests against a live Llama Stack server to catch API drift early.... canonical: https://daily.dev/posts/how-we-built-integration-testing-for-fast-moving-ai-backend-leywrsnnc twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: How we built integration testing for fast-moving AI backend | daily.dev og:description: A Red Hat OpenShift AI team shares how they replaced mocked unit tests with real integration tests against a live Llama Stack server to catch API drift early.... og:url: https://daily.dev/posts/how-we-built-integration-testing-for-fast-moving-ai-backend-leywrsnnc og:image: https://api.daily.dev/og/posts/LEywrsnNc.png og:image:alt: How we built integration testing for fast-moving AI backend og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # How we built integration testing for fast-moving AI backend **[Red Hat Developer](https://daily.dev/sources/rhdev)** · 14 min read · 3 upvotes · 0 comments ## Summary A Red Hat OpenShift AI team shares how they replaced mocked unit tests with real integration tests against a live Llama Stack server to catch API drift early. The core insight: mocks test your assumptions, not reality. Their solution uses `uv run` to spin up a real Llama Stack subprocess (no Docker), leverages Llama Stack's built-in record-replay system to avoid LLM API costs on every test run, and wires a daily GitHub Actions 'Compatibility Sentinel' that tests both stable and dev versions of Llama Stack. The sentinel uses a two-phase strategy: try replay first (zero cost), fall back to record mode with real Gemini calls only when fixtures don't match. Results are posted to Slack each morning. Since March 2026, the sentinel has caught every breaking upstream change weeks before it reached stable, turning potential production outages into planned sprint tasks. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Similar posts on daily.dev - [Continuous integration and continuous delivery for agentic AI](https://daily.dev/posts/continuous-integration-and-continuous-delivery-for-agentic-ai-lhegkveiq) · Red Hat Developer · 5 upvotes · 0 comments - [The AI-Native SDLC Starts With Your Infrastructure](https://daily.dev/posts/the-ai-native-sdlc-starts-with-your-infrastructure-2fviwpotm) · MetalBear · 1 upvotes · 1 comments - [Untitled](https://daily.dev/posts/untitled-6mrgpcxc1) · SitePoint · 0 upvotes · 0 comments - [How Cloud-Native Complexity is Outpacing Test Automation Strategy](https://daily.dev/posts/how-cloud-native-complexity-is-outpacing-test-automation-strategy-aizwf7ic6) · Cloud Native Now · 0 upvotes · 0 comments --- Tags: [#testing](https://daily.dev/tags/testing), [#golang](https://daily.dev/tags/golang), [#github-actions](https://daily.dev/tags/github-actions) [View this post on daily.dev](https://daily.dev/posts/how-we-built-integration-testing-for-fast-moving-ai-backend-leywrsnnc) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"How we built integration testing for fast-moving AI backend","url":"https://daily.dev/posts/how-we-built-integration-testing-for-fast-moving-ai-backend-leywrsnnc","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/how-we-built-integration-testing-for-fast-moving-ai-backend-leywrsnnc"},"datePublished":"2026-05-27T07:46:50.621Z","dateModified":"2026-05-27T07:47:32.494Z","description":"A Red Hat OpenShift AI team shares how they replaced mocked unit tests with real integration tests against a live Llama Stack server to catch API drift early....","image":"https://developers.redhat.com/sites/default/files/styles/share/public/Red%20Hat%20Openshift%20AI_0.png?itok=EGHltugp","thumbnailUrl":"https://developers.redhat.com/sites/default/files/styles/share/public/Red%20Hat%20Openshift%20AI_0.png?itok=EGHltugp","isAccessibleForFree":true,"articleSection":"Red Hat Developer","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Red Hat Developer","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/2a8895f3216c4900bdc455585a4ad3e4","url":"https://daily.dev/sources/rhdev"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/how-we-built-integration-testing-for-fast-moving-ai-backend-leywrsnnc","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":3},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"testing,golang,github-actions","timeRequired":"PT14M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Red Hat Developer","item":"https://daily.dev/sources/rhdev"},{"@type":"ListItem","position":3,"name":"How we built integration testing for fast-moving AI backend"}]} ```