--- title: Our PR gate has just 17 tests, and here's why | daily.dev description: A test automation vendor's engineering team describes how they scaled down their pull request gate from 288 candidate tests to just 17 high-value tests. They... canonical: https://daily.dev/posts/our-pr-gate-has-just-17-tests-and-here-s-why-71xuiiwf7 twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Our PR gate has just 17 tests, and here's why | daily.dev og:description: A test automation vendor's engineering team describes how they scaled down their pull request gate from 288 candidate tests to just 17 high-value tests. They... og:url: https://daily.dev/posts/our-pr-gate-has-just-17-tests-and-here-s-why-71xuiiwf7 og:image: https://api.daily.dev/og/posts/71xuiiWF7.png og:image:alt: Our PR gate has just 17 tests, and here's why og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Our PR gate has just 17 tests, and here's why **[Mabl Engineering Blog](https://daily.dev/sources/mabl)** · 8 min read · 0 upvotes · 0 comments ## Summary A test automation vendor's engineering team describes how they scaled down their pull request gate from 288 candidate tests to just 17 high-value tests. They selected tests based on real usage data, ranked user journeys by failure impact into three tiers, and required each test to prove it checked a durable outcome (not just that something happened). They spent months eliminating flaky tests so the gate's false-failure rate stayed low, since failure rates multiply when many tests gate a single merge (a 288-test gate at 99% per-test reliability passes only ~5% of the time versus ~84% for 17 tests). The gate was rolled out gradually as advisory before becoming required, and the team tracks trust metrics weekly, including via an MCP server integration for developers. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### Why does adding more tests to a CI blocking gate make it fail more often instead of less? Because failure rates multiply across tests in a blocking gate: if each test passes 99% of the time independently, a 17-test gate passes cleanly about 84% of the time, but a 288-test gate at the same per-test reliability only passes about 5% of the time. Widening a gate means fixing flaky tests, not adding more coverage to it. _Anyone designing a CI gate can compare notes on test reliability tradeoffs through daily.dev._ ### How do you decide which tests belong in a blocking pull request gate versus a full regression suite? Pick user journeys first, ranked into three impact tiers (blocked work, lost time, or merely noticed), based on 30 days of real usage data rather than instinct. For each journey, keep a test in the gate only if it covers the journey on the customer-facing surface, actually runs, checks a durable outcome rather than just a state change, and matches the journey's tier. Everything else runs in a larger regression suite after merge. _Teams rethinking their PR gate strategy can explore test design approaches like this on daily.dev._ ## Similar posts on daily.dev - [We halved our continuous integration pipeline](https://daily.dev/posts/we-halved-our-continuous-integration-pipeline-w7assusoy) · LeadDev · 0 upvotes · 1 comments - [How We Ship Quality at mabl](https://daily.dev/posts/how-we-ship-quality-at-mabl-yx9gy8bal) · Mabl Engineering Blog · 0 upvotes · 0 comments - [We now auto-approve and merge 15% of PRs](https://daily.dev/posts/we-now-auto-approve-and-merge-15-of-prs-8bygwtyr3) · swizec.com · 42 upvotes · 14 comments - [How MuleSoft Is Raising the Trust Bar for AI-Generated Code](https://daily.dev/posts/how-mulesoft-is-raising-the-trust-bar-for-ai-generated-code-q2wkjy8ak) · Salesforce Engineering · 1 upvotes · 0 comments --- Tags: [#testing](https://daily.dev/tags/testing), [#cicd](https://daily.dev/tags/cicd) [View this post on daily.dev](https://daily.dev/posts/our-pr-gate-has-just-17-tests-and-here-s-why-71xuiiwf7) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Our PR gate has just 17 tests, and here's why","url":"https://daily.dev/posts/our-pr-gate-has-just-17-tests-and-here-s-why-71xuiiwf7","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/our-pr-gate-has-just-17-tests-and-here-s-why-71xuiiwf7"},"datePublished":"2026-10-02T20:33:38.517Z","dateModified":"2026-10-02T20:34:00.399Z","description":"A test automation vendor's engineering team describes how they scaled down their pull request gate from 288 candidate tests to just 17 high-value tests. They...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/640e0c653ca4d3c295f5fda5eae7b940?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/640e0c653ca4d3c295f5fda5eae7b940?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Mabl Engineering Blog","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Mabl Engineering Blog","logo":"https://media.daily.dev/image/upload/s--LPBlphgx--/f_auto,q_auto/v1780213707/logos/mabl?_a=BAMAMiWQ0","url":"https://daily.dev/sources/mabl"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/our-pr-gate-has-just-17-tests-and-here-s-why-71xuiiwf7","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"testing,cicd","timeRequired":"PT8M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Mabl Engineering Blog","item":"https://daily.dev/sources/mabl"},{"@type":"ListItem","position":3,"name":"Our PR gate has just 17 tests, and here's why"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/our-pr-gate-has-just-17-tests-and-here-s-why-71xuiiwf7#faq","mainEntity":[{"@type":"Question","name":"Why does adding more tests to a CI blocking gate make it fail more often instead of less?","acceptedAnswer":{"@type":"Answer","text":"Because failure rates multiply across tests in a blocking gate: if each test passes 99% of the time independently, a 17-test gate passes cleanly about 84% of the time, but a 288-test gate at the same per-test reliability only passes about 5% of the time. Widening a gate means fixing flaky tests, not adding more coverage to it. Anyone designing a CI gate can compare notes on test reliability tradeoffs through daily.dev."}},{"@type":"Question","name":"How do you decide which tests belong in a blocking pull request gate versus a full regression suite?","acceptedAnswer":{"@type":"Answer","text":"Pick user journeys first, ranked into three impact tiers (blocked work, lost time, or merely noticed), based on 30 days of real usage data rather than instinct. For each journey, keep a test in the gate only if it covers the journey on the customer-facing surface, actually runs, checks a durable outcome rather than just a state change, and matches the journey's tier. Everything else runs in a larger regression suite after merge. Teams rethinking their PR gate strategy can explore test design approaches like this on daily.dev."}}]} ```