--- title: From postmortems to guardrails at scale | daily.dev description: A production outage traced back to a release pipeline rebuilding an image from 'python:latest', which pulled a base image missing a required system package.... canonical: https://daily.dev/posts/from-postmortems-to-guardrails-at-scale-i8cect33g twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: From postmortems to guardrails at scale | daily.dev og:description: A production outage traced back to a release pipeline rebuilding an image from 'python:latest', which pulled a base image missing a required system package.... og:url: https://daily.dev/posts/from-postmortems-to-guardrails-at-scale-i8cect33g og:image: https://api.daily.dev/og/posts/I8cECT33g.png og:image:alt: From postmortems to guardrails at scale og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # From postmortems to guardrails at scale **[Earthly](https://daily.dev/sources/earthly)** · 6 min read · 0 upvotes · 0 comments ## Summary A production outage traced back to a release pipeline rebuilding an image from 'python:latest', which pulled a base image missing a required system package. The fix: pin base images by digest and deploy the exact image that passed testing. The post argues postmortem lessons like this can be lost when enforcement depends on tickets and reminders, not automated checks. It describes Earthly Lunar's guardrails engine, which collects signals from repos and CI/CD to centrally enforce rules like 'no-latest' and 'digest-pinned' across hundreds of repos, blocking merges or releases when a requirement fails. Lunar offers 300+ pre-built guardrails and its built-in AI skills can help draft custom ones. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### why did my docker build break after passing in staging but failing in production A common cause is using a floating tag like 'python:latest' as the base image, which can resolve to a different image on a later build. If the pipeline rebuilds the application image before deploying instead of reusing the one that passed testing, a fresh pull can introduce a missing system package and break the deploy even though staging still runs the old build. _daily.dev surfaces practical fixes like digest pinning for developers debugging flaky container builds._ ### what is the difference between pinning a docker base image by tag versus by digest A version tag can still move to point at a different image later, while a digest identifies one specific, unchanging image. Pinning by digest means a build is reproducible, but it also means someone must deliberately update the digest to pick up future base image fixes, trading automatic updates for consistency. _developers weighing reproducibility against staying current can track this tradeoff on daily.dev._ ### how can a platform team enforce that all teams pin docker base images by digest across many repositories One approach is a guardrails engine that centrally collects and normalizes signals from repos and CI/CD, then evaluates rules like 'digest-pinned' deterministically against every component. Earthly Lunar applies such checks via PR checks and release gates, blocking merges or releases for components tagged 'production' that fail the requirement, rather than relying on tickets or reminders. _platform teams standardizing build rules across hundreds of repos can follow this approach via daily.dev._ ## Similar posts on daily.dev - [One canary rule, three systems to reconcile](https://daily.dev/posts/one-canary-rule-three-systems-to-reconcile-8zhacdbbe) · Earthly · 0 upvotes · 0 comments - [Software Supply Chain Security: Why 99% of Your Container is Mystery Code](https://daily.dev/posts/software-supply-chain-security-why-99-of-your-container-is-mystery-code-0o8ohjmsz) · Cloud Native Now · 0 upvotes · 0 comments - [Why traditional CI/CD fails for LLMs \(and the release gates we built to fix it\)](https://daily.dev/posts/why-traditional-ci-cd-fails-for-llms-and-the-release-gates-we-built-to-fix-it--e3jggw3ud) · The New Stack · 0 upvotes · 0 comments --- Tags: [#cicd](https://daily.dev/tags/cicd), [#docker](https://daily.dev/tags/docker) [View this post on daily.dev](https://daily.dev/posts/from-postmortems-to-guardrails-at-scale-i8cect33g) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"From postmortems to guardrails at scale","url":"https://daily.dev/posts/from-postmortems-to-guardrails-at-scale-i8cect33g","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/from-postmortems-to-guardrails-at-scale-i8cect33g"},"datePublished":"2026-10-28T21:19:55.680Z","dateModified":"2026-10-28T21:43:51.074Z","description":"A production outage traced back to a release pipeline rebuilding an image from 'python:latest', which pulled a base image missing a required system package....","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/e72bff4f1c10bfec7c35cf77c4233bf7?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/e72bff4f1c10bfec7c35cf77c4233bf7?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Earthly","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Earthly","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/139a6e6902684ea1aa97ca01c0c8c92e","url":"https://daily.dev/sources/earthly"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/from-postmortems-to-guardrails-at-scale-i8cect33g","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"cicd,docker","timeRequired":"PT6M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Earthly","item":"https://daily.dev/sources/earthly"},{"@type":"ListItem","position":3,"name":"From postmortems to guardrails at scale"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/from-postmortems-to-guardrails-at-scale-i8cect33g#faq","mainEntity":[{"@type":"Question","name":"why did my docker build break after passing in staging but failing in production","acceptedAnswer":{"@type":"Answer","text":"A common cause is using a floating tag like 'python:latest' as the base image, which can resolve to a different image on a later build. If the pipeline rebuilds the application image before deploying instead of reusing the one that passed testing, a fresh pull can introduce a missing system package and break the deploy even though staging still runs the old build. daily.dev surfaces practical fixes like digest pinning for developers debugging flaky container builds."}},{"@type":"Question","name":"what is the difference between pinning a docker base image by tag versus by digest","acceptedAnswer":{"@type":"Answer","text":"A version tag can still move to point at a different image later, while a digest identifies one specific, unchanging image. Pinning by digest means a build is reproducible, but it also means someone must deliberately update the digest to pick up future base image fixes, trading automatic updates for consistency. developers weighing reproducibility against staying current can track this tradeoff on daily.dev."}},{"@type":"Question","name":"how can a platform team enforce that all teams pin docker base images by digest across many repositories","acceptedAnswer":{"@type":"Answer","text":"One approach is a guardrails engine that centrally collects and normalizes signals from repos and CI/CD, then evaluates rules like 'digest-pinned' deterministically against every component. Earthly Lunar applies such checks via PR checks and release gates, blocking merges or releases for components tagged 'production' that fail the requirement, rather than relying on tickets or reminders. platform teams standardizing build rules across hundreds of repos can follow this approach via daily.dev."}}]} ```