--- title: Implementing Durable Workflows on Postgres Without an... description: Building Kestrel Workflows, a durable-execution system for automating incident response and CI/CD, led to using plain Postgres instead of an external... canonical: https://daily.dev/posts/implementing-durable-workflows-on-postgres-without-an-external-orchestrator-hn7fsiivi twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Implementing Durable Workflows on Postgres Without an External Orchestrator | daily.dev og:description: Building Kestrel Workflows, a durable-execution system for automating incident response and CI/CD, led to using plain Postgres instead of an external... og:url: https://daily.dev/posts/implementing-durable-workflows-on-postgres-without-an-external-orchestrator-hn7fsiivi og:image: https://api.daily.dev/og/posts/hn7FSIiVI.png og:image:alt: Implementing Durable Workflows on Postgres Without an External Orchestrator og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Implementing Durable Workflows on Postgres Without an External Orchestrator **[InfoQ](https://daily.dev/sources/infoq)** · 12 min read · 0 upvotes · 0 comments ## Summary Building Kestrel Workflows, a durable-execution system for automating incident response and CI/CD, led to using plain Postgres instead of an external orchestrator like Temporal or AWS Step Functions. SELECT ... FOR UPDATE SKIP LOCKED turns a table into a concurrent work queue with exactly-once claiming, primary-key constraints on step checkpoints enforce idempotency, and a lease-and-sweeper pattern handles crash recovery. Durable sleeps and human approvals are stored as rows with wake-at timestamps, surviving restarts indefinitely. The approach reportedly handles tens of thousands of workflows per second on a single Postgres instance, collapses observability into plain SQL queries, and reduces the security surface to one dependency, though PgBouncer pooling, autovacuum tuning, and partitioning are needed at scale, and Temporal may still be better for very high fan-out with sub-millisecond latency needs. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How does SELECT FOR UPDATE SKIP LOCKED prevent two workers from processing the same queue row in Postgres? It locks the rows a worker selects and instructs any other concurrent worker to skip past those locked rows instead of blocking on them, so two workers polling the same table simultaneously can never claim the same row. Combined with a short transaction that flips the row's status and writes a lease, this gives exactly-once dispatch without a message broker, leader election, or external lock service. _Developers weighing queue designs can compare Postgres-based patterns like this one on daily.dev before picking an architecture._ ### How can I implement idempotent step checkpoints for a workflow engine using Postgres constraints instead of application code? Key a checkpoints table by a composite primary key such as (execution_id, step_id), then insert results with ON CONFLICT DO NOTHING so the unique constraint blocks duplicate writes. If a recovering worker reruns a completed step, it reads the existing row and returns the prior output instead of repeating the side effect, letting the database rather than application logic enforce idempotency. _Anyone designing crash-safe pipelines can dig into idempotency patterns like this through daily.dev._ ### What is a lease-and-sweeper pattern for recovering crashed workers in a Postgres-backed job queue? Workers that own a running row periodically extend a lease_expires timestamp via heartbeat updates, and a separate sweeper query resets any row whose status is still 'running' but whose lease has expired back to 'enqueued' with no owner. A healthy worker then reclaims the row and resumes from its last completed checkpoint, with lease duration trading off wasted work against recovery delay. _Teams building crash-recovery logic for job queues can find implementation patterns like this via daily.dev._ ## Similar posts on daily.dev - [Making Postgres queues scale](https://daily.dev/posts/making-postgres-queues-scale-otqspthk6) · Hacker News · 1 upvotes · 0 comments - [I Tried the “Just Use Postgres” Queue for Six Weeks. SKIP LOCKED Paid Until It Didn’t](https://daily.dev/posts/i-tried-the-just-use-postgres-queue-for-six-weeks-skip-locked-paid-until-it-didn-t-sni9qhs1s) · Medium · 2 upvotes · 0 comments - [Potential Consequences of Using Postgres as a Job Queue](https://daily.dev/posts/potential-consequences-of-using-postgres-as-a-job-queue-kcv0c9q1v) · Lobsters · 70 upvotes · 0 comments --- Tags: [#postgresql](https://daily.dev/tags/postgresql), [#distributed-systems](https://daily.dev/tags/distributed-systems), [#workflow-orchestration](https://daily.dev/tags/workflow-orchestration) [View this post on daily.dev](https://daily.dev/posts/implementing-durable-workflows-on-postgres-without-an-external-orchestrator-hn7fsiivi) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Implementing Durable Workflows on Postgres Without an External Orchestrator","url":"https://daily.dev/posts/implementing-durable-workflows-on-postgres-without-an-external-orchestrator-hn7fsiivi","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/implementing-durable-workflows-on-postgres-without-an-external-orchestrator-hn7fsiivi"},"datePublished":"2026-09-14T11:06:48.433Z","dateModified":"2026-09-14T11:07:13.424Z","description":"Building Kestrel Workflows, a durable-execution system for automating incident response and CI/CD, led to using plain Postgres instead of an external...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/c469e089739c9b00cfe27182fc13e72c?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/c469e089739c9b00cfe27182fc13e72c?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"InfoQ","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"InfoQ","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/afc3bced3e1e4b188dd9127017a60e0c","url":"https://daily.dev/sources/infoq"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/implementing-durable-workflows-on-postgres-without-an-external-orchestrator-hn7fsiivi","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"postgresql,distributed-systems,workflow-orchestration","timeRequired":"PT12M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"InfoQ","item":"https://daily.dev/sources/infoq"},{"@type":"ListItem","position":3,"name":"Implementing Durable Workflows on Postgres Without an External Orchestrator"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/implementing-durable-workflows-on-postgres-without-an-external-orchestrator-hn7fsiivi#faq","mainEntity":[{"@type":"Question","name":"How does SELECT FOR UPDATE SKIP LOCKED prevent two workers from processing the same queue row in Postgres?","acceptedAnswer":{"@type":"Answer","text":"It locks the rows a worker selects and instructs any other concurrent worker to skip past those locked rows instead of blocking on them, so two workers polling the same table simultaneously can never claim the same row. Combined with a short transaction that flips the row's status and writes a lease, this gives exactly-once dispatch without a message broker, leader election, or external lock service. Developers weighing queue designs can compare Postgres-based patterns like this one on daily.dev before picking an architecture."}},{"@type":"Question","name":"How can I implement idempotent step checkpoints for a workflow engine using Postgres constraints instead of application code?","acceptedAnswer":{"@type":"Answer","text":"Key a checkpoints table by a composite primary key such as (execution_id, step_id), then insert results with ON CONFLICT DO NOTHING so the unique constraint blocks duplicate writes. If a recovering worker reruns a completed step, it reads the existing row and returns the prior output instead of repeating the side effect, letting the database rather than application logic enforce idempotency. Anyone designing crash-safe pipelines can dig into idempotency patterns like this through daily.dev."}},{"@type":"Question","name":"What is a lease-and-sweeper pattern for recovering crashed workers in a Postgres-backed job queue?","acceptedAnswer":{"@type":"Answer","text":"Workers that own a running row periodically extend a lease_expires timestamp via heartbeat updates, and a separate sweeper query resets any row whose status is still 'running' but whose lease has expired back to 'enqueued' with no owner. A healthy worker then reclaims the row and resumes from its last completed checkpoint, with lease duration trading off wasted work against recovery delay. Teams building crash-recovery logic for job queues can find implementation patterns like this via daily.dev."}}]} ```