--- title: "The Secrets Behind Plumber: Managing Thousands of Pipelines at Semaphore" url: https://daily.dev/posts/the-secrets-behind-plumber-managing-thousands-of-pipelines-at-semaphore-bldkiidbo source_url: https://semaphore.io/blog/plumber-semaphore type: article source: "Semaphore" published: 2025-06-04T09:23:23.878Z updated: 2025-06-04T09:23:46.944Z tags: ["kubernetes", "cicd", "postgresql", "elixir"] reading_time: 4 upvotes: 2 comments: 0 language: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # The Secrets Behind Plumber: Managing Thousands of Pipelines at Semaphore **[Semaphore](https://daily.dev/sources/semaphore)** · 4 min read · 2 upvotes · 0 comments ## Summary Semaphore's Plumber engine orchestrates CI/CD pipelines using a finite state machine architecture built in Elixir. The system manages pipeline execution through distinct states (initializing, pending, running, done) with dedicated Looper processes handling each state. Key components include PostgreSQL for transactional integrity, RabbitMQ for service communication, and Kubernetes for deployment. The architecture provides fault tolerance through supervision trees, recovery mechanisms for stuck pipelines, and horizontal scalability across distributed agents. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Similar posts on daily.dev - [Don’t just attend KubeCon \+ CloudNativeCon, Merge Forward your experience\!](https://daily.dev/posts/don-t-just-attend-kubecon-cloudnativecon-merge-forward-your-experience--l0rpp73x8) · CNCF · 0 upvotes · 0 comments - [Announcing H2 2026 KCDs](https://daily.dev/posts/announcing-h2-2026-kcds-m96goajm1) · CNCF · 1 upvotes · 0 comments - [Two months of Open Community Groups](https://daily.dev/posts/two-months-of-open-community-groups-asf52zhbs) · CNCF · 0 upvotes · 0 comments --- Tags: [#kubernetes](https://daily.dev/tags/kubernetes), [#cicd](https://daily.dev/tags/cicd), [#postgresql](https://daily.dev/tags/postgresql), [#elixir](https://daily.dev/tags/elixir) [View this post on daily.dev](https://daily.dev/posts/the-secrets-behind-plumber-managing-thousands-of-pipelines-at-semaphore-bldkiidbo)