--- title: Rewind VM: a flaky build you only catch once | daily.dev description: Rewind VM is a new open-source tool that makes the execution of a Nix build, test suite, or any Linux command fully deterministic by running it inside a KVM... canonical: https://daily.dev/posts/rewind-vm-a-flaky-build-you-only-catch-once-a2u3ugkt4 twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Rewind VM: a flaky build you only catch once | daily.dev og:description: Rewind VM is a new open-source tool that makes the execution of a Nix build, test suite, or any Linux command fully deterministic by running it inside a KVM... og:url: https://daily.dev/posts/rewind-vm-a-flaky-build-you-only-catch-once-a2u3ugkt4 og:image: https://api.daily.dev/og/posts/A2u3UgKT4.png og:image:alt: Rewind VM: a flaky build you only catch once og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Rewind VM: a flaky build you only catch once **[Farid Zakaria’s Blog](https://daily.dev/sources/fzakaria)** · 11 min read · 0 upvotes · 0 comments ## Summary Rewind VM is a new open-source tool that makes the execution of a Nix build, test suite, or any Linux command fully deterministic by running it inside a KVM virtual machine whose thread schedule becomes a controllable input. It replays races and deadlocks exactly, lets you perturb schedules to surface rare interleavings, fork a run at any step under a different schedule, and attach gdb or drop into a shell at any point in the recorded run without altering it. The post walks through a dining philosophers deadlock example end-to-end: reproducing it, narrowing the failing schedule window, inspecting locked mutexes with gdb, forking under alternate schedules, exporting a replayable .rwd file, and verifying a fix with rewind check --all. It also lists real bugs found in Nix, nixd, and jujutsu, plus current limitations like single-vCPU execution, AMD-only exact clocks, and no networking beyond loopback. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How can I reproduce a flaky test failure caused by a thread scheduling race that only happens on CI? Rewind VM runs the build or test suite inside a deterministic KVM virtual machine where the thread schedule itself becomes a recorded input, so the exact same run can be replayed, scrubbed through, or forked under a different interleaving. The rewind check command perturbs schedules to surface rare failures, and a failing run can be exported as a .rwd file and replayed bit-for-bit on another machine. _Developers chasing one-in-a-hundred CI failures can find deeper concurrency debugging approaches like this through daily.dev._ ### What causes the dining philosophers deadlock in a naive multithreaded C implementation and how is it fixed? Deadlock occurs when every philosopher thread picks up its left fork mutex before trying for the right one, so all five forks get taken and every thread waits on a neighbor who is also waiting. The fix orders lock acquisition by fork number instead of position (always locking the lower-numbered fork first), which breaks the circular wait and eliminated the deadlock across 64 perturbed schedules in testing. _Anyone debugging mutex ordering bugs can keep up with concurrency patterns and tooling like this via daily.dev._ ## Similar posts on daily.dev - [Nix wrote half of my debugger](https://daily.dev/posts/nix-wrote-half-of-my-debugger-gnc47givc) · Farid Zakaria’s Blog · 0 upvotes · 0 comments - [Your Logs Are Lying to You — So I Built an Open-Source LogRocket Alternative That Runs on a $6 VPS](https://daily.dev/posts/your-logs-are-lying-to-you-so-i-built-an-open-source-logrocket-alternative-that-runs-on-a-6-vps-x5p4shdwz) · Medium · 0 upvotes · 0 comments - [Docker versus Nix: The quest for true reproducibility](https://daily.dev/posts/docker-versus-nix-the-quest-for-true-reproducibility-r1crtumla) · The New Stack · 1 upvotes · 0 comments - [Exports, checkpoints or branches: what should you reuse? · SmolBox](https://daily.dev/posts/exports-checkpoints-or-branches-what-should-you-reuse-smolbox-ilupevugm) · ElixirStatus · 0 upvotes · 0 comments - [OpenAI Fixes 18-Year-Old GNU libunwind Bug by Treating Crash Debugging Like Epidemiology](https://daily.dev/posts/openai-fixes-18-year-old-gnu-libunwind-bug-by-treating-crash-debugging-like-epidemiology-laln47lxr) · InfoQ · 0 upvotes · 0 comments --- [View this post on daily.dev](https://daily.dev/posts/rewind-vm-a-flaky-build-you-only-catch-once-a2u3ugkt4) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Rewind VM: a flaky build you only catch once","url":"https://daily.dev/posts/rewind-vm-a-flaky-build-you-only-catch-once-a2u3ugkt4","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/rewind-vm-a-flaky-build-you-only-catch-once-a2u3ugkt4"},"datePublished":"2026-10-04T05:21:21.607Z","dateModified":"2026-10-04T05:21:46.289Z","description":"Rewind VM is a new open-source tool that makes the execution of a Nix build, test suite, or any Linux command fully deterministic by running it inside a KVM...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/efe971f43c13f618301674b99f4c7f84?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/efe971f43c13f618301674b99f4c7f84?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Farid Zakaria’s Blog","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Farid Zakaria’s Blog","logo":"https://media.daily.dev/image/upload/s--YqqvRt90--/f_auto/v1723789280/logos/fzakaria","url":"https://daily.dev/sources/fzakaria"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/rewind-vm-a-flaky-build-you-only-catch-once-a2u3ugkt4","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"","timeRequired":"PT11M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Farid Zakaria’s Blog","item":"https://daily.dev/sources/fzakaria"},{"@type":"ListItem","position":3,"name":"Rewind VM: a flaky build you only catch once"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/rewind-vm-a-flaky-build-you-only-catch-once-a2u3ugkt4#faq","mainEntity":[{"@type":"Question","name":"How can I reproduce a flaky test failure caused by a thread scheduling race that only happens on CI?","acceptedAnswer":{"@type":"Answer","text":"Rewind VM runs the build or test suite inside a deterministic KVM virtual machine where the thread schedule itself becomes a recorded input, so the exact same run can be replayed, scrubbed through, or forked under a different interleaving. The rewind check command perturbs schedules to surface rare failures, and a failing run can be exported as a .rwd file and replayed bit-for-bit on another machine. Developers chasing one-in-a-hundred CI failures can find deeper concurrency debugging approaches like this through daily.dev."}},{"@type":"Question","name":"What causes the dining philosophers deadlock in a naive multithreaded C implementation and how is it fixed?","acceptedAnswer":{"@type":"Answer","text":"Deadlock occurs when every philosopher thread picks up its left fork mutex before trying for the right one, so all five forks get taken and every thread waits on a neighbor who is also waiting. The fix orders lock acquisition by fork number instead of position (always locking the lower-numbered fork first), which breaks the circular wait and eliminated the deadlock across 64 perturbed schedules in testing. Anyone debugging mutex ordering bugs can keep up with concurrency patterns and tooling like this via daily.dev."}}]} ```