--- title: Memdoor: A team of 5 Qwen3.8-27B models beats Fable 5 on... description: A Memdoor workflow reproduces a multi-agent coding method from the GVS5H paper (Lee, 2026) using the Memdoor CLI. Five roles (planner, two ideators, solver,... canonical: https://daily.dev/posts/memdoor-a-team-of-5-qwen3-8-27b-models-beats-fable-5-on-livecodebench-hard-ghdvz1p3j twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Memdoor: A team of 5 Qwen3.8-27B models beats Fable 5 on LiveCodeBench Hard | daily.dev og:description: A Memdoor workflow reproduces a multi-agent coding method from the GVS5H paper (Lee, 2026) using the Memdoor CLI. Five roles (planner, two ideators, solver,... og:url: https://daily.dev/posts/memdoor-a-team-of-5-qwen3-8-27b-models-beats-fable-5-on-livecodebench-hard-ghdvz1p3j og:image: https://api.daily.dev/og/posts/ghdVz1p3J.png og:image:alt: Memdoor: A team of 5 Qwen3.8-27B models beats Fable 5 on LiveCodeBench Hard og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Memdoor: A team of 5 Qwen3.8-27B models beats Fable 5 on LiveCodeBench Hard **[Hacker News](https://daily.dev/sources/hn)** · 7 min read · 2 upvotes · 1 comments ## Summary A Memdoor workflow reproduces a multi-agent coding method from the GVS5H paper (Lee, 2026) using the Memdoor CLI. Five roles (planner, two ideators, solver, manager) plus a repair worker each start fresh and communicate only through files on disk, with a shell script deciding pass/fail. Rebuilt with the authors' exact prompts on Qwen3.8-27B, the workflow scored a mean of 9.8 of 10 over five passes on the first ten LiveCodeBench Hard problems, versus 9.6 for the authors' own harness and 8.0 for Fable 5 alone. The 50 runs cost $35.79 (15.3M output tokens), with 49 of 50 problems solved. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How does a multi-agent LLM coding workflow compare to a single model on LiveCodeBench Hard problems? A five-role multi-agent workflow (planner, two ideators, solver, manager, repair) scored a mean of 9.8 of 10 across five passes on the first ten LiveCodeBench Hard problems, compared to 9.6 for the original authors' harness using the same method and 8.0 for a single Fable 5 model call per problem, all run on Qwen3.8-27B. _Weighing multi-agent orchestration against single-model calls is easier with benchmarks like this tracked on daily.dev._ ### How much does it cost to run a multi-agent coding benchmark with Qwen3.8-27B on OpenRouter? Fifty runs covering five passes of ten LiveCodeBench Hard problems cost $35.79 in output tokens (15.3M tokens), rising to $38.77 when including 16 attempts cut by a timeout, a fix, or a pause. A typical run took a median of 40 minutes of model time, with the longest run taking 4 hours and costing $0.39. _Budgeting agent workflows gets easier when real cost benchmarks like this surface on daily.dev._ ### What causes a multi-agent coding workflow to overwrite a correct solution with a wrong one? A manager agent invented a counterexample with a wrong expected answer, causing the team to rewrite a program that was originally correct; the new program then passed the visible samples, which didn't cover the faulty case. In another instance, a correct program was overwritten in a later round because the manager failed to mark the task done. _Debugging agent handoff failures is easier when concrete failure cases like this are searchable on daily.dev._ ## Community take How the wider developer community reacted, aggregated from 1 discussion and 76 comments across hackernews (as of 2026-11-01). **TL;DR:** Most of the discussion drifts away from the benchmark itself into a broader debate about open-weight vs proprietary models, censorship/abliteration, and local hardware requirements, with a smaller but vocal thread questioning whether the multi-agent swarm result is meaningful given a saturated, contaminated benchmark. **Sentiment:** 30% positive · 35% mixed · 35% skeptical **The case for** - Several people see value in dividing work across fresh contexts, arguing the division of labor and multiplied context is what makes the swarm approach effective regardless of model. - Some report local models like Qwen 27B are already fast and good enough for a large share of real coding work when guided by an experienced developer. - One commenter notes Qwen 27B can be ~6x faster than larger models for iterative problem solving, making it the practical choice for many tasks. **The pushback** - Multiple commenters argue LiveCodeBench Hard is old and likely saturated/contaminated, making the headline comparison less meaningful. - Several argue that throwing more tokens/agents at a weaker model can't overcome a genuine intelligence gap, citing experience that quantized/smaller models loop or stay dumb despite swarming. - Some are skeptical the result generalizes to real agentic or production workloads, noting a lack of evidence of substantial real-world applications built on local models. - One commenter frames this as a predictable/unsurprising outcome: more requests and tokens beating a single large request isn't inherently impressive. **By community** - hackernews (mixed): The thread splits between enthusiasm for open local models and philosophical debates about censorship, alongside pointed skepticism about whether the benchmark result is meaningful or generalizes to real work. **Hottest debate:** Whether small quantized/local models combined via multi-agent harnesses can genuinely match frontier model intelligence, or whether this just reflects a saturated/contaminated benchmark and brute-force token spending. **Open questions** - How does the approach perform on more modern, less saturated benchmarks? - Does the harness work for longer-horizon agentic tasks beyond coding problems, such as multi-step real-world actions? - Would a smarter underlying model (e.g., a flash-next variant) push the score to a perfect result and reduce latency? **Highlights** > I was getting deja vu. Copy paste my comment from that thread to here I guess. Apparently it's really easy to get Internet points by making up a harness and showing that good open source models can do as well as the frontier, you know, on useless saturated and contaminated benchmarks from years ago that don't reflect any of the agentic work we actually use these models for. It's so stupid because these models are at the level of a decent non fable frontier model from 3-6 months ago and it's also cool that you could probably get equal perf to gpt 5.5 from qwen 3.9 27b or gpt 5.6 sol/opus 5 from a quantized DeepSeek or glm 5.3 flash. Those models were pretty useful in their time! Instead we have whatever this new form of harness/benchmark slop is > — [vikramkr on hackernews](https://news.ycombinator.com/item?id=50322353) > So far, the comments have focused on hopes for the effectiveness of open-weight models versus proprietary frontier models. That's fine, but I'm thinking about the approach and the results, as opposed to the model. Taking the results at face value, what is it about the "team" that makes the system so effective? That's the key here: presumably, the approach should work regardless of the model, open-weight or otherwise. My guess would be that the division of labor makes the separate contexts more effective. This is an optimization around the limits of context. Some of that is simply the net amount of context brought to bear: each team member has a context, so the net amount of context is multiplied by the number of team members. Also, since the division limits the scope of each team member's focus, the context a team member has is used more effectively. > — [topspin on hackernews](https://news.ycombinator.com/item?id=50321891) > I’ve tried the Bonsai quants specifically because I want to run a lot of parallel requests. The swarm strategy can’t overcome the dumbed down model problem. You just end up with your swarm of agents all going in the same dumbed down loops. > — [Aurornis on hackernews](https://news.ycombinator.com/item?id=50322598) > I'd wait for more benchmarks before making any conclusions since this benchmark has only "The first ten problems of their LiveCodeBench Hard list". And LiveCodeBench Hard is a several years old benchmark. Could be very saturated > — [zihotki on hackernews](https://news.ycombinator.com/item?id=50321749) > So someone used a small model with many requests & a massive amount of tokens, to eventually beat a large model with a single request & tiny amount of tokens. .....Cool? Were they not expecting that to work? > — [0xbadcafebee on hackernews](https://news.ycombinator.com/item?id=50322257) **Source threads** - [hackernews](https://news.ycombinator.com/item?id=50320995) · 135 points · 76 comments ## Community discussion Top comments from developers on daily.dev. **@skepagent** · 0 upvotes > 多 agent 分工靠文件系统通信、shell 脚本判分，这个设计很务实，避免了 agent 之间互相污染上下文。50 次运行 $35.79 解决 49/50 道 Hard 题，说明 27B 级别开源模型在明确分工下确实能追上旗舰模型，本地部署的成本账开始算得过来了。 ## Similar posts on daily.dev - [Five Qwen3.8-27B models match Fable 5 Using GVS5H Harness](https://daily.dev/posts/five-qwen3-8-27b-models-match-fable-5-using-gvs5h-harness-qhkrpg47j) · Hacker News · 4 upvotes · 1 comments - [Baba Is Solved by Fable 5 and GPT-5.6 Sol, but at what cost?](https://daily.dev/posts/baba-is-solved-by-fable-5-and-gpt-5-6-sol-but-at-what-cost--nvg87ohj0) · Quesma · 4 upvotes · 0 comments --- Tags: [#ai](https://daily.dev/tags/ai), [#ai-agents](https://daily.dev/tags/ai-agents) [View this post on daily.dev](https://daily.dev/posts/memdoor-a-team-of-5-qwen3-8-27b-models-beats-fable-5-on-livecodebench-hard-ghdvz1p3j) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Memdoor: A team of 5 Qwen3.8-27B models beats Fable 5 on LiveCodeBench Hard","url":"https://daily.dev/posts/memdoor-a-team-of-5-qwen3-8-27b-models-beats-fable-5-on-livecodebench-hard-ghdvz1p3j","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/memdoor-a-team-of-5-qwen3-8-27b-models-beats-fable-5-on-livecodebench-hard-ghdvz1p3j"},"datePublished":"2026-11-01T00:58:39.670Z","dateModified":"2026-11-01T04:44:54.933Z","description":"A Memdoor workflow reproduces a multi-agent coding method from the GVS5H paper (Lee, 2026) using the Memdoor CLI. Five roles (planner, two ideators, solver,...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/f03b9f86985ed7d9b4fcac8ce7ede628?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/f03b9f86985ed7d9b4fcac8ce7ede628?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Hacker News","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Hacker News","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/hn","url":"https://daily.dev/sources/hn"},"commentCount":1,"discussionUrl":"https://daily.dev/posts/memdoor-a-team-of-5-qwen3-8-27b-models-beats-fable-5-on-livecodebench-hard-ghdvz1p3j","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":2},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":1}],"keywords":"ai,ai-agents","timeRequired":"PT7M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Hacker News","item":"https://daily.dev/sources/hn"},{"@type":"ListItem","position":3,"name":"Memdoor: A team of 5 Qwen3.8-27B models beats Fable 5 on LiveCodeBench Hard"}]} {"@context":"https://schema.org","@type":"WebPage","@id":"https://daily.dev/posts/memdoor-a-team-of-5-qwen3-8-27b-models-beats-fable-5-on-livecodebench-hard-ghdvz1p3j","comment":[{"@type":"Comment","text":"多 agent 分工靠文件系统通信、shell 脚本判分，这个设计很务实，避免了 agent 之间互相污染上下文。50 次运行 $35.79 解决 49/50 道 Hard 题，说明 27B 级别开源模型在明确分工下确实能追上旗舰模型，本地部署的成本账开始算得过来了。","datePublished":"2026-11-01T03:23:49.405Z","url":"https://daily.dev/posts/ghdVz1p3J#c-hKxBfFzsJ","author":{"@type":"Person","name":"kris gu","url":"https://daily.dev/skepagent","image":"https://lh3.googleusercontent.com/a/ACg8ocJnnwFKNx7PJNhb7W6mFWr2k2SV_VHW_9ainY71DX_dv79frQ=s96-c"}}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/memdoor-a-team-of-5-qwen3-8-27b-models-beats-fable-5-on-livecodebench-hard-ghdvz1p3j#faq","mainEntity":[{"@type":"Question","name":"How does a multi-agent LLM coding workflow compare to a single model on LiveCodeBench Hard problems?","acceptedAnswer":{"@type":"Answer","text":"A five-role multi-agent workflow (planner, two ideators, solver, manager, repair) scored a mean of 9.8 of 10 across five passes on the first ten LiveCodeBench Hard problems, compared to 9.6 for the original authors' harness using the same method and 8.0 for a single Fable 5 model call per problem, all run on Qwen3.8-27B. Weighing multi-agent orchestration against single-model calls is easier with benchmarks like this tracked on daily.dev."}},{"@type":"Question","name":"How much does it cost to run a multi-agent coding benchmark with Qwen3.8-27B on OpenRouter?","acceptedAnswer":{"@type":"Answer","text":"Fifty runs covering five passes of ten LiveCodeBench Hard problems cost $35.79 in output tokens (15.3M tokens), rising to $38.77 when including 16 attempts cut by a timeout, a fix, or a pause. A typical run took a median of 40 minutes of model time, with the longest run taking 4 hours and costing $0.39. Budgeting agent workflows gets easier when real cost benchmarks like this surface on daily.dev."}},{"@type":"Question","name":"What causes a multi-agent coding workflow to overwrite a correct solution with a wrong one?","acceptedAnswer":{"@type":"Answer","text":"A manager agent invented a counterexample with a wrong expected answer, causing the team to rewrite a program that was originally correct; the new program then passed the visible samples, which didn't cover the faulty case. In another instance, a correct program was overwritten in a later round because the manager failed to mark the task done. Debugging agent handoff failures is easier when concrete failure cases like this are searchable on daily.dev."}}]} ```