--- title: Astra can read and write weights to improve Qwen | daily.dev description: Frontier LLMs can read and write neural network weights directly, without gradient descent or training loops, a capability the authors call Weight-Space... canonical: https://daily.dev/posts/astra-can-read-and-write-weights-to-improve-qwen-swdnk0yoe twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Astra can read and write weights to improve Qwen | daily.dev og:description: Frontier LLMs can read and write neural network weights directly, without gradient descent or training loops, a capability the authors call Weight-Space... og:url: https://daily.dev/posts/astra-can-read-and-write-weights-to-improve-qwen-swdnk0yoe og:image: https://api.daily.dev/og/posts/SWDNk0yoe.png og:image:alt: Astra can read and write weights to improve Qwen og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Astra can read and write weights to improve Qwen **[Hacker News](https://daily.dev/sources/hn)** · 15 min read · 0 upvotes · 0 comments ## Summary Frontier LLMs can read and write neural network weights directly, without gradient descent or training loops, a capability the authors call Weight-Space Programming. GPT-6 Astra identifies what a network was trained to do just from raw weights, and writes small game-playing networks from scratch that beat PPO agents trained for 100 million steps. Asked to fix Qwen3.5-0.8B's reasoning loops, Astra edits just 0.016% of its weights, raising MATH-500 from 7.9% to 48.1% and IFEval from 21.0% to 62.3%. On Gemma 3 270M, which rarely loops, the same approach failed to improve math scores, showing it removes obstacles rather than adding new abilities. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How much did editing weights improve Qwen3.5-0.8B's math performance without any training? GPT-6 Astra edited just 0.016% of Qwen3.5-0.8B's weights (131,072 out of 0.8 billion) without gradients or training, raising MATH-500 accuracy from 7.9% to 48.1% and GSM8K from 13.8% to 67.1%. The fix addressed a reasoning-loop failure mode, and the improvement generalized to IFEval instruction-following, which rose from 21.0% to 62.3%. _Developers evaluating weight-level model fixes versus retraining can follow emerging techniques like this on daily.dev._ ### Can an LLM write a working neural network from scratch without training it? Yes, GPT-6 Astra wrote game-playing neural networks from scratch in a single zero-shot attempt, with no training or environment feedback, and these networks beat PPO agents trained for 100 million steps in all three MinAtar games tested. Against PPO trained for 500 million steps, Astra's networks scored 20 times as much in Breakout and came close in Asterix. _Anyone tracking alternatives to gradient-based training can keep up with results like this on daily.dev._ ### Why did editing neural network weights with an LLM fail to improve Gemma 3 270M's math accuracy? The edits failed because Gemma 3 270M's errors were ordinary reasoning mistakes, such as wrong arithmetic and invalid algebraic steps, with no single weight component causing them that could be switched off. Unlike Qwen3.5-0.8B, which looped on 93% of answers, Gemma rarely looped (8.5%), so there was no isolated failure to fix, and no edit moved its MATH-500 score by more than about a point. _Teams weighing weight-editing against retraining for model fixes can track findings like this on daily.dev._ ## Community take How the wider developer community reacted, aggregated from 1 discussion and 240 comments across hackernews (as of 2026-11-02). **TL;DR:** Reactions are a mix of genuine astonishment that a frontier model can intuitively read and hand-edit raw weights, and pushback that the Qwen 'fix' was really just removing a known looping/refusal-style activation pattern rather than teaching new math ability. **Sentiment:** 40% positive · 45% mixed · 15% skeptical **The case for** - Many find it genuinely surprising that a model shows apparent intuitive understanding of raw weights with no training data humans could have produced on that exact task. - Several note the qualitative leap from prior human/LLM-assisted interpretability tools (like activation steering or abliteration) to a model doing this unaided in one pass. - Some see this as a plausible, if early, step toward richer recursive self-improvement than code-based retraining loops. **The pushback** - Multiple commenters argue the Qwen improvement is overstated: it mainly removed a known looping/refusal-style direction in activation space rather than adding real math ability. - Several point out a large (likely trillion-parameter) model editing a much smaller one isn't evidence it can improve itself or models of similar size. - Some suspect the capability is less emergent 'intuition' than a predictable result of training on ML research, interpretability literature, or even weights from related larger Qwen versions. - A few are skeptical of the hype itself, noting similar weight-editing work (jailbreak/ablation, mechanistic interpretability) has been done by humans before. **By community** - hackernews (heated): Discussion splits sharply between people stunned by the apparent emergent weight-reading ability and others insisting it's an expected, explainable extension of known interpretability/ablation techniques rather than new knowledge creation. **Hottest debate:** Whether Astra's edits reflect genuine emergent 'understanding' of weights versus a predictable application of known interpretability tricks (like removing a refusal/looping direction) possibly aided by prior exposure to related model weights. **Open questions** - Is this capability trained explicitly by labs for ML/interpretability tasks, or does it emerge naturally from general reasoning ability? - Does the editing approach scale to models of similar or larger size than the editor, or only to much smaller target networks? - Can edits be made without degrading other capabilities, or does reliable generalization still require something equivalent to retraining? - How vulnerable is this to malicious use, e.g., hostile weight-patching of open-weight models without integrity checks? **Highlights** > This is insane. Astra can, without tooling, determine what games small networks are trained on, can edit the weights of networks directly to improve outcomes, and can hand edit a small qwen model to improve its math skills by like 5x. I'm shook. If you'd asked me this morning whether LLMs have native understanding of their weights that is less than human <-> neuron understanding, the same as human or more than human, I would have said "same to less" depending on definitions. No human can go edit a brain to become better at math. This is just very, very surprising to me and extremely interesting. I would guess this upsets quite a number of interp theories and philosophical takes on what's going on with these networks. > — [vessenes on hackernews · 22 comments](https://news.ycombinator.com/item?id=50341841) > I think the "taught qwen math" part is not exactly correct though, even if the post makes it sound like that. Qwen already knew math, but it was bugging out and getting into an infinite loop when it tried to solve the problems (85 of its first 96 answers hit the token limit). With numpy and pytorch, Astra found where the loop was happening and fixed it. They also tried the same with Gemma 3 270M, which doesn't loop and is just bad at math, and Astra couldn't improve it. Still impressive, but it's more in the realm of what I'd expect, given labs are probably training their models on ML research and interpretability work. > — [l9o on hackernews · 2 comments](https://news.ycombinator.com/item?id=50342593) > It's worth pointing out that this is a multi-trillion (probably) parameter model optimizing an 800 million parameter model. There is no indication that it can improve itself with this same methodology. > — [Wheen on hackernews · 1 comments](https://news.ycombinator.com/item?id=50342060) > LLMs have been used to investigate and improve LLMs for a long time. Drawing parallels to human brains and neurons isn't all that interesting other than as a philosophical abstraction. The work done in this experiment could have also been done by a sufficiently motivated human with enough time to explore, though obviously we would need more tools to visualize and track things. There have been a lot of interesting LLM introspection tools that allow you to peek into the model and see what's being activated and run experiments to trigger it. Are you familiar with abliterated or "uncensored" models where weights have been modified to avoid triggering refusal patterns? You take an LLM, create test cases that trigger the refusal pattern, observe weights involved in that response, then tweak those weights until the response is reduced. This is the same type of work, applied to a specific type of error that was impeding math within Qwen. That work can also be done with an LLM in a harness running the experiments. The amazing part here is that Astra was large enough to walk through the problem without other tooling doing it experimentally. So I don't think your analogy of a human modifying a neuron is quite applicable. LLM weights are not neurons and they've been inspectable, by humans or LLMs in a harness, for a long time. > — [Aurornis on hackernews](https://news.ycombinator.com/item?id=50342896) > I'm baffled this is surprising. This has been the goal and every data point on the trajectory has been leading here and beyond. Opus 5.5 was created by many research engineers using Opus 4.6 extensively. Opus 6 is being created by using even more intelligence and agents on Opus 5.5. They are approaching autonomy and Opus X+1 will soon be rapidly created autonomously from Opus X. Given we have practically zero interpretability into the models, and we have not solved alignment, they will recursively improve out of our control. The engineers in A.I. labs are actually generally scared of what's unfolding. They don't know at what point this RSI takeoff occurs and what will happen after. > — [reducesuffering on hackernews · 2 comments](https://news.ycombinator.com/item?id=50342304) **Source threads** - [hackernews](https://news.ycombinator.com/item?id=50341174) · 687 points · 240 comments ## Similar posts on daily.dev - [OpenAI’s new LLM exposes the secrets of how AI really works](https://daily.dev/posts/openai-s-new-llm-exposes-the-secrets-of-how-ai-really-works-iepupajzg) · MIT Technology Review · 0 upvotes · 0 comments - [6 Automatic Optimization Methods for LLM Systems](https://daily.dev/posts/6-automatic-optimization-methods-for-llm-systems-c17ntahzv) · Daily Dose of Data Science \| Avi Chawla \| Substack · 3 upvotes · 0 comments - [Can LLMs Be Computers?](https://daily.dev/posts/can-llms-be-computers--wuueagzci) · Hacker News · 3 upvotes · 0 comments --- Tags: [#ai](https://daily.dev/tags/ai), [#llm](https://daily.dev/tags/llm), [#reinforcement-learning](https://daily.dev/tags/reinforcement-learning) [View this post on daily.dev](https://daily.dev/posts/astra-can-read-and-write-weights-to-improve-qwen-swdnk0yoe) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Astra can read and write weights to improve Qwen","url":"https://daily.dev/posts/astra-can-read-and-write-weights-to-improve-qwen-swdnk0yoe","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/astra-can-read-and-write-weights-to-improve-qwen-swdnk0yoe"},"datePublished":"2026-11-02T19:11:24.015Z","dateModified":"2026-11-03T03:26:13.368Z","description":"Frontier LLMs can read and write neural network weights directly, without gradient descent or training loops, a capability the authors call Weight-Space...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/ffa30ca6a4260935ae08b2be732bd0c3?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/ffa30ca6a4260935ae08b2be732bd0c3?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Hacker News","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Hacker News","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/hn","url":"https://daily.dev/sources/hn"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/astra-can-read-and-write-weights-to-improve-qwen-swdnk0yoe","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"ai,llm,reinforcement-learning","timeRequired":"PT15M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Hacker News","item":"https://daily.dev/sources/hn"},{"@type":"ListItem","position":3,"name":"Astra can read and write weights to improve Qwen"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/astra-can-read-and-write-weights-to-improve-qwen-swdnk0yoe#faq","mainEntity":[{"@type":"Question","name":"How much did editing weights improve Qwen3.5-0.8B's math performance without any training?","acceptedAnswer":{"@type":"Answer","text":"GPT-6 Astra edited just 0.016% of Qwen3.5-0.8B's weights (131,072 out of 0.8 billion) without gradients or training, raising MATH-500 accuracy from 7.9% to 48.1% and GSM8K from 13.8% to 67.1%. The fix addressed a reasoning-loop failure mode, and the improvement generalized to IFEval instruction-following, which rose from 21.0% to 62.3%. Developers evaluating weight-level model fixes versus retraining can follow emerging techniques like this on daily.dev."}},{"@type":"Question","name":"Can an LLM write a working neural network from scratch without training it?","acceptedAnswer":{"@type":"Answer","text":"Yes, GPT-6 Astra wrote game-playing neural networks from scratch in a single zero-shot attempt, with no training or environment feedback, and these networks beat PPO agents trained for 100 million steps in all three MinAtar games tested. Against PPO trained for 500 million steps, Astra's networks scored 20 times as much in Breakout and came close in Asterix. Anyone tracking alternatives to gradient-based training can keep up with results like this on daily.dev."}},{"@type":"Question","name":"Why did editing neural network weights with an LLM fail to improve Gemma 3 270M's math accuracy?","acceptedAnswer":{"@type":"Answer","text":"The edits failed because Gemma 3 270M's errors were ordinary reasoning mistakes, such as wrong arithmetic and invalid algebraic steps, with no single weight component causing them that could be switched off. Unlike Qwen3.5-0.8B, which looped on 93% of answers, Gemma rarely looped (8.5%), so there was no isolated failure to fix, and no edit moved its MATH-500 score by more than about a point. Teams weighing weight-editing against retraining for model fixes can track findings like this on daily.dev."}}]} ```