--- title: You Could Have Come Up with Kimi Delta Attention | daily.dev description: A derivation-first walkthrough of the DeltaNet family of linear attention mechanisms, culminating in Kimi Delta Attention (KDA). Starting from vanilla linear... canonical: https://daily.dev/posts/you-could-have-come-up-with-kimi-delta-attention-lrfzt6svl twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: You Could Have Come Up with Kimi Delta Attention | daily.dev og:description: A derivation-first walkthrough of the DeltaNet family of linear attention mechanisms, culminating in Kimi Delta Attention (KDA). Starting from vanilla linear... og:url: https://daily.dev/posts/you-could-have-come-up-with-kimi-delta-attention-lrfzt6svl og:image: https://api.daily.dev/og/posts/lRFzT6svL.png og:image:alt: You Could Have Come Up with Kimi Delta Attention og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # You Could Have Come Up with Kimi Delta Attention **[Hacker News](https://daily.dev/sources/hn)** · 8 min read · 0 upvotes · 0 comments ## Summary A derivation-first walkthrough of the DeltaNet family of linear attention mechanisms, culminating in Kimi Delta Attention (KDA). Starting from vanilla linear attention's fixed-size recurrent state, the post motivates each successive improvement: DeltaNet's targeted delta-rule write, Gated DeltaNet's scalar forgetting gate, and finally KDA's per-key-channel retention vector (a diagonal matrix). The post also covers both the token-by-token recurrent implementation and the chunkwise formulation that reorganizes the same recurrence into matrix multiplications for efficient training on tensor cores, including pseudocode for the chunkwise forward pass. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Community take How the wider developer community reacted, aggregated from 3 discussions and 141 comments across hackernews, lobsters (as of 2026-07-28). **TL;DR:** The community is largely amused and self-deprecating about the audacious title — most commenters freely admit they could not have come up with Kimi Delta Attention — while side discussions about notation (bra-ket vs. standard), the nature of ML innovation, and whether the post is LLM-written add a mixed but good-natured tone. **Sentiment:** 35% positive · 45% mixed · 20% skeptical **The case for** - The math/physics notation toggle is widely praised as a genuinely useful UX feature. - Several commenters note that bra-ket notation actually makes the shapes and operations clearer once you're familiar with it. - The incremental derivation style is appreciated for making the algorithm feel more motivated than a raw paper would. **The pushback** - The title 'You Could Have Come Up With Kimi Delta Attention' is seen as misleading or even insulting, since the content is inaccessible to most readers including many ML practitioners. - Multiple commenters suspect the post is LLM-generated based on its prose style. - The article is criticized for never explicitly defining key variables (q, k, S), making it opaque without prior domain knowledge. - Some argue the innovation required access to massive compute, not just clever math, so the 'you could have' framing is fundamentally disingenuous. **By community** - hackernews (mixed): Mostly good-natured self-deprecating humor about the title's hubris, with genuine side debates about notation, ML innovation, and possible LLM authorship. - lobsters (mixed): No comments available to assess sentiment. **Hottest debate:** Whether the 'you could have come up with it' framing is a legitimate pedagogical device or an insulting mismatch between the article's target audience and its actual accessibility. **Open questions** - Is the incremental linear-attention approach fundamentally limited compared to full quadratic attention for capturing meaning at scale? - Does bra-ket notation genuinely aid understanding for non-physicists, or does it add unnecessary barrier? - Was the post written with significant LLM assistance? **Highlights** > "you could have..." is among the top insulting phrases used by maths-adjacent people. Others in that league are "it should now be obvious...", "it's abundantly clear...", "it can be easily shown that...", "this is nothing but..." etc. The rest of us reading this are like, holy batman, what the fuck was that?! > — [penguin\_booze on hackernews · 5 comments](https://news.ycombinator.com/item?id=49087637) > Math educators like Grant Sanderson (3blue1brown) use it in a very specific way: the goal of a mathematical explanation is to make the learner feel like they could have come up with something. And a really good mathematical communicator can absolutely do that. A piece like this which uses it in a headline but in no way makes an average reader feel like they could have come up with it is just badly misjudging how good of an explanation it is. > — [jameshart on hackernews · 1 comments](https://news.ycombinator.com/item?id=49087944) > Doubleword AI is conducting a classic textbook marketing trick called newsjacking. Writing a detailed technical post behind the news of Kimi K3 and KDA algorithm with an audacious title like "You Could Have Invent Breakthrough It too" they are pre-filtering out the ones who couldn't comprehend with quick read (myself included) and attracting the ones who agreed with the blog post. At the end with a strong CTA to promoting their 10x cheapter open weight model AI inference and hiring too. Good job Doubleword, I see what you are doing there. > — [devy on hackernews · 2 comments](https://news.ycombinator.com/item?id=49087996) > lol - co-founder of Doubleword here. honoured you think we have sophisticated enough marketing to 'newsjack'. What actually happened is my cofounder wrote it over the weekend because he's a mega-nerd and put it live yesterday. I hadn't even read it until I saw this hacker news thread lol We're just a group of guys and gals who like inference! > — [mezark on hackernews](https://news.ycombinator.com/item?id=49088490) > You could've also came up with Newton's laws. After all, they look trivial in retrospect. But, there's an important lesson in a story about balancing an egg here that can teach us something. Filippo Brunelleschi said he could build the large dome for the church that had stood unfinished for a century. Skeptical, other's demanded he'd explain how. He refused. Instead he challenged everyone to balance an egg on its tip. Nobody could do it. He then demonstrated by lightly tapping the egg on the table, flattening the tip, making it stand. "Anyone could've done that! You never said we could break the egg!". And that's the point. Anyone could've done it. But nobody did. Nobody thought 'outside the box'. And likewise, his solution to building the dome is as simple, and as ingenious. It's called Egg of Columbus. (there's a similar story about Columbus that's more famous, but apparently fictitious). It teaches us that hindsight is 20/20. > — [MrFiskarBengt on hackernews](https://news.ycombinator.com/item?id=49088559) **Source threads** - [hackernews](https://news.ycombinator.com/item?id=49085909) · 130 points · 141 comments - [hackernews](https://news.ycombinator.com/item?id=49068166) · 3 points · 0 comments - [lobsters](https://lobste.rs/s/jjap0n/you_could_have_come_up_with_kimi_delta) · 1 points · 0 comments ## Similar posts on daily.dev - [Designing Hardware-Aware Algorithms with Kimi Linear: Kimi Delta Attention](https://daily.dev/posts/designing-hardware-aware-algorithms-with-kimi-linear-kimi-delta-attention-8ir1lhbc7) · DigitalOcean Community · 3 upvotes · 0 comments - [Kimi Linear: An Expressive, Efficient Attention Architecture](https://daily.dev/posts/kimi-linear-an-expressive-efficient-attention-architecture-znopemqzl) · Hacker News · 3 upvotes · 0 comments - [MoonshotAI/Kimi-Linear](https://daily.dev/posts/moonshotai-kimi-linear-penxtotqu) · Hacker News · 21 upvotes · 1 comments --- Tags: [#machine-learning](https://daily.dev/tags/machine-learning), [#deep-learning](https://daily.dev/tags/deep-learning) [View this post on daily.dev](https://daily.dev/posts/you-could-have-come-up-with-kimi-delta-attention-lrfzt6svl) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"You Could Have Come Up with Kimi Delta Attention","url":"https://daily.dev/posts/you-could-have-come-up-with-kimi-delta-attention-lrfzt6svl","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/you-could-have-come-up-with-kimi-delta-attention-lrfzt6svl"},"datePublished":"2026-07-28T17:06:54.413Z","dateModified":"2026-07-28T19:34:30.146Z","description":"A derivation-first walkthrough of the DeltaNet family of linear attention mechanisms, culminating in Kimi Delta Attention (KDA). Starting from vanilla linear...","image":"https://media.daily.dev/image/upload/s--1KxV4ohY--/f_auto/v1722860400/public/Placeholder%2007","thumbnailUrl":"https://media.daily.dev/image/upload/s--1KxV4ohY--/f_auto/v1722860400/public/Placeholder%2007","isAccessibleForFree":true,"articleSection":"Hacker News","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Hacker News","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/hn","url":"https://daily.dev/sources/hn"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/you-could-have-come-up-with-kimi-delta-attention-lrfzt6svl","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"machine-learning,deep-learning","timeRequired":"PT8M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Hacker News","item":"https://daily.dev/sources/hn"},{"@type":"ListItem","position":3,"name":"You Could Have Come Up with Kimi Delta Attention"}]} ```