--- title: Open-Weight AI Releases Are Coming Fast Right Now description: A wave of open-weight AI model releases is happening across text, image, audio, and other modalities. Hugging Face's Clément Delangue and AI commentator AK... canonical: https://daily.dev/posts/open-weight-ai-releases-are-coming-fast-right-now-ovyubagrj twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Open-Weight AI Releases Are Coming Fast Right Now | daily.dev og:description: A wave of open-weight AI model releases is happening across text, image, audio, and other modalities. Hugging Face's Clément Delangue and AI commentator AK... og:url: https://daily.dev/posts/open-weight-ai-releases-are-coming-fast-right-now-ovyubagrj og:image: https://api.daily.dev/og/posts/OVYubAGRj.png og:image:alt: Open-Weight AI Releases Are Coming Fast Right Now og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Open-Weight AI Releases Are Coming Fast Right Now **[Collections](https://daily.dev/sources/collections)** · 2 min read · 1 upvotes · 0 comments ## Summary A wave of open-weight AI model releases is happening across text, image, audio, and other modalities. Hugging Face's Clément Delangue and AI commentator AK amplified an observation from Victor Mustar that open source is currently dominating the pace of AI releases, though the post offers no specific models or data, just a general observation worth watching. ## Content Mira Murati's Thinking Machines dropped Inkling on July 15, 2026 — an open-weight model (Apache 2.0) trained from scratch, not fine-tuned off someone else's base. That distinction matters more than it sounds: building from scratch means the architecture choices below are deliberate, not inherited. And there are a lot of choices to dig into. ## The architecture Inkling is a Mixture-of-Experts model with 975B total parameters but only 41B active per token. It has 256 experts total, with 8 running per token (2 of those are shared experts that always fire, regardless of routing). Balancing which experts get used is handled through a bias-based routing mechanism rather than the usual auxiliary loss tricks. For context length, the model uses a 5:1 ratio of sliding-window to full-attention layers, which gets it to a 1M-token context window without the compute cost of full attention everywhere. It's a reasonable trade: most tokens don't need to see the entire context, so let the expensive layers be the exception. A more surprising choice is the position encoding. Instead of RoPE, which has become close to a default in recent models, Inkling uses Shaw-style relative position encoding. The reasoning seems to be better extrapolation behavior — RoPE has known issues when you push context length past what the model was trained on, and this sidesteps that. The catch is that RoPE has years of tooling and optimization built around it (flash attention kernels, etc.), so going a different route means some of that tooling has to be rebuilt. There's also convolution layers mixed in for local token mixing, which is a bit of a callback to pre-transformer architecture ideas, now repurposed to complement attention rather than replace it. ## Multimodality without an encoder Rather than bolting on a separate vision or audio encoder, Inkling handles multimodal input natively: dMel for audio, hMLP patches for images. This encoder-free approach keeps the architecture more unified, though it's still an open question how it compares in practice to dedicated encoders on modality-specific benchmarks. ## A dial for reasoning length One of the more practical additions is a trainable ## Questions this post answers ### What architecture does the Inkling model from Thinking Machines use? Inkling is a Mixture-of-Experts model with 975B total parameters and 41B active per token, spread across 256 experts with 8 active per token (2 always-on shared experts). It was released July 15, 2026 under Apache 2.0, trained from scratch rather than fine-tuned from an existing base model. _Follow daily.dev for the latest open-weight model architecture releases like this one._ ### How does Inkling achieve a 1M-token context window without full attention everywhere? It uses a 5:1 ratio of sliding-window attention layers to full-attention layers, letting most layers handle local context cheaply while reserving expensive full attention for the layers that need long-range dependencies. This keeps compute costs down while still reaching a 1M-token context length. _Engineers evaluating long-context models can track architecture trade-offs like this via daily.dev._ ### Why did Inkling use Shaw-style relative position encoding instead of RoPE? Shaw-style relative position encoding was chosen for better extrapolation behavior beyond the trained context length, since RoPE has known degradation issues when context is pushed past training length. The trade-off is that RoPE has years of accumulated tooling and kernel optimizations (like flash attention) that need to be rebuilt for a different position encoding scheme. _daily.dev helps developers weighing RoPE alternatives keep up with emerging position encoding choices._ ## Community take How the wider developer community reacted, aggregated from 1 discussion and 11 comments across x (as of 2026-09-13). **TL;DR:** Replies mostly riff on how relentless and exhausting the release cadence has become, with a few noting that constant new drops make it hard to actually evaluate or commit to any one model. **Sentiment:** 15% positive · 45% mixed · 40% skeptical **The case for** - More models being released means more choice, which some see as good for everyone. **The pushback** - The pace of releases itself is described as a real cost, since teams finish evaluating one model only to have it superseded before they can act on it. - Some feel fatigued by the nonstop cadence, joking about lost sleep and diminishing returns. - One commenter argues 'breakthrough' claims need actual benchmark differences rather than just another launch announcement. **By community** - x (mixed): Replies mix exhausted jokes about nonstop releases with a few pointing out the practical cost of constantly re-evaluating new models, alongside some optimism about increased choice. **Open questions** - Whether the release pace is sustainable or will hit diminishing returns. - How teams should adapt their evaluation and adoption workflows to a cadence that outpaces integration cycles. **Highlights** > @levie the cadence itself is the actual cost. by the time you've run evals and integrated one release, the next one lands and resets the decision. teams end up shipping on whatever was current when they started, not whatever's newest when they finish. > — [vsaietta on x · 1 points](https://x.com/vsaietta/status/2088150736685191598) > @levie By the time you will wake up another model will move up the leaderboard. What used to be an annual thing is now happening daily and many times multiple times a day. Things are getting wild. > — [\_RanjanSoni on x](https://x.com/_RanjanSoni/status/2088148185122521227) > @levie at this rate, "breakthrough" needs a benchmark diff, not another launch post 😅 > — [i\_mika\_el on x](https://x.com/i_mika_el/status/2088146129409978740) > @levie Can we hit diminishing returns already, some of us have jobs in the morning > — [SaileshMukil on x](https://x.com/SaileshMukil/status/2088151738159370360) > @levie Absolutely! Although, the more models, the more choice, the better for all of us. 🙏🏾 > — [ldondeti on x](https://x.com/ldondeti/status/2088148069317746975) **Source threads** - [x](https://x.com/levie/status/2088145829261709556) · 0 points · 11 comments --- Tags: [#open-source](https://daily.dev/tags/open-source) [View this post on daily.dev](https://daily.dev/posts/open-weight-ai-releases-are-coming-fast-right-now-ovyubagrj) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Open-Weight AI Releases Are Coming Fast Right Now","url":"https://daily.dev/posts/open-weight-ai-releases-are-coming-fast-right-now-ovyubagrj","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/open-weight-ai-releases-are-coming-fast-right-now-ovyubagrj"},"datePublished":"2026-08-12T16:54:06.650Z","dateModified":"2026-09-13T19:24:41.575Z","description":"A wave of open-weight AI model releases is happening across text, image, audio, and other modalities. Hugging Face's Clément Delangue and AI commentator AK...","isAccessibleForFree":true,"articleSection":"Collections","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Collections","logo":"https://media.daily.dev/image/upload/s--fk_6ycEi--/f_auto,q_auto/v1780996001/logos/collections?_a=BAMAMiWQ0","url":"https://daily.dev/sources/collections"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/open-weight-ai-releases-are-coming-fast-right-now-ovyubagrj","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"open-source","timeRequired":"PT2M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Collections","item":"https://daily.dev/sources/collections"},{"@type":"ListItem","position":3,"name":"Open-Weight AI Releases Are Coming Fast Right Now"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/open-weight-ai-releases-are-coming-fast-right-now-ovyubagrj#faq","mainEntity":[{"@type":"Question","name":"What architecture does the Inkling model from Thinking Machines use?","acceptedAnswer":{"@type":"Answer","text":"Inkling is a Mixture-of-Experts model with 975B total parameters and 41B active per token, spread across 256 experts with 8 active per token (2 always-on shared experts). It was released July 15, 2026 under Apache 2.0, trained from scratch rather than fine-tuned from an existing base model. Follow daily.dev for the latest open-weight model architecture releases like this one."}},{"@type":"Question","name":"How does Inkling achieve a 1M-token context window without full attention everywhere?","acceptedAnswer":{"@type":"Answer","text":"It uses a 5:1 ratio of sliding-window attention layers to full-attention layers, letting most layers handle local context cheaply while reserving expensive full attention for the layers that need long-range dependencies. This keeps compute costs down while still reaching a 1M-token context length. Engineers evaluating long-context models can track architecture trade-offs like this via daily.dev."}},{"@type":"Question","name":"Why did Inkling use Shaw-style relative position encoding instead of RoPE?","acceptedAnswer":{"@type":"Answer","text":"Shaw-style relative position encoding was chosen for better extrapolation behavior beyond the trained context length, since RoPE has known degradation issues when context is pushed past training length. The trade-off is that RoPE has years of accumulated tooling and kernel optimizations (like flash attention) that need to be rebuilt for a different position encoding scheme. daily.dev helps developers weighing RoPE alternatives keep up with emerging position encoding choices."}}]} ```