Thinking Machines, founded by ex-OpenAI CTO Mira Murati, released Inkling on July 15, 2026, an open-weight (Apache 2.0) model trained from scratch and designed for customization. A deep architectural breakdown covers its 975B total / 41B active parameter Mixture-of-Experts design with 256 experts (8 running per token including 2 shared), a bias-based routing balance mechanism, a 5:1 ratio of sliding-window to full-attention layers enabling a 1M-token context, relative position encoding (Shaw-style) chosen over RoPE for better extrapolation, convolution layers for local mixing, an encoder-free multimodal approach using dMel for audio and hMLP patches for images, and a trainable 'effort' setting controlling reasoning length. Trade-offs include high memory floor despite sparse compute, narrow per-layer context, new tooling requirements for the position encoding, and closed training data despite open weights.
Table of contents
Cut Your Token Usage by Up to 36% (Sponsored)GroundworkSparsityWhat Makes an FDE Role Credible? (Sponsored)RoutingAttentionPositionConvolutionsMultimodalityEffortTradeoffsConclusionQuestions this post answers
How many parameters does Thinking Machines' Inkling model activate per token versus store in total?
Inkling stores 975 billion total parameters but activates only about 41 billion, roughly 4 percent, per token. This is achieved through a Mixture-of-Experts design with 256 experts per layer across 66 layers, where a router selects six experts per token plus two shared experts that always run, for eight experts total per token. daily.dev surfaces architecture breakdowns like this for engineers evaluating which model to build on.
Why did Thinking Machines use relative position encoding instead of RoPE in Inkling?
Relative position encoding, in the style of Shaw and colleagues, was chosen because it extrapolates better to sequence lengths beyond training data. Unlike RoPE, which rotates queries and keys by angles tied to absolute position and fails on unseen positions, this scheme learns a value per distance between tokens, so a pair 900,000 tokens apart reuses a distance value seen frequently during training, avoiding extrapolation problems for the 1 million token context window. track architectural trade-offs like this on daily.dev when choosing a long-context model for your stack.
How does Inkling handle images and audio without a separately pretrained encoder?
Inkling processes audio as a mel spectrogram rounded to fixed levels using the dMel method, requiring no separate trained audio model, and images as 40x40 pixel patches passed through a small four-stage hMLP stem network. Both are trained from scratch alongside the rest of the 66-layer model rather than using a large pretrained encoder, which the launch materials describe as an 'encoder-free' architecture. daily.dev helps engineers compare multimodal architecture choices before picking a model to integrate.