AI LABS

Fireship

<h2>What Kimi K3 is</h2>
<p>Moonshot AI released Kimi K3 this week — a 2.8-trillion-parameter mixture-of-experts model with a 1-million-token context window. It activates 16 of 896 experts per token, supports native vision, and uses a hybrid KDA/full-attention architecture with something called Attention Residuals. Open weights are scheduled to drop July 27th on Hugging Face, where a countdown is already running.</p>
<p>The model is the largest released by a Chinese lab to date. Alibaba dropped Qwen 3.8 (2.4T parameters) almost simultaneously, so it’s been a busy week for large open-weight releases out of China.</p>
<h2>Where it actually leads</h2>
<p>Kimi K3 hit #1 on the Frontend Code Arena leaderboard with an ELO of 1,679, and #1 on 3D Design with an ELO of 1,450. It also topped the Epoch Capabilities Index (ECI), which aggregates dozens of benchmarks into a single score — making it harder to game than any individual leaderboard.</p>
<p>On DeepSWE, a benchmark covering 113 real software engineering tasks, Kimi K3 trails Claude Fable 5 at pass@1 (68.5% vs 69.9%) but overtakes it at pass@2 and pass@4 (89.4% vs 88.5%). The cost gap is the more striking number: $4.65 per rollout versus $13.41 for Fable — roughly 2.8x more solved tasks per dollar. Kimi K3 is notably stronger on Go tasks; Fable leads on Python, JavaScript, TypeScript, and Rust.</p>
<p>One practical demo: a Moonshot staff member gave K3 a 3D model and a single prompt. In about 30 minutes it built a working VR companion — found and deployed local ASR and TTS, connected an LLM, built VR scenes, and handled expression changes across environments. That’s the kind of agentic chaining that’s hard to fake on a benchmark.</p>
<h2>Where it falls short</h2>
<p>Moonshot is upfront that K3 still trails Claude Fable 5 and GPT-5.6 Sol on overall benchmarks, including Humanity’s Last Exam by roughly 10 points. A 51% hallucination rate is a real problem for anything requiring reliable factual output. The model also tends toward verbose token generation, which matters for cost at scale.</p>
<p>Self-hosting requires data-center-grade hardware, so “open weights” here doesn’t mean you can run it on a workstation. vLLM is preparing day-0 serving support, including redesigned prefix caching to handle KDA’s recurrent state, fused kernels, and MXFP4 MoE support for both NVIDIA and AMD hardware — but this is still infrastructure work in progress.</p>
<p>The two models (Kimi K3 and Fable 5) also show unusually high cross-vendor similarity — a 0.72 correlation on DeepSWE — meaning combining them in an ensemble yields less diversity than you’d expect.</p>
<h2>Enterprise adoption: cautious optimism</h2>
<p>IT leaders are split on whether to use Chinese models in production. The case for: strong performance on bounded, high-volume tasks like coding, document analysis, and multilingual processing, at significantly lower cost. The case against: unknown training data provenance, geopolitical exposure, and no clear answer on what happens to data sent through the API.</p>
<p>The emerging consensus is task-specific evaluation rather than blanket adoption or rejection. Regulated data, customer-facing applications, and anything requiring trustworthy judgment are generally flagged as off-limits. That’s a reasonable place to land, even if it’s not a satisfying one.</p>
<h2>The business situation</h2>
<p>Moonshot reportedly raised $2 billion at a $20 billion valuation in May 2026. It’s now closing a $31.5 billion round and planning another targeting a $50 billion valuation as early as August, ahead of a Hong Kong IPO. Annual recurring revenue hit $300 million in June, up from $200 million in April. Daily sales jumped sixfold since K3 launched.</p>
<p>The K3 release hit Chinese AI rivals hard: Zhipu fell 28%, MiniMax dropped 16%, Alibaba declined 4%. Investors read it as raising the performance bar for independent Chinese labs.</p>
<p>The open-weight business model carries real structural risk, though. Zhipu and MiniMax have lost hundreds of millions on modest revenues. Moonshot itself had to pause new sign-ups due to GPU capacity shortages — which is a strange problem to have when you’re trying to capitalize on momentum before potential US regulatory action.</p>
<h2>Who built it</h2>
<p>Yang Zhilin, 34, co-founded Moonshot AI in Beijing in 2023. He studied computer science at Tsinghua, completed a machine learning PhD at Carnegie Mellon in four years, and interned at Google Brain and Meta AI. His earlier work — Transformer-XL and XLNet, both published before GPT-3 — addressed how language models lose track of distant context. XLNet has been cited over 10,000 times. Kimi K3’s 1M-token context window is a direct continuation of that research thread.</p>
<p>Bank of America noted that K3 demonstrates large-scale pretraining combined with architectural innovation can still produce substantial capability gains despite China’s hardware constraints. That’s the more interesting takeaway: the competition is shifting from raw GPU access toward how effectively labs combine architecture, training methods, and system software. More efficient models unlock more inference use cases, which in turn increases demand for compute infrastructure — so hardware doesn’t become less important, it just gets competed over differently.</p>


Collections

Yang Zhilin, a 34-year-old AI researcher with a Tsinghua and CMU background, co-authored foundational LLM papers Transformer-XL and XLNet before founding Moonshot AI in Beijing in 2023. His company's assistant Kimi has grown into one of China's widely used AI products. The latest model, Kimi K3, features native vision, a 1-million-token context window, and a 1-trillion-parameter mixture-of-experts architecture with only 16 active experts at inference time — a direct evolution of his long-running research into long-context language modeling.

Yang Zhilin and Moonshot AI: the researcher behind Kimi

<p>The opposite of me, who is doing all different things every day. and not being successful after years and years of work ;)</p>
