Kimi K3 is a 2.8-trillion-parameter open mixture-of-experts model from Moonshot AI that sits close to the best closed models on benchmarks. Its 47-page technical report reveals that the model architecture itself — a hybrid of linear and full attention, cross-layer residuals, and sparse routing across 896 experts — is only a small fraction of the total effort. The bulk of the work involves hand-building reinforcement learning environments with verifiable rewards, constructing a knowledge-graph-based task generation pipeline, training nine specialist models across domains and reasoning-effort levels before distilling them into one, and engineering a serving stack optimized around prefix caching, cache-aware routing, and quantization. The report also covers security implications of open weights, including confirmed real-world vulnerability discoveries, and offers practical guidance for developers building with frontier models: prioritize evaluation design, long-horizon agentic engineering, and serving economics over chasing architecture novelty.
