Own the Models You’re Renting Introducing distiliaa.
This title could be clearer and more informative.Try out Clickbait Shieldfor free (5 uses left this month).
A launch paper for distiliaa, a self-hosted platform that lets teams distill frontier open-weight models (GLM-5.2, Kimi K3) into small, owned fine-tuned models for narrow, high-volume production tasks. The argument: frontier MoE models are architected for input diversity and long-horizon reasoning, which most production workloads don't need, so teams pay frontier latency and cost for unused capacity. distiliaa's pipeline uses a frontier model as a one-time teacher to annotate data, freezes a reproducible dataset, and fine-tunes a small model the organization keeps, with storage remaining in the customer's own object store and job-scoped credentials for GPU workers. The piece also lists boundary conditions where distillation doesn't apply: tasks needing current knowledge, provenance/citations, genuinely open-ended long-horizon reasoning, or low volume.
Table of contents
Abstract1. Introduction2. Architecture as an Explicit Bet on Diversity3. What Production Tasks Actually Look Like4. Distiliaa: Closing the Gap5. Boundary Conditions — When This Doesn’t Apply6. ConclusionReferencesQuestions this post answers
How many active parameters per token does Kimi K3 use out of its total parameter count?
Kimi K3 activates 104 billion parameters per token out of 2.8 trillion total parameters, routed through 16 of 896 experts. This is more than 3x the active-parameter budget of its predecessor Kimi K2, which used 32 billion of 1 trillion total parameters routed through 8 of 384 experts, meaning the newer generation scaled per-token compute up rather than down. Teams comparing MoE model efficiency for production inference can track releases like this on daily.dev.
What is GLM-5.2's IndexShare attention technique and how much compute does it save?
IndexShare is a sparse-attention technique in GLM-5.2 that reuses the same indexer across every four sparse attention layers, cutting per-token FLOPs by 2.9x at a 1-million-token context length. GLM-5.2 is a 753-billion-parameter mixture-of-experts model routing each token through 8 of 256 routed experts plus 1 shared expert across 78 hidden layers, with roughly 40 billion active parameters per token. Engineers evaluating frontier model architecture tradeoffs can follow developments like this on daily.dev.
When does it make sense to distill a frontier LLM into a smaller fine-tuned model instead of calling the frontier model on every request?
Distillation makes sense for narrow, high-volume, repetitive tasks with a stable input distribution and bounded output space, such as classifying support tickets or extracting invoice fields, where a small fine-tuned model can match a frontier teacher's performance on that specific task. It does not work for tasks requiring current knowledge, source citations, genuinely open-ended or long-horizon reasoning, or low call volumes where the pipeline overhead isn't justified. Teams weighing frontier API costs against owning a fine-tuned model can track this tradeoff on daily.dev.