A deep dive into the 'Kimi K3 in C' open-source repository that enables running a 2.78 trillion parameter Mixture-of-Experts LLM on a CPU with as little as 8GB of RAM. The project achieves this through four key techniques: streaming model weights from disk on demand, loading only the active experts (16 out of 896 per token), an LRU expert cache to reduce repeated disk reads, and MXFP4 compressed weight representations. Written in pure C99 with no GPU, PyTorch, or CUDA dependencies, it trades inference speed for accessibility. Token generation is slow and NVMe storage speed becomes critical, but the project serves as an engineering proof-of-concept showing that software architecture can dramatically lower hardware barriers for trillion-parameter models.
