WASTE (Weight-Aware Streaming Tensor Engine) is a dependency-free, embeddable C inference engine that runs the full 2.78-trillion-parameter Kimi K3 model on a consumer laptop by streaming expert weights directly from NVMe storage. Instead of requiring the full 982 GB model in RAM, it keeps the model trunk resident (~27 GB) and streams only the activated experts per token from disk, using remaining RAM as a bounded expert cache. On a 64 GB MacBook Pro M5 Pro, it achieves 0.49–0.54 tokens/second. The engine uses 3-bit residual vector quantization for experts, bypasses the OS page cache for direct NVMe reads, and implements an absorbed KV cache for K3's hybrid attention. It also supports multimodal inputs (images via a 27-layer ViT), an OpenAI-compatible HTTP server, and runs on macOS, Linux, and Windows. The project documents all measured optimizations including those that were tried and rejected, with detailed performance tables and engineering rationale.
