DigitalOcean's engineering team details how they launched Kimi K3 (a 2.78 trillion parameter MoE model) on day zero, covering GPU selection (NVIDIA HGX B300 and AMD MI350X), vLLM serving stack tuning, and the specific OpenAI-compatibility fixes required to pass Moonshot's Kimi Vendor Verifier (KVV) benchmark suite. Key engineering challenges included implementing Moonshot's dynamic tools extension (fixing 26 of 30 initial KVV failures), handling tool_choice schema-constrained decoding for auto mode, mapping K3's custom thinking/reasoning_effort fields, fixing four streaming response deviations from the OpenAI spec, resolving prompt token count discrepancies from injected system messages, and correcting temperature range validation. The post emphasizes that serving open-weight models correctly is as important as the weights themselves, since misconfigured serving stacks can significantly degrade benchmark performance.
