vLLM now supports TML Inkling on Day 0 — a 1T-parameter multimodal model from Thinking Machines Lab that accepts text, image, and audio inputs with up to 1M context length. Key architecture features include relative attention, short convolution (sconv), MoE with expert sinks, and 8 chained MTP heads for speculative decoding. vLLM achieves up to 380 tok/s/user with MTP and 140 tok/s/user without on 4× GB200 GPUs through optimizations including sconv-aware TP sharding, low-latency fused collectives (5x kernel time reduction), FA4 with sheared-bias for relative attention, and MTP KV cache recomputation. Full feature parity is provided including LoRA, TP/DP/EP/PP parallelism, prefix caching, and disaggregated serving. Accuracy was verified across MMAU, MMMU-Pro, BFCL, HLE, and NIAH benchmarks, with vLLM matching or slightly exceeding the reference implementation up to 513K context length.
