vLLM and TileRT announce an integration that pairs vLLM's prefill with TileRT's specialized decode engine for latency-critical LLM serving workloads. The integration uses vLLM V1's public KVConnectorBase_V1 interface with zero modifications to vLLM itself — no forks, no patches. A lightweight router directs requests to either the TileRT decode pool (optimized for per-user token speed) or native vLLM decode (optimized for aggregate throughput), both behind the same OpenAI-compatible API surface. KV state is transferred via RDMA one-sided writes using Mooncake or NIXL, fully overlapped with prefill. Current limitations include one in-flight request per TileRT decode node and model support limited to GLM-5/5.1 and DeepSeek-V3.2. TileRT 0.1.5 is available on PyPI.
