vLLM拆开预填充和解码，TileRT提供低延迟解码，单用户618 tok/s，适合实时助手。
vLLM项目实现了预填充与解码分离，TileRT通过vLLM V1连接器将自身低延迟解码引擎与vLLM预填充配对，无需修改vLLM代码。原生vLLM解码默认用于吞吐场景，TileRT解码用于延迟敏感场景（如智能体、实时助手）。TileRT在GLM-5.1-FP8（8×B200）上单用户解码达618 tok/s，启用推测解码（MTP）后约为无MTP基线的2倍，峰值接近800 tok/s。
Once prefill and decode are disaggregated, the dec…
Once prefill and decode are disaggregated, the decode side becomes a choice: pluggable, swappable per workload. The @TileRT_AI team just shipped a concrete example: vLLM prefill paired with TileRT's latency-optimized decode engine through vLLM V1's connector interface, with zero changes to vLLM.
Native vLLM decode stays the default for throughput. TileRT decode is there for latency-bound work (agents, real-time assistants). Same OpenAI-compatible surface, same prefix caching, so switching is a routing change. No fork, no patches, both decode pools behind one stock vLLM prefill pool.
TileRT reports around 618 tok/s single-user decode on GLM-5.1-FP8 (8× B200) with MTP, roughly 2× its no-MTP baseline. At peak acceptance, up to nearly 800 tok/s.
Thanks to the @TileRT_AI team and @inferact for the collaboration.
🔗 https://t.co/dYp052byCr