VLM推理成本高是实际部署的痛点，Reroute用零训练代价解决了令牌缩减中信息丢失的问题，做多模态模型优化或部署的团队可以直接集成到现有方案中，值得一试。
视觉语言模型（VLM）将图像投影为数百到数千个视觉令牌，导致解码器推理成本高昂。现有方法通常采用“排序并移除”范式，永久丢弃低分令牌。但研究发现，视觉令牌的重要性会随解码器深度变化，早期低分令牌可能在后续层变得重要。为此，研究者提出Reroute，一种无需训练的插件，将移除改为可恢复路由：被延迟的令牌在后续阶段重新进入候选池。该方法在FastV、PDrop等方案上，在LLaVA-1.5和Qwen骨干上，在激进令牌缩减下提升了接地性能，同时保持VQA性能。这表明VLM令牌缩减应视为可恢复路由，而非不可逆修剪。代码已开源。
Reroute, Don't Remove: Recoverable Visual Token Routing for Vision-Language Models
Vision-language models (VLMs) project images into hundreds to thousands of visual tokens, making decoder inference expensive in both attention computation and KV-cache memory. Existing visual-token reduction methods largely follow a rank-and-remove paradigm: they score visual tokens, keep a compact subset, and permanently discard the rest. We show that this irreversible action is fragile because visual-token importance changes across decoder depth; tokens ranked low at one stage may become relevant in later layers, especially for grounding-sensitive queries. We propose Reroute, a training-free plug-in that replaces removal with recoverable routing. At each routing stage, selected vision tokens pass through decoder blocks, while deferred tokens bypass the stage and re-enter the candidate pool at the next routing decision. Reroute reuses existing attention-score ranking rules and stage-wise schedules, preserving the theoretical TFLOPs and KV-cache budget class of the pruning method it augments. Across FastV, PDrop, and Nüwa variants on LLaVA-1.5 and Qwen backbones, reroute improves grounding under aggressive token reduction while maintaining general VQA performance. These results suggest that VLM token reduction should not be viewed only as irreversible pruning, but also as recoverable routing. The code can be found here: https://github.com/elmma/mllm-reroute/