ReToken用一个小token当检索目标，从KV缓存里挑关键视觉token，Qwen3VL-8B在Visual Haystacks涨13.4分，单卡H100就能跑。
ReToken训练一个可学习嵌入作为检索目标，从预填充的视觉KV缓存中选取与查询相关的稀疏视觉token。它在小型图像问答数据集上训练，在Visual Haystacks基准上让Qwen3VL-8B提升13.4分，InternVL3.5提升12.4分。在LVBench上，模型零样本迁移到长视频任务，Qwen3VL-8B取得8.0分增益。由于设计轻量，训练与长视频推理可在一块H100上完成。
ReToken: One Token to Improve Vision-Language Models for Visual Retrieval
Long visual context poses a challenge for vision-language models: performance degrades as the number of distractors grows, and processing all tokens at once is computationally infeasible under GPU memory constraints. We present ReToken, a single learnable embedding trained as an explicit retrieval target that selects a sparse set of query-relevant visual tokens from a pre-filled visual KV cache. Trained on only a small image-QA dataset, ReToken yields consistent gains across image and video benchmarks: on Visual Haystacks it improves Qwen3VL-8B by 13.4 points and InternVL3.5 by 12.4 points (>20% relative), and on LVBench it transfers zero-shot to long video for an 8.0-point gain with Qwen3VL-8B. Thanks to its lightweight design, both training and long-video inference fit on a single H100. Code is available at: https://github.com/avaxiao/ReToken
- Yangyi07-30 02:00原文