C$^2$-Routing：裁剪与压缩互补，让 VLM 视觉推理 FLOPs 降 36.53%
C$^2$-Routing: Complementary Cropping and Compression for Efficient Vision-Language Inference
做 VLM 推理优化的开发者可以看看这篇。它不训模型，靠按问题选裁剪还是压缩来省算力，FLOPs 省三成多、精度只掉不到 1%，RTX 3090 就能复现，比单纯砍视觉 token 上限更细。
arXiv 论文提出 C$^2$-Routing，一种面向视觉问答（VQA）的模型无关输入缩减方法，可在黑盒设置下套用于多种 VLM。核心观察是裁剪与压缩互补：裁剪突出小目标和局部细节，整图压缩更适合大目标或证据分散的场景，方法据此按图片和问题自适应选择策略。
由于两种策略都可能丢弃关键信息，论文设计了安全门，利用原始输入与缩减输入下预测的联合正确性分布来决定回退，保护准确率。在视觉预算统一为 1,024 视觉 token 的全图基线上，C$^2$-Routing 平均再降低端到端 FLOPs 36.53%、视觉 token 42.36%，任务分数平均相对下降仅 0.77%。评测覆盖 3 个 VLM 和 18 个数据集，涉及局部细节、全局场景、场景文字和文档，所有训练与评测均在 RTX 3090 消费级 GPU 上完成。
C$^2$-Routing: Complementary Cropping and Compression for Efficient Vision-Language Inference
Visual Question Answering (VQA) imposes substantial computational overhead on vision-language models (VLMs), motivating model-agnostic input reduction methods that can be readily applied across diverse VLMs, including in black-box settings. Simply enforcing a visual-token cap reduces computation, but it can discard information needed to answer the question. A single reduction strategy also struggles to accommodate different scenarios. Our key observation is that cropping and compression are complementary: cropping highlights small targets and local details, whereas whole-image compression better preserves context for large targets or distributed evidence. Based on this observation, we propose C$^2$-Routing, which adaptively selects cropping or compression according to the input image and question. Since both strategies may still discard key information, we further design a safety gate that uses the predicted joint correctness distribution under original and reduced inputs to guide fallback and protect accuracy. To accommodate consumer GPU resource constraints, all training and evaluation experiments run on RTX 3090 GPUs. To measure gains beyond token capping, we use full-image baselines with the same cap of 1,024 visual tokens. We conduct extensive evaluation on three VLMs and 18 datasets spanning local details, global scenes, scene text, and documents. With the visual budget already capped, C$^2$-Routing further reduces end-to-end FLOPs by 36.53% and visual tokens by 42.36% on average, with a mean relative task-score decrease of only 0.77%.
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include VLM, 视觉问答, C$^2$-Routing, 推理优化.
Source attribution: curated from arXiv cs.LG (@Guorui Lu, Qinyu Chen).