做3D重建或视觉Transformer的开发者，这篇论文用两阶段令牌选择解决了计算瓶颈，85%的加速效果值得直接参考实现。
视觉几何Transformer在多视图3D重建中表现出色，但全局注意力层导致计算成本随输入序列长度二次增长，限制了可扩展性和效率。本文提出一种简单通用的策略：限制每个查询在全局注意力中交互的键/值令牌数量。通过两阶段框架实现有效令牌选择：帧间选择基于多样性策略确保场景覆盖，帧内选择利用注意力熵指导层感知稀疏化。实验表明，该方法在500张图像场景下加速超过85%，同时保持甚至提升基线性能，为视觉几何Transformer的未来应用提供了关键优化思路。
Good Token Hunting: A Hitchhiker's Guide to Token Selection for Visual Geometry Transformers
Visual geometry transformers have become powerful architectures for multi-view 3D reconstruction, enabling joint prediction of multiple 3D attributes in a feed-forward manner. However, their computational cost grows quadratically with the input sequence length due to the global attention layers inside these models. This limits both their scalability and efficiency. In this work, we address this challenge with a simple yet general strategy: restricting the number of key/value tokens that each query interacts with during global attention. To achieve effective token selection, we introduce a two-stage framework. First, an inter-frame selection step operates at the frame level to identify frames that should be preserved. Second, an intra-frame selection step further discards more redundant tokens within the selected frames. Our analysis highlights the advantage of a diversity-based strategy for inter-frame selection, which ensures broad coverage of the scene. For intra-frame selection, we show that layer-aware sparsification is necessary, with the selection process guided by the entropy of the global attention pattern. Our approach offers a superior speed-accuracy trade-off compared to existing solutions. Extensive experiments show that it accelerates visual geometry transformers by over 85% for scenes with 500 images while maintaining, or even improving, baseline performance, which hints that how our token selection strategy can play a crucial role in future applications of visual geometry transformers. Our project website is available at https://zsh2000.github.io/good-token-hunting.github.io.