这篇论文查了6个模型的视觉工具调用：多数只是白花钱，只有少数轨迹真有用。别被表面准确率骗了。
论文提出视觉证据增益（Visual Evidence Gain）估计量，在策略、轨迹、步骤三个层级干预，审计多模态大模型的裁剪缩放等视觉工具调用。对 6 个代表性模型和 5 个细粒度感知基准的测试显示，工具调用相对直接推理的准确率提升有限，甚至为负，而 token 成本明显更高。作者识别出两种失败模式：Calling Without Looking 中返回的观察对答案无因果作用，Looking Without Planning 中观察有信息量但调用次序不合理。轨迹级诊断表明，策略级准确率增益集中在少数校准良好的轨迹上，作者称之为视觉工具使用的幻觉。代码已在 GitHub 的 OpenCausaLab/CauAudit 发布。
The Illusion of Visual Tool-Use: A Causal Audit of Thinking with Images
The "thinking-with-images" paradigm equips multimodal LLMs with active visual operations such as crop-and-zoom. However, models using these operations often achieve only marginal or negative gains over direct inference at substantially higher token cost. They may also repeatedly crop irrelevant regions and fail on questions that direct inference answers correctly. We ask whether the returned visual evidence causally affects the answer. To answer this question, we formulate visual tool-use as a causal graph that separates observation-mediated paths from action-induced shortcuts. We then audit it through interventions at the three levels: policy (comparing tool-use with direct inference), trajectory (corrupting all observations during rollout), and step (counterfactually replacing one individual observation under a fixed prefix). Our step-level estimand, Visual Evidence Gain, isolates the contribution of each returned observation. Across six representative models and five fine-grained perception benchmarks, we uncover policy miscalibration with two failure modes. In Calling Without Looking, returned observations have no causal effect on the answer. In Looking Without Planning, observations are informative but the call schedule is incoherent. A trajectory-level diagnostic decomposes the policy-level accuracy gain and shows that the gain is concentrated in a Calibrated minority. We term this discrepancy the illusion of visual tool-use: despite aggregate accuracy gains, visual tool-use is not causally effective across a broad range of rollouts. The code is available at https://github.com/OpenCausaLab/CauAudit.