IGG 基准：评估 GUI 智能体在目标不可见时的交互定位能力
IGG: A Benchmark for Interactive GUI Grounding under Visibility Constraints
评测 GUI 智能体老盯着静态截图，这篇造了 601 个目标先藏起来的任务，人能拿 94.6%，GPT-5.5 只有 79.4%，差距到底卡在哪写得很清楚。
arXiv 论文 IGG 提出 601 个任务、16 个可控 GUI 环境、7 种可见性约束子类型，要求智能体先通过交互让目标显现再点击。人类参与者准确率达 94.6%，GPT-5.5 为 79.4%，最强的开源 VLM 仅 31.6%。对 Qwen3-VL 的干预实验显示，告诉它执行什么交互只能把准确率从 30.8% 提到 33.9%，替它执行交互则可提升到 81.9%，说明瓶颈在执行显现交互本身。论文还证明该能力可通过训练在多个模型家族上稳定提升。
IGG: A Benchmark for Interactive GUI Grounding under Visibility Constraints
GUI grounding benchmarks ask whether an agent can localize a target that is already visible in a static screenshot. Real interfaces often violate this assumption: a target may be off-screen, appear only after a state change or hover, remain occluded, or become actionable after a delay. This leaves the ability to recover target visibility unevaluated. To measure this reveal step, we introduce Interactive GUI Grounding (IGG), a benchmark in which an agent must first make the target visible through interaction and then click it. IGG contains 601 tasks across 16 controllable GUI environments and seven visibility-constraint sub-types, with a compact action space of scrolling, moving, dragging, waiting, and clicking. In our human study, participants achieve 94.6% accuracy, while GPT-5.5 reaches 79.4% and the strongest evaluated open-source VLM reaches only 31.6%. To identify the source of this gap, we conduct two controlled interventions. Telling Qwen3-VL which interaction to perform raises its accuracy only from 30.8% to 33.9%, whereas performing the interaction for the model raises it to 81.9%. The bottleneck is executing the reveal interaction, not recognizing the target once it is exposed. We further show that this capability is learnable, demonstrating that visibility recovery can be strengthened consistently across model families.