多视图训练让GUI定位更准
GRPO在GUI接地训练中因单视图采样导致有效信号不足。VISTA框架从多个保持目标元素可见的裁剪视图中构建比较组,并添加自验证跨视图锚点。在五个GUI接地基准上持续提升,ScreenSpot-Pro上Qwen3-VL 4B/8B/30B-A3B从55.5/52.7/53.7分别升至63.4/65.8/67.0。鲁棒性分析显示最差视图准确率更高、预测翻转率更低。
VISTA: View-Consistent Self-Verified Training for GUI Grounding
When applying Group Relative Policy Optimization (GRPO) for GUI Grounding, rollouts are sampled from a single screenshot view; groups often become either all failures on difficult instances or all successes on easy ones, yielding no useful relative advantage. We propose VISTA (View-Consistent Self-Verified Training), a GRPO-based training framework that constructs each comparison group from multiple target-preserving views of the same GUI instance.Each view is generated by a crop that keeps the target element visible and remaps its box exactly, so model rollouts are compared across semantically equivalent but geometrically different inputs. To stabilize short coordinate generation without turning reinforcement learning into unconditional imitation, VISTA further adds a self-verified cross-view anchor: an oracle answer optimized with an advantage-weighted loss, excluded from the group baseline and activated only when the model has produced a maximum-reward rollout. Across five GUI-grounding benchmarks and multiple Qwen backbones, VISTA consistently improves grounding accuracy.On ScreenSpot-Pro, it raises Qwen3-VL 4B/8B/30B-A3B from 55.5/52.7/53.7 to 63.4/65.8/67.0. Robustness analyses further show higher worst-view accuracy and lower prediction flip rates.