PACT 训练框架修复 GUI 智能体的计划-动作执行偏差
PACT: Plan-Action Consistency Training by Preserving Correct Plans and Repairing Actions in GUI Agents
做 GUI 智能体训练的可以看看这篇:它发现模型想得对但动作做错的老问题,用屏蔽计划区间的负梯度和偏好修复来解决,AndroidWorld 成功率直接提了 8.63 个百分点,移动端和桌面端都适用。
GUI 智能体常出现推理中给出正确计划、执行动作却偏离计划的情况,一个错误动作会让多步任务整体失败。PACT(Plan-Action Consistency Training)针对这一执行缺口设计:先用带标注的参考动作定位执行错误,在计划 token 区间上屏蔽负的策略梯度贡献,保护已验证的计划;再以生成的推理作为固定上下文,通过带 stop-gradient 权重的偏好损失,让参考动作优于错误动作,且不改动原有的标量动作奖励。
实验数据上,相比仅用结果奖励的基线,PACT 将 AndroidWorld 任务成功率从 60.34% 提升到 68.97%(+8.63 个百分点),AndroidControl High 上的规则化计划-动作一致性从 83.71% 提升到 89.74%(+6.03 个百分点);迁移到桌面端后,在 369 个任务的 OSWorld 基准上任务成功率从 40.92% 提升到 44.17%(+3.25 个百分点)。该方法解决的是 outcome-supervised RL 中把负优势广播到全部 token、误伤正确规划的信用分配冲突,对训练 GUI 智能体的工程团队有直接参考价值。
PACT: Plan-Action Consistency Training by Preserving Correct Plans and Repairing Actions in GUI Agents
GUI agents can formulate a correct immediate plan in their reasoning yet emit an action that fails to execute it, creating an execution gap. This recurring mismatch undermines reliable task completion, as a single execution error can derail a multi-step workflow. In outcome-supervised reinforcement learning, broadcasting a negative response-level advantage across all generated tokens risks penalizing valid planning together with erroneous execution. To address this credit-assignment conflict, we introduce PACT (Plan-Action Consistency Training), a training framework that combines reference-aligned plan protection and plan-aligned action repair without altering the original scalar action reward. Using annotated reference actions to identify execution gaps, PACT protects the verified plan by masking negative policy-gradient contributions across its token span. Keeping the generated reasoning unchanged as context, it then favors the reference action over the emitted erroneous action through a preference surrogate with stop-gradient pair weights. Compared with the outcome-only baseline, PACT raises AndroidWorld task success from 60.34% to 68.97% (+8.63 pp), while improving rule-based plan-action consistency on AndroidControl High from 83.71% to 89.74% (+6.03 pp). Adapting PACT to desktop agents also improves task success from 40.92% to 44.17% (+3.25 pp) on the full 369-task OSWorld benchmark. Together, these results establish PACT as an effective credit-assignment framework for bridging execution gaps and improving task completion across mobile and desktop GUI environments.
This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include GUI agents, PACT, AndroidWorld, OSWorld.
Source attribution: curated from arXiv cs.AI (@Zhuohan Cai, Zhangxuan Gu, Runze Li, Beitong Zhou, Ming Wen, Hongbo Yu, Shuheng Shen, Changhua Meng).