做LLM智能体开发的团队终于有了计划表示的系统性对比——选对计划形式能直接提升任务成功率,建议做Web Agent的开发者点开看看具体指标差异。
LLM网页智能体在探索、关键步骤遗漏和任务约束敏感性上存在不足,现有研究认为这些失败源于规划弱点,但自然语言计划表示的影响尚未被系统探索。PlanAhead提出静态规划-执行框架,自动将WebArena任务分为三个难度级别,并在困难任务上评估四种计划表示(顺序子目标、叙事、伪代码、清单)对多模态LLM智能体(OpenAI、阿里巴巴、Google)的影响。引入两个新指标:达成率和解决任务一致性,发现计划形式和底层LLM都显著影响智能体的鲁棒性和任务成功率。
Does The Way You Plan Matter? An Empirical Study of Planning Representations for LLM Web Agents
Despite recent advances, LLM-based web agents still struggle with limited exploration, omission of critical steps, and sensitivity to task constraints. Prior work suggests that many of these failures stem from weaknesses in planning, yet the impact of alternative natural language plan representation remains unexplored. To address this, we introduce PlanAhead, a static planner-executor framework that evaluates the impact of plan representation in agent performance. We first automatically categorize WebArena tasks into 3 difficulty levels, enabling consistent difficulty grading without human annotation. Then we systematically evaluate 4 different plan representations on the tasks categorized as hard: sequential subgoals, narrative, pseudocode, and checklist; across different families of multimodal LLM powered agents (OpenAI, Alibaba, and Google). To account for stochastic variability, we introduce two novel evaluation metrics: Achievement Rate (AR) and Solved-Task Consistency (STC). Our results show that both, the plan formulation and the underlying LLM generating the plan, significantly influence web-agent robustness and task success.