这篇论文提出了Otap，一个用最优传输来评估Agent轨迹的新方法。它不再只看任务成功与否，而是分析执行结构，还能容忍不同的规划顺序和冗余步骤。
当前Agent轨迹评估依赖二进制成功标志或精确匹配，无法区分侥幸成功或分析失败原因。Otap将轨迹评估转化为执行图与有效解图之间的最优传输距离，基于不平衡融合Gromov-Wasserstein问题。该伪度量对保持依赖的重新排序具有不变性，对冗余步骤敏感度有限，并能处理缺失或幻觉步骤。在受控扰动和三个公开基准上，Otap在语义指标低于随机水平的场景中仍能有效区分有效与无效轨迹。其准确性在依赖图精确恢复时最高，在从自由文本推断时下降。
Otap:Structure-Aware Optimal Transport for Evaluating Planning and Execution in Agent Trajectories
Large language model agents solve tasks by generating trajectories that interleave planning, tool calls, and intermediate results. Current evaluation metrics reduce such a trajectory to a binary success flag or compare it against a reference by exact matching. A success flag cannot distinguish a sound solution from one that succeeds by luck, and says nothing about why a failed run went wrong. Exact matching penalizes plans that are valid but reordered or decomposed differently from the reference. We reframe trajectory evaluation as a distance between the agent's execution graph and a set of valid solution graphs, and instantiate it via an unbalanced fused Gromov-Wasserstein transport problem over attributed dependency graphs. The resulting score, termed \otap{} (Optimal Transport for Agentic Planning), is a pseudo-metric that is provably invariant to dependency-preserving reorderings and has bounded sensitivity to redundant steps. Its unbalanced marginals handle missing or hallucinated steps without forcing a match, and its soft coupling accommodates variation in plan granularity. On controlled perturbations and three public benchmarks, \otap{} separates valid from invalid trajectories in a regime where semantics-only metrics score below chance. Its accuracy is highest when the dependency graph is recovered exactly, and drops only when the graph is inferred heuristically from free-text traces.