这篇论文提出了ASV框架，能比传统方法更细粒度地诊断智能体每一步的好坏，特别适合做Agent评估的朋友。
ASV是一个状态转换测量框架，为每个观察到的动作评分，基于状态锚定评估器在固定候选结果上的分布变化。在100个开放问答任务中，使用实时PubMed检索和部分实时的DeepSeek执行器与log-probability评分，评估了1100步和2200个状态。在固定布局理由条件下，平均金标准边际增益为-2.335（轨迹自举95% CI [-3.395, -1.272]），熵值变化为0.000，平均贝叶斯惊喜为2.693。ASV能够定位最终答案分数或纯熵步指标遗漏的建设性和破坏性信念转变。
Agent Step Value: State-Transition Measurement with State-Grounded LLM Evaluators
Most agent evaluations collapse a multi-step trace into a final answer, a success flag, or a trajectory-level score. These aggregates obscure the diagnostic question developers need most: which action changed the state in a useful direction? We introduce Agent Step Value (ASV), a state-transition measurement framework that scores each observed action by the change it induces in a state-grounded evaluator's distribution over fixed candidate outcomes. ASV renders redacted before/after state projections, uses a stateless LLM evaluator to assign candidate log scores, and reports both gold-free belief diagnostics and offline oracle validation metrics. A label-free rationale pass separates evaluator deliberation from one-token option scoring, preserving candidate likelihoods while exposing leakage and floor-score events. On 100 reviewed open-QA evidence-seeking tasks with live PubMed retrieval, a partially live DeepSeek actor, and DeepSeek log-probability scoring, ASV evaluates 1,100 steps and 2,200 states. Under the fixed-layout rationale-conditioned protocol, mean gold-margin gain is -2.335 (trajectory-bootstrap 95\% CI [-3.395, -1.272]), entropy movement is 0.000, and mean Bayesian surprise is 2.693. ASV therefore localizes constructive and destructive belief pivots that final-answer scores and entropy-only step metrics miss. We release the standalone ASV Eval toolkit.