这篇论文搞了个新基准 OpenRCA 2.0，用 PAVE 协议给每一步因果关系打标签，发现 LLM 猜对根因容易，但连对因果路径很难——这比只看结果靠谱多了。
OpenRCA 2.0 引入了 PAVE 协议，通过故障注入重建因果传播路径，标注了 500 个跨系统实例的步骤级因果链。在 11 个前沿 LLM 上测试，完全恢复根因集的成功率平均仅 20.7%。放宽条件后发现，模型在 76.0% 的案例中能识别至少一个正确根因服务，但只有 61.5% 能将服务与观察到的症状通过验证的因果路径关联起来。该基准揭露了仅靠结果标签评估时隐藏的未接地诊断失败模式。
OpenRCA 2.0: From Outcome Labels to Causal Process Supervision
Root cause analysis (RCA) poses a holistic test of LLM agentic capabilities, such as long-context understanding, multi-step reasoning, and tool use. However, existing datasets suffer from a fundamental gap: they label only the root cause, not the propagation path connecting it to the observed symptom, which largely simplifies the task to naive pattern matching. To support rigorous evaluation, we introduce PAVE, a step-wise labeling protocol that leverages known interventions from fault injection to reconstruct causal propagation paths. The mechanism is forward verification: reasoning from cause to effect rather than inferring backward from symptoms. Applying PAVE yields OpenRCA 2.0 (500 instances), the first cross-system RCA benchmark with step-wise causal annotations for LLM agents. Across 11 frontier LLMs, recovering the exact root-cause set succeeds in only 20.7% of cases on average. To locate where this difficulty lies, we relax the criterion and find what we call the ungrounded diagnosis: agents identify at least one correct root-cause service in 76.0% of cases, but ground that service in a verified causal propagation path to the observed symptom in only 61.5%. Outcome-only evaluation hides this failure mode; step-wise causal ground truth is the missing piece for trustworthy LLM-based RCA agents.