简单 LLM 工作流在根因分析上胜过手工设计的智能体
Free to Look, Free to Reason: Simple LLM Harnesses Outdiagnose Hand-Crafted Root Cause Analysis Agents
跑微服务排障智能体的可以看看：不写固定流程，让 Codex 自己查，准确率反超五套精心设计的智能体，还更省时间。
一项研究在 RCAtlas 基准（来自八个公开数据集的 90 个验证案例）上，将三种不预设调用流程的简单工作流与五个已发表的 LLM 根因分析智能体对比。简单工作流在三个中档模型上达到 PR@1 65-77%，而已发表智能体为 16-53%，非 LLM 统计与因果方法为 6-39%。案例研究显示，已发表智能体固定了一成到全部的模型调用，导致跳过关键证据。提高推理强度最多带来几个点的准确率变化，却最多增加七倍耗时，而在成本或时间上所有 Pareto 最优的运行都来自简单工作流。
Free to Look, Free to Reason: Simple LLM Harnesses Outdiagnose Hand-Crafted Root Cause Analysis Agents
Root cause analysis (RCA) of microservice failures has attracted many LLM agents whose investigation is hand-crafted, through standard operating procedures, fixed stages and role agents. Each is evaluated on its own data against few baselines, so it is unknown whether this design beats letting a capable model investigate on its own. We compare three simple harnesses that prescribe none of the model's calls (one call over an evidence package, a SQL agent and the Codex coding agent) with five published LLM RCA agents and with statistical and causal methods on RCAtlas, 90 validated cases from eight public datasets, across six models and three reasoning efforts. The simple harnesses are the most accurate (PR@1 65-77% on three mid-tier models), ahead of the published agents (16-53%) and the non-LLM methods (6-39%), on every model on which both families were run. The published agents fix from a tenth to all of their model calls in the program, and a case study shows how these prescriptions skip the decisive evidence. A stronger model mainly improves reliability, a multi-turn harness mainly improves aptitude, and more reasoning effort changes accuracy by a few points at most while multiplying the time by up to seven. Among runs more accurate than the best non-LLM method, every Pareto-optimal run in cost or time uses a simple harness.