FIRE 运行时策略让 Agent 重复成功率最高提升 9.2 个百分点
FIRE: Failure-Informed Runtime Engineering for Reliable Language-Model Agents
一篇很实用的论文：不改模型、不改提示词，只在运行时注入失败 informed 的策略指令，GPT-5.6 跑 Terminal-Bench 的重复成功率就能涨近 10 个点，做 Agent 的可以看看。
arXiv 论文 FIRE 提出运行时策略：Agent 框架在失败发生前的状态注入自然语言指令或拒绝某些动作，不改模型权重和用户提示。在 Terminal-Bench 2.1 全部 87 个任务上，GPT-5.6 三个档位的 pass^2 均提升：Luna 从 50.6% 到 54.0%，Terra 从 55.2% 到 60.9%，Sol 从 64.4% 到 73.6%。五臂随机对照实验显示，真实策略在符合条件任务上达到 61%，无策略时为 39%，时间匹配的假策略仅 36%，排除了解释为额外验证或时间的可能。成本方面，Terra 加策略达到 71.4%，超过无辅助的 Sol 的 64.3%，成本约为其一半。
FIRE: Failure-Informed Runtime Engineering for Reliable Language-Model Agents
Language-model agents often reach a working solution and then fail to consistently deliver it. We study runtime policies: targeted natural-language instructions and action denials applied by the agent harness at states that preceded observed failures, without changing model weights or the user prompt. With this, keeping capability constant, we observe a meaningful unlock in delivered reliability. Across the complete 87-task Terminal-Bench 2.1 suite, with two attempts per task, policies increase repeated success (pass^2) in all three GPT-5.6 tiers: 50.6% to 54.0% for Luna, 55.2% to 60.9% for Terra, and 64.4% to 73.6% for Sol. Sol's best-of-two success changes by 1.2 points while repeated success rises by 9.2, showing that policies chiefly convert reachable solutions into dependable delivery. We further cover 14 tasks under Terra's frozen portfolio. Policy-guided Terra reaches 71.4%, compared with 64.3% for unassisted Sol, at about half the cost, demonstrating how engineering around models could unlock dependability for a use case. To isolate the mechanism we run a randomized five-arm experiment: real policies reach 61% on eligible tasks, versus 39% without a policy, 36% with a timing-matched sham, and 39 to 43% with generic verification or reconsideration. The intended corrective behavior appears in 22 of 24 coded policy attempts, against at most 14 in any other arm. Runtime policies are therefore a practical reliability layer: they make capabilities an agent already possesses substantially more repeatable.
