论文精选

同一模型和提示词,换 Agent Harness 单次成功成本相差 5-30 倍

Picking the right agent harness is now a crucial skill for any AI engineer. Imagine using the same ...

精选理由

别急着换模型,先看看 agent harness。同一任务成功成本能差 5 到 30 倍,论文还给了省 token 的提示模板。

AI 摘要

新基准对比了 6 个大型推理模型在两种真实 agent harness 上的表现。同一模型、同一任务和提示词,换一个 harness 后单次成功成本可相差 5 到 30 倍。实验包含 24 个带隐藏评估器的确定性编码任务和 4,643 次有效运行。让模型自行开发并比较多种方案会使推理 token 增加 2.4 到 7.4 倍且无正确性提升;泛泛的 think-deeply 提示再增加 1.6 到 2.2 倍。采用限定范围、验收标准和停止条件的模板可做到成本中性,有时将推理消耗减半。

原文 · elvis

Picking the right agent harness is now a crucial skill for any AI engineer. Imagine using the same ...

Picking the right agent harness is now a crucial skill for any AI engineer. Imagine using the same model, same task, and same prompt. Now move it between two agent harnesses and the cost per success can swing by 5 to 30x. This benchmark measured this across six large reasoning models, two real harnesses, 24 deterministic coding tasks with hidden evaluators, and 4,643 valid runs. Asking a model to develop and compare several approaches raised reasoning tokens by 2.4 to 7.4x with no correctness gain. Generic think-deeply cues added another 1.6 to 2.2x. A bounded-efficiency template that specifies scope, acceptance criteria, and a stop condition came out cost-neutral and sometimes halved reasoning. Harness design and prompt wording decide most agent spend before the model reasons at all, and both are cheap to change. Paper: arxiv.org/abs/2608.01347 Track more trending AI papers in our academy: academy.dair.ai 💬 12 🔄 8 ❤️ 27 👀 3387 📊 19 ⚡