UC Berkeley 论文：自改 harness 的智能体提分同时丢失已能完成的任务
做智能体的朋友可以看看这篇。它用 GPT-5.4 跑了 GEPA、HyperAgents 等 4 种进化方法，发现平均分涨了但任务可能偷偷变少、成本变高。如果你在用自动改 prompt 或 skill 的框架，这篇给出的三个评估指标很实用。
UC Berkeley 等多个顶级实验室的论文发现，让智能体自行改写 harness（提示词、技能、工具或代码，模型本身保持冻结）虽然平均基准分数上升，但会掩盖任务得失：所有被测智能体都同时获得了新任务、丢掉了原本能解决的任务。
团队用 4 种进化方法（含 GEPA 和 HyperAgents）搭配 GPT-5.4，在 2 个 harness 和 2 个基准上做实验，每个进化后的智能体相对初始版本重测 8 次。
16 个进化后的智能体全部出现任务此消彼长的情况，其中一个可靠性提高 5.2 分，却丢掉了 10 个原本能完成的任务；
在 11 个可靠性提升的智能体中，有 8 个的单次运行成本也随之上升。论文建议评估 harness 进化时同时追踪任务丢失数、单次运行成本以及在未见任务上的表现，而不是只看平均分。
New University of California, Berkeley+other top labs paper finds that agents that rewrite their own harness to score higher also lose tasks they once solved, which the average score hides.
Harness evolution lets an agent edit its own prompts, skills, tools, or code while the model stays frozen. Results usually come as 1 higher benchmark score, which can hide lost tasks, higher costs, and gains that don't carry over.
They ran 4 evolution methods, including GEPA and HyperAgents, with GPT-5.4 on 2 harnesses and 2 benchmarks. Then they re-tested each evolved agent 8 times against its starting version.
All 16 evolved agents gained some tasks and lost others, and 1 raised reliability by 5.2 points while losing 10 tasks it used to cover. Of 11 agents that became more reliable, 8 also cost more per run.
When you evolve an agent harness, track tasks lost, cost per run, and results on unseen tasks, not just the average score.
This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include GEPA, HyperAgents, GPT-5.4, 智能体.
Source attribution: curated from rohanpaul_ai (@rohanpaul_ai).
