字节跳动新出的HarnessDev框架，让AI从完成任务进化到构建智能体，在写作和ML实验上已能匹敌人工工程。
字节跳动Seed团队提出HarnessDev方法，不再评估模型完成任务的表现，而是评估其构建的智能体框架。该框架从弱种子和少量案例开始，构建完整执行系统，再通过下游反馈改进。实验涵盖6个创作者LLM、4个领域和2207个保留下游实例。结果显示，生成的框架在代码和搜索研究方面落后于成熟的人工工程参考，但在写作和机器学习实验方面与之相当或更优。
Banger paper from ByteDance Seed. If you are curious about self-evolving agent harnesses, this one ...
Banger paper from ByteDance Seed. If you are curious about self-evolving agent harnesses, this one is worth your time. (bookmark it) The proposes method, HarnessDev, stops scoring a model on the tasks it completes and scores it on the harness it builds. The agent starts from a weak but runnable seed plus a handful of cases, then builds a full execution system. A second stage hands that harness back and asks it to improve on downstream feedback. Both stages are scored on capability and on execution-token cost, so there is awareness of efficiency and spend. The experiments include six creator LLMs, four domains, 2,207 held-out downstream instances. The result splits by domain. Generated harnesses stay well behind mature human-engineered references on code and on search and research, while matching or beating them on writing and machine-learning experimentation. Evolution produces gains, but they are unstable and transfer only partially to held-out tasks, and they depend heavily on which model runs the harness. Paper: arxiv.org/abs/2609.01437 Chat with Paper: academy.dair.ai/papers/harness… 💬 1 🔄 5 ❤️ 10 👀 1131 📊 5 ⚡