研究:LM Harness 是组合泛化器,可让 LLM 超越训练分布
Alex Zhang 团队的新论文,把 harness 从"给弱模型打补丁"重新定位成泛化机制,还给出 surrogate training 这种省钱训法。做智能体工程、纠结要不要自己写 harness 的开发者应该看看,论文、代码和日志全公开。
Alex Zhang 与同事发布论文《LM Harnesses are Compositional Generalizers》,核心论点是 harness 并非弱模型的拐杖,而是让 LLM 把已有能力组合起来、在任务长度、任务策略、递归深度等维度泛化到训练环境之外的机制。
论文的关键技术是让所有单次 LLM 调用保持局部在分布内(Locally In-Distribution, LID),使轨迹更适合数据高效训练。对需要超长 rollout 的任务,作者提出 surrogate harness training:只用纯合成任务(伪造的工具调用和子智能体)训练单次 LLM 调用,不必跑完整的长任务流程。
论文直接回应了"模型变强后 harness 会被吸收而淘汰"的说法,认为 harness 的价值远未被探索充分。文中提到 RLM 是近期代表,并指出 post-training 与 harness 协同进化(co-evolve)正在显现效果, Omar 社区中也有人在 meta harness 里用动态工作流复现了这一现象。
Highly recommended read. Learn to build your own harness, folks. My belief—and that of many fully AI-native builders and companies—is that harnesses remain largely underexplored. Research shows a huge edge in custom harnesses. But little is known about why. Alex and colleagues have been exploring this direction and have noted that harnesses can act as compositional generalizers. In other words, harnesses can push LLMs further (with stronger generalization effects) through existing LLM capabilities. Harnesses like RLMs are a good recent example, with abstractions that unlock this. I have recently started seeing success in post-training used to co-evolve harnesses and models. I think this could explain why this is happening, but more research is needed to understand this better. I have experienced this with a variant of dynamic workflows I have implemented in my meta harness. alex zhang @a1zhang Harnesses used to be a crutch for weak LLMs. In 2026 and onward, with stronger models, harnesses now offer a mechanism for composing an LLM’s existing and new capabilities to generalize well beyond their training environments. In our new paper, LM Harnesses are Compositional Generalizers, we explore how various harness abstractions enable LM systems to generalize over task length, task strategies, recursive task depth, and more through composing existing LLM capabilities. These abstractions provide a means for more data-efficient training, especially over long tasks, by keeping trajectories locally in-distribution (LID) — i.e., shaping all individual LLM calls to be in-distribution. For tasks that take a very long time to roll out, this implies that we can even perform surrogate harness training, meaning we train an individual LLM call instead of doing the very long harness rollout, using purely synthetic tasks, with fake tool calls + sub-agents! A common idea floating around is that LLMs will absorb harnesses and make them obsolete. We disagree, and argue that we haven’t sufficiently explored what harnesses can provide. The full paper, logs, codebase, and data are available: alphaxiv.org/abs/2611.harne… w/ @SOURADIPCHAKR18 @lateinteraction 🔗 View Quoted Tweet📊 9 ⚡
This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include LLM Harness, agents, RLM, compositional generalization.
Source attribution: curated from elvis (@omarsar0).