arXiv 论文:LLM 智能体无需人工指定即可自主构建自我改进算法
Do LLM Agents Still Need to Be Told How to Self-Improve?
斯坦福系研究团队把训练循环的设计权交给了模型本身,冻结的 LLM 只靠改文本就变强了,还能迁移到 3D 导航。做 Agent 开发和强化学习的人应该看看,它可能预示下一阶段 post-training 的写法。
arXiv 论文 2611.23219 研究了一个自主自我改进范式:给定基础工具、计算预算和通用目标后,一个冻结的 LLM 智能体能否自行设计自我学习算法。实验以游戏为测试环境,智能体只能收集经验并更新文本文档,没有外部强加的训练循环,也无法访问真实世界数据。
结果显示,能力足够的模型确实在这种条件下实现自我改进,其采用的策略与经典自我调节学习认知框架以及自生成课程(self-generated curricula)相关研究一致。论文还考察了学习到的文本产物能否迁移:在更长游戏时间线、不同游戏机制以及 3D 导航的外部基准上均观察到提升。
这项工作回应了一个核心问题:随着模型能力增强,人类是否还需要为它们指定自我改进的方式。论文给出的答案是基本机制可以交给模型自己,人类只需提供原语、资源和目标。
Do LLM Agents Still Need to Be Told How to Self-Improve?
LLM agents increasingly drive their own learning. Self-improvement methods delegate various components of model optimization to the models themselves. As models become more capable, this motivates a basic question: do these increasingly capable models still need us to specify how they should self-improve? In this paper we study an autonomous paradigm in which, given basic primitives, resources, and a general goal for self-improvement, agents construct their own self-learning algorithm for context-based optimization. Using the setting of gameplay as a testbed, a frozen LLM agent is given a compute budget and tools for collecting experience and updating text documents, and allowed to freely experiment without an externally imposed loop or access to real-world data. We find that sufficiently capable models do self-improve under these conditions, demonstrating strategies that align with classic cognitive frameworks for self-regulated learning and constructs from machine learning literature on self-generated curricula. Furthermore, we study the extent to which the learned text artifacts transfer to external benchmarks, observing improvement across different game horizons, mechanics, and 3D navigation.
This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include LLM Agent, recursive self-improvement, arXiv.
Source attribution: curated from arXiv cs.AI (@Shobhita Sundaram, Cansu Sancaktar, Phillip Isola).