这篇论文直击 LLM Agent 在陌生环境中的核心短板——过早行动，做智能体开发或强化学习的研究者值得一读，提出的探索-执行范式可以直接启发你的训练策略。
论文指出基于大语言模型的智能体在陌生环境中常因过早利用先验知识而失败，提出自主探索能力是关键但被忽视的环节。作者引入可验证的探索检查点覆盖率指标，量化智能体发现关键状态、物体和功能的能力。实验发现标准任务导向强化学习训练出的智能体行为狭窄重复，阻碍下游性能。为此提出探索-执行交替训练策略，并构建先探索后行动的范式，让智能体先利用交互预算获取环境知识再执行任务。结果表明系统学习探索对构建通用、真实世界可用的智能体至关重要。
Look Before You Leap: Autonomous Exploration for LLM Agents
Large language model based agents often fail in unfamiliar environments due to premature exploitation: a tendency to act on prior knowledge before acquiring sufficient environment-specific information. We identify autonomous exploration as a critical yet underexplored capability for building adaptive agents. To formalize and quantify this capability, we introduce Exploration Checkpoint Coverage, a verifiable metric that measures how broadly an agent discovers key states, objects, and affordances. Our systematic evaluation reveals that agents trained with standard task-oriented reinforcement learning consistently exhibit narrow and repetitive behaviors that impede downstream performance. To address this limitation, we develop a training strategy that interleaves task-execution rollouts and exploration rollouts, with each type of rollout optimized by its corresponding verifiable reward. Building on this training strategy, we propose the Explore-then-Act paradigm, which decouples information-gathering from task execution: agents first utilize an interaction budget to acquire grounded environmental knowledge, then leverage it for task resolution. Our results demonstrate that learning to systematically explore is imperative for building generalizable and real-world-ready agents.