这篇论文提出了一种不用人工任务标注的训练方法，让LLM智能体自己从环境里找目标学，在ALFWorld和ScienceWorld上有效，还能给后续强化学习打底子。
State2State是一种从环境交互中派生训练目标的方法，无需外部指定的任务或人工监督。它通过规则匹配状态来验证智能体是否到达目标状态，从而提供可扩展且可验证的训练信号。在ALFWorld和ScienceWorld基准上，State2State作为独立训练阶段在多数设置下提升了智能体性能。用作下游强化学习的初始化时，它进一步提高了最终性能和采样效率，并展现出跨环境泛化的潜力。
State2State: Environment-Derived Mid-Training for LLM Agents
Training LLM agents commonly relies on supervised fine-tuning from expert trajectories or online reinforcement learning over human-specified tasks with handcrafted verifiers. Though effective, both remain bottlenecked by externally specified tasks and supervision signals, limiting the scalability and diversity of agent training. We study an environment learning paradigm in which agents acquire interaction and manipulation capabilities solely through environment interaction, without externally specified tasks. We propose State2State, an environment-derived mid-training method that converts explored environment states into training objectives, challenging agents to reach a specified target state. By deriving tasks from environment exploration and verifying success through rule-based state matching, State2State provides scalable and verifiable training objectives without expert supervision or manual task design. Experiments on ALFWorld and ScienceWorld show that State2State improves agent performance as a standalone environment-learning stage in most settings. As initialization for downstream RL, it further improves final performance and learning efficiency, with promising evidence of cross-environment generalization.