无需示范的分层强化学习技能组合方法用于导航与机械臂操作
Demonstration-Free Skill Composition for Navigation and Manipulation
这篇论文展示了不靠人类示范也能把任务拆成小技能分别训练再组合,导航上比 flat DQN 省交互,机械臂上比 flat SAC 更稳。做机器人 RL 或分层控制的研究者可以看看它的管线细节。
一篇 arXiv 论文评估了不依赖人类示范的分层强化学习管线,任务涵盖 MiniGrid 导航和机器人操作。在导航任务中,两个参数化技能在人工指定的高层计划下复用,跨未见过的 FourRooms 布局测试,分层管线大幅超过 flat DQN 基线,且所需训练交互更少。
在操作任务中,reaching、grasp alignment 和 lifting 三个技能分别独立训练,下游技能用前序技能 rollout 收集的状态做初始化。结果显示分层链条比 flat SAC 需要更少训练步数,且跨随机种子的表现波动更低。
论文还把冻结的 Pick 和 Lift 技能直接复用于堆叠任务,只需额外训练一个 Place 技能。
Demonstration-Free Skill Composition for Navigation and Manipulation
Hierarchical reinforcement learning enables sequential tasks to be represented as compositions of separately trained skills. This paper evaluates demonstration-free hierarchical pipelines for MiniGrid navigation and robotic manipulation, comparing their training requirements and task performance with flat reinforcement learning baselines. The evaluation concerns complete pipelines with task-specific observation, reward and control structures. In navigation, two parameterised skills are reused across unseen FourRooms layouts under manually specified high-level plans. In manipulation, reaching, grasp alignment and lifting are trained separately, with downstream skills initialised from states collected during predecessor rollouts. Across unseen navigation layouts, the hierarchical pipeline substantially outperforms the evaluated flat DQN baselines while requiring fewer training interactions. In manipulation, the hierarchical chain requires fewer training steps than flat SAC and exhibits lower observed variation across seeds. The frozen Pick and Lift skills are further reused for stacking by training an additional Place skill.
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include reinforcement learning, Hierarchical RL, MiniGrid, robot manipulation.
Source attribution: curated from arXiv cs.AI (@Madeleine Nouri, Gelareh Mohammadi, Francisco Cruz).