这篇论文教你用分层SAC解决稀疏奖励的长期任务，比普通SAC成功率更高，代码已开源。
论文提出一种两层分层强化学习（HRL）框架，高层负责策略规划，底层采用连续控制算法Soft Actor-Critic（SAC），并通过熵正则化策略优化。该框架在Search-and-Rescue-2（SAR-2）数据集上训练和评估。HRL-SAC在成功率、覆盖效率和收敛速度上均优于普通SAC基线，有效解决延迟奖励和连续控制问题。实验表明分层熵正则化策略是处理长视野稀疏奖励任务的有前景方案。
Hierarchical Soft Actor-Critic for Sparse-Reward Long-Horizon Reinforcement Learning
Exploration in sparse-reward long-horizon tasks poses significant challenges for reinforcement learning. To address these challenges, we propose a two-level Hierarchical Reinforcement Learning (HRL) framework. The first level handles high-level strategic planning, while the low-level uses the continuous-control Soft Actor-Critic (SAC) algorithm, and they utilize entropy-regularized policy optimization. The proposed framework was trained and evaluated using the Search-and-Rescue-2 (SAR-2) dataset. HRL-SAC effectively addresses sparse-reward long-horizon search problems characterized by delayed rewards and continuous control, and its outperforming the flat SAC baseline reinforcement learning in terms of success rates, coverage efficiency, and convergence. These findings indicate that hierarchical entropy-regularized policies are a promising solution to tackle long-horizon sparse-reward reinforcement learning tasks.