ASALT这个新方法解决了MARL中状态空间维度不同时知识迁移的难题,实验效果不错,值得做多智能体迁移学习的同学看看。
ASALT是一种针对多智能体强化学习(MARL)的迁移学习方法,解决了源域和目标域观测空间与全局状态空间维度不匹配的问题。该方法引入观测层和状态层适配器,将目标域观测和全局状态映射到共享嵌入空间,支持跨异构域的策略迁移。在标准基准环境的多个配置上,ASALT在合作场景中相比现有基线提升了样本效率和全局回报,但效果依赖于源域与目标域的不匹配程度。实验还表明ASALT能缓解负迁移,这是跨不同观测和动作空间域迁移时的常见障碍。
ASALT: Adaptive State Alignment for Lateral Transfer in Multi-agent Reinforcement Learning
Multi-agent reinforcement learning (MARL) addresses the problem of training multiple agents that pursue collaborative, competitive, or mixed objectives. Prior work has investigated transfer learning between source and target domains in MARL; however, the majority of existing approaches impose the constraint that the dimensionalities of the observation space and the global state space must be identical across domains. In this paper, we introduce a method that explicitly accommodates mismatched state-space dimensionalities between source and target domains. The proposed approach, ASALT, incorporates both observation-level and state-level adapters that map the target-domain observations and global states into a shared embedding space, thereby enabling more effective transfer of knowledge across both actors and critics. These adapters can generate embeddings that support efficient strategy transfer across heterogeneous domains. Experimental results on multiple configurations in standard benchmark environments demonstrate that ASALT surpasses existing baselines in terms of sample efficiency and global return in cooperative settings, but its effectiveness depends on the degree of mismatch between source and target domains. Furthermore, our findings indicate that ASALT mitigates negative transfer, which frequently constitutes a major obstacle when transferring policies between domains with differing observation and action spaces.