IPR：结合动作前缀预测与跨模态对齐的多模态强化学习方法
Integrating Action Prefix Prediction and Alignment Across Modalities for Multimodal Reinforcement Learning
做强化学习表征学习的值得看看这篇。它改进了 MJPR 那种依赖递归 rollout 的预测方式，改成从观测锚点直接预测，在 DeepMind Control Suite 上平均回报高了 10%。如果你的智能体只有部分模态数据，收益可能更明显。
一篇 arXiv 论文提出 Improved Predictive Representation（IPR），针对多模态联合预测表征（MJPR）在递归 rollout 依赖早期预测、配对对齐只关注模态间对应关系的缺陷进行改进。
IPR 的做法是：基于对比多模态学习，在每个预测步长上直接从观测锚点和对应动作序列预测未来表征，并引入动作前缀预测。论文测试了两种裁剪配置：最小保留裁剪面积为 10% 的 IPR-S 和 50% 的 IPR-H。
在 DeepMind Control Suite 的五个任务上，IPR-H 相比基线在使用视觉与深度图像时平均回报高出 10.0%，使用视觉图像与位置向量时高出 3.4%。在 Webots 仿真环境中，两种 IPR 配置在没有 LiDAR 时平均最终回报均高于基线，但有 LiDAR 时则不再有优势，说明该方法在部分模态缺失场景下收益更明显。
Integrating Action Prefix Prediction and Alignment Across Modalities for Multimodal Reinforcement Learning
Learning representations from multimodal observations is difficult in reinforcement learning with limited interaction data. Multimodal Joint Predictive Representation combines multimodal latent prediction with contrastive learning across modalities, but its recursive rollout depends on earlier predictions and its pairwise alignment focuses on correspondence between modalities. We propose Improved Predictive Representation, which combines prediction from action prefixes with alignment at the observed anchor based on Contrastive Multimodal learning. At each horizon, it predicts future representations directly from the observed anchor and the corresponding sequence of actions. We evaluate two crop configurations: IPR-S with a minimum retained crop area of 10%, and IPR-H with 50%. Across five DeepMind Control Suite tasks, IPR-H achieves an average return across tasks that is 10.0% higher than the baseline with visual and depth images and 3.4% higher with visual images and position vectors. The relative performance of the two configurations varies across tasks. In Webots, both IPR configurations achieve higher mean final returns than the baseline without LiDAR, but not when LiDAR is available.
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include 强化学习, 多模态, DeepMind Control Suite, 对比学习.
Primary source verification: published directly by arXiv: Google DeepMind with first-party authority.
