Papers

RIFT:基于奖励改进场迁移的跨域离线强化学习方法

RIFT: Reward Improvement Field Transfer for Cross-Domain Offline RL

Why we picked it

强化学习论文,作者提出 RIFT,用最优传输配对源域数据学出奖励改进场,再搬到目标域合成数据,在重力、形态等偏移的运动控制任务上跑赢了基线。做离线 RL 或跨域迁移的研究者可以看看误差上界那部分的推导。

arXiv 上的一篇论文提出 RIFT(Reward Improvement Field Transfer),针对跨域离线强化学习中目标域数据稀缺的问题。与传统复用源域转移样本的做法不同,RIFT 学习源域中低奖励与高奖励转移之间的差异结构:将源域与目标域转移编码到共享潜空间,用最优传输配对低、高奖励的源域转移,并对位移拟合一个仿射场。

Body

该仿射场随后作用于目标域的低奖励锚点,解码后生成增广数据加入目标数据集,核心假设是相关域之间即使动力学不同,也可共享“低奖励到高奖励”的改进结构。论文在共享类几何假设下给出合成误差上界,分析了目标域锚定优于源域高奖励移植的条件,并推导了与下游回报的条件性关联。

实验在重力、形态、摩擦三种偏移下的密集奖励运动控制任务上验证,聚合性能优于现有基线。对做离线 RL 数据增广或跨域泛化的研究者来说,这套场迁移思路提供了一条不依赖直接转移复用的替代路线。

Original · arXiv cs.LG

RIFT: Reward Improvement Field Transfer for Cross-Domain Offline RL

Learning from limited target data is a central challenge in cross-domain offline reinforcement learning. Abundant source data can help, but dynamics differences can make direct transition reuse unreliable. We propose Reward Improvement Field Transfer (RIFT), which instead learns how low- and high-reward source transitions differ. RIFT encodes source and target transitions in a shared latent space, pairs low- and high-reward source transitions using optimal transport, and fits an affine field to their displacements. It applies this field to low-reward target anchors and decodes the results to augment the target dataset. The approach builds on the hypothesis that related domains can share useful low-to-high reward structure despite different dynamics. We bound synthesis error under shared class geometry, identify conditions favoring target anchoring over source-high transplant, and derive a conditional connection to downstream return. Experiments on dense-reward locomotion tasks under gravity, morphology, and friction shifts show improved aggregate performance over existing baselines.

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include offline reinforcement learning, optimal transport, RIFT, domain generalization.

Source attribution: curated from arXiv cs.LG (@Jinhee Won, Mardavij Roozbehani, Munther Dahleh).