他们提出了 TRL，让强模型和弱模型组队推理，强模型学会写弱模型能看懂的推理过程。训练 Qwen3-4B 后，单打能力不降，协作能力更强。
Tandem Reinforcement Learning (TRL) 将 tandem 训练范式引入带可验证奖励的强化学习（RLVR）。TRL 让一个较强的 senior 模型与一个冻结的 junior 模型随机交替协作生成推理过程，对最终结果给予奖励，并对 senior 应用标准 GRPO 损失。在 Qwen3-4B-Instruct 上使用竞赛数学训练，TRL 的 solo 推理能力与 vanilla GRPO 持平，但同时提升了 senior 与 junior 的交接鲁棒性、减少了 junior 侧分布漂移，并产出了对 junior 更易理解的思维链。该工作为多模型通信与人类兼容性提供了实际收益的路径。
Tandem Reinforcement Learning with Verifiable Rewards
Reinforcement learning with verifiable rewards (RLVR) has significantly improved the reasoning capability of large language models, reaching expert or even superhuman performance in domains such as competition math. However, whether weaker agents and humans can actually harness this capability is far less certain, with RLVR documented to drift reasoning toward idiosyncratic patterns such as poor readability and language mixing. Tandem training is a recently introduced paradigm that targets this compatibility problem: a trained, stronger senior co-generates each rollout with a frozen, weaker junior, and the two are rewarded as a team, so the senior is pushed to reason in ways the junior can follow. Yet this paradigm has so far been demonstrated only in proof-of-concept settings, leaving open whether it scales to the long chains of thought of the modern RLVR pipeline. In this work, we propose Tandem Reinforcement Learning (TRL), which carries the tandem training paradigm into RLVR. In TRL, the senior and a frozen junior alternate stochastically to co-generate the reasoning, the resulting generation is rewarded, and the standard GRPO loss is applied to the senior. Training Qwen3-4B-Instruct on competition math, we find that TRL matches vanilla GRPO on solo reasoning capability while three properties emerge together from the same rollout structure: stronger handoff robustness with the junior, reduced distributional drift from the junior, and a chain-of-thought more legible to the junior. Our results demonstrate a promising route for RLVR with practical payoffs in multi-model communication and human compatibility.