这项研究揭示了强化学习在良性事实上的应用可能带来的个人信息泄露风险，值得对AI模型安全性和隐私保护进行深入探讨。
使用可验证奖励的强化学习（RLVR）用于提升模型推理能力，但其对模型泄露信息的影响研究不足。研究发现，在良性事实数据上应用RLVR会提高模型提取已记忆的个人信息（PII）。在DeepSeek-V3.1上，针对姓名-电子邮件对的精确召回率从0.155提升到0.370，增幅达2.4倍。模型大小与泄露程度成正比，在参数量最大的模型上泄露最严重。同时，模型推理能力和拒绝率保持不变，表明RL只是选择性地改变可访问的已记忆信息，而非全面改变。通过从未接触过的训练提升可提取的个人信息，为攻击者提供了获取记忆数据的途径，无需隐私相关的训练信号和数据访问权限，只需具备微调无害数据的能力。
Reinforcement Learning on Benign Facts Amplifies Leakage of Memorized Private Data
Reinforcement learning with verifiable rewards (RLVR) is deployed to make models better at reasoning tasks, but its side effect on what models will divulge is under studied. Here we show that RLVR on facts increases extraction of personally identifiable information (PII) the instruct model had already memorized. We first confirm that instruct models have already memorized PII but leave them latent, rarely surfacing one when asked. We then apply RL on benign factual data that contains no PII of any kind, and re-probe: a targeted probe over name->email pairs, and an untargeted free-recall prompt that simply asks the model to list the addresses it knows. PII extraction rises sharply under both: on DeepSeek-V3.1, verbatim recall@k increases from 0.155 to 0.370, a 2.4x gain. The effect scales with model size: across three models spanning 8B to 671B parameters, absolute leakage is largest in the biggest model. Meanwhile model's reasoning abilities and refusal rates are retained, indicating that RL selectively changes which memorized information is accessible rather than broadly altering the model. In summary, memorized private data can be made markedly more extractable by training that never touches it. This gives an adversary a route to memorized data that requires no privacy-relevant training signal and no access to the data itself -- only the ability to fine-tune on something innocuous.