研究显示 RLVR 只是强化模型已有能力，并未带来新推理知识
RLVR Sharpens the Pencil, Not the Mind
有人在 OLMo-2-1B 的 13 个训练检查点上做了逐层解剖，用探针数据证明 RLVR 只是让模型挑出本来会的解法，没教会它新东西，数据和结论都挺扎实。
一篇 arXiv 论文追踪了 OLMo-2-1B 在 RLVR 训练全部 13 个检查点上的内部变化。在 GSM8K 上，首次尝试准确率从 58.0% 升至 61.3%，但让模型作答 256 次后，解法覆盖率从 94.5% 升至 96.9%，说明解题知识在训练前就已存在。研究者向题目中植入推理错误并训练探针检测，13 个检查点、3 类错误下 AUROC 保持在 0.798-0.840，区间差异小于置信区间，即模型内部对错误的认识没有变。唯一可检测的内部变化出现在最终答案 token 处，区分早期与晚期检查点的探针 AUROC 达 0.93，而在实际推理步骤上接近随机。结论支持 sharpening 假说：RLVR 让模型优先采用最优解法路径，而非获得新推理能力。
RLVR Sharpens the Pencil, Not the Mind
Models like DeepSeek-R1 made Reinforcement Learning with Verifiable Rewards (RLVR) famous: reward a model for correct answers and it gets dramatically better at reasoning without being taught how. The leading explanation is sharpening---that the base model already knows how to solve these problems and RLVR just makes it more likely to reach for its best solution paths. This has only ever been checked by comparing before and after training, never by watching what changes inside across it. We tracked OLMo-2-1B across all 13 training checkpoints on GSM8K. First-try accuracy climbs from 58.0% to 61.3%, but ask the model 256 times and solution coverage actually rises from 94.5% to 96.9%, proving the knowledge was already there. We then planted reasoning mistakes into problems and trained simple probes to detect them. Those probes held between AUROC 0.798 and 0.840 across all 13 checkpoints and 3 error types, a spread smaller than the confidence interval; the model never builds a sharper internal picture of what a reasoning mistake looks like. The one place we find real internal change is at the final answer token, where a probe distinguishing early from late checkpoints hits an AUROC of 0.93, while the same probe on the actual reasoning steps lands near chance. RLVR makes the model lead with its best thinking. It does not change what that thinking is.