推理模型开发者长期受困于验证器失效导致自改进停滞，STV用参考答案不对称性巧妙破解，在困难数学和科学任务上效果显著，做自训练或测试时搜索的团队值得深入看。
论文提出自训练验证（STV）方法，解决推理模型在测试时验证-精炼循环和训练时自训练中验证器失效的瓶颈。核心发现是模型单独无法捕捉自身错误，但看到参考答案后可以，利用这一不对称性训练验证器模仿更知情版本。STV在困难数学题上准确率翻倍，科学推理任务从1.5%提升至21%。结合验证器在循环中的强化学习（ViL），使pass@1再提升33%，且生成器独立推理能力也超越标准RL收敛点。这表明推理模型的下一个前沿在于如何训练验证及利用验证。
Self-Trained Verification for Training- and Test-Time Self-Improvement
Self-improvement at scale has been a longstanding goal for reasoning models, and there are two natural places to do it: at test time, through verification-refinement (V-R) loops; and at training time, through self-training methods. Both are gated by the same bottleneck: the verifier. V-R loops stall when verifier scores inflate while accuracy stagnates, and when feedback is too generic to act on; self-training fails similarly when bad self-generated data are added to training. Better verification would unlock both, but the capability we want to train, i.e., catching self-generated errors, lacks training signal. To address this challenge, we propose self-trained verification (STV). Our key observation is that, while a model cannot catch these errors alone, it can when shown the reference solution. We turn this asymmetry into a supervision target and train the verifier to imitate a more informed version of itself. At test time, STV substantially improves V-R loops on hard problems, while alternatives (e.g., SFT, RL on verifier scores, and even meta-verifiers) do not. STV roughly doubles accuracy on hard math and lifts it 14x on scientific reasoning tasks (1.5% to 21%). At training time, we additionally train the generator using RL with STV verifier's feedback inside the V-R loop - a procedure we call verifier-in-the-loop training (ViL). Starting from an RL-converged generator, ViL yields a further 33% gain in pass@1. More notably, the generator's standalone pass@1, with no verifier at test time, climbs 30% relative past where standard RL had converged. Hence, the next frontier in reasoning on hard problems may lie in how we train for and with verification.