一篇教你用强化学习优化代码执行速度的论文，针对测量噪声和奖励稀疏问题设计了DMC-Optim基准和三阶段方法，让Qwen 2.5 7B的pass@1从18%提到31%，CWM 32B提到50%。
论文提出DMC-Optim基准，用于评估代码优化RL。通过三阶段方法解决测量噪声和奖励稀疏问题：构建校准沙箱测试，组合正确性与速度奖励，并适配GRPO到稀疏场景。在DMC-Optim上，Qwen 2.5 7B的top-50% pass@1从18.0%提升至31.3%，CWM 32B从30.7%提升至50.4%。top-30%时CWM 32B相对提升125%，且保持纯正确性分数不变。在LCB上，CWM 32B赢得83%的中位数样本速度对比。
Reinforcement Learning for Code Optimization
RL for code correctness is now established: have the model generate a program, run it against hidden test cases, and reward solutions that pass. Extending this to code optimization seems straightforward: just add execution time to the reward. But in practice, once timing drives the reward, small problems in measurement noise, reward sparsity, or GRPO instability overwhelm the signal and make RL fail: generated solutions are barely faster, and more of them can fail. We make execution time learnable through three stages: (1) how code is tested, by building DMC-Optim with large optimization tests and a calibrated sandbox; (2) how speed is turned into reward, by composing correctness and speed in the RL environment and using an offline simulator to predict the most promising configurations; and (3) how the model learns from that reward, by adapting GRPO and evaluation to the sparser, noisier timed-execution setting. On DMC-Optim, the strongest optimization-aware configurations improve strict top-50% pass@1 from 18.0% to 31.3% on Qwen 2.5 7B and from 30.7% to 50.4% on CWM 32B. These gains further increase at stricter percentiles such as top-30%, with 125% relative improvement for CWM 32B, while preserving pure-correctness scores. When the timing sandbox is degraded, robust optimization RL reaches 100% to 200% improvement over standard RLVR, depending on the evaluation criterion. On LCB, CWM 32B wins up to 83% of median-sample speed comparisons against standard RLVR. Relative to the fastest correct human submissions per problem, it reaches about half the human rate of complexity-class improvements (14% vs. 28%).