MGE-PO：用熵感知信用分配改进数学推理训练
MGE-PO: Entropy-Aware Credit Assignment for Mathematical Reasoning
这篇论文针对 GRPO 对同奖励响应一视同仁的问题，用生成熵给对错响应加不同权重的修正项，Qwen2.5-7B 上 avg@32 从 41.5% 提到 45.4%。
论文提出 MGE-PO，核心是四象限（4Q）优势整形，将验证的正确性与 rollout 生成熵结合：正确性决定额外修正的方向，熵决定幅度，低熵成功获得更大奖励、高熵失败受到更大惩罚，并配合基于在线熵的问题课程。相比 GRPO 对同组等奖励响应赋予相同优势的做法，该方法利用了生成不确定性。在 Qwen3-8B-Base 上训练八个 epoch 后，三种策略优化设置在六基准 macro avg@32 上带来 3.23-3.69% 的绝对提升，macro pass@32 提升 4.24-11.06%。在 Qwen2.5-7B-Base 上，完整配置将四基准 macro avg@32 从 41.5% 提升到 45.4%。
MGE-PO: Entropy-Aware Credit Assignment for Mathematical Reasoning
GRPO assigns the same group-relative advantage to equally rewarded responses within a question, leaving their generation uncertainty unused. We introduce MGE-PO, centered on four-quadrant (4Q) advantage shaping that combines verified correctness with rollout generation entropy. Correctness determines the sign of an additional correction, while entropy controls its magnitude: lower-entropy successes receive larger bonuses and higher-entropy failures receive larger penalties. An online entropy-based question curriculum supports this credit rule using cached rollout measurements. At the final checkpoint after eight training epochs on Qwen3-8B-Base, adding 4Q across three policy optimization settings yields absolute gains of 3.23-3.69% in six-benchmark macro avg@32 and 4.24-11.06% in macro pass@32. On Qwen2.5-7B-Base, the complete configuration raises four-benchmark macro avg@32 from 41.5% to 45.4%.
