这篇论文提出了一种新方法，能解决多模态模型推理时中间步骤乱但结果对的问题，效果SOTA还更高效，做推理增强的可以看看。
多模态大语言模型在推理任务中常出现中间步骤有缺陷但最终答案正确的问题，影响可解释性。论文提出O^2-CritiCuRL框架，在离线阶段通过多轮次分析步骤标注轨迹估计步骤重要性，提取关键推理步骤；在线阶段采用逐步强化学习策略，用截断链引导模型推理缺失步骤。在多个多模态推理基准上达到SOTA，同时训练和推理效率更高。代码已开源。
Offline-Online Curriculum RL for Multimodal Reasoning
Multimodal large language models exhibit capabilities on reasoning tasks, yet often produce flawed intermediate steps while yielding correct final answers. This behavior undermines interpretability and reliability, suggesting reliance on spurious shortcuts rather than faithful reasoning. Although efforts have explored step-level supervision, distinguishing decisive steps from redundant ones remains challenging. We propose $O^2$-CritiCuRL, a novel curriculum reinforcement learning framework that introduces critical-step awareness through an iterative offline-online paradigm. In the offline stage, $O^2$-CritiCuRL conducts multi-rollout analysis over step-annotated trajectories to estimate step-level importance, allowing the framework to distill critical reasoning steps and filter out redundant ones. In the online stage, we employ a progressive step-level reinforcement learning strategy, where truncated chains guide the model to infer missing steps and refine its reasoning, thereby sharpening its focus on critical steps and overcoming the limitations of static supervision. Extensive experiments on multimodal reasoning benchmarks show that our method achieves state-of-the-art performance while delivering superior training and inference efficiency. Code is available at https://github.com/kk0013/CritiCuRL.