这篇论文教模型用自己脑内的几何结构来挑正确答案，不用额外奖励模型，在GSM8K和MATH上比原版ScPO高出4个多点。
Cloud-ScPO 提出从大模型内部表征的几何结构中挖掘偏好信号，无需外部奖励模型。它利用少量标注样本构建正确与错误的参考云，将每条推理轨迹映射为平均池化隐藏状态，并用组件级软k近邻打分。在GSM8K和MATH-Numeric数据集上，Cloud-ScPO相较ScPO最高提升4.49%和4.19%。对配对轨迹的分析显示，该方法在保持正确率的同时，能更有效筛选出高质量的正确轨迹。
Cloud-ScPO: Hidden-State Geometry for Semi-Supervised Preference Optimization in LLM Reasoning
Preference optimization improves mathematical reasoning in large language models (LLMs), but reliable chosen-rejected pairs usually require verified answers, human annotations, or external reward models. We investigate whether preference supervision can instead be derived from the model's internal representation geometry in a semi-supervised setting. Our analysis shows that reasoning trajectories generated across different mathematical problems form structured global point clouds in which correct and incorrect trajectories exhibit different geometric organization. Based on this observation, we propose Cloud--ScPO, a topology-guided preference-mining framework that uses a small labeled set to construct multiple correct and incorrect reference Clouds. Each trajectory is represented by a mean-pooled hidden state and scored against connectivity-induced components using a component-level soft $k$-nearest-neighbor measure averaged across reference banks. We combine this cross-problem Cloud signal with prompt-level self-consistency: self-consistency determines the answer-level preference direction, while Cloud scoring selects concrete trajectories and filters pairs by their score margin. Experiments on GSM8K and MATH-Numeric across four model settings show that Cloud--ScPO consistently improves over ScPO, with gains of up to 4.49\% on GSM8K and 4.19\% on MATH-Numeric. Pair-level analyses further show that Cloud--ScPO maintains comparable correctness reliability while more effectively separating informative chosen trajectories from incomplete, repetitive, or otherwise low-quality rejected responses.