Not All Test-Time Compute Is Evidence：DIA 方法用异构模型仲裁修正 LLM 推理偏差
Not All Test-Time Compute Is Evidence: Model-Specific Bias in LLM Reasoning
一篇挺有意思的论文：发现同一个模型反复算也不算独立证据，提出 DIA 用几个不同模型盲投仲裁，MuSiQue 上涨 7 个点，token 还省一半。
论文指出测试时扩展的隐含假设不成立：同一模型的重复采样、不同推理深度和自我评估中的错误是相关的，追加算力并不等于增加独立证据。作者提出 DIA（Disagreement-triggered Independent Arbitration），把模型中间阶段与最终阶段预测的分歧作为推理不稳定信号，再由一组异构小模型在看不到候选答案的情况下独立求解并聚合。在 7 个基准和 2 个基础模型上，DIA 在 14 个设置中的 12 个超过完整思维链，其中 MuSiQue 提升 7.00%。该方法只对约 17% 的样本触发，比多智能体基线少用最多 47.2% 的 token。消融实验显示独立求解和求解器异构性缺一不可。
Not All Test-Time Compute Is Evidence: Model-Specific Bias in LLM Reasoning
Test-time scaling methods implicitly assume that additional computation from the same model provides independent evidence. We show that this assumption fails within a single reasoning model: errors are correlated across repeated samples, across reasoning depth, and in the model's own self-evaluation, so more compute from the same model is not more evidence. We introduce DIA (Disagreement-triggered Independent Arbitration), which treats disagreement between an intermediate-stage and the final-stage prediction as a per-question signal of reasoning instability and resolves it by asking a small panel of heterogeneous models to solve the question independently, without seeing either candidate, and aggregating their answers directly. Across seven benchmarks and two base models, DIA outperforms full chain-of-thought in 12 of 14 settings, including a 7.00% gain on MuSiQue, while activating on only $\sim 17\%$ of examples and using up to 47.2% fewer tokens than a multi-agent baseline. Ablations show that both independent solving and solver heterogeneity are necessary for these gains.
