Papers

FLCVA 审计框架检验自适应推理:继续思考的真实价值有限

A Full-Ledger Counterfactual Audit of Adaptive Reasoning

Why we picked it

这篇论文拿数据拆穿了自适应推理里的一个常见假设:预测当前答案错了,不代表多想一步就能改对。作者用 Qwen3-8B、DeepSeek-R1-Distill-Qwen-7B 和 Llama-8B 做对照实验,结论对做测试时算力分配的人很有参考价值。做推理调度或 agent 工程的开发者建议看看。

arXiv 论文 2611.22778 提出 Full-Ledger Counterfactual Value Audit(FLCVA),把自适应推理拆成三个环节:测量继续推理的局部价值、从可观测信号预测该价值、再用预测结果分配算力。方法是在同一个冻结前缀上,将立即作答与四种固定长度续写进行对照,并对策略使用的每个信号计入成本。

Body

在 GSM8K 和 MATH-500 上,随机 256 token 续写能在 26.6% 的 Qwen3-8B 错误案例和 18.9% 的 DeepSeek-R1-Distill-Qwen-7B 错误案例中纠正答案,同时只有 1.4% 和 1.9% 的正确答案被破坏。来自独立采样续写的正确性标签可以预测另一条续写的结果,说明这种信号本身有可预测性。

但在离线回放中用这些标签分配算力,只在 DeepSeek 上带来性能提升,Qwen3 的收益不明确。论文结论是:在全成本核算下,测试的轻量短视控制器主要只是识别当前错误,除此之外对续写本身几乎没有额外价值。研究还用 Llama-8B 复现和动作/成本消融验证了稳健性。

Original · arXiv: DeepSeek

A Full-Ledger Counterfactual Audit of Adaptive Reasoning

Adaptive reasoning must decide whether to answer now or spend more compute. Predicting that the current answer is wrong does not establish whether further reasoning will improve it. We introduce the Full-Ledger Counterfactual Value Audit (FLCVA), which compares an immediate answer with four fixed-length continuations from the same frozen prefix under one answer procedure and accounts for the cost of every signal used by a policy. On GSM8K and MATH-500, a random 256-token continuation recovers a wrong answer in 26.6% of Qwen3-8B cases and 18.9% of DeepSeek-R1-Distill-Qwen-7B cases, while damaging a correct answer in 1.4% and 1.9%. Correctness labels from independently sampled continuations predict a separate continuation's outcome in both primary models. In offline replay, using these labels to allocate compute improves performance on DeepSeek; the Qwen3 gain remains inconclusive. Under full cost accounting, the tested lightweight, myopic controllers mainly recognize current errors and add little continuation-specific value beyond that shortcut. A Llama-8B replication and action/cost ablations examine the robustness of these findings. FLCVA separates measuring local continuation value, predicting it from observable signals, and using those predictions to allocate computation.

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include adaptive inference, Qwen3, DeepSeek-R1, GSM8K.

Primary source verification: published directly by arXiv: DeepSeek with first-party authority.