Papers

精确验证器下的小模型学习:可算出正确但不泛化

Procedure without Judgment: What Small Language Models Learn under an Exact Verifier

Why we picked it

这篇论文做了件少见的事:给解释任务装上零误差验证器,看小模型到底学到了什么。结果反直觉——4B 模型在训练策略上 92% 精确,但换 3 个新策略就只剩 48% 最小性,RL 还把策略练反了。做可解释性或微调的人都该看看这组对照数据。

arXiv 论文研究一个可精确打分的解释任务:授权策略阻止智能体调用工具时,模型要找出迫使该决策的最小充分事实集。由于上下文空间可枚举,正确答案可被精确计算,解释直接对照 ground truth 评分。前沿模型仅凭工具约一半时间能找到精确原因,一句“把答案缩短”的提示将 Haiku 4.5 从 58% 提到 75%;在 2000 条删除算法轨迹上微调的 4B 模型在训练策略上达到 92%。

Body

实验的关键发现是“教师类型”决定效果:同等数据量下,算法教出的学生在全部 3 个训练种子上都领先于用前沿模型教出的学生,且两组结果完全不重叠。学生学到的是一套固定的充分性检查时间表,但这套时间表不迁移。在 3 个从未见过的公开策略上,其答案保持充分(94%)却不再最小(48%),而 Haiku 在同一句提示下于全部 47 个测试回合中精确命中。

用精确奖励做强化学习并未提升 held-out 准确率,反而把策略从“探测策略”反转成“把预算全部花在充分性检查上”。论文结论是:oracle 让训练的效果变得可观察,但没有让训练变好。

Original · arXiv cs.LG

Procedure without Judgment: What Small Language Models Learn under an Exact Verifier

When an authorization policy blocks an agent's tool call, the audit log should name the facts that forced the decision and nothing more. For an executable policy over an enumerable context space this minimal sufficient reason can be computed exactly, so an explanation is scored against ground truth rather than a proxy for it. We use that exactness to ask what a model can learn from a verifier that has no error in it. The explainer sees one request and one decision, never the policy, and has thirty turns in which it may call the policy on altered requests and ask whether a candidate set of facts is sufficient. Frontier models find the exact reason about half the time with the tools alone, and one sentence telling them to shrink the answer lifts Haiku 4.5 from 58% to 75%. A 4B model fine-tuned on 2000 traces of the deletion algorithm reaches 92% on the policy it was trained on. Teacher kind is what moves this: at matched data size the algorithm-taught student leads the frontier-taught one in all three training seeds, and the two arms never overlap. What the student learns is a fixed schedule of sufficiency checks, and the schedule does not travel. On three public policies it never saw, its answers stay sufficient (94%) and stop being minimal (48%), while Haiku under the same sentence is exact on all 47 episodes. Reinforcement learning from the exact reward leaves held-out accuracy where it was and inverts the strategy, from probing the policy to spending the entire budget on sufficiency checks. The oracle made the effect of training legible, but it did not make training better.

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include Haiku 4.5, fine-tuning, reinforcement learning, interpretability.

Source attribution: curated from arXiv cs.LG (@Arunkumar V, Manoranjan Gandhudi, Gangadharan G. R., Arun Prakash, S. Senthilkumar).