Best-of-Depth 训练方法让 Looped Transformers 中间层输出更多有效解
Best-of-Depth: Learning Exploratory Dynamics in Looped Transformers
一篇 arXiv 论文，教 Looped Transformers 在循环推理的中途就产出有用候选解，N-Queens 覆盖率从 45.75% 拉到 91.03%，做循环模型训练的可以看看。
arXiv 论文提出 Best-of-Depth（BoD）训练方法，通过最小化各深度的预测损失，把 Looped Transformers 的循环轨迹当作候选解集合来训练，无需改动架构或添加深度标签。在 N-Queens 补全任务上，BoD 的解覆盖率达到 91.03%，而传统的最终深度训练只有 45.75%；图着色任务同样超过最终深度训练和中间监督两种基线。实验还显示模型能发现超出训练深度的解，并在熟悉输入上恢复未包含在教师池中的解。将 BoD 用于微调预训练 looped 语言模型后，在各固定深度分别生成代码的 pooled 成功率也高于最终深度训练。
Best-of-Depth: Learning Exploratory Dynamics in Looped Transformers
Looped Transformers extend inference-time computation through repeated latent-state updates, but final-depth training does not directly encourage useful intermediate alternatives. We introduce Best-of-Depth (BoD), which trains a recurrent trajectory as a candidate set by minimizing prediction loss across depths. Different valid targets can be matched at different depths without architectural changes or depth-specific labels. An exact-match analysis connects this objective to solution coverage. On N-Queens completion and graph coloring, BoD improves coverage over final-depth and intermediate-supervision objectives. N-Queens coverage reaches 91.03%, versus 45.75% for final-depth training. Further experiments show discovery beyond the training depth and recovery of solutions withheld from teacher pools on familiar inputs. Fine-tuning a pretrained looped language model with BoD yields higher pooled code-generation success than final-depth training when programs are generated separately at evaluated fixed depths. These results support training recurrent computation for solution discovery along its trajectory.
