一篇用百万参数模型研究思维链的论文，提出Protoreasoning，能有效提升分布外泛化，比大模型实验更好分析。
该研究在约100万参数的微型Transformer上测试了一种简单思维链（Chain of Thought）形式，命名为Protoreasoning。他们使用Dyck语言（正确嵌套括号的句子）定义推理友好任务，发现Protoreasoning轨迹大幅缩小了分布外泛化差距。消融实验证实，增益来自轨迹内容本身，而非额外标记数量。这为研究逐步推理提供了低成本实验平台。
Protoreasoning in Tiny Transformers
We show that tiny transformers can profitably employ a simple form of Chain of Thought, which we call protoreasoning, allowing us to study step-by-step reasoning on ~1M-parameter models and opening up opportunities for much more detailed experimentation and analysis than is feasible for larger models. Current Large Language Models exhibit impressive step-by-step reasoning, but we have yet to understand its generality, i.e., when and how LLMs learn genuinely general algorithms rather than "bags of heuristics." Such questions are hard to settle on compute-intensive frontier models trained on opaque data. To work at model scales far below the threshold for natural-language competence, we define reasoning-friendly tasks on Dyck languages (sentences of correctly nested brackets). We find that protoreasoning traces substantially close the out-of-distribution generalization gap, and ablations confirm that the trace's content, not merely its extra tokens, drives the gain.