这篇论文搞清了Grokking延迟的可控关键：用无标签特征族先验就能加速17倍，而且只在训练早期用一小会儿就够，比你想的简单实用。
该论文通过188次新实验研究了表征先验在Grokking中的作用。错误特征族（幅度带）像随机划分一样阻止泛化（1/15 vs 0/20 grok，p=0.43）。完全无标签的不变性先验（仅交换对）在15/15次运行中泛化，中位加速2.7倍，且比标签监督先验更可靠（p=0.038）。早期窗口（前2000个epoch，占4%预算）即可实现10/10泛化及2.7倍加速，优于连续应用（8/10，1.25倍）和后期窗口（2.1倍）。结合权重范数钳制的先验达到中位17倍加速（5/5），而普通交叉熵仅在精确临界范数下才匹配此速度。
What Makes a Representational Prior Work? Feature Families, Label-Free Invariances, and Critical Windows in Grokking
Companion work showed the grokking delay is causally the time to form task-structured representations, injectable via a contrastive prior. Here we characterize what makes such a prior work, across four axes, in 188 new runs. Content: a coherent, learnable prior built from the wrong feature family (magnitude bands) blocks generalization like a random partition (1/15 vs 0/20 grok; $p=0.43$ between them), confirming the companion's prediction that priors act at the level of the circuit's features. Supervision: a fully label-free invariance prior -- positives are commuted pairs $(a,b)\sim(b,a)$ only -- generalizes in 15/15 runs at a median $2.7\times$ speedup, more reliably than the label-supervised prior itself ($p=0.038$), and combined with a weight-norm clamp yields the strongest method we test (median $17\times$, 5/5) -- strongest meaning reliably fast: plain cross-entropy with a clamp matches this speed only at the exact critical norm, while the prior keeps it fast across the entire clamp range. Timing: the prior is only needed early -- applied solely during the first 2000 epochs (4% of budget) it generalizes 10/10 at $2.7\times$, beating continuous application (8/10, $1.25\times$) and a duration-matched later window ($2.1\times$). Setting: the dissociation replicates on modular multiplication and across depths and normalization variants, and a clamp sweep quantifies the companion's central claim: structure injection flattens the weight-norm delay-law exponent about 17-fold (plain cross-entropy slows $31\times$ per +10 norm units, a lower bound as higher cells are censored, versus $1.22\times$ with the prior). Honest boundary: tasks that generalize before memorizing have no delay to control. Feature-family alignment decides whether a prior permits generalization; invariance content suffices for acceleration without labels; a brief early window captures nearly all of the benefit.