单头注意力位置实现性理论：解释长度泛化为何依赖位置编码
A Positional Realizability Theory of Length Generalization in Single-Head Attention
解释了为什么绝对位置编码难做长度泛化、相对的却行，还给出训练前就能算出来的判据 α_L，做理论的人都该看看推导。
一篇 arXiv 论文针对单头仅位置注意力提出长度泛化的实现性理论，用模拟转移算子 T_L 和标量 α_L 在训练前判定给定位置核能否外推。理论给出相对头深行刻画（Theorem 2）、绝对/相对对偶（Proposition 4）以及充分性定理（Theorem 1），并指出一般确定性情形仍是开放猜想。实验用 2625 个训练模型验证：在 vocab 64、d=128 的受控单头设置下，α_L 能精确区分可泛化与不可泛化的（PE，任务）组合。论文明确该理论只覆盖位置通道，加法、排序等依赖 token 内容的任务不在 Theorem 2 范围内。
A Positional Realizability Theory of Length Generalization in Single-Head Attention
Why does length generalization (LG) depend so sharply on the positional encoding (PE)? Relative schemes extrapolate fixed offsets, absolute schemes fix positions, and neither handles growing offsets. We give a realizability theory for a deliberately controlled abstraction, a single positional-only attention head, that derives this well-documented empirical pattern and says precisely where it holds and breaks. The device is a simulation transfer operator $T_L$ that re-evaluates a selected length-$L_0$-realizing positional kernel on the longer grid, with a scalar $α_L \in [0,1]$ read off before any training from $(Φ, t_n)$ alone. Here $α_L$ serves as the computable witness of that taxonomy's derivation, not as a stronger empirical predictor. Our results are a relative-head deep-row characterization for kernels with a strict unique global maximizer (Theorem 2), an absolute/relative duality that derives the pattern (Proposition 4), a learned-absolute underdetermination result, and a static expressivity bound; a sufficiency theorem (Theorem 1) bridges $α_L$ to LG, and the general deterministic case is a precise open conjecture (correcting a tempting but invalid "outside-colspan" argument). Experiments confirm the structure: in the controlled single-head regime $α_L$ exactly separates the taxonomy's generalizing and non-generalizing (PE, task) cells (2625 trained-model measurements; stable at vocab 64/d=128). Together, these results turn the received PE taxonomy into a positional-kernel transfer theorem, and identify the content and depth mechanisms that escape it. The theory concerns the positional channel alone: tasks whose targets depend on token content, including the standard length-generalization suite such as addition and sorting, fall outside Theorem 2.