这篇论文证明了Adam没有梯度下降的低秩隐式偏置，在Transformer里能让等价初始化差56%，不是调参小事。
论文研究矩阵分解模型 W=UV^T 下不同优化器的隐式偏置。梯度下降、动量、共享标量Adam、Muon、Shampoo具有规范等变性，Adam和RMSProp等坐标级方法不具备，因此后者无法继承低秩偏置。结构定理刻画了无记忆等变规则等价于Gram决定的左预条件，传递定理把梯度流的路径性质推广到公共标量流。在欠定矩阵感知的9种更新规则实验中，从坐标级到共享标量预条件的单参数族单调恢复偏置，说明各向异性是主因。Transformer实验中，Adam第一步就让两个规范等价初始化分离，最终 W_Q^T W_K 的相对Frobenius距离相差56%。高光谱数据集在匹配训练损失下，梯度下降在最低采样密度时将留出误差降低43-44%。
The Loss Does Not See the Basis, but Adam Does
Gradient descent on a factored model $W = UV^\top$ is implicitly biased toward low-rank solutions, while Adam, starting from the same small initialization, is not. We trace the difference to the gauge symmetry of the loss, its invariance under $(U, V) \mapsto (UQ, VQ)$. Gradient flow's low-rank mechanism is available to an optimizer only if that optimizer is gauge-equivariant, a condition necessary for the transfer but not sufficient for low-rank recovery. Gradient descent, momentum, "shared-scalar" Adam, Muon, and Shampoo satisfy it. Adam, RMSProp, and the other coordinate-wise methods do not. A structure theorem characterizes the memoryless equivariant rules as exactly the Gram-determined left preconditioners, and a transfer theorem carries gradient flow's pathwise properties to common-scalar flows. We then sort nine update rules on underdetermined matrix sensing by recovery error against the planted ground truth. A one-parameter family from coordinate-wise to shared-scalar preconditioning restores the bias monotonically, isolating anisotropy as the cause. A "spectral schedule" reconciles two opposing reports about Muon: equal-rate updates recover exactly low-rank targets but lose their edge as the spectral tail grows. In transformers, Adam separates two gauge-equivalent initializations at the first step, where the equivariant optimizers stay at float precision, and ends with the per-head invariants $W_Q^\top W_K$ 56% apart in relative Frobenius distance, a gap no per-head rotation can close. On two hyperspectral datasets at matched training loss, gradient descent cuts held-out error by 43-44% at the lowest sampling density, and at lower effective rank. Basis choice is therefore not a tuning detail but a decision about which interpolant the optimizer selects.