做模型微调或可解释性研究的团队会感兴趣——它揭示了LoRA和激活导向的线性假设为何局部成立但全局不成立，建议在调试任务向量时参考其轨迹前缀基方法。
该研究探讨了预训练模型中的线性结构是否真实存在及其尺度。通过合成多任务Transformer和LoRA适配器实验，发现任务梯度具有强局部低秩结构，但静态基无法捕捉恢复方向，有用基在100步内显著漂移。首次恢复更新形成的轨迹前缀基可捕获77%的LoRA恢复位移。研究还提出高斯局部线性定理，解释了高维随机参数搜索的有效性，并发现单步梯度产生的激活偏移与CAA导向向量余弦相似度为0.58。结果表明，线性结构并非全局任务方向，而是部分跨参数和激活空间演化的局部几何。
Recoverable but Not Stationary:Local Linear Structures in Weights and Activations
Task vectors, LoRA, activation steering, and random search around pretrained weights all suggest that learned behaviour can be controlled by linear directions. We ask which linear structures actually exist and on what scale. In a synthetic multitask transformer and LoRA adapters on DistilGPT-2 / GPT-2 we find strong local low-rank task-gradient structure but reject the fixed-task-plane hypothesis: static bases miss the recovery direction, and the useful basis drifts substantially within 100 steps. However, the first recovery updates form a trajectory-prefix basis capturing 77% of the LoRA recovery displacement. We develop random search theory with a Gaussian local-linear theorem that justifies the effectiveness of random parameter search even in very high dimensions. We also study the relation between parameter perturbations and activation steering: a single gradient step produces an activation shift with 0.58 cosine to a labelled-contrast CAA steering vector, with a similar steering effect on Qwen-0.5B BoolQ statements. We validate our results with experiments on synthetic Transformers and LLMs. Our results suggest that linear structures in trained networks are not global task directions, but evolving local geometries that partially persist across parameter and activation spaces.