论文精选

匹配原则:一种针对干扰鲁棒表示学习的损失函数几何理论

The Matching Principle: A Geometric Theory of Loss Functions for Nuisance-Robust Representation Learning

精选理由

这篇论文把鲁棒性、域适应等一堆看似无关的问题统一成了一个统计框架,做表示学习或模型泛化的研究者可以直接用匹配原则指导正则化器设计,省去试错成本。

AI 摘要

本文提出“匹配原则”,将鲁棒性、域适应、光度不变性、组合泛化、时间鲁棒性、对齐安全等看似独立的问题统一为同一个统计问题:估计部署干扰的协方差,然后沿着覆盖该协方差的矩阵正则化编码器雅可比矩阵。在线性高斯模型中,作者证明了闭式最优性(定理A),包括匹配范围内的立方根水填充;证明了二次雅可比惩罚需要范围覆盖(定理G);并提供了七个条件一致性引理。作者引入轨迹偏差指数(TDI),一种无标签的嵌入敏感性探针。在从经典ML到Qwen2.5-7B的13个预注册实验中,12个通过了预测的匹配-各向同性-错误W顺序测试。在7B规模上,匹配风格PMH提高了选择性诚实性,并保持了风格TDI,而标准DPO则使其退化。

原文 · arXiv cs.AI

The Matching Principle: A Geometric Theory of Loss Functions for Nuisance-Robust Representation Learning

Robustness, domain adaptation, photometric and occlusion invariance, compositional generalisation, temporal robustness, alignment safety, and classical anisotropic regularisation are usually treated as separate problems with separate method families. This paper argues that much of their shared structure is one statistical problem: estimate the covariance of label-preserving deployment nuisance, then regularise the encoder Jacobian along a matrix whose range covers that covariance (the matching principle). CORAL, adversarial training, IRM, augmentation, metric learning, Jacobian penalties, and alignment-style constraints are different estimators of that object, not independent robustness tricks. In the linear-Gaussian model we prove closed-form optimality (Theorem A), including cube-root water-filling within the matched range; necessity of range coverage for quadratic Jacobian penalties (Theorem G); the same range dichotomy at deep global minima; and two falsification controls (Lemma C; Corollaries E), with seven conditional consistency lemmas (D1-D7) for estimation under standard identifiability assumptions. We introduce the Trajectory Deviation Index (TDI), a label-free probe of embedding sensitivity when task accuracy or Jacobian Frobenius norm is insufficient. Thirteen pre-registered blocks from classical ML through Qwen2.5-7B test the predicted matched, then isotropic, then wrong-W ordering on geometry and deployment drift; twelve pass, and the sole exception (Office-31) is an eigengap failure named before the run. At 7B scale, matched style-PMH improves selective honesty and preserves Style TDI where standard DPO degrades it. The contribution is naming the deployment nuisance covariance, stating what the regulariser must do, and supplying a closed-form falsifiable theory once that object is identified, not universality on every leaderboard.