这篇论文戳破了JEPA中“各向同性正则化无害”的默认假设，做自监督表示学习的团队值得关注——它用哈密顿几何给出了更优的耦合方式，实验提升显著且理论扎实。
论文指出JEPA（联合嵌入预测架构）通常将单视图嵌入正则化为各向同性高斯分布，这隐含地引入了欧几里得对称性，但并非无害。当下游几何结构已知时，最优协方差应为哈密顿能量预算下的(c/d)H^{-1}，各向同性会导致可量化的性能损失。当下游几何未知时，任何固定边际目标都可能与某些结构严重不匹配。作者提出HamJEPA，将每个视图编码为相空间状态(q,p)，并用可学习的哈密顿跳蛙映射预测视图间转换，非各向同性尺度和谱底防止崩溃。在CIFAR-100上，HamJEPA在30轮时比SIGReg提升+4.89 kNN@20和+3.52线性探针点，80轮时提升+6.45 kNN@20和+10.64线性探针点。在ImageNet-100上，45轮时提升+4.82 kNN@20和+7.52线性探针点。
Beyond Isotropy in JEPAs: Hamiltonian Geometry and Symplectic Prediction
JEPAs often regularize one-view embeddings toward an isotropic Gaussian, implicitly baking Euclidean symmetry into the representation. We show that this is not merely a benign default. For a known structured downstream geometry $H\succ0$, the minimax and maximum-entropy covariance under a Hamiltonian energy budget is $(c/d)H^{-1}$, and Euclidean isotropy incurs a closed-form price of isotropy. More importantly, when the downstream geometry is unknown, no geometry-independent fixed marginal target is canonical: every fixed covariance shape can be maximally misaligned for some structured geometry. We further show that even oracle one-view marginals do not identify the JEPA view-to-view predictive coupling. These results suggest that the structural bias in JEPAs should enter the cross-view coupling rather than a fixed encoder marginal. We instantiate this principle with \textbf{HamJEPA}, which encodes each view as a phase-space state $(q,p)$ and predicts view-to-view transitions with a learned Hamiltonian leapfrog map, while non-isotropic scale and spectral floors prevent collapse. In a deliberately headless token protocol, HamJEPA improves over SIGReg on CIFAR-100 by $+4.89$ kNN@20 and $+3.52$ linear-probe points at 30 epochs, and by $+6.45$ kNN@20 and $+10.64$ linear-probe points at 80 epochs, while a matched MLP predictor ablation shows that the symplectic coupling is the ingredient driving the neighborhood-geometry gain. On ImageNet-100, HamJEPA-$q$ improves by $+4.82$ kNN@20 and $+7.52$ linear-probe points at 45 epochs.