Papers

Hyperspherical Simplex 表征:让模型升级后检索免重建索引

Hyperspherical Simplex Representations for Backward-Compatible Retrieval

Why we picked it

如果你维护检索系统,模型一升级就得重做全量索引,这篇论文给了个不用重索引的思路,连旧模型都不用依赖,很实用。

论文提出基于 softmax 输出和 logits 的 hyperspherical simplex 表征,解决模型更新后表征空间变化、需要全量重建索引的问题。方法利用 softmax 交叉熵训练与 one-hot 标签诱导的单纯形结构作为跨模型几何参照,针对新增类别导致的类原型错位,推导出一个确定性线性变换恢复对齐。该方案无需兼容性专用损失、学习的映射函数或架构修改,也不依赖旧模型。理论部分基于 von Mises–Fisher 类分布假设,给出向后兼容不等式在期望意义下成立的两个几何条件。在标准兼容性基准上(含类别扩充、更强架构和公开仓库中独立训练的模型)取得有竞争力的结果。

Original · arXiv cs.LG

Hyperspherical Simplex Representations for Backward-Compatible Retrieval

Training modern AI models has become increasingly expensive, and updating deployed models can alter their internal feature representations, affecting applications built upon them. In retrieval systems, such updates often require complete gallery re-indexing, a computationally expensive process at scale. Existing backward-compatibility methods enable direct comparison of representations from updated and earlier models, avoiding re-indexing. However, they typically require compatibility-specific training, learned mapping functions, or architectural modifications, and often depend explicitly on the previous model. We propose hyperspherical simplex representations derived from softmax outputs and logits for backward-compatible retrieval across independently trained models. They leverage the simplex structure induced by softmax cross-entropy training with one-hot labels as a common geometric reference across model updates. Since this reference depends on the number of classes, adding new ones expands the simplex, misaligning existing class prototypes with their earlier counterparts. We therefore derive a deterministic linear transformation to restore their alignment. Our theoretical analysis of the formal backward-compatibility definition identifies two geometric conditions under which its inequalities hold in expectation, assuming von Mises--Fisher class distributions with increasing concentration across updates. Aligned regular-simplex representations satisfy these conditions, providing a theoretical basis for the proposed representations under the stated assumptions. Without using additional compatibility-specific losses, learned mappings, or architectural modifications, the proposed representations achieve competitive results on standard compatibility benchmarks, including expanded class sets, more expressive architectures, and independently trained models from public repositories.