一篇理论论文，解释了为什么对比嵌入模型的向量长度能反映语义，以后做检索任务可以白捡校准信号。
该论文研究了对比嵌入模型中，尽管训练损失是尺度不变的且通常使用余弦相似度，但嵌入向量的范数（norm）却意外地与概念特异性、词频和人类不确定性等语义属性相关。作者通过分析优化动力学，推导出一个解析公式，证明嵌入长度作为训练过程的副产品自然地编码了这些信息。研究还展示了这种信号如何在特定模型和检索任务中作为“免费”校准工具使用，为之前仅基于经验的观察提供了理论解释。
Optimization Dynamics Imprint Semantic Specificity in Contrastive Embedding Norms
Contrastive embedding models trained with scale-invariant losses are typically paired with distance metrics like cosine similarity, effectively ignoring embedding magnitudes. However, surprisingly, empirical studies reveal that despite this, these "discarded" norms seem to correlate with semantic properties such as concept specificity, token frequency, and human uncertainty. In this work, we provide a formal theoretical framework explaining this phenomenon. By analyzing the optimization dynamics, we derive an analytic formula demonstrating that embedding length naturally encodes this information as a byproduct of the training process. We also show how this gives rise to signals that can serve as "free" calibration tools in specific models and retrieval tasks, providing a grounded explanation for a previously heuristic observation.