多语言检索不对称是实际部署中的常见痛点,做跨语言 NLP 或搜索的团队可以直接用 CSLS 替换余弦相似度,效果提升显著且无需重新训练模型。
多语言嵌入模型假设跨语言检索是对称的,但实际中并非如此。研究使用 6,518 条英、孟、印、阿语习语和谚语平行语料,测试了 Gemini、Mistral、OpenAI-L、OpenAI-S、Qwen 五个编码器,发现 hubness(中心性)是导致检索不对称的主要几何病理,而非各向异性、质心漂移或向量幅度。在预注册实验中,hub mass 对互惠性的联合回归主导份额达 49.5%,是次优预测因子的 1.68 倍,而基于 hub 感知的 CSLS 评分修正可缩小 63.5% 的最差-最佳互惠差距。研究建议用 CSLS 替代余弦相似度作为多语言嵌入管道的默认检索指标。
Hubness, Not Anisotropy, Drives Cross-Lingual Retrieval Asymmetry in Multilingual Embedding Models
Multilingual embedding models are deployed under the assumption that cross-lingual retrieval is symmetric: if a query in language A retrieves its translation in language B, the reverse should also hold. In practice it does not. Using a parallel corpus of 6,518 idiomatic and proverbial expressions in English, Bangla, Hindi, and Arabic, embedded by five production-grade encoders (Gemini, Mistral, OpenAI-L, OpenAI-S, Qwen), we formalise this failure as a deficit in mutual nearest-neighbour reciprocity and test a single mechanistic claim: among the geometric pathologies of multilingual spaces, hubness, not anisotropy, centroid drift, or magnitude, is the dominant causal driver. Across five pre-registered experiments with falsification conditions specified in advance, hub mass dominates a joint regression on reciprocity (49.5% dominance share, 1.68x the next predictor; partial R^2 = 0.302 versus 0.003 for anisotropy), while a hub-aware score correction (CSLS) closes 63.5% of the worst-to-best reciprocity gap and yields a mean within-model effect size 130x larger than surgical hub-vector ablation. The latter contrast pinpoints the mechanism: hubness is a pathology of the similarity metric, not of individual hub vectors. We resolve the well-known anisotropy-hubness paradox by showing the two are statistically dissociable, and we recommend replacing cosine similarity with CSLS as the default retrieval metric for multilingual embedding pipelines.