做向量检索的团队注意了——LEMUR 在长尾语料上会悄悄按文档长度排序而非相关性，LoTTE 上召回率从 98% 掉到 30.5%。如果你的语料长度差异大（P90/P10 > 20），建议先跑文中的两个检查再决定是否用 LEMUR。
LEMUR 是一种将多向量压缩为单向量的策略，声称无需调参、数据驱动。但研究发现，在文档长度差异大的语料（如 LoTTE，长度范围 400 倍）上，LEMUR 会学习到“长文档得分更高”的偏差，导致按长度而非相关性排序。在 LoTTE 上，LEMUR 的 nDCG@10 仅 0.109，召回率 30.5%，而 Exact MaxSim 方法达 0.722 和 98%。问题根源在于 LEMUR 的训练标签（MaxSim 分数）天然偏向长文档，MLP 学到了长度信号。建议用户检查语料的长度分布（P90/P10 比）和模型的可分离性（MaxSim 标准差），若长度差异大或可分离性高，应改用 TokenANN 或 MUVERA。已使用 LEMUR 的团队可通过长度分层采样缓解偏差。
𝗟𝗘𝗠𝗨𝗥 𝗶𝘀 𝘁𝗵𝗲 𝗺𝘂𝗹𝘁𝗶-𝘃𝗲𝗰𝘁𝗼𝗿 𝘀𝘁𝗿𝗮𝘁𝗲𝗴𝘆 𝘁𝗵𝗮𝘁 𝗹𝗼𝗼𝗸𝘀 𝗲𝗮𝘀𝗶𝗲𝘀𝘁 ...
𝗟𝗘𝗠𝗨𝗥 𝗶𝘀 𝘁𝗵𝗲 𝗺𝘂𝗹𝘁𝗶-𝘃𝗲𝗰𝘁𝗼𝗿 𝘀𝘁𝗿𝗮𝘁𝗲𝗴𝘆 𝘁𝗵𝗮𝘁 𝗹𝗼𝗼𝗸𝘀 𝗲𝗮𝘀𝗶𝗲𝘀𝘁 𝘁𝗼 𝗮𝗱𝗼𝗽𝘁 — 𝗻𝗼 𝘁𝘂𝗻𝗶𝗻𝗴, 𝗱𝗮𝘁𝗮-𝗱𝗿𝗶𝘃𝗲𝗻, 𝗰𝗼𝗺𝗽𝗿𝗲𝘀𝘀𝗲𝘀 𝘆𝗼𝘂𝗿 𝗺𝘂𝗹𝘁𝗶-𝘃𝗲𝗰𝘁𝗼𝗿𝘀 𝗶𝗻𝘁𝗼 𝘀𝗶𝗻𝗴𝗹𝗲 𝘃𝗲𝗰𝘁𝗼𝗿𝘀 𝗼𝗻 𝗶𝘁𝘀 𝗼𝘄𝗻. 𝗕𝘂𝘁 𝗼𝗻 𝗹𝗼𝗻𝗴-𝘁𝗮𝗶𝗹𝗲𝗱 𝗰𝗼𝗿𝗽𝗼𝗿𝗮, 𝗶𝘁 𝗰𝗮𝗻 𝗾𝘂𝗶𝗲𝘁𝗹𝘆 𝘀𝘁𝗮𝗿𝘁 𝗿𝗮𝗻𝗸𝗶𝗻𝗴 𝗯𝘆 𝗱𝗼𝗰𝘂𝗺𝗲𝗻𝘁 𝗹𝗲𝗻𝗴𝘁𝗵 𝗶𝗻𝘀𝘁𝗲𝗮𝗱 𝗼𝗳 𝗿𝗲𝗹𝗲𝘃𝗮𝗻𝗰𝗲. 𝗟𝗼𝗧𝗧𝗘, 𝗝𝗶𝗻𝗮-𝗖𝗼𝗹𝗕𝗘𝗥𝗧-𝘃𝟮: 📈 𝗘𝘅𝗮𝗰𝘁 𝗠𝗮𝘅𝗦𝗶𝗺 — nDCG @10 = 0.722, R @100 = 98% 📉 𝗟𝗘𝗠𝗨𝗥 — nDCG @10 = 0.109, R @100 = 30.5% 𝗪𝗵𝘆? 𝗜𝘁'𝘀 𝗻𝗼𝘁 𝗮𝗻 𝗶𝗺𝗽𝗹𝗲𝗺𝗲𝗻𝘁𝗮𝘁𝗶𝗼𝗻 𝗯𝘂𝗴 — 𝗶𝘁'𝘀 𝗮 𝗹𝗲𝗻𝗴𝘁𝗵 𝗯𝗶𝗮𝘀 𝗯𝗮𝗸𝗲𝗱 𝗶𝗻𝘁𝗼 𝗵𝗼𝘄 𝗟𝗘𝗠𝗨𝗥 𝘁𝗿𝗮𝗶𝗻𝘀. LEMUR's labels are MaxSim scores — for a sampled vector, its highest inner product against any token in a document. A longer document has more tokens, so it's more likely to hold a high value regardless of relevance, a pure statistical effect. The MLP learns "longer scores higher" instead of "more relevant scores higher," and the ranking follows length. 𝗜𝘁 𝗼𝗻𝗹𝘆 𝗯𝗿𝗲𝗮𝗸𝘀 𝘄𝗵𝗲𝗻 𝘁𝘄𝗼 𝘁𝗵𝗶𝗻𝗴𝘀 𝗹𝗶𝗻𝗲 𝘂𝗽: • First, a length-sensitive model, which comes with high separability. On Jina, docs under 50 tokens average a MaxSim of 0.492; docs over 1,000 average 0.666 — a 0.174 gap the model assigns to length alone. • Second, heavy length variance. LoTTE runs 10 to 4,109 tokens, about a 400x range. 𝗟𝗼𝗧𝗧𝗘 𝗵𝗮𝘀 𝗯𝗼𝘁𝗵, 𝗮𝗻𝗱 𝗶𝘁 𝗳𝗮𝗶𝗹𝘀 𝗮𝘁 𝗰𝗼𝗺𝗽𝗿𝗲𝘀𝘀𝗶𝗼𝗻, 𝗻𝗼𝘁 𝗿𝗲𝗿𝗮𝗻𝗸𝗶𝗻𝗴 — 𝗠𝗮𝘁𝗵 𝗥@𝟭𝟬𝟬 𝘁𝗵𝗲𝗿𝗲 𝗶𝘀 𝟮𝟰.𝟳%, 𝘀𝗼 𝘁𝗵𝗲 𝗿𝗶𝗴𝗵𝘁 𝗱𝗼𝗰𝘂𝗺𝗲𝗻𝘁𝘀 𝗮𝗿𝗲 𝗱𝗿𝗼𝗽𝗽𝗲𝗱 𝗳𝗿𝗼𝗺 𝘁𝗵𝗲 𝗰𝗮𝗻𝗱𝗶𝗱𝗮𝘁𝗲 𝘀𝗲𝘁 𝗯𝗲𝗳𝗼𝗿𝗲 𝗿𝗲𝗿𝗮𝗻𝗸𝗶𝗻𝗴 𝗰𝗮𝗻 𝗵𝗲𝗹𝗽. 𝗧𝗵𝗲 𝗰𝗼𝗻𝘁𝗿𝗼𝗹 𝗿𝘂𝗻𝘀 𝗶𝘀𝗼𝗹𝗮𝘁𝗲 𝘁𝗵𝗲 𝗰𝗮𝘂𝘀𝗲: For length-uniform data (SciFact): same LEMUR behaves normally — 0.651 vs BruteForce 0.719. For a low-separability model (AnswerAI on LoTTE): unaffected — R @100 91.5%. 𝗧𝘄𝗼 𝗰𝗵𝗲𝗰𝗸𝘀 𝗳𝗼𝗿 𝘆𝗼𝘂𝗿 𝗼𝘄𝗻 𝗰𝗼𝗿𝗽𝘂𝘀: → 𝗟𝗲𝗻𝗴𝘁𝗵: if P90/P10 is under 5 it's fairly uniform and LEMUR is usually safe; over 20 (LoTTE is ~400), be cautious. → 𝗦𝗲𝗽𝗮𝗿𝗮𝗯𝗶𝗹𝗶𝘁𝘆: sample ~1,000 token vectors, take the std of their MaxSim scores. Under 0.08 is low, LEMUR is safe; over 0.12 is high — use TokenANN or MUVERA. 𝗜𝗳 𝘆𝗼𝘂'𝗿𝗲 𝗮𝗹𝗿𝗲𝗮𝗱𝘆 𝗰𝗼𝗺𝗺𝗶𝘁𝘁𝗲𝗱 𝘁𝗼 𝗟𝗘𝗠𝗨𝗥, 𝗹𝗲𝗻𝗴𝘁𝗵-𝗯𝗮𝗹𝗮𝗻𝗰𝗲𝗱 𝘀𝗮𝗺𝗽𝗹𝗶𝗻𝗴 𝗵𝗲𝗹𝗽𝘀: 𝘀𝘁𝗿𝗮𝘁𝗶𝗳𝘆 𝗯𝘆 𝗹𝗲𝗻𝗴𝘁𝗵 (𝟬–𝟭𝟬𝟬, 𝟭𝟬𝟬–𝟯𝟬𝟬, 𝟯𝟬𝟬–𝟭,𝟬𝟬𝟬, >𝟭,𝟬𝟬𝟬 𝘁𝗼𝗸𝗲𝗻𝘀) 𝗮𝗻𝗱 𝗱𝗿𝗮𝘄 𝗲𝗾𝘂𝗮𝗹 𝘀𝗮𝗺𝗽𝗹𝗲𝘀 𝗽𝗲𝗿 𝗯𝗮𝗻𝗱. 𝗜𝘁 𝘄𝗲𝗮𝗸𝗲𝗻𝘀 𝘁𝗵𝗲 𝗹𝗲𝗻𝗴𝘁𝗵 𝘀𝗶𝗴𝗻𝗮𝗹 𝘄𝗶𝘁𝗵𝗼𝘂𝘁 𝗳𝘂𝗹𝗹𝘆 𝗿𝗲𝗺𝗼𝘃𝗶𝗻𝗴 𝗶𝘁. 💬 0 🔄 0 ❤️ 0 👀 4 ⚡