做 RAG 或智能体检索的团队,终于不用被五个语义相同的 chunk 塞满上下文了——Weaviate 的 MMR 一行参数就能让结果既相关又多样,值得直接上手试。
Weaviate 1.37 版本新增了最大边际相关性(MMR)算法,用于解决向量搜索中返回高度相似重复结果的问题。通过一个参数 selection= Diversity.MMR(limit=5, balance=0.5),算法在每次选择结果时惩罚与已选结果过于相似的候选,确保最终结果既相关又多样。balance 参数可调节多样性与相关性的权重,0.0 为最大多样性,1.0 为标准搜索。该功能适用于所有 near_* 查询,特别适合检索密集型智能体和标准 RAG 管道,能有效利用上下文窗口,避免浪费 slots。
Your vector search just returned five pizzas. You queried "Italian food" and got margherita, marghe...
Your vector search just returned five pizzas. You queried "Italian food" and got margherita, margherita, margherita, margherita, and in a bold twist 𝗺𝗮𝗿𝗴𝗵𝗲𝗿𝗶𝘁𝗮. All technically correct. All useless together. This is what happens when relevance is the only thing scoring the results… You get a tight cluster of near-identical objects, ranked by how much they agree with each other. Weaviate 1.37 ships Maximum Marginal Relevance (MMR). One new parameter: 𝘀𝗲𝗹𝗲𝗰𝘁𝗶𝗼𝗻=𝗗𝗶𝘃𝗲𝗿𝘀𝗶𝘁𝘆.𝗠𝗠𝗥(𝗹𝗶𝗺𝗶𝘁=𝟱, 𝗯𝗮𝗹𝗮𝗻𝗰𝗲=𝟬.𝟱), and the algorithm stops letting the fifth result coast on similarity to the first four. It iteratively picks the most relevant item first, then penalises candidates that are too similar to what has already been selected, so each new result has to earn its place by adding something new. The 𝗯𝗮𝗹𝗮𝗻𝗰𝗲 is the control knob: - 𝟬.𝟬 = go full throttle and maximise the difference between results. - 𝟭.𝟬 = don't touch anything, give back standard search results. - 𝟬.𝟱 = the middle, where each result has to justify itself on both axes. MMR works across all 𝗻𝗲𝗮𝗿_* queries in Weaviate, and in the query agent it uses 𝗱𝗶𝘃𝗲𝗿𝘀𝗶𝘁𝘆_𝘄𝗲𝗶𝗴𝗵𝘁 for more accurate relevance score from the reranker. The use cases that get the most benefits are retrieval-heavy agents and standard RAG pipelines. Because if the retrieval step pulls five semantically identical chunks into context, we've essentially wasted four of our five slots. MMR fixes this without changing the schema, index, or collection. We set a limit at the query level (the candidate pool), then 𝗗𝗶𝘃𝗲𝗿𝘀𝗶𝘁𝘆.𝗠𝗠𝗥(𝗹𝗶𝗺𝗶𝘁=𝟱) for how many objects come back after reranking. Bigger pool = more material to work with = better diversity. 3-4× y weaviate.io/blog/weaviate-… od starting point. Learn more about what’s new in the latest release: https://t.co/kE2omv8g3r Five identical pizzas is a solved problem now. 💬 0 🔄 0 ❤️ 1 👀 80 ⚡