ColBERT 这类多向量检索很好用,但每个 token 都存一个向量,成本吃不消。这篇博客讲怎么在训练时就学会合并 token 向量,少存很多还不掉点。自己在搭 RAG 检索的工程师可以看看。
#embedding
The non-obvious insight is that embedding model swaps are a data-migration problem: old vectors must stay searchable until the new index is fully backfilled and validated, or search quality collapses mid-rollout. Teams running RAG or semantic search in production can follow the walkthrough to plan migrations with built-in evaluation and rollback points, comparing their own Recall numbers against the SciFact baseline before committing.
如果你维护检索系统,模型一升级就得重做全量索引,这篇论文给了个不用重索引的思路,连旧模型都不用依赖,很实用。
Weaviate 这篇教你按自己的数据选 embedding 模型,还拿 OpenAI 大小模型实测对比,方法可直接照抄。
Embedding compatibility across model sizes means teams can swap between a fast, cheap query encoder and a higher-quality one without re-embedding their corpus, which cuts both cost and maintenance work. Engineers running RAG or search pipelines with tight latency budgets should evaluate where Nano and Zero land against their current setup.
Target 公开了自家电商搜索的完整架构,从 embedding 训练到融合策略都讲了,做电商搜索的直接抄作业。
Qdrant 开源了 Supernova 工具和一个 10B 的 FineWeb 数据集,做向量搜索调优和评测的人可以直接拿来用。