做检索系统的可以看看:Milvus 3.0 改表结构不用下线重建了,还能自动补 BM25 向量,省事不少。
Milvus 3.0 支持在服务流量不断的情况下在线增删字段,无需重建集合或安排迁移窗口。新增字段会创建可空列,不重写已有数据;废弃字段可运行时更新元数据。Milvus 3.0 还支持内部回填,例如直接从文本字段生成 BM25 稀疏向量,省去单独的客户端编码器。外部回填已列入路线图,计划通过 Spark 对快照计算新列值并增量更新索引。
𝗠𝗶𝗹𝘃𝘂𝘀 𝟯.𝟬 𝗶𝗻𝘁𝗿𝗼𝗱𝘂𝗰𝗲𝘀 𝗼𝗻𝗹𝗶𝗻𝗲 𝘀𝗰𝗵𝗲𝗺𝗮 𝗲𝘃𝗼𝗹𝘂𝘁𝗶𝗼𝗻 𝗮𝗻𝗱 ...
𝗠𝗶𝗹𝘃𝘂𝘀 𝟯.𝟬 𝗶𝗻𝘁𝗿𝗼𝗱𝘂𝗰𝗲𝘀 𝗼𝗻𝗹𝗶𝗻𝗲 𝘀𝗰𝗵𝗲𝗺𝗮 𝗲𝘃𝗼𝗹𝘂𝘁𝗶𝗼𝗻 𝗮𝗻𝗱 𝗯𝗮𝗰𝗸𝗳𝗶𝗹𝗹, 𝗮𝗹𝗹𝗼𝘄𝗶𝗻𝗴 𝘁𝗲𝗮𝗺𝘀 𝘁𝗼 𝗰𝗵𝗮𝗻𝗴𝗲 𝗮 𝗰𝗼𝗹𝗹𝗲𝗰𝘁𝗶𝗼𝗻’𝘀 𝗱𝗮𝘁𝗮 𝗺𝗼𝗱𝗲𝗹 𝘄𝗵𝗶𝗹𝗲 𝗶𝘁 𝗰𝗼𝗻𝘁𝗶𝗻𝘂𝗲𝘀 𝘀𝗲𝗿𝘃𝗶𝗻𝗴 𝘁𝗿𝗮𝗳𝗳𝗶𝗰. In production retrieval systems, schemas rarely remain static. As retrieval requirements evolve, teams frequently need to add new embedding models, sparse vectors, labels, metadata fields, text columns, and retention policies. Before Milvus 3.0, these data model changes often came with high operational costs, requiring disruptive collection rebuilds or carefully scheduled migration windows to avoid service impact. To resolve this pain point, Milvus 3.0 enables teams to add, populate, and drop fields without taking the collection offline. Adding a field creates a new nullable column without rewriting existing data. Dropping a deprecated or experimental field updates the collection metadata at runtime. Both operations simply update the collection manifest instead of forcing a full data rewrite, significantly reducing operational overhead. While online schema changes support structural updates, backfill complements the capability by completing the full schema-evolution workflow and populating newly added fields with valid data. Milvus 3.0 supports inner backfill for values derived from data already stored in the collection. For example, Milvus can generate BM25 sparse vectors directly from a text field, removing the need for a separate client-side encoder in dense-and-sparse hybrid retrieval pipelines. In addition to in-engine backfill, external backfill is on the roadmap for scenarios where new column values need to be computed outside Milvus. The workflow is straightforward: take a snapshot, run Spark against the stable view, compute the new column, write the values back, and let Milvus update the relevant indexes incrementally. Combined together, online schema evolution and backfill enable retrieval pipelines to iterate flexibly. Teams can continuously adjust data models to fit evolving busi milvus.io/blog/announcin… uilding the entire collection every time schema updates occur. More in the Milvus 3.0 launch blog: https://t.co/hq4qqZASuX 💬 0 🔄 0 ❤️ 0 👀 68 ⚡