Milvus 3.0 Storage V3 引入 Manifest

Milvus 3.0 Storage V3 introduces 𝗠𝗮𝗻𝗶𝗳𝗲𝘀𝘁, a versioned source of truth for datasets shared a...

Milvus 3.0 Storage V3 introduces 𝗠𝗮𝗻𝗶𝗳𝗲𝘀𝘁, a versioned source of truth for datasets shared across multiple compute paths. In AI systems, the same data often needs to serve vector search, real-time analytics, batch processing, and model training. A common way to optimize each workload is to maintain separate copies for different engines. That works, but it comes with a cost: more storage, more I/O, and a harder question to answer over time: 𝗪𝗵𝗶𝗰𝗵 𝗳𝗶𝗹𝗲𝘀 𝗮𝗰𝘁𝘂𝗮𝗹𝗹𝘆 𝗿𝗲𝗽𝗿𝗲𝘀𝗲𝗻𝘁 𝘁𝗵𝗲 𝗰𝘂𝗿𝗿𝗲𝗻𝘁 𝗱𝗮𝘁𝗮𝘀𝗲𝘁? A Parquet file can tell you what it contains. It cannot tell you, by itself, which dataset version it belongs to, which deletes apply to it, which statistics and indexes are still valid, or which set of files a query should read. As the number of workloads, file types, and update paths grows, those relationships become difficult to manage with file-level conventions alone. Manifest gives Milvus a single place to describe dataset state: the files in a version, the schema they follow, the deletes that apply, and the statistics and indexes associated with that version. That gives the storage layer a much cleaner contract. Reading a Manifest tells Milvus how to interpret a specific dataset version. Writes, updates, and compaction can commit against that state, resolve conflicts, and produce a new version. Higher-level components no longer need to reconstruct those relationships independently. For multimodal AI infrastructure, Manifest reduces unnecessary data copies and duplicated plumbing while giving every compute path a consistent view of dataset state. 💬 0 🔄 2 ❤️ 1 👀 117 📊 1 ⚡