Jev tested across three Zilliz open-source RAG projects
The benchmarks show where cheap small models can replace larger ones in RAG pipelines: stopping decisions and relationship filtering see large latency and cost savings with matched recall. Engineers building retrieval systems can use the published code to pick where to route between Jev, GPT-5-mini and Voyage rerank-3. The evaluation includes Chinese-language queries, useful for teams serving bilingual users.
Zilliz integrated Jev into DeepSearcher, MemSearch and a Vector Graph RAG pipeline to evaluate it on frequent, bounded judgments. In DeepSearcher, Jev matched DeepSeek V4 Flash at 93.25% Recall @5 on 100 multi-hop questions while cutting median decision latency from 2.23s to 0.55s at roughly one-seventh of the API cost. In MemSearch, Jev raised reranking Recall @5 from 74.71% to 79.41% on 2,172 Chinese and English queries, behind Voyage rerank-3 at 81.87%. In Vector Graph RAG, Jev outperformed GPT-4o-mini but trailed GPT-5-mini by 1 point on HotpotQA and 4.13 points on MuSiQue. The team concludes Jev suits search stopping, filtering, routing and simple reranking, while heavier tasks still favour larger models and specialist rerankers.
𝗪𝗲 𝗶𝗻𝘁𝗲𝗴𝗿𝗮𝘁𝗲𝗱 𝗝𝗲𝘃 𝗶𝗻𝘁𝗼 𝘁𝗵𝗿𝗲𝗲 𝗼𝗽𝗲𝗻-𝘀𝗼𝘂𝗿𝗰𝗲 𝗽𝗿𝗼𝗷𝗲𝗰𝘁𝘀 𝘁𝗼 𝘁𝗲𝘀𝘁 𝘁𝗵𝗮𝘁 𝗶𝗱𝗲𝗮. First, in 𝗗𝗲𝗲𝗽𝗦𝗲𝗮𝗿𝗰𝗵𝗲𝗿, we used Jev to decide whether an Agentic Search workflow had gathered enough evidence to stop. Across 100 multi-hop questions, Jev matched DeepSeek V4 Flash at 93.25% Recall @5 . Yet median decision latency fell from 2.23s to 0.55s, with estimated API cost at roughly one-seventh. Second, in 𝗠𝗲𝗺𝗦𝗲𝗮𝗿𝗰𝗵, we tested coding-agent memory reranking on 2,172 private Chinese and English queries. Jev improved Recall @5 from 74.71% to 79.41%, but remained behind the specialist Voyage rerank-3 at 81.87%. Third, in 𝗩𝗲𝗰𝘁𝗼𝗿 𝗚𝗿𝗮𝗽𝗵 𝗥𝗔𝗚, Jev reranked and filtered candidate relationships for multi-hop retrieval. It outperformed GPT-4o-mini, but trailed GPT-5-mini—by just 1 point on HotpotQA and 4.13 points on the harder MuSiQue benchmark. Jev is compelling for frequent, bounded judgments such as search stopping, filtering, routing, and simpler reranking. But as tasks become more domain-specific or reasoning-heavy, stronger generative models and specialist rerankers still earn their place. Want to di github.com/zilliztech/dee… • DeepSearcher — search-stopping experiment, code, and e github.com/zilliztech/mem… /aqPCKQnhMi • MemSearch — Jev reranking integration and evaluation: https://t.co/ github.com/zilliztech/vec… ph RAG — Jev relationship filtering, evaluation, and cost analysis: https://t.co/7z9kGZsKuO 💬 0 🔄 0 ❤️ 0 👀 37 ⚡
