Harvey开源了个合成律所数据集，专测AI Agent处理法律检索的硬问题，比单纯RAG更狠。
Harvey与EngramLab开源了一个包含1亿+token的合成律所数据集，覆盖46个客户、250多个虚拟事项，约1万份文件。该环境用于评测智能体能否像资深律师一样检索和理解事务所过往工作。联合创始人Dan Biderman指出，像“今年哪些并购交易未完成”这类问题无法靠RAG搜索，因为不存在“未完成”的记录，必须逐案阅读文件。数据集是Harvey迈向深度理解律所工作流的第一步。
The types of hard problems @EngramLab works on ft. co-founder @dan_biderman: "Clients do financing,...
The types of hard problems @EngramLab works on ft. co-founder @dan_biderman : "Clients do financing, mergers, acquisitions and things like take loans and do deals. And there's many queries that agents might run into which are these kinds of ambient, hard questions that are not easily searchable with RAG. For example, if you want to ask, which M&A deals haven't we completed this year? To actually solve this problem, you have to go client matter by client matter [and] read all the files. You can't read in any place that it was not completed." Your browser does not support the video tag. 🔗 View on Twitter Harvey @harvey We're open sourcing a 100M+ token synthetic law firm we built with @EngramLab . The firm contains work product from 250+ synthetic matters across 46 clients, spanning ~10k files. We built this environment to evaluate an agents' ability to search and understand a firm's past practice to inform present work - the same knowledge that a tenured associate or partner would have. It's our first step towards building agents that deeply understand a firm's work and processes. More to come soon Deep dive by @ItsJulioPereyra and @nikogrupen : 🔗 View Quoted Tweet 💬 1 🔄 0 ❤️ 1 👀 305 📊 1 ⚡