LlamaIndex 发布 ParseBench-2 文档解析基准,3 天下载量破 3 万
做 RAG 或文档处理的可以看看,LlamaIndex 这套基准把 330 多种解析错误按严重度打分,比 OmniDocBench 和 LLM 评委那套更贴近人工判断。顺手还发了 OCRia-1.5 模型,表格表单解析明显升级,已经接入 LlamaParse 可以直接试用。
LlamaIndex 发布 ParseBench-2,定位为最细粒度的文档解析评估基准,上线 3 天月下载量达 30k,登上 Hugging Face trending 榜第 3。基准覆盖 4.3k+ 文档页,比 ParseBench-1 翻倍,并新增表单(forms)维度,对 330+ 种错误类型逐一标注严重度分数,再据此校准每项指标,使分数与人工判断对齐。
与现有基准相比,ParseBench-2 避开了两条路线的缺陷:OmniDocBench 依赖脆弱的确定性指标,ParseArena 的 LLM-as-a-judge 则粗糙、随机且受提示词和模型偏差影响。上一代 ParseBench 已有 50k+ 月下载量、测评过 150+ 个 OCR 方案,被 NVIDIA、Cohere、Perplexity、Aleph Alpha 采用,发布后团队已合并数十个社区 PR,持续修复标注和指标问题。
同期发布的还有 OCRia-1.5 文档解析模型家族,驱动 LlamaParse 全部解析模式,在复杂表格、表单、图表、版式和 bounding box 标注上达到当前最优水平。对需要在生产环境做文档处理的团队来说,可以按需挑选关心的指标子集直接接入。
ParseBench-2 hit 30k monthly downloads in 3 days, and is #3 on @huggingface trending benchmarks! 📈 In ParseBench-2, we tried hard to create a comprehensive taxonomy of metrics to evaluate document processing, while creating a severity-based framework to align these metrics with human judgment. Whether you’re deeply familiar with document OCR and document OCR benchmarks, or just getting started, we encourage you to check out the ParseBench-2 methodology. We’re happy to help you pick whatever subset of metrics you care about if you’re looking to process documents in production. We’ve merged dozens of PRs since the release date, including fixes/metric updates from some of our peers. We’re aggressively fixing any lingering annotation and metric issues. Report: llamaindex.ai/files/parseben… e Hugging Face: huggingface.co/datasets/llama… d Github: github.com/run-llama/Pars… f Jerry Liu @jerryjliu0 Today we’re introducing ParseBench-2 - our most ambitious document parsing benchmark to date. It is the only benchmark that provides both extreme granularity into every single type of production parsing error, while also making sure that metric scores are highly aligned with human judgment. We do this by painstakingly assigning severity scores to 330+ error types, and tuning each metric to match the severity scores. Existing benchmarks either use brittle deterministic metrics (e.g. OmniDocBench) or LLM-as-a-judge (ParseArena), which can help recognize equivalent representations but are coarse/stochastic/subject to bias by the prompt+model. ParseBench-2 also contains: ✅ 4.3k+ document pages (double from ParseBench-1). ✅ An extra dimension (forms) ✅ Much better ground-truth across all dimensions compared to ParseBench-1. This is the next evolution of ParseBench, our hugely popular benchmark that has 50k+ monthly downloads, over 150+ OCR solutions benchmarked, and has been adopted by companies like NVIDIA, Cohere, Aleph Alpha, Perplexity, and others. Alongside ParseBench-2, we are also releasing OCRia-1.5 - a new family of document parsing models that powers all our modes in LlamaParse. They represent a massive step up in capabilities from our latest release, and are state of the art in parsing complex tables, forms, charts, formatting, and assigning bounding boxes to complex layouts. Come check out our blog post, technical report, and website. Blog: llamaindex.ai/blog/introduci… Technical Report: llamaindex.ai/files/parseben… Website: parsebench.ai If you’re interested in trying out our OCRia-1.5 capabilities, come sign up for LlamaParse: cloud.llamaindex.ai
This report is categorized under AITOP's "Models" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include LlamaIndex, ParseBench, OCRia, document parsing.
Source attribution: curated from Jerry Liu (@jerryjliu0).