Products

LlamaIndex 发布 ParseBench-2:可自定义权重的文档解析基准

Why we picked it

解析 PDF 表单和解析图表幻灯片根本是两回事,LlamaIndex 这次让 19 个解析器按你自己定的权重实时重排,还能导出 policy JSON 复用到自家评估里。做文档处理 RAG 选型的团队值得花十分钟玩玩。

LlamaIndex 上周发布 ParseBench-2,一个覆盖文档解析全部维度的综合基准,评估范围包括阅读顺序、格式、表格、图表、表单和视觉定位,共对 19 个解析器进行排名。基准默认将“严重”错误按 100 分制扣 10 分惩罚,但不同业务场景对错误的容忍度并不相同:处理后台单据的团队主要关心表格和表单而非图表,处理法律文档的团队则更在意精确格式。

Image · Jerry Liu
Body

为此 LlamaIndex 推出 ParseBench-2 Metric Lab(parsebench.ai/lab),一个可交互的界面,用户可以自行设定各类别(表格、图表、表单、文本)的权重,并调整 critical、major、minor 三档错误的扣分幅度,浏览器内 19 个解析器的排名会实时重排。确定评分方案后可导出 policy JSON,直接用于自有文档的评估。

对开发者而言,这个基准承认“最佳解析器取决于你的文档”这一前提:解析保险理赔表单的团队和解析财报幻灯片的团队需要的能力不同。Metric Lab 把选型从看通用榜单转向按业务需求定制评分,导出的 JSON 也可复用到团队自己的评估流程中。

Original · Jerry Liu

We launched ParseBench-2 last week as a comprehensive benchmark to evaluate all dimensions of document parsing: reading order, formatting, tables, charts, forms, and visual grounding. While we strive to create some standardization, it's very clear that not everyone weights the same errors equally: - If you're dealing with backoffice paperwork, you probably don't care about charts, and mostly care about tables and forms - If you're not dealing with legal documents, you likely don't care about precise formatting. Also, you may care about different severities differently. By default we penalize "severe" errors by 10 points out of 100, but for instance if getting dollar values wrong is catastrophic, you can turn the slider way up. Introducing the ParseBench-2 Metric Lab: it's a simple interface where you can define how much you care about different evaluation metrics, and how you weigh different severities. Once you've settled on a dimension, you can export the policy JSON, so that you can use it on evaluations over your own documents! Come check the Metric Lab out: parsebench.ai/lab/index.html For more information on the benchmark itself and overall methodology, come check it out here: llamaindex.ai/blog/introduci…

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Products" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include LlamaIndex, ParseBench, document parsing, RAG.

Source attribution: curated from Jerry Liu (@jerryjliu0).