Datalab releases OmniExtractBench for document extraction

The benchmark measured their 'Agentic Plus' extraction model, and after a compatibility fix, it achieved a score of 93.

94%.This score is slightly higher than the highest published results.

Recently, Datalab released OmniExtractBench, a benchmark that combines several benchmarks (our own ExtractBench, LongExtractBench, other vendors) to measure structured document quality. The benchmark measured our "Agentic Plus" extraction model, our most powerful mode for complex document extraction. We found a compatibility workaround in the benchmark’s integration that stripped null from fields that allowed it - removing a valid way to represent missing information. We restored that option while preserving the schema’s structure and required fields. With the same scorer, we were able to get to 93.94% on the benchmark. This puts it slightly higher (or at least equal) to the highest published scores. There's a lot of vendor benchmarks, but ultimately it's also important to evaluate over your own data. We'd be happy to help you get that set up. PR: github.com/datalab-to/omn… Our Agentic Plus mode for LlamaExtract is legitimately quite good, would love to you to try it out and let us know your thoughts! cloud.llamaindex.ai📊 6 ⚡

This report is categorized under AITOP's "Models" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to US-China foundation model benchmarks and commercialization comparison and AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include Datalab, OmniExtractBench, document extraction, benchmark.

Source attribution: curated from Jerry Liu (@jerryjliu0).