ParseBench-2 lets users define custom penalty policies for parsing metrics
Customizable severity policies shift parsing evaluation from a single fixed leaderboard toward scoring that mirrors a specific deployment's error tolerance — a model other benchmark builders may copy. Teams choosing between parsing vendors should audit each benchmark's penalty design and judge prompts, since both can be tuned to favor the sponsor. The spat also shows evaluation itself becoming a competitive battleground worth tracking.