Vik Paruchuri explains why OCR benchmark scores mislead model choice
Flawed ground-truth answers, scoring scripts that reject correct output, and postprocessing tricks can shift rankings more than real capability differences. Engineers choosing between OCR models should run task-specific evals on their own documents instead of trusting vendor-cited aggregate scores. The talk's final chapter walks through building private tests with verified answers and consistent scoring.
