HIRA模型在监管行业文档分类中表现出色,通过人机协同的方式提高了分类准确率,减少了模型训练的次数,值得一看。
在监管行业,文档分类受到数据驻留、冷启动标签有限、审查能力稀缺和模型治理程序昂贵等限制。本文提出了一种名为HIRA的训练免费、本地部署的文档分类检索增强级联方法,结合了BM25 OCR文本检索、密集文本嵌入和图像级表示。在80类贸易金融语料库上,HIRA处理了全部30,233个文档的生产流,仅请求人类对1,945个文档(6.4%)进行纠正,将宏-F1从0.6218提升到0.8548。在修正的Tobacco-3482基准上,HIRA使用本地部署的DeepSeek-R1-Distill-Qwen-32B验证器达到了0.9423的宏-F1,比零样本LLM基线高出17.4个百分点,同时仅对约40%的文档调用验证器,减少了约60%的LLM调用。通过518次人类纠正(纠正池的24.8%),HIRA与完全标记的池预言家相匹配,其中所有2,086个池文档都索引了它们的真实标签。这些结果表明,选择性人类反馈和检索记忆适应可以成为监管部署中长尾文档分类的实用替代方案。
HIRA: A Human-in-the-Loop Retrieval-Augmented Cascade for Document Classification in Regulated Industries
Document classification in regulated industries is constrained by data residency, limited cold-start labels, scarce review capacity, and costly model-governance procedures. We present HIRA, a training-free, on-premises retrieval-augmented cascade for document classification in regulated deployments that combines BM25 over OCR text, dense text embeddings, and image-level representations through validation-calibrated weighted reciprocal-rank fusion. Confident documents are classified directly by retrieval; uncertain or visually confusable documents are passed to a locally hosted LLM verifier, which receives the OCR text, retrieved exemplars, label descriptions, and confusion-specific terms. When the verifier remains uncertain, the document is sent to human review. Each correction is stored as a margin-weighted retrieval exemplar and updates a Dirichlet-smoothed confusion graph, letting the system improve without updating model weights. On a private 80-class trade-finance corpus, HIRA processes the full 30,233-document production stream while requesting human correction for only 1,945 documents (6.4%), improving Macro-F1 from 0.6218 to 0.8548. On the corrected Tobacco-3482 benchmark, HIRA reaches 0.9423 Macro-F1 with a locally hosted DeepSeek-R1-Distill-Qwen-32B verifier, 17.4 percentage points above the zero-shot LLM baseline, while invoking the verifier for only about 40% of documents and reducing LLM calls by approximately 60%. With 518 human corrections (24.8% of the pool), HIRA matches the fully labelled pool oracle, in which all 2,086 pool documents are indexed with their ground-truth labels. These results show that selective human feedback and retrieval-memory adaptation can be a practical alternative to repeated model retraining for long-tail document classification in regulated deployments.