Theia:面向无数据蒸馏的Incidents1M大规模多模态描述与自动验证

Theia: Large-Scale Multimodal Captioning and Automated Validation of the Incidents1M Dataset for Data-Free Distillation

精选理由

想把纯图片的Incidents1M变成图文对?这篇用Qwen3.5做生成和验证,10万条描述加自动打分,干活靠谱。

AI 摘要

该论文针对灾难管理场景的视觉语言模型数据需求,从仅含图像的Incidents1M数据集中恢复了10万张图像,并使用Qwen3.5-4B和Qwen3.5-35B-MoE两种架构生成文本描述。为验证描述可靠性,作者提出图像不可见的LLM-as-a-Judge流水线,用Qwen3.5-9B模拟无数据蒸馏中学生的模态差距。在173,179个标签对上的评估显示,两种架构的语义一致性得分78.65/100,自动评估发现高精确率(77.6%)和低召回率(46.0%)的保守描述行为。该工作提供了可扩展的LLM验证多模态数据集,并揭示了原始标注中的人类不一致性。

原文 · arXiv cs.AI

Theia: Large-Scale Multimodal Captioning and Automated Validation of the Incidents1M Dataset for Data-Free Distillation

The deployment of Vision-Language Models (VLMs) in critical domains like disaster management requires high-quality multimodal datasets, especially for transferring knowledge via Data-Free Knowledge Distillation (DFKD). However, existing datasets in this domain either entirely lack descriptive text, such as Incidents1M, or suffer from severe text-image semantic misalignment, such as CrisisMMD. In this work, we present a novel methodology to construct and automatically validate a large-scale multimodal dataset for disaster response. Starting from the vision-only Incidents1M, we successfully recovered 100,000 images and generated high-fidelity textual descriptions using two distinct Qwen3.5 architectures: a 4B dense model and a 35B Mixture-of-Experts (MoE) model. To ensure the generated captions provide reliable semantic anchoring for DFKD, we introduce an image-blind LLM-as-a-Judge validation pipeline leveraging Qwen3.5-9B. By intentionally obscuring the original image from the judge, this evaluator accurately simulates the modality gap of the student model during data-free distillation. Our evaluation across 173,179 label pairs demonstrates a high semantic agreement (78.65/100) between the two architectures. Furthermore, the automated evaluation reveals a conservative captioning behaviour, characterized by a high Precision (77.6%) and low Recall (46.0%). This minimizes the false positive noise, while simultaneously exposing underlying human annotation inconsistencies in the original ground truth. This work provides a scalable, LLM-validated multimodal dataset and a reproducible framework to advance cross-modal knowledge distillation.