这篇论文开源了120万对的放射学双语数据集RefRad2D，训练出的RadGrounder能同时做报告生成、VQA和空间定位，空间定位还不影响语言质量，搞医疗AI的值得看看。
论文提出RefRad2D数据集，包含120万CT和MR图像-文本对，覆盖德语和英语。该数据集通过LLM标注和自动分割生成任务特定的VQA和空间定位子集。基于此训练的RadGrounder模型同时支持报告生成、视觉问答和边界框检测/分割。在Slake和VQA-RAD外部基准上，RadGrounder取得与专用医学VLM竞争的结果。加入临床数据训练可提升开放VQA表现，且增加空间定位监督不降低语言质量。
Scalable Training of Spatially Grounded 2D Vision-Language Models for Radiology
We study how to train visually grounded vision-language models (VLMs) for radiology without manual spatial annotations. We introduce RefRad2D, a large-scale bilingual (German/English) dataset of 1.2M CT and MR image-text pairs derived from clinical practice, with task-specific VQA and spatial grounding subsets generated automatically via LLM-based curation and automated segmentation. Trained on this data, our model RadGrounder jointly performs report generation, visual question answering, and spatial grounding via bounding-box detection or segmentation. On external VQA benchmarks (Slake, VQA-RAD), RadGrounder achieves competitive results with specialized medical VLMs. Adding our clinical data to the training mixture improves open-ended VQA over fine-tuning on the downstream datasets alone, showing the transferability of our dataset. Crucially, adding grounding supervision does not degrade language quality, enabling spatially verifiable outputs at no cost to VQA performance.