轻量级VLM在复杂场景中经常胡编乱造,DRScaffold用结构化监督解决了这个痛点,做视觉推理或部署小模型的团队可以直接用它的框架和基准来提升可靠性。
轻量级视觉语言模型在标准基准上表现不错,但在需要多步推理的密集场景中(如多个物体、属性、关系)系统性地失败。为此,研究者首先构建了DRBench基准,包含14,573个问题、2,943张图像,覆盖5类任务和3个推理层次。然后提出DRScaffold框架,通过将监督目标分解为四个因果有序阶段,在不改动模型架构的情况下强制进行有依据的推理。实验表明,使用DRScaffold训练的Qwen2.5-VL-3B在DRBench上超越了冻结的Qwen2.5-VL-32B,证明结构化监督可以替代大量模型规模。代码和模型已开源。
DRScaffold: Boosting Dense-Scene Reasoning in Lightweight Vision Language Models
Lightweight vision-language models perform competitively on standard benchmarks yet fail systematically in dense-scene reasoning, where multiple objects, attributes, and relations must be jointly grounded and resolved through multi-step inference. Such capability is critical for real-world applications where models must reliably interpret cluttered environments. Yet existing training signals provide no explicit grounding between reasoning steps and the underlying visual entities and relations, leaving lightweight models free to generate fluent but visually unanchored reasoning chains. To address this gap, we first introduce DRBench, a benchmark of 14,573 questions across 2,943 images, organized into five task categories spanning three progressive reasoning layers. Building on DRBench, we propose DRScaffold, a supervised fine-tuning framework that decomposes the supervision target into four causally ordered stages, enforcing grounded reasoning without architectural modification. Experiments on three lightweight VLMs demonstrate substantial gains on DRBench while preserving or improving performance on general-purpose benchmarks. Notably, Qwen2.5-VL-3B trained with DRScaffold surpasses the frozen Qwen2.5-VL-32B on DRBench, demonstrating that structured supervision can substitute for a significant portion of model scale in dense-scene reasoning. Our code and models are available at https://github.com/irene-shi/DRScaffold .