HAD-VLM:用扩散机制解决医学影像测量中的数值连续性问题
Beyond Discrete Tokens: Preserving Numerical Continuity in Medical Vision-Language Models
一篇解决医疗 VLM 量尺问题的论文:离散 token 测不准毫米级数值,HAD-VLM 用扩散头预测坐标和长度,误差比 Vicuna 基线降了最高 37.6%。
HAD-VLM 是一个混合自回归-扩散框架,在同一主干中结合文本生成与连续测量预测,用于超声心动图定量测量。在 EchoNet-LVH 数据集的 IVSd、LVIDd、LVIDs、LVPWd 四项测量任务上,平均绝对误差分别为 1.36、2.77、3.05 和 1.16 mm,相比基于 Vicuna 的自回归基线降低 21.4%-37.6%。该模型同时超越了微调后的 Qwen3-VL 和直接的 MLP 回归方法。在 X 射线椎体测量任务上,平均绝对误差从 11.4 像素降至 6.7 像素,医学问答能力则保持不变。
Beyond Discrete Tokens: Preserving Numerical Continuity in Medical Vision-Language Models
Objective: To develop a unified vision-language model for prompt-conditioned quantitative ultrasound measurement that addresses the mismatch between continuous measurement targets and discrete token-based numerical prediction. Methods: This article presents HAD-VLM, a hybrid autoregressive-diffusion framework that combines autoregressive text generation with continuous measurement prediction within a shared backbone. Visual prompt tuning adapts the vision encoder to ultrasound images. A diffusion process predicts a five-component representation comprising caliper endpoint coordinates and measurement length. A post-processing strategy combining the proposed coordinate-length consistency mechanism with ensemble prediction is introduced to refine measurement estimates. We evaluate four echocardiographic measurement tasks on EchoNet-LVH, with additional assessments of medical question answering and X-ray vertebral measurement. Results: HAD-VLM achieves mean absolute errors of 1.36, 2.77, 3.05, and 1.16 mm for IVSd, LVIDd, LVIDs, and LVPWd, respectively, reducing errors by 21.4-37.6\% relative to the Vicuna-based autoregressive baseline. It also outperforms fine-tuned Qwen3-VL and direct MLP regression across all four tasks. Medical question answering performance is preserved on the evaluated dataset. For X-ray vertebral measurement, HAD-VLM reduces mean absolute error from 11.4 to 6.7 pixels. Conclusion: Integrating autoregressive language generation with diffusion-based continuous prediction improves quantitative medical image measurement while retaining prompt-conditioned task flexibility. Significance: HAD-VLM provides a unified approach to combining language interaction with anatomically grounded numerical prediction for quantitative medical image analysis.