这篇论文发现VLM其实知道个数但说不准，用内部探测+自纠正就能白嫖15%精度提升，不调参数，挺实用。
研究者发现视觉语言模型（VLM）在计数任务中常输出错误答案，但内部表示已编码正确数量。通过训练非线性探针，在四个VLM和五个计数数据集上可检测到计数错误。SVCCA分析显示探针沿特定方向读取的表示与正确计数方向未对齐。因果干预实验证实增强计数识别方向可提升模型计数性能。提出的探测器引导自纠正方法在推理时仅对检测到错误的输入重提示，使计数准确率最高提升15.6个百分点，且无需更新参数。
The Count Is There, but Misaligned: Understanding and Correcting Counting Failures in VLMs
Despite strong performance on many multimodal tasks, vision-language models (VLMs) still struggle with basic object counting. We investigate whether this reflects missing internal knowledge or a gap between internal representations and verbalized outputs. Training simple probes on activations from four VLMs across five counting datasets reveals that nonlinear probes can reliably detect counting errors, suggesting that VLMs often encode the correct count even when they output the wrong answer. SVCCA analysis shows that probes trained on ground-truth counts and probes trained on model outputs occupy a partially shared activation subspace but read out along misaligned directions. We further validate our findings using a causal steering intervention, proving that strengthening the direction of count-identified probes does improve model counting performance. Motivated by this result, we propose a detector-guided self-correction method that selectively re-prompts the model only when an internal error detector predicts failure. This simple inference-time intervention improves counting accuracy by up to 15.6 absolute percentage points, without any parameter updates. Our results establish activation-based error probing as both a practical tool for improving VLM counting and a mechanistic lens on the gap between internal knowledge and model outputs.