这篇论文把VLM做图像编辑时定位不准的老问题挖到底了——原来不是模型不行，是提取信号的方式错了。他们用了个'代理分析'的巧招，把隐藏的定位信号给揪了出来，搞编辑管线的值得一看。
本文研究VLM在扩散图像编辑管线中作为条件编码器时的定位能力下降问题。作者提出Analysis-by-Proxy框架，训练轻量级代理模型在VLM中间表示上执行辅助定位任务。实验发现，在单次前向传递约束下，定位信号未可靠传播到预设条件层配置，而是隐藏在随输入提示变化的中间表示中。该框架揭示了现有编辑管线条件提取策略的根本失败，为设计更合理的条件架构提供了方向。
Analysis-by-Proxy: Localization Signals in VLMs Operating as Condition Encoders
Vision-Language Models (VLMs) are increasingly utilized as the conditioning backbone for diffusion-based image editing due to their remarkable multimodal reasoning capabilities. While standalone VLMs demonstrate strong localization capabilities, editing pipelines frequently struggle to maintain this accuracy, particularly in complex, multi-entity scenes. In this work, we investigate this performance gap, hypothesizing that it stems from treating the VLM as a condition encoder. In this role, the model is restricted to a single forward pass, preventing the autoregressive generation process for which it was optimized, thereby failing to fully expose its capabilities. To investigate whether this spatial understanding persists when the VLM is used as a condition encoder, we introduce Analysis-by-Proxy. In this framework, we train a lightweight, interpretable proxy model on the VLM's intermediate representations using an auxiliary localization task. By analyzing the VLM through this proxy, we uncover the specific VLM representations that encode localization information. Our findings expose a fundamental mismatch between how spatial knowledge is represented within a VLM condition encoder and how it is extracted by current editing pipelines. We reveal that under single-pass constraints, the localization signal does not reliably propagate to the predefined layer configurations commonly used for conditioning. Instead, this crucial signal remains hidden within intermediate representations, at locations that vary depending on the input prompt. Using our introduced Analysis-by-Proxy framework, we reveal the fundamental failures of existing condition extraction strategies in editing pipelines, opening the door to more principled design of conditioning architectures.