这篇论文搞了个Appearance Pointers，能让DiT模型精确指定图片不同区域该长什么样，不用重训模型，效果还比那些专门的方法好。
论文提出Appearance Pointers，一种紧凑的额外token，用于指导扩散变换器（DiT）在指定空间区域获取正确的纹理、对象身份等外观信息。该方法通过区域对应网络生成token，并用空间聚合机制优化，无需重新训练基础DiT模型即可处理多个区域描述。在多个基准测试上，单模型性能达到或超越针对特定模态的现有方法，实现了首个模态无关的局部多模态控制接口。
Appearance Pointers -- Multimodal Region Control of Diffusion Transformers
Controllable image generation remains challenging for creative professionals, who often require precise regional control over materials, object identities, and spatial arrangements that cannot be reliably achieved through text prompting alone. Diffusion Transformers (DiTs) can natively ingest heterogeneous tokens stemming from texts and images, but they lack mechanisms for determining where and how these tokens should influence the output. We introduce appearance pointers, compact tokens that guide DiTs toward the correct appearance cues at the correct spatial locations by aligning text or image inputs with user-specified masks. Appearance pointers are produced by a region correspondence network and refined through a spatial aggregation mechanism, enabling the model to handle multiple regional descriptions without significantly increasing token load. Our approach introduces the first modality-agnostic interface for localized multimodal control in a DiT without retraining the base model from scratch. Across a range of metrics, our single model reaches or surpasses the performance of modality-specific state of the art methods, offering a simple and extensible path toward precise, region-aware, multimodal guidance in generative image synthesis.