做图像生成或AI绘画的开发者，这篇论文解决了主体身份保留与文本指令跟随的长期矛盾，提出的DLA模块和多阶段去噪策略可以直接参考，值得点开看看具体实现。
主体驱动图像生成旨在根据文本指令生成保留给定主体身份的新图像。现有方法通常分别编码文本和参考图像，限制了跨模态推理能力并导致复制粘贴伪影。本文提出了一种新框架，通过将扩散模型与多模态大语言模型（MLLM）结合，并引入基于VAE的身份条件，实现了文本指令与身份保留的平衡。其中，双层级聚合（DLA）模块用于融合MLLM的多层特征，多阶段去噪策略在推理时逐步平衡语义信息与细节身份。实验表明，该方法在主体驱动图像生成中优于现有方法，有效缓解了复制粘贴问题，并更符合人类偏好。
Squeezing Capacity from Multimodal Large Language Models for Subject-driven Generation
Subject-driven image generation aims to synthesize new images that preserve the identity of the given subject while following textual instructions. Existing approaches often encode text and reference images separately. This limits cross-modal reasoning abilities and causes copy-paste artifacts. Recent frameworks that connect multimodal models and diffusion models improve instruction following, but largely overlook identity preservation. To address these limitations, we condition diffusion models on Multimodal Large Language Models (MLLMs) that jointly encode text and reference images, and augment it with VAE-based identity conditioning. A novel Dual Layer Aggregation (DLA) module is designed to aggregate multi-level MLLM features for optimal conditioning, and a multi-stage denoising strategy is applied to progressively balance the semantic information from MLLM and fine-detail identity from VAE during inference. Extensive experiments demonstrate that our approach harmonizes multimodal understanding with identity preservation, mitigates copy-paste issues, and achieves superior performance regarding human preference on subject-driven image generation. Our project website is available at https://zsh2000.github.io/squeeze-mllm-subject-gen/.