想要生成同一主题下不同设计的图像?这篇论文教你用VLM在文本层面控制多样性,比随机抽噪声靠谱多了。
该论文提出一种名为Semantic Browsing的方法,解决文本到图像模型生成样本多样性不足的问题。传统方法依赖随机噪声产生无意义变化,而Semantic Browsing通过Vision Language Model(VLM)在文本层面施加结构化语义变异。用户可沿可解释的语义轴(如物体属性、场景布局)导航图像集,每个变体对应一个具体可理解的语义决策。实验表明该方法能生成多样且可浏览的设计空间。
Semantic Browsing: Controllable Diversity for Image Generation
Modern text-to-image models excel in visual fidelity and prompt adherence. However, this strict adherence comes at the cost of diversity: generated samples tend to collapse into a single visual interpretation. Existing methods to improve diversity produce outputs driven by incidental variations rather than meaningful design choices. This motivates a new variant of the diversity task where structure is enforced on the generated samples. We introduce a method for controlled diversity that enables Semantic Browsing, where users can navigate structured image galleries and experience creative exploration through a systematic traversal of meaningful, interpretable axes of variation. Achieving this level of semantic control requires a deep understanding of the scene. We exploit the fact that recent text-to-image models are trained on elaborated captions, effectively decoupling semantic decision-making from pixel generation. This enables a paradigm shift: instead of relying on stochastic variation within the text-to-image model, we induce diversity directly at the text level. By leveraging rich textual representations, we allow a Vision Language Model (VLM) to operate on the full scene context. To overcome the generic outputs typical of standard VLMs, we employ an agentic workflow that explicitly enforces structured variation attuned to the original prompt. We demonstrate that our method produces diverse and navigable design spaces where every variation corresponds to a specific, user-understandable semantic decision.