IV-CoT:隐式视觉思维链用于结构感知文本到图像生成

IV-CoT: Implicit Visual Chain-of-Thought for Structure-Aware Text-to-Image Generation

精选理由

这篇论文解决了文生图模型在物体数量、空间位置等结构细节上经常翻车的问题,用隐式思维链单次前向传播搞定,在GenEval和T2I-CompBench上效果更好。

AI 摘要

统一多模态大语言模型在文本到图像生成中仍难以精确遵循结构提示(如物体计数、空间关系、属性绑定、粗略布局)。IV-CoT提出的隐式视觉思维链框架将视觉条件分解为结构查询和语义查询的级联,结构查询先形成隐式视觉计划,语义查询再基于该计划渲染外观。训练时引入草图监督信号,无需推理时草图提取或中间解码,在单个前向传播中完成隐式CoT推理。该方法在GenEval和T2I-CompBench基准上取得更优结果,可视化分析验证了结构和语义查询的互补作用。

原文 · arXiv cs.AI

IV-CoT: Implicit Visual Chain-of-Thought for Structure-Aware Text-to-Image Generation

Unified multi-modal large language models (MLLMs) have achieved strong text-to-image generation quality, but still struggle with structure-aware prompt following, where object counts, spatial relations, attribute bindings, and coarse layouts must be preserved. We attribute this limitation in part to the entanglement of structural planning and appearance rendering within a single conditioning stream. To address this issue, we propose Implicit Visual Chain-of-Thought (IV-CoT), a latent visual reasoning framework for query-conditioned image generation. IV-CoT decomposes the visual conditioning queries into a structural-to-semantic cascade, where structural queries first form a latent visual plan and semantic queries then render appearance conditioned on this plan. To guide the structural queries, we introduce training-only sketch supervision, which encourages them to capture structure from sketches without requiring sketch extraction or intermediate decoding at inference time. IV-CoT performs implicit CoT reasoning in a single forward pass and achieves superior results on GenEval and T2I-CompBench. Visualizations and analyses demonstrate that the learned structural and semantic queries play complementary roles in structure-aware generation.