论文精选

FaithRewriter:用视觉锚点对齐提示重写,提升文生图忠实度

Seeing is Believing: Aligning Prompt Rewriting with Visual Anchors for Text-to-Image Generation

精选理由

做文生图应用或研究的人会关心——FaithRewriter 用视觉锚点解决了提示重写中的过度推断问题,生成的提示更贴近用户真实意图,值得在项目中尝试集成。

AI 摘要

FaithRewriter 是一种新的提示增强框架,旨在解决文生图模型中用户意图与生成结果之间的差距。现有方法主要优化提示的流畅性和可读性,但缺乏视觉基础,容易过度推断缺失细节。FaithRewriter 首先利用多模态大模型根据原始提示生成中间图像作为视觉线索,然后将该图像与提示结合输入大规模语言模型,生成视觉上更合理的增强内容。最后,这些增强内容被蒸馏到小规模语言模型中实现高效部署。实验表明,FaithRewriter 生成的提示更忠实于用户意图,视觉上更合理,有效缩小了意图-生成差距。

原文 · arXiv cs.AI

Seeing is Believing: Aligning Prompt Rewriting with Visual Anchors for Text-to-Image Generation

Despite the impressive capabilities of text-to-image (T2I) models, an intent-generation gap often persists due to the brevity and ambiguity of user prompts. Existing approaches primarily polish the prompt for fluency and readability. However, the enhancement process still lacks visual grounding. As a result, the rewriter may over-infer missing details, causing an intent-generation gap. To address this limitation, we propose FaithRewriter, a novel prompt-enhancement framework for T2I generation. Specifically, FaithRewriter first leverages a multimodal MLLM to generate an image from the original prompt as an intermediate visual cue. This cue is then combined with the prompt and fed into a large-scale LLM to produce visually grounded augmentations that better reflect how the intended content should appear in images. Finally, these augmentations are distilled into a small-scale LLM for efficient deployment, enhancing its ability to generate effective T2I prompts. Experiments show that FaithRewriter yields prompts that are more faithful to the user intent and more visually plausible than strong baselines, helping narrow the intent-generation gap.

FaithRewriter:用视觉锚点对齐提示重写,提升文生图忠实度 · AI 热点