想让视频生成听你的话?这篇论文用多模态大模型加贝叶斯优化调提示词和参数,胜率最高69%,告别瞎试。
该论文提出Agentic Self-Improvement框架,将图像到视频(I2V)生成重构为闭环优化。第一阶段用多模态大模型迭代优化提示词,通过Davidsonian Scene Graph(DSG)查询和Common Mistake Questions(CMQ)分别检测语义一致性与伪影。第二阶段用贝叶斯优化联合调优随机种子和CFG尺度,并引入Video-Text Adherence(VTA)评分作为质量指标。在人类偏好测试中,该方法生成的视频对比无引导搜索方法最高获得69%的胜率。
Beyond Trial-and-Error: Agentic Optimization for Image-to-Video Adherence
Modern black-box Image-to-Video (I2V) models offer powerful capabilities in automated content creation, yet their lack of fine-grained control and reliability presents significant challenges in professional workflows. Their inherent stochasticity causes minor variations in textual prompts or hyperparameters to yield drastically different outputs often necessitating inefficient, brute-force trial-and-error processes. To address these limitations, we introduce the ``Agentic Self-Improvement" framework, which reframes video synthesis into a closed-loop, goal-directed optimization. Our framework systematically navigates the generation parameter space using a novel two-stage approach. In the first stage, an iterative prompt optimization loop uses a multimodal Large Language Model (mLLM) to refine the input prompt. This refinement implements two automated evaluations: Davidsonian Scene Graph (DSG) queries ensure semantic adherence, and Common Mistake Questions (CMQ) for artifact detection. At the second stage, we use Bayesian optimization to efficiently co-optimize stochastic seeds and CFG scales. This search is guided by a suite of quality metrics, including the novel Video-Text Adherence (VTA) score derived from the DSG and CMQ evaluations. Our framework significantly outperforms unguided search methods: in human preference studies, videos generated via our agentic approach were strongly preferred over baseline outputs, achieving win rates up to 69\%. This work provides a practical and extensible methodology for enhancing the predictability and control of state-of-the-art video generation models, moving the field beyond speculative curiosities toward reliable, production-ready tools.