PACR-Video:参数高效视频外推框架通过提示适配器路由

Prompt-Adapter Context Routing for Parameter-Efficient Multi-Shot Long Video Extrapolation

精选理由

看看这篇论文,PACR-Video用提示路由和轻量适配器让长视频生成更稳定,不需要全量微调,在6个基准上全面领先。

AI 摘要

PACR-Video提出一种参数高效的多镜头长视频外推框架,冻结文本到视频扩散变换器,仅添加低秩时间适配器并由学习到的镜头角色提示令牌调节。它构建递归提示库,存储前一镜头的实体、位置、动作和风格提示,通过适配器门根据预测叙事依赖路由。在6个多镜头和长视频基准上,PACR-Video在分布质量、语义对齐、身份一致性、时间平滑性、运动稳定性、过渡连贯性和人类偏好上均优于文本到视频、基于微调、记忆增强、流式和递归上下文基线。实验表明,紧凑提示路由和轻量时间适配提供了足够可控能力实现稳定长视频外推。

原文 · arXiv cs.AI

Prompt-Adapter Context Routing for Parameter-Efficient Multi-Shot Long Video Extrapolation

We present PACR-Video, a parameter-efficient framework for multi-shot long video extrapolation that preserves recurring entities, scene structure, visual style, and causal progression without full generator fine-tuning. PACR-Video keeps a text-to-video diffusion transformer frozen and augments it with low-rank temporal adapters conditioned by learned shot-role prompt tokens. To maintain long-horizon coherence, it builds a recursive prompt bank that stores compact entity, location, action, and style prompts from previous shots, then routes them through adapter gates according to predicted narrative dependencies. A Shot-Local/Story-Global tuning objective combines next-shot reconstruction, cross-shot identity contrast, and prompt sparsity regularization, while an adapter composition schedule balances early-shot visual consistency with later-shot event progression and viewpoint change. Across six multi-shot and long-video benchmarks, PACR-Video outperforms text-to-video, tuning-based, memory-augmented, streaming, and recursive-context baselines on distributional quality, semantic alignment, identity consistency, temporal smoothness, motion stability, transition coherence, and human preference. These results show that compact prompt routing and lightweight temporal adaptation provide sufficient controllable capacity for stable long video extrapolation.