Papers

FVScore:用神经符号方法做视频生成的时序忠实度评估

FVScore: Specification-Driven Neuro-Symbolic Video Evaluation

Why we picked it

视频生成的自动评测一直是个坑,MLLM 打分跟人评经常对不上。这篇把 LTL 时序逻辑搬进视频评估,能精确报出第几秒哪里违背了提示词,做视频模型研发的同学可以直接参考它的工具链设计。

研究者提出 FVSCORE,一个规格驱动的视频生成评估框架,解决现有 MLLM 整体打分难以与人类判断对齐、无法定位具体时序违规的问题。

Body

框架把每段视频编译为显式命题(捕捉提示词要求)和隐式命题(捕捉未言明的物理、时序、跨模态约束),再用专门的感知工具为每个命题提供依据。验证结果通过受线性时序逻辑(LTL)启发的软时序算子进行评估,随后由 Significance-Aware Calibration 步骤只保留感知上显著的违规,最后输出带时间戳和证据的报告。

在公开基准和复杂多事件基准上,FVSCORE 在与人类评分的相关性上超过整体式和基于智能体的评估器,且反馈可解释。

Original · arXiv cs.AI

FVScore: Specification-Driven Neuro-Symbolic Video Evaluation

We introduce Faithfulness Video Score (FVSCORE), a neuro-symbolic framework for specification-driven video generation evaluation. Existing evaluators rely on holistic MLLM judgments or vision-language metrics, yet aligning their scores with human judgments and tracing them to specific temporal violations remain open challenges. FVSCORE addresses these limitations by compiling each video into explicit propositions capturing prompt requirements and implicit propositions capturing unstated physical, temporal, and cross-modal constraints. It grounds each proposition using specialized perceptual tools and evaluates the resulting confidence traces with soft temporal operators inspired by Linear Temporal Logic. A Significance-Aware Calibration step then aligns verification with human perception by retaining only perceptually significant violations before producing a timestamp- and evidence-grounded report. We conduct extensive experiments across public and complex multi-event benchmarks with the strongest video models to date, demonstrating that FVSCORE outperforms holistic and agent-based evaluators in human correlation while providing highly accurate and interpretable feedback.

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Key entities and technical topics under continuous tracking include video generation, FVSCORE, Linear Temporal Logic, benchmarks.

Source attribution: curated from arXiv cs.AI (@Do Xuan Long, Long T. Le, Yiwen Song, Min-Yen Kan, Tomas Pfister, Yale Song).