视频生成模型常被当作世界模型,但几何一致性是硬伤——PDI-Bench用定量方法暴露了现有模型在3D结构上的系统性失败,做视频生成或世界模型研究的团队值得用它来诊断自己的模型。
生成式视频模型常被当作隐式世界模型,但现有评估方法依赖人工判断或学习评分器,难以诊断几何错误。研究者提出PDI-Bench框架,通过分割、点跟踪和单目重建,将生成视频中的物体提升到3D世界坐标,计算尺度-深度对齐、3D运动一致性和3D结构刚性三个维度的残差。配套的PDI-Dataset覆盖多种几何约束场景,测试发现当前最先进的视频生成器存在一致的几何特定失败模式,这些模式不被常见感知指标捕获。该框架为迈向物理可信的视频生成提供了诊断信号。
Quantitative Video World Model Evaluation for Geometric-Consistency
Generative video models are increasingly studied as implicit world models, yet evaluating whether they produce physically plausible 3D structure and motion remains challenging. Most existing video evaluation pipelines rely heavily on human judgment or learned graders, which can be subjective and weakly diagnostic for geometric failures. We introduce PDI-Bench (Perspective Distortion Index), a quantitative framework for auditing geometric coherence in generated videos. Given a generated clip, we obtain object-centric observations via segmentation and point tracking (e.g., SAM 2, MegaSaM, and CoTracker3), lift them to 3D world-space coordinates via monocular reconstruction, and compute a set of projective-geometry residuals capturing three failure dimensions: scale-depth alignment, 3D motion consistency, and 3D structural rigidity. To support systematic evaluation, we build PDI-Dataset, covering diverse scenarios designed to stress these geometric constraints. Across state-of-the-art video generators, PDI reveals consistent geometry-specific failure modes that are not captured by common perceptual metrics, and provides a diagnostic signal for progress toward physically grounded video generation and physical world model. Our code and dataset can be found at https://pdi-bench.github.io/.