游戏QA团队和VLM研究者终于有了专门测试时间性故障的基准——当前模型表现接近随机，说明这是个硬骨头，做自动化测试的值得关注。
现有视觉语言模型（VLM）在游戏故障检测评估中，大多将故障视为静态视觉异常，忽略了时间性故障——这类故障需通过帧间变化才能识别。研究者提出TempGlitch基准，包含五种时间性故障类型及配对的无故障视频，用于系统评估。对12个开源和闭源VLM的测试显示，当前模型在TempGlitch上表现接近随机，要么过于保守漏检，要么过于敏感误报。增加帧采样密度或模型规模并不能可靠解决这些问题。该基准为时间推理、游戏理解和自动化故障检测提供了聚焦测试平台。
TempGlitch: Evaluating Vision-Language Models for Temporal Glitch Detection in Gameplay Videos
Vision-language models (VLMs) are increasingly being explored for video game quality assurance, especially gameplay glitch detection. Most existing evaluations, however, treat glitches as static visual anomalies, asking models to detect failures from a single frame. We argue that this framing misses a key distinction: some glitches are spatial and visible in an isolated frame, whereas others are temporal and become evident only through changes across ordered frames. A preliminary study confirms this gap, showing that temporal glitches are substantially harder for VLMs to detect than spatial ones. To enable systematic evaluation of this underexplored setting, we introduce TempGlitch, a controlled gameplay video benchmark for temporal glitch detection. TempGlitch covers five temporal glitch types with balanced per-category samples, together with paired glitch-free videos that enable reliable binary evaluation. We evaluate 12 proprietary and open-weight VLMs across multiple frame-sampling settings. Our results show that current VLMs remain near chance on TempGlitch, often collapsing into either overly conservative behavior that misses most glitches or overly sensitive behavior that flags clean videos as glitchy. Moreover, denser frame sampling and larger model size do not reliably resolve these failures. TempGlitch provides a focused testbed for temporal reasoning, robust gameplay understanding, and automated glitch detection with VLMs. Code and data are available at the project website.