VQ-VAD:基于向量量化的运动表示学习,用于人体中心视频异常检测

VQ-VAD: Vector-quantized Motion Representation Learning for Human-centric Video Anomaly Detection

精选理由

VQ-VAD把动作量化成码本,重建误差高就是异常。HR-SHT上81.83%,跨域不重训76.69%,挺能打。

AI 摘要

VQ-VAD提出把人体关键点序列通过VQ-GAN量化为离散运动码本,只在正常动作上训练。VQ-VAD推理时若某段动作在码本里重建误差过高,就判定为异常。在HR-SHT上达到81.83%的域内准确率,从CMU Panoptic跨域到HR-SHT无需重训仍有76.69%。该方法在HR-SHT、CMU Panoptic等四个基准上做了域内、跨域和跨数据集三类评估,代码已开源。

原文 · arXiv cs.AI

VQ-VAD: Vector-quantized Motion Representation Learning for Human-centric Video Anomaly Detection

Video Anomaly Detection (VAD) is inherently challenging due to the scarcity of anomalies and the large visual variability in surveillance footage, including changes in lighting, viewpoint, and human appearance. To mitigate visual noise and address privacy concerns, recent work has shifted to pose-based VAD, which focuses on motion dynamics rather than raw video data. However, existing pose-based approaches model human behavior in continuous latent spaces, limiting their ability to learn compact motion patterns necessary for robust behavior analysis. We address this by proposing Vector-Quantized Video Anomaly Detection (VQ-VAD), a novel human-centric anomaly detection framework that learns discrete motion representations. VQ-VAD adapts Vector-Quantized GAN (VQ-GAN), originally developed for image generation, to operate on keypoint sequences and construct a motion codebook of normal behavior. Trained exclusively on normal motion sequences, VQ-VAD detects anomalies by identifying high reconstruction errors when an observed motion sequence cannot be mapped to the learned codebook. We conduct extensive experiments across three complementary evaluation settings, including in-domain, cross-domain, and cross-dataset generalization, on four anomaly detection benchmarks. VQ-VAD achieves strong in-domain accuracy (81.83% on HR-SHT [15]), effective cross-domain transfer from CMU Panoptic [14] (76.69% on HR-SHT [15] without retraining), and competitive cross-dataset robustness. The code base for this work is available at https://github.com/TeCSAR-UNCC/VQ-VAD.