LogCDM:用行为图漂移与自适应信号融合做可解释日志异常检测
LogCDM: Interpretable Log Anomaly Detection via Behavioral Graph Drift and Adaptive Signal Fusion
做日志监控的同学可以看看:8M 参数、CPU 上 10ms 内打分,F1 却逼近 8B 参数的 LLM 检测器。亮点是卡方散度视角给出逐边告警解释,适合部署在无法上 GPU 的监控节点上。
LogCDM 是一个紧凑型日志异常检测器,参数量仅 8.1-8.8M,推理时无需 GPU,在 CPU 上对单个日志会话的打分耗时 5.5-10.5 ms。它在 HDFS 上达到 F1 0.984,按时间顺序切分的 BGL 上达到 0.870,Thunderbird 上达到 0.850,与同一协议下评估的 8.1B 参数 LLM 检测器相差 0.013-0.12,并在 BGL 上超过所有紧凑基线。
技术核心是三路信号融合:一个小型因果 Transformer 计算下一事件惊喜度;图注意力嵌入衡量会话转移图与正常参考图的距离;转移漂移统计量被证明等价于转移核之间的频率加权卡方散度。散度视角带来了渐近零分布、长度标准化分数和逐边分解,使每条告警都可解释。一个 sigmoid 门控负责融合三路信号,在共享协方差高斯模型下即贝叶斯后验,其学习到的权重能显示各数据集依赖哪种信号。
消融实验显示结构信号在 BGL 和 Thunderbird 上带来 0.025-0.030 的 F1 提升,但在 HDFS 上几乎无贡献,与融合权重一致。相较 BERT 级编码器到 8B 语言模型这类需要 GPU 推理的大模型,LogCDM 可直接部署在产生日志的监控节点上。
LogCDM: Interpretable Log Anomaly Detection via Behavioral Graph Drift and Adaptive Signal Fusion
Log-based anomaly detection is increasingly dominated by large pretrained models, from BERT-scale encoders to 8B-parameter language models, which require GPUs at inference time and are difficult to deploy on the monitoring nodes where logs are produced. We present LogCDM, a compact detector that scores each log session using three complementary signals: the next-event surprise of a small causal Transformer, the distance between a graph-attention embedding of the session's own transition graph and that of a normal reference graph, and a transition-drift statistic that we show is exactly a frequency-weighted chi-squared divergence between transition kernels. The divergence view yields an asymptotic null distribution, a length-standardized score, and a per-edge decomposition that explains each alarm. A sigmoid gate fuses the three signals; under a shared-covariance Gaussian model, this gate is the Bayes posterior, and its learned weights indicate which signal each dataset relies on. With 8.1-8.8M parameters and no GPU required at inference, LogCDM reaches F1 scores of 0.984 on HDFS, 0.870 on chronologically split BGL, and 0.850 on Thunderbird, within 0.013-0.12 of an 8.1B-parameter LLM detector evaluated under the same protocol and above all compact baselines on BGL. It scores a session in 5.5-10.5 ms on a CPU. Ablations show that the structural signals add 0.025-0.030 F1 on BGL and Thunderbird but almost nothing on HDFS, consistent with the learned fusion weights.
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include LogCDM, 异常检测, 日志分析, Transformer.
Source attribution: curated from arXiv cs.AI (@Suryansh Talukdar, Taha Sarwat, Jin Soung Yoo).