论文精选

FAME:面向消息级日志异常检测的故障感知混合专家模型

FAME: Failure-Aware Mixture-of-Experts for Message-Level Log Anomaly Detection

精选理由

运维团队终于有了兼顾精度和效率的日志异常检测方案——FAME 用一次 LLM 离线标注就解决了消息级检测的痛点,标注量减少 76 倍还能发现未知故障,做日志分析或运维自动化的开发者值得关注。

AI 摘要

FAME 提出了一种标签高效的混合专家框架,用于消息级日志异常检测。传统方法在会话或窗口级别检测异常,粒度粗糙,迫使运维人员检查大量常规日志行。FAME 通过离线使用一次大语言模型,为每个模板标注最多 K 行,生成正常/异常指示符和代表性示例,并自动将模板划分为故障域。训练后的轻量路由器和领域专家可在本地运行,输出异常预测和故障域标签。在 BGL 数据集上,FAME 在 K=100 时达到 F1=98.16,标注量减少 76 倍,并检测出 86.3% 的未见 EventID 异常;在 Thunderbird 上达到 F1=99.95 且召回率完美。

原文 · arXiv cs.LG

FAME: Failure-Aware Mixture-of-Experts for Message-Level Log Anomaly Detection

Production systems generate millions of log lines daily, yet most anomaly detectors operate at the session or window-level, flagging groups of lines rather than identifying the specific message responsible. This coarse granularity forces operators to inspect many routine lines per alert. Message-level detection offers finer granularity, but remains challenging. A single event template may correspond to both normal and anomalous messages, failures arise from heterogeneous subsystems, and line-level labeling at scale is impractical. Although large language models (LLMs) can reason over log semantics, applying them to every line is too costly for continuous monitoring. We present FAME (Failure-Aware Mixture-of-Experts), a label-efficient message-level mixture-of-experts framework that uses an LLM only once offline. We annotate at most K labeled lines per template to derive binary normal/anomaly indicators and representative examples. The LLM proposes a partition of templates into failure domains, and a certification step validates the proposal before training. FAME trains a lightweight router and domain experts that run on-premise and output anomaly predictions and failure-domain labels. On BGL, FAME achieves F1 = 98.16 at K = 100 reducing annotation effort by 76x and detects 86.3% of anomalies from unseen EventIDs. On Thunderbird, FAME reaches F1 = 99.95 with perfect recall.