研究人员发现日志异常检测器过度自信问题，提出LoRD框架能有效校准模型，提高系统可靠性。
研究显示，基于语言模型的日志异常检测器在严重类别不平衡情况下对错误预测过度自信。作者提出LoRD校准框架，通过从正确分类的验证样本中学习预测路径特定可靠性模型。在四个大规模日志基准数据集上的实验表明，LoRD显著提高了置信度可靠性，减少了过度自信的异常相关错误，同时保持了异常检测性能。
Too Sure to Be Safe: Model Calibration for Reliable Log Anomaly Detection
Online log anomaly detection is critical for maintaining the reliability of large-scale computing systems. Although recent language model-based log anomaly detectors achieve strong detection performance, their confidence estimates remain poorly calibrated. We show that these detectors frequently assign excessive confidence to incorrect predictions, particularly for anomalous logs under severe class imbalance. Moreover, confidence on erroneous predictions remains persistently high even when conventional calibration metrics indicate good calibration, creating a critical reliability gap for operational monitoring systems. To address this issue, we propose Log Reconstruction and Distance (LoRD), a lightweight post-hoc calibration framework for reliable log anomaly detection. LoRD learns prediction-route-specific reliability models from latent representations of correctly classified validation samples and estimates prediction reliability through route-wise reconstruction distances. Based on the estimated reliability, LoRD selectively recalibrates high-risk predictions to suppress overconfident errors while preserving reliable predictions. Extensive experiments on four large-scale log benchmark datasets and multiple language model-based detectors demonstrate that LoRD consistently improves confidence reliability and substantially reduces overconfident anomaly-related errors without sacrificing anomaly detection performance.