做模型可解释性研究的团队终于有了可靠的置信度评估方法——bootstrap模式频率比传统log-prob校准误差低近5倍，建议做LLM内部机制分析的开发者直接参考论文代码。
该研究针对激活预言机（activation oracles）的自然语言输出，探索了6种不确定性量化方法。实验基于6000个样本，发现bootstrap模式频率在Qwen3-8B和Qwen3.6-27B上校准误差最低（ECE 5.7% vs 25.5%），而log-prob基线可作为低成本快速筛选信号。这项工作填补了激活预言机置信度评估的空白，对依赖模型内部解释的AI安全研究有直接意义。
Confidence and Calibration of Activation Oracles for Reliable Interpretation of Language Model Internals
Activation oracles aim to make the activations of other models legible to humans and yield promising results compared to white-box interpretability techniques. However, uncertainty quantification (UQ) for the natural-language outputs of such activation oracles is so far understudied. Here, we investigate 6 different methods for estimating the confidence of activation oracles and evaluate how well-calibrated their confidence scores are. Our experiments on 6,000 samples per oracle (varying verbalizer and context prompts) reveal that bootstrap mode frequency is the best-calibrated method among those tested (ECE 5.7% vs. 25.5% for the answer-word log-probability on Qwen3-8B; 10.3% vs. 13.1% on Qwen3.6-27B), and that the log-prob baseline can serve as a fast triage signal at a fraction of the cost. Code and the patched trainer are available at https://github.com/federicotorrielli/probabilistic_activation_oracles.