Papers

机制可解释性研究解释 Web 智能体的幻觉失败:线性探针达 0.80 AUC

Can Mechanistic Interpretability Help Explain LLM Agent Failures? A Case Study on Web Element Hallucination

Why we picked it

如果你做过浏览器操作智能体就会懂:它老点不存在的按钮。这篇用 Qwen2.5-7B-Instruct 做了很扎实的机制分析,探针预测 0.80 AUC、steering 纠 85% 幻觉,但格式一变就失效,把可解释性方法的局限讲得很诚实。做智能体评测和可解释性研究的人都该看看。

一篇 arXiv 论文研究了机制可解释性(MI)工具能否解释 LLM 智能体的决策失败,聚焦 Web 智能体幻觉出不存在的网页操作这一具体场景。研究对象是 Qwen2.5-7B-Instruct 在真实网站上的交互过程。

Body

实验结果显示,基于最后一个 prompt token 训练的线性探针可在生成开始前预测幻觉,AUC 达 0.80;沿探针方向进行 steering 可以纠正 85% 的幻觉,但会破坏 7% 的有效操作。不过这只是部分解释:探针方向对 prompt 格式高度敏感,操作表示方式稍作改动后 steering 效果大幅下降。

论文还指出,电路识别存在困难,因为计算分布在多个组件和位置上,智能体依赖的信息也随样例变化。作者讨论了这些观察如何挑战标准 MI 方法在电路发现和特征识别上的底层假设,并提出了让这些方法适配智能体失败分析的未来方向。

Original · arXiv cs.AI

Can Mechanistic Interpretability Help Explain LLM Agent Failures? A Case Study on Web Element Hallucination

Mechanistic Interpretability (MI) has made significant progress in explaining the internal computations of Large Language Models (LLMs); however, these advances have yet to demonstrate comparable success in explaining the decision-making processes of LLM-based agents. This paper examines the challenges MI tools face when applied to realistic agentic failures. We focus on one such failure: web agents hallucinating actions that are not available on a website. We apply a series of mechanistic analyzes to Qwen2.5-7B-Instruct as it interacts with live websites. We find that a linear probe trained on the last prompt token can predict hallucinations before generation begins (0.80 AUC) and steering along a probe direction corrects 85\% of hallucinations, but disrupts 7\% of valid actions. However, these results only provide a partial explanation of why the agent hallucinates. The probe direction is sensitive to prompt formatting, losing much of its steering effect after a minor change to how actions are represented. Additionally, circuit identification can be difficult as the computation is distributed across multiple components and positions, and the information the agent relies on varies across examples. We discuss how these observations challenge the assumptions underlying standard MI methods for circuit discovery and feature identification, and outline future directions for making these methods better suited to explain failures in LLM-based agents.

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include Qwen2.5, mechanistic interpretability, agents, hallucinations.

Source attribution: curated from arXiv cs.AI (@Mohamed Aghzal, Mor Geva, Gregory J. Stein, Ziyu Yao).