这篇论文揭露了一个容易被忽视的坑:智能体工具用对了,但可能找错对象。测试中有24-26%的出错率,很值得关注。
工具增强语言模型智能体在选取正确工具后仍可能对错误的外部实体执行操作。例如,请求“给Alex发邮件关于发布事宜”可能导致联系错误的Alex或附加错误文档。该研究区分了工具正确性与实体正确性,提出了企业工作流中错误实体失败的分类法。在60个任务、5个模型后端和6种工具方法的评估中,所有方法实现0.0%工具错误,但动作基线仍有24.0-26.0%的运行出现错误实体动作。实体感知方法消除了错误实体动作,但会因模糊延迟降低直接任务完成率。
Entity Binding Failures in Tool-Augmented Agents
Tool-augmented language-model agents are often evaluated by whether they select the correct tool, produce valid API arguments, and complete the requested task. However, an agent may choose the right tool and still act on the wrong external entity. For example, a request to "email Alex about the launch" may lead the agent to contact the wrong Alex, attach the wrong launch document, reply in the wrong thread, or update the wrong customer account. We call these errors entity binding failures. This paper studies entity binding failures as a distinct reliability and safety problem in tool-augmented agents. We formalize the separation between tool correctness and entity correctness, introduce a taxonomy of wrong-entity failures in enterprise workflows, and evaluate entity-aware execution mechanisms including entity-resolution preconditions, confidence-gated binding, clarification under ambiguity, and provenance tracking. In a controlled diagnostic evaluation across 60 tasks, five model backends, and six tool-use methods, all methods achieved 0.0 percent wrong-tool error, yet action-oriented baselines still produced wrong-entity actions in 24.0-26.0 percent of runs. Entity-aware methods eliminated wrong-entity actions and risk-weighted wrong-entity exposure in this setting, but reduced direct task completion by deferring under ambiguity. These findings show that safe tool use requires not only selecting the correct tool, but also reliably binding natural-language references to the correct real-world entity before action.