论文精选

Self-Reflective APIs:结构化建议让AI Agent恢复率提升40个百分点

Self-Reflective APIs: Structure Beats Verbosity for AI Agent Recovery

精选理由

做Agent系统或API设计的开发者,这个方案直接解决了Agent调用API时频繁失败、需要人工介入的痛点——用结构化建议代替自然语言错误,让Agent自己就能修复重试,实测效果显著,建议直接参考其设计思路。

AI 摘要

当AI Agent调用API遇到验证错误时,传统做法返回自然语言错误描述,但Agent往往无法有效修复。该论文提出Self-Reflective API,在验证失败时返回机器可读的结构化恢复建议(recovery_feedback.suggestions[]),使Agent能直接修复请求并重试,无需外部推理。在30个样本、3个LLM、10个对抗任务的实验中,结构化建议在Anthropic模型上将任务完成率提升36.7-40个百分点,且每个成功token效率提升1.8-2.2倍。在gpt-4o-mini上效果不显著,但计费API的二次验证确认了模式。研究还发现了LLM基准测试中两类未记录的答案泄露问题,并开源了审计工具。

原文 · arXiv: Anthropic

Self-Reflective APIs: Structure Beats Verbosity for AI Agent Recovery

When an AI agent calls an API and hits a validation error, it needs more than what went wrong -- it needs what to do next. A self-reflective API returns, on validation failure, a machine-readable recovery\_feedback.suggestions[] payload sufficient for the agent to repair the request and retry without external reasoning. On a leak-audited pilot ($N{=}30$ per cell, 3 LLMs, 10 adversarial tasks), structured suggestions lift task-completion rate by $+36.7$--$40.0$pp over plain-English diagnoses on Anthropic models (Fisher's exact $p \le 0.0022$), at $1.8$--$2.2\times$ better per-success token efficiency. The lift is not significant on gpt-4o-mini ($p{=}0.435$); a second-domain replication on a billing API confirms the pattern. The comparison only holds after auditing two undocumented classes of answer leakage in LLM benchmarks. We shipaudit\_prompt\_leakage.py as reusable CI infrastructure. Code and data: https://github.com/arquicanedo/self-reflective-apis.