这篇论文用大量实验告诉你,加技能反而可能让Agent变笨。三种坑要记住:技能描述渗透、基础位移、验证位移。别再盲目加技能了
该论文通过近6000次配对运行,在两个办公自动化基准和三个模型栈上对比有技能和无技能的Agent。研究发现添加程序化技能会导致任务回归率显著上升,最佳技能主要通过减少回归来区分自身。论文识别了三种失败机制:技能描述渗透、基础位移和验证位移。现有技能几乎全是程序性的,程序化引导反而较少导致失败。
Are agent skills always worth using? The answer is no? This paper provides some important insights...
Are agent skills always worth using? The answer is no? This paper provides some important insights to understand this more. (bookmark it) Paper summary: Adding procedural skills to an agent is usually scored by average task success. That number nets gains against damage and hides half of what happened. Setup: nearly 6,000 paired runs across two office automation benchmarks and three model harness stacks, comparing the same agent with and without skills. A regression is a task the agent solved without skills and then failed once skills were added. Regressions are large enough that the best performing skills separate themselves mainly through fewer regressions. Larger gains contribute much less. Three mechanisms drive it. Skill description osmosis, where a skill changes agent behavior just by sitting in context even when it is never invoked. Grounding displacement, where a prescribed procedure overrides how the agent reads its inputs. Verification displacement, where the procedure suppresses checks the agent would otherwise run on its own output. Trace analysis surfaces that procedural guidance is the stage least often responsible for failure, while grounding and verification dominate the errors that remain. Existing skills are almost entirely procedure. Paper: arxiv.org/abs/2607.22520 Learn to build effective AI agents in our academy: academy.dair.ai 💬 3 🔄 4 ❤️ 12 👀 2237 📊 8 ⚡