一篇教你如何让AI智能体更聪明的研究:不用改权重,光靠整理技能库就能让DeepSeek-V3和GPT-4.1冲上榜单第一,方法还开源了。
论文提出ASSAY框架,通过随机遮蔽测量技能库中每个技能的因果贡献,发现个体技能对某些任务类型有帮助但对其他任务有害,全局筛选效果欠佳。ASSAY在AppWorld和tau-bench两个基准上,对DeepSeek-V3、GPT-4.1等7个基础模型进行测试。在AppWorld最难分岔上,DeepSeek-V3达到69.3%任务目标完成率,相对提升47.4%,超越所有已发表方法包括权重微调方法。在tau-bench零售环境中,GPT-4.1相对提升8.7%,超越o4-mini、o1和GPT-4.5。消融实验表明主要增益来自推理时按任务遮蔽技能,而非全局移除坏技能。
Not All Skills Help: Measuring and Repairing Agent Knowledge
LLM agents can improve without weight updates by accumulating natural-language skills from experience, but current systems entrust every decision about which skills to keep and how to apply them to LLM judgment alone. We argue that this conflates two distinct roles: generating a skill from experience is a creative act that judgment handles well, while deciding whether that skill actually helps requires empirical evidence across many tasks. Measuring per-skill causal contributions via randomized masking, we find that skill libraries exhibit pervasive causal heterogeneity: individual skills routinely help on some task types while hurting on others, yet their opposing effects cancel in aggregate, making them invisible to global curation methods. We propose ASSAY, a framework that separates generation from curation: it computes a per-skill causal attribution on a small development set, restructures the library offline, and suppresses skills with negative predicted effect for each test task. Across seven base models spanning four providers and two benchmarks (AppWorld and tau-bench), ASSAY consistently improves over prior skill-curation approaches. On AppWorld's hardest split, DeepSeek-V3 achieves 69.3% task-goal completion (47.4% relative improvement), a new state of the art among all published methods including weight-tuned approaches. On tau-bench retail, GPT-4.1 improves by 8.7% relative, advancing past o4-mini, o1, and GPT-4.5 on the public leaderboard without any weight modification. Ablation traces the dominant gain to per-task masking, confirming that the bottleneck is matching skills to tasks at inference time, not removing bad skills globally. Code is available at https://github.com/aiming-lab/assay.