WikiSkill：Agent经验持久化知识沉淀

WikiSkill：把 Agent 的经验沉淀为持久知识，让 Skills 持续进化 来自 Google Research 的论文，在 "Agent Skills 自动进化" 这一方向上，论文提出在...

WikiSkill：把 Agent 的经验沉淀为持久知识，让 Skills 持续进化 来自 Google Research 的论文，在 "Agent Skills 自动进化" 这一方向上，论文提出在原始经验与可执行 Skills 之间插入一个持久化知识层（Wiki），让经验被持续编译、沉淀为结构化知识，从而支撑 Skills 的长期、复利式进化——实验表明这一改动带来了一致且显著的性能提升。 arxiv.org/pdf/2608.27454 # 论文要解决的问题 "Agent Skills" 是把领域 know-how 打包成文件系统模块（SKILL.md + 脚本）供 Agent 复用的机制，近期工作（EvoSkill、Trace2Skill、SkillOpt）已能自动从执行轨迹中进化 Skills。但这些方法有一个共同的结构性缺陷： 经验是"用过即散"的——每轮迭代中从轨迹里学到的洞察，散落在提案历史、反馈记录等临时产物中，没有被维护成一个独立、持续演化的知识表示。后续迭代的 Skills 更新无法系统性地建立在"至今为止学到的一切"之上。 作者受 Karpathy 的 "LLM Wiki" 观点启发，提出核心问题：能否把 Agent 经验编译成持久、可复利的知识，来支撑 Skills 的长期进化？ # 方法：三层架构 + 进化闭环 三层知识架构 · Raw Layer (raw/): 完整执行轨迹（推理、工具调用、反馈） · Wiki Layer (wiki/): 模式页（失败根因/成功策略）、演化日志、Skills 影响追踪表 · Skill Layer (skills/): 可执行 Skills（SKILL.md + 溯源用的 PURPOSE.md） 关键设计是 Wiki 层中的 skill-impact.md：它程序化地记录每次 Skills 提案的 diff、验证分数、接受/拒绝结果，形成一条客观的"审计轨迹"，使后续提案不会重复已被否决的修改 每轮迭代的四个角色 1. 推理 Agent：用当前 Skills 在训练集上跑 rollout（刻意禁止访问 Wiki）； 2. Wiki 维护者：对采样轨迹做根因分析，把成败模式以增量补丁方式写入 Wiki； 3. Skills 提案者：以 ReAct 方式自主探索 Wiki 索引、影响追踪表和原始轨迹，每次只提出一个原子化的 Skills 创建/修改； 4. 门控与回滚：候选 Skills 在验证集上评测，优于历史最佳才接受，否则回滚 Skills ——但 Wiki 始终保留。 # 实验与核心发现 在 5 个基准（数学推理 LiveMath、网络搜索 SealQA、电子表格 SpreadsheetBench、长文档问答 OfficeQA、具身交互 ALFWorld）× 5 个模型（Qwen 4B/9B/27B、Gemma-4-31B、Gemini-3.5-Flash）上评估，全部跑 3 次独立实验并做 bootstrap 显著性检验。 发现一：一致优于现有方法。 WikiSkill 在所有模型上取得最高平均分，比最强基线高 3.3–12.0 分；而 EvoSkill、SkillOpt 等基线在部分设置下反而损害性能（如 EvoSkill 让 Gemma 在 LiveMath 上 33.9%→29.8%）。WikiSkill 的优势是"更强且更稳"。 发现二：Skills 进化与模型规模互补。 Qwen 家族中，WikiSkill 的平均提升随规模增大：4B +12.3 → 9B +17.5 → 27B +23.9。同时，9B + Skills（47.4%）可以超过无 Skills 的 27B（39.4%）——Skills 能在一定程度上"替代"模型规模。 发现三：Skills 可跨模型迁移，甚至优于自进化 Skills。 例如 Qwen-9B 用 27B 进化的 Skills 在 ALFWorld 上达 70.2%，高于用自己的 Skills（63.4%）；小模型进化的 Skills 也能帮助大模型。这揭示了一个重要区分："发现有用程序知识"与"执行这些知识"是两种独立的能力，自进化框架通常把二者混为一谈。但迁移也有陷阱：小模型 Skills 中的"低级变通"（如单行 Python 命令）会束缚强模型，造成负迁移。 发现四（消融）：持久 Wiki 是关键。 给提案者访问 Wiki 使平均分从 48.7% 升至 63.7%（+15）；反之，若让推理智能体在训练时也能看 Wiki，反而降到 60.9%——因为轨迹的知识来源被"污染"，降低了其对 Skills 开发的诊断价值。 Akshay 🚀 @akshay_pachaar Google just dropped a banger paper. If you write or curate skills for your agents, this one is for you. (bookmark it) Agent skills are just folders holding the instructions an agent follows for a task, and people increasingly let agents write those instructions themselves. The loop is simple. Run the agent on some tasks, read the runs that failed, rewrite the instructions, and keep the rewrite only if the score goes up on a held-out validation set. That last check matters, because an edit that quietly makes the agent worse would pile up over time. So the system always applies the edit, tests it, and reverts the file if the score drops. Here is where it breaks. Working out why the agent failed is the expensive part, since it means reading full traces and comparing failed runs against successful ones. But that diagnosis never gets written down. The only thing saved is the new instruction text it produced. So when that instruction reverts, the finding behind it goes too. The next round reads the same failures, reaches the same conclusion, and often proposes the fix that already lost, with no memory that it was ever tried. WikiSkill fixes this by splitting the one folder into three: 1. Raw traces, written once and never touched. 2. A wiki of accumulated knowledge, holding what keeps breaking, what has worked, and every edit tried so far with the reason it passed or failed. Nothing here is ever deleted. 3. The skills themselves, reverted whenever an edit makes things worse. So the instructions can roll back, but the understanding behind them never does. Their own example makes it click. An early skill gets rejected for being too vague, and instead of vanishing, that rejection is logged. The next round reads the log, sees the vague version failed, and writes a concrete rule in its place. That one gets accepted. Same failure, same analysis, but the second attempt starts from knowing what already did not work. The authors confirm this is where most of the gain comes from by turning the knowledge layer off. The average score drops by about a quarter. Two other results are worth the read on their own. Smaller models with evolved skills beat much larger models running without any, and skills evolved by one model transfer across families, sometimes working better than the ones a model wrote for itself. Paper → arxiv.org/abs/2608.27454 I wrote about a related idea (GEPA) earlier, which improves a model by having it reflect on its own mistakes in plain language and rewrite its prompt, with no weight updates involved. The detailed article is quoted below. 🔗 View Quoted Tweet 💬 0 🔄 0 ❤️ 0 👀 33 ⚡