看智能体记忆到底行不行
AGENTCL 提出一个评估语言智能体持续学习能力的新基准，通过构建任务流来测试智能体是否真正从经验中学习。该基准对比了“组合式”任务流（后续任务可复用前序任务的代码函数、研究证据或工作流）与“朴素”任务流（任务同领域但无明确复用关系）。研究发现，当前记忆方法在任务连接明显时能复用过往经验，但在任务差异较大时仍难以避免混淆。AGENTCL 旨在解决现有基准中任务关系不清晰导致的评估偏差问题。
Most AI agents do not forget because they lack mem…
Most AI agents do not forget because they lack memory; they fail because they remember badly.
AGENTCL asks a simple question: does an AI agent really learn from experience, or merely carry clutter forward?
Today's agents can spend enormous effort solving one task, then enter the next one almost as if nothing happened.
AGENTCL says AI agents need better tests for whether their memory actually helps them learn across tasks.
The paper’s main idea is to build task streams where earlier tasks clearly contain pieces that later tasks can reuse, such as a small coding function, evidence for a research question, or a useful workflow.
It compares these careful “compositional” streams with normal “naive” streams, where tasks come from the same area but do not have a guaranteed reuse link.
Agent memory is easy to overrate when the benchmark is messy.
If tasks are not carefully connected, a memory system may look good for the wrong reason, or bad for a reason the test cannot explain.
AGENTCL tries to fix that by making the task relationships clear, then measuring whether memory helps on later tasks, stays useful, and transfers to unseen tasks.
The key finding is that today’s memory methods can reuse past work when the connection is obvious, but they still struggle to avoid confusion when the next task is different.
----
Link – arxiv. org/abs/2606.02461
Title: "AGENTCL: Toward Rigorous Evaluation of Continual Learning in Language Agents"