StreamMemBench:流式智能体记忆评估基准

StreamMemBench: Streaming Evaluation of Agent Memory for Future-Oriented Assistance

精选理由

测测你的智能体记性

AI 摘要

StreamMemBench是一个针对智能体记忆的流式评估基准,基于EgoLife自我中心流构建两步任务序列。初始任务测试证据使用,后续任务测试反馈与交互经验的复用。基准包含证据回忆、初始证据使用、反馈整合和后续复用四项指标。实验在8个记忆系统、2个基础模型上显示,当前系统在证据使用和反馈转化为可靠行为方面常失败。

原文 · arXiv cs.AI

StreamMemBench: Streaming Evaluation of Agent Memory for Future-Oriented Assistance

A central role of personal-agent memory is to turn stored information and prior interactions into future-oriented assistance. In daily use, useful cues come from what the agent observes and how the user interacts with the agent, and the agent must carry them forward from the current request to similar future tasks. Existing memory benchmarks usually test dialogue recall or task improvement in isolation, leaving the trajectory from streaming observations to later assistance largely untested. We introduce StreamMemBench, a streaming benchmark that constructs a two-step task sequence around each evidence anchor from EgoLife egocentric streams. The initial task tests evidence use, while the follow-up task tests whether feedback and interaction experience are reused. Four metrics diagnose evidence recall, initial evidence use, feedback incorporation, and follow-up reuse. Experiments with eight memory systems across two backbones show that current systems often fail to use observed evidence or turn feedback into reliable follow-up behavior, even when evidence is stored or feedback is incorporated locally. StreamMemBench is publicly available at https://github.com/landian60/StreamMemBench.