超越检索:多模态智能体的分析记忆框架AdaMM

Beyond Retrieval: Analytic Memory for Multimodal Agents

精选理由

AdaMM让智能体能对历史多模态观测做筛选、聚合和排序,不止检索。MemEye、MemGallery上比现有高11.3%和7.3%,适合多模态记忆研究者。

AI 摘要

AdaMM是一个面向多模态智能体的分析记忆框架。AdaMM从对话、图像和元数据中提取带溯源信息的属性-值观察,并发现重复字段结构。在MemEye等基准中,现有系统侧重检索记忆,而AdaMM同时支持检索与分析。推理时,AdaMM的记忆感知规划器将查询分解为检索和分析操作。实验显示,在MemEye和MemGallery上AdaMM分别比基线提升11.3%和7.3%。

原文 · arXiv cs.AI

Beyond Retrieval: Analytic Memory for Multimodal Agents

Long-term multimodal memory must support not only retrieving relevant information but also computing over observations accumulated across interactions. Existing systems largely emphasize \emph{retrieval memory}, organizing interaction histories through summaries and indexes to return query-relevant information at multiple granularities, from high-level abstractions to underlying records. In this paper, we formulate \emph{analytic memory} as a complementary abstraction that organizes recurring multimodal observations into queryable structures supporting filtering, aggregation, ranking, and temporal comparison. We present AdaMM, a framework that jointly supports retrieval and analytic memory. Rather than relying on application-defined schemas, AdaMM extracts provenance-linked attribute-value observations from dialogue, images, and contextual metadata, discovers recurring field structures, and materializes them for analytical access. At inference time, a memory-aware planner decomposes queries into retrieval and analytic operations and routes each operation to the appropriate tools. Experiments on two long-term multimodal memory benchmarks, MemEye and MemGallery, show that AdaMM improves performance by up to 11.3\% and 7.3\%, respectively.