HiVe-Mem 提出多模态智能体内存框架，token 消耗降低约 90%
HiVe-Mem: Interleaving Horizontal Memory Expansion and Vertical Evidence Composition for Multimodal Agents
一篇多模态智能体记忆机制的论文：HiVe-Mem 用关系图扩展检索范围，再用可训练选择器按查询挑证据，token 省了约 90%，性能还在三个基准上超基线。做长程多模态 Agent 的人值得照着实现试试。
论文提出 HiVe-Mem，一个面向多模态智能体的内存框架，在三个多模态记忆基准上持续提升任务表现，同时 token 消耗比最高效的竞争基线降低约 90%。
其核心机制分为两个维度：横向内存扩展将记忆 episode 组织成带有文本与视觉锚点的关系图，用相关邻居扩展语义检索到的种子记忆，找回互补信息；纵向证据组合则让每条记忆保留紧凑摘要、caption、视觉视图、原始图像和交互历史等多种证据形式，由一个轻量可训练的选择器按查询独立组合证据子集。
这一设计同时拓宽了记忆覆盖面并按需适配单条记忆贡献的证据，针对现有系统依赖压缩记忆的语义检索、错过相关记忆的问题给出改进，对构建长程交互多模态智能体的开发者有直接参考价值。
HiVe-Mem: Interleaving Horizontal Memory Expansion and Vertical Evidence Composition for Multimodal Agents
As multimodal agents accumulate increasingly long interaction histories, effectively retrieving and utilizing past information becomes critical for long-term interaction. Existing memory systems typically rely on semantic retrieval over compressed memories and expose predefined representations, which may miss relevant memories and overlook heterogeneous evidence needs across queries. We introduce \textbf{\hivemem}, a multimodal memory framework that interleaves \textbf{\textsc{H}}or\textbf{\textsc{i}}zontal Memory Expansion with \textbf{\textsc{Ve}}rtical Evidence Composition. For horizontal memory expansion, \hivemem organizes memory episodes into a relational graph using textual and visual anchors, expanding semantically retrieved seeds with related neighbors to recover complementary information. For vertical evidence composition, each retrieved memory retains multiple evidence forms, including compact summaries, captions, grounded visual views, raw images, and interaction history. A lightweight trainable selector independently composes an evidence subset for each retrieved memory, enabling query-adaptive utilization of heterogeneous multimodal information. This two-dimensional design jointly broadens memory coverage and adapts the evidence contributed by individual memories. Experiments across three multimodal memory benchmarks show that \hivemem consistently improves task performance across base models while reducing token consumption by about 90\% compared with the most efficient competing multimodal baselines.
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include HiVe-Mem, 多模态, 智能体, RAG.
Source attribution: curated from arXiv cs.AI (@Haozhen Zhang, Bohan Liu, Jianzhu Bao, Qingyuan Liu, Haodong Yue, Xiao Li, Zi Pong Lim, Wenya Wang).
