DialogMem 基准：评测对话与视觉结合的具身记忆
DialogMem: Evaluating Dialogue-Grounded Embodied Memory in 3D Environments
这个新基准挺有意思，142 个 episode 测试智能体怎么把人说的话和看到的东西绑定起来，实验发现对话对问答有用、对导航基本没用，问题出在物体级关联上。
DialogMem 是一个两阶段基准，智能体从时间对齐的 RGB-D 观测流与对话流中构建记忆，再用于实例级物体导航和问答任务。
基准包含 142 个 episode，覆盖 27 个真实世界 3D 场景，提供 14,193 个 QA 对和 11,890 条导航指令。研究者在统一的记忆形成、证据检索与推理框架下评测了 9 种记忆表示。
实验显示，在评估时将对话追加到视觉记忆能显著提升 QA 成绩，但对导航提升微乎其微，说明相关对话虽在记忆中，却未与具体物体建立关联。
DialogMem: Evaluating Dialogue-Grounded Embodied Memory in 3D Environments
Embodied agents in human-centered environments require memory that integrates what they observe with what humans tell them over time. However, existing embodied memories are built primarily from visual experience, leaving open how dialogue with humans can be grounded in the objects encountered throughout that experience. We introduce DialogMem, a two-phase benchmark in which agents build memory from time-aligned streams of posed RGB-D observations and dialogue, and later use it for instance-level object navigation and question answering (QA). DialogMem provides 142 episodes across 27 real-world 3D scenes, with 14,193 QA pairs and 11,890 navigation instructions. We evaluate nine memory representations within a unified framework of memory formation, evidence retrieval, and reasoning for task outputs. In experiments, appending dialogue to visual memory at evaluation time substantially improves QA but yields negligible navigation gains, suggesting that relevant dialogue is present in memory but not grounded to specific objects. Controlled diagnostics show that navigation improves with link quality but falls short of ground-truth object-level memory even under perfect linking. Retrieving the correct object entry and recovering its spatial location remain further bottlenecks. These results identify three bottlenecks for dialogue-grounded embodied memory: grounding and organizing dialogue at object-level entries during formation, retrieving the correct entry, and localizing it accurately.
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include DialogMem, 具身智能, 基准评测, 物体导航.
Source attribution: curated from arXiv cs.AI (@SeongJun Jeong, Junghyun Kim, Gi-Cheon Kang, Byoung-Tak Zhang).