论文精选

BabyCL:通过儿童自我中心视频实现持续多模态学习

Continual Visual and Verbal Learning Through a Child's Egocentric Input

精选理由

BabyCL解决了持续学习场景下多模态对齐的难题,做认知科学或持续学习的研究者可以直接参考其双回放缓冲区设计,值得关注。

AI 摘要

儿童从连续的自我中心经验流中学习词汇,而现有神经网络模型通常对数据进行数百轮随机打乱训练,与真实学习过程不符。研究者提出BabyCL框架,以单次时间顺序处理SAYCam数据集,结合流式视觉表示学习和图像-文本对比目标。BabyCL采用多阶段时间分割和双回放缓冲区,在匹配优化预算下,在SAYCam Labeled-S 4AFC基准上优于流式学习基线,显著缩小了与离线训练的差距。消融实验表明,其增益对时间分割窗口长度和回放缓冲区驱逐规则具有鲁棒性。这项工作表明,在更接近儿童实际体验的训练条件下,有意义的词-指代映射可以涌现。

原文 · arXiv cs.AI

Continual Visual and Verbal Learning Through a Child's Egocentric Input

Children learn the meanings of words from a continuous, temporally structured stream of egocentric experience. Recent work shows that neural networks can also learn word-referent mappings from a child's egocentric video recordings, but they cycle through the shuffled data for hundreds of epochs, contrasting with how children actually encounter their environment. We introduce BabyCL, a continual multimodal learning framework that processes the SAYCam dataset in a single chronological pass, combining streaming visual representation learning with an image-text contrastive objective. BabyCL combines a multi-stage temporal segmentation of the stream with a dual replay buffer that independently manages visual and multimodal histories, and it is jointly trained with three contrastive losses on a shared backbone. Under a matched optimization budget, BabyCL outperforms streaming learning baselines on the SAYCam Labeled-S 4AFC benchmark, substantially narrowing the gap to an upper bound of offline training. Ablations show that the gains are robust to the length of the online temporal segmentation window and the eviction rule of the replay buffer. Together, these results show that meaningful word-referent mappings can emerge under training conditions much closer to a child's actual experience.