论文精选

ASAM:对抗子空间对齐实现鲁棒多模态知识编辑

Beyond Binary Edits Robust Multimodal Knowledge Editing with Adversarial Subspace Alignment

精选理由

做多模态大模型知识更新的研究者会关注——ASAM解决了编辑后泛化差的痛点,用对抗子空间对齐让模型对语义等价变体也生效,值得在MLLM编辑任务上试试。

AI 摘要

多模态大模型需要高效更新知识,但现有方法在语义等价变体上泛化不足。论文提出ASAM框架,包含Latent Adversarial Robustification(LAR)生成对抗变体,以及Rank-Constrained Subspace Learning(RCSL)通过低秩对齐增强编辑鲁棒性。实验表明该方法在保持可靠性和局部性的同时,显著提升了跨视觉和语言变体的泛化能力。这项工作为多模态知识编辑的鲁棒性提供了新思路。

原文 · arXiv cs.AI

Beyond Binary Edits Robust Multimodal Knowledge Editing with Adversarial Subspace Alignment

Multimodal large language models (MLLMs) need efficient mechanisms to update knowledge without degrading existing capabilities. While intrinsic multimodal knowledge editing achieves strong reliability and locality, it often exhibits limited generality, failing to propagate edits across semantically equivalent visual and linguistic variations. This issue arises from the lack of explicit semantic supervision, rigid editing scopes, and biased anchoring to individual samples in high-dimensional multimodal spaces. We address robust intrinsic multimodal knowledge editing by explicitly targeting generalization. We formalize robustness through knowledge units that group semantically equivalent multimodal inputs and define generality as consistent predictions within each unit. To expose fragile semantic regions, we introduce Latent Adversarial Robustification (LAR), which generates adversarial yet semantically coherent variants in the joint latent space. We further propose Rank-Constrained Subspace Learning (RCSL), enforcing low-rank alignment of adversarial representations at the edit layer via a singular value-based objective. Extensive analysis demonstrates the effectiveness of ASAM empirically.

ASAM:对抗子空间对齐实现鲁棒多模态知识编辑 · AI 热点