LION:一种用于多模态属性图学习的Clifford神经网络范式

LION: A Clifford Neural Paradigm for Multimodal-Attributed Graph Learning

精选理由

LION基于Clifford代数,在多模态属性图学习上取得了显著成果,值得一看。

AI 摘要

近期,多模态领域快速发展推动图机器学习进入数据中心化范式,从文本属性图转向多模态属性图。LION基于Clifford代数和图神经网络范式,提出了一种新的多模态属性图学习方法。实验表明,LION在9个文本-图像MAG数据集上显著优于现有SOTA基准方法。

原文 · arXiv cs.LG

LION: A Clifford Neural Paradigm for Multimodal-Attributed Graph Learning

Recently, the rapid advancement of multimodal domains has driven a data-centric paradigm shift in graph ML, transitioning from text-attributed to multimodal-attributed graphs. This advancement significantly enhances data representation and expands the scope of graph downstream tasks, such as modality-oriented tasks, thereby improving the practical utility of graph ML. Despite its promise, limitations exist in the current neural paradigms:(1) Neglect Context in Modality Alignment: Most existing methods adopt topology-constrained or modality-specific operators as tokenizers.These aligners inevitably neglect graph context and inhibit modality interaction, resulting in suboptimal alignment.(2) Lack of Adaptation in Modality Fusion: Most existing methods are simple adaptations for 2-modality graphs and fail to adequately exploit aligned tokens equipped with topology priors during fusion, leading to poor generalizability and performance degradation.To address the above issues, we propose LION (c\underline{LI}ff\underline{O}rd \underline{N}eural paradigm) based on the Clifford algebra and decoupled graph neural paradigm (i.e., propagation-then-aggregation) to implement alignment-then-fusion in multimodal-attributed graphs. Specifically, we first construct a modality-aware geometric manifold grounded in Clifford algebra.This geometric-induced high-order graph propagation efficiently achieves modality interaction, facilitating modality alignment.Then, based on the topology-aware Clifford components of aligned tokens, we propose adaptive holographic aggregation. This module integrates component-wise energy and propagation-scale information with learnable parameters to improve modality fusion. Extensive experiments on 9 text-image MAG datasets demonstrate that LION significantly outperforms SOTA baselines across 3 graph and 3 modality downstream tasks.