Dynamic Neural Graph Encoder 提升 INR 分类准确率约 10%

Dynamic Neural Graph Encoding of Inference Processes in Deep Weight Space

精选理由

想搞权重空间分析的可以看看,这篇用动态图编码推理过程,在 INR 分类上直接提了 10 个点,挺实在的。

AI 摘要

新提出的 DNG-Encoder 用动态图表示神经网络推理的层间时序特征,解决了现有权重空间方法忽略顺序处理的问题。基于该编码器构建的 INR2JLS 框架,在 CIFAR-100-INR 分类任务上达到当前最优,准确率提升约 10%。实验涵盖多个下游任务,展示了权重空间动态建模的有效性。

原文 · arXiv cs.LG

Dynamic Neural Graph Encoding of Inference Processes in Deep Weight Space

The rapid advancements in using neural networks as implicit data representations have attracted significant interest in developing machine learning methods that analyze and process the weight spaces of other neural networks. However, efficiently handling these highdimensional weight spaces remains challenging. Existing methods often overlook the sequential nature of layer-by-layer processing in neural network inference. In this work, we propose a novel approach using dynamic graphs to represent neural network parameters, capturing the temporal dynamics of inference. Our Dynamic Neural Graph Encoder (DNG-Encoder) processes these graphs, preserving the sequential nature of neural processing. Additionally, we also leverage DNG-Encoder to develop INR2JLS (Implicit Neural Representation to Joint Latent Space) for facilitate downstream applications, such as classifying Implicit Neural Representations (INRs). Our approach demonstrates significant improvements across multiple tasks, surpassing the state-of-the-art INR classification accuracy by approximately 10% on the CIFAR-100-INR.