Reusable Neural Bases for Network Compression
A Mathematical Theory of Reusable Neural Bases for Network Compression
LRNBA enables wider and deeper networks under the same parameter budget, addressing memory costs in AI applications.
The Linear Reusable Neural Bases Architecture (LRNBA) introduces a novel framework for improving parameter efficiency in large AI models. The approach represents each network block as a linear combination of shared neural bases, inspired by RNN designs. Experiments show the model achieves comparable or faster convergence and lower loss than classical architectures while maintaining stable training dynamics.
A Mathematical Theory of Reusable Neural Bases for Network Compression
As large AI models become increasingly prevalent across a wide range of applications, memory cost has become a critical bottleneck in both training and inference. To mitigate this issue, we introduce the Linear Reusable Neural Bases Architecture (LRNBA), a novel framework aimed at improving parameter efficiency and reducing memory cost. Inspired by recurrent neural network (RNN) designs, the core idea of our approach is to represent each network block as a linear combination of a shared set of neural bases, thereby enjoying highly network compression rate while maintaining stable training. The proposed architecture allows for the construction of significantly wider and deeper networks under the same parameter budget. Extensive experiments demonstrate that our model achieves comparable or even faster convergence and lower loss than classical architectures, while maintaining stable training dynamics.
