这项研究为处理大规模数据的神经网络训练提供了并行化新思路,做机器学习和数据挖掘的开发者可以关注其如何平衡精度与效率。
该研究提出两种新方法:多列RBF神经网络结合PSO(MC-PSO)和结合自适应PSO(MC-APSO),旨在解决传统RBF神经网络在大数据集上的可扩展性问题。传统梯度下降和PSO方法在处理大数据时面临核计算过多和隐藏层结构过大的挑战。新方法通过并行部署多个小型RBF网络,每个网络独立训练于数据子集,仅选择与测试实例邻近的网络参与输出,从而提升准确率和速度。实验表明,MC-PSO和MC-APSO在多个基准数据集上优于现有方法,训练和测试时间也更快。
Multi-Column RBF Neural Network Using Adaptive and Non-Adaptive Particle Swarm Optimization
The radial basis function neural network (RBFN) trained with a gradient descending algorithm provides an effective fully connected structure in both shallow and deep networks. The error correction (ErrCor), a state-of-the-art gradient-based training method, selects optimal hidden units to improve accuracy. Alternatively, as a population-based algorithm, the particle swarm optimization algorithm (PSO) uses the swarm experience to optimize RBFN parameters, offering global search and robustness to local minima. Adaptive PSO (APSO) has emerged as an improved variant of PSO. APSO algorithm improves convergence speed by dynamically adjusting swarm parameters during optimization. Both ErrCor and PSO demonstrate improved results and competitive convergence. However, with large datasets, these methods face scalability challenges such as excessive kernel computations and large hidden layer structures. A recent multi-column RBFN approach (MCRN) improves ErrCor performance by deploying small RBFNs in a parallel system. Inspired by MCRN's success, we propose two novel approaches to improve PSO performance: the multi-column RBFN with PSO (MC-PSO) and the multi-column RBFN with APSO (MC-APSO). These methods introduce parallel RBFN structures trained using evolutionary swarm methods. Each RBFN is independently trained on a specific spatial subset of the dataset using either PSO or APSO algorithms. These resulting specialist-trained RBFNs are tailored to their respective subsets. During testing, only selected RBFNs, where the test instance neighbors are located, contribute to the multi-column output. This specialization improves accuracy, while parallelism enhances speed. We evaluate the proposed methods on various benchmark datasets. The MC-PSO and MC-APSO outperform ErrCor, PSO, APSO, and MCRN in terms of accuracy and recall. They also demonstrate faster training and testing times in most experiments.