A regularized stochastic configuration network based on weighted mean of vectors for regression

Yang Wang; Tao Zhou; Guanci Yang; Chenglong Zhang; Shaobo Li

doi:10.7717/peerj-cs.1382

A regularized stochastic configuration network based on weighted mean of vectors for regression

PeerJ Comput Sci. 2023 May 30:9:e1382. doi: 10.7717/peerj-cs.1382. eCollection 2023.

Authors

Yang Wang¹, Tao Zhou¹, Guanci Yang², Chenglong Zhang³, Shaobo Li¹

Affiliations

¹ State Key Laboratory of Public Big Data, Guizhou University, Guiyang, Guizhou, China.
² Key Laboratory of Advanced Manufacturing Technology of the Ministry of Education, Guizhou University, Guiyang, Guizhou, China.
³ School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, Jiangsu, China.

Abstract

The stochastic configuration network (SCN) randomly configures the input weights and biases of hidden layers under a set of inequality constraints to guarantee its universal approximation property. The SCN has demonstrated great potential for fast and efficient data modeling. However, the prediction accuracy and convergence rate of SCN are frequently impacted by the parameter settings of the model. The weighted mean of vectors (INFO) is an innovative swarm intelligence optimization algorithm, with an optimization procedure consisting of three phases: updating rule, vector combining, and a local search. This article aimed at establishing a new regularized SCN based on the weighted mean of vectors (RSCN-INFO) to optimize its parameter selection and network structure. The regularization term that combines the ridge method with the residual error feedback was introduced into the objective function in order to dynamically adjust the training parameters. Meanwhile, INFO was employed to automatically explore an appropriate four-dimensional parameter vector for RSCN. The selected parameters may lead to a compact network architecture with a faster reduction of the network residual error. Simulation results over some benchmark datasets demonstrated that the proposed RSCN-INFO showed superior performance with respect to parameter setting, fast convergence, and network compactness compared with other contrast algorithms.

Keywords: Residual error feedback; Stochastic configuration networks; Swarm intelligence optimization; Weighted mean of vectors.

Grants and funding

This work is supported by the National Natural Science Foundations of China (No. 62163007 and No. 62166005). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.