Random pruning: channel sparsity by expectation scaling factor

Chuanmeng Sun; Jiaxin Chen; Yong Li; Wenbo Wang; Tiehua Ma

doi:10.7717/peerj-cs.1564

Random pruning: channel sparsity by expectation scaling factor

PeerJ Comput Sci. 2023 Sep 5:9:e1564. doi: 10.7717/peerj-cs.1564. eCollection 2023.

Authors

Chuanmeng Sun^{1

2}, Jiaxin Chen^{1

2}, Yong Li³, Wenbo Wang^{1

2}, Tiehua Ma^{1

2}

Affiliations

¹ North University of China, State Key Laboratory of Dynamic Measurement Technology, Taiyuan, Shanxi, China.
² North University of China, School of Electrical and Control Engineering, Taiyuan, Shanxi, China.
³ Chongqing University, State Key Laboratory of Coal Mine Disaster Dynamics and Control, Chongqing, China.

Abstract

Pruning is an efficient method for deep neural network model compression and acceleration. However, existing pruning strategies, both at the filter level and at the channel level, often introduce a large amount of computation and adopt complex methods for finding sub-networks. It is found that there is a linear relationship between the sum of matrix elements of the channels in convolutional neural networks (CNNs) and the expectation scaling ratio of the image pixel distribution, which is reflects the relationship between the expectation change of the pixel distribution between the feature mapping and the input data. This implies that channels with similar expectation scaling factors ( $δ_{E}$ ) cause similar expectation changes to the input data, thus producing redundant feature mappings. Thus, this article proposes a new structured pruning method called EXP. In the proposed method, the channels with similar $δ_{E}$ are randomly removed in each convolutional layer, and thus the whole network achieves random sparsity to obtain non-redundant and non-unique sub-networks. Experiments on pruning various networks show that EXP can achieve a significant reduction of FLOPs. For example, on the CIFAR-10 dataset, EXP reduces the FLOPs of the ResNet-56 model by 71.9% with a 0.23% loss in Top-1 accuracy. On ILSVRC-2012, it reduces the FLOPs of the ResNet-50 model by 60.0% with a 1.13% loss of Top-1 accuracy. Our code is available at: https://github.com/EXP-Pruning/EXP_Pruning and DOI: 10.5281/zenodo.8141065.

Keywords: Channel pruning; Image classification; Model compression; Random sparse.

Grants and funding

This work was supported by the National Key Research and Development Program of China (2022YFC2905700), the National Key Research and Development Program of China (2022YFB3205800), and the Fundamental Research Programs of Shanxi Province (202103021224199, 202203021221106). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.