Pruning and quantization algorithm with applications in memristor-based convolutional neural network

Mei Guo; Yurui Sun; Yongliang Zhu; Mingqiao Han; Gang Dou; Shiping Wen

doi:10.1007/s11571-022-09927-7

Pruning and quantization algorithm with applications in memristor-based convolutional neural network

Cogn Neurodyn. 2024 Feb;18(1):233-245. doi: 10.1007/s11571-022-09927-7. Epub 2023 Jan 19.

Authors

Mei Guo¹, Yurui Sun¹, Yongliang Zhu¹, Mingqiao Han², Gang Dou¹, Shiping Wen³

Affiliations

¹ College of Electrical Engineering and Automation, Shandong University of Science and Technology, Qingdao, 266590 China.
² The University of Nottingham Ningbo China, Ningbo, 315100 China.
³ The Australian AI Institute, Faculty of Engineering and Information Technology, University of Technology Sydney, Sydney, NSW 2007 Australia.

PMID: 38406206
PMCID: PMC10881922 (available on 2025-02-01)
DOI: 10.1007/s11571-022-09927-7

Abstract

The human brain's ultra-low power consumption and highly parallel computational capabilities can be accomplished by memristor-based convolutional neural networks. However, with the rapid development of memristor-based convolutional neural networks in various fields, more complex applications and heavier computations lead to the need for a large number of memristors, which makes power consumption increase significantly and the network model larger. To mitigate this problem, this paper proposes an SBT-memristor-based convolutional neural network architecture and a hybrid optimization method combining pruning and quantization. Firstly, SBT-memristor-based convolutional neural network is constructed by using the good thresholding property of the SBT memristor. The memristive in-memory computing unit, activation unit and max-pooling unit are designed. Then, the hybrid optimization method combining pruning and quantization is used to improve the SBT-memristor-based convolutional neural network architecture. This hybrid method can simplify the memristor-based neural network and represent the weights at the memristive synapses better. Finally, the results show that the SBT-memristor-based convolutional neural network reduces a large number of memristors, decreases the power consumption and compresses the network model at the expense of a little precision loss. The SBT-memristor-based convolutional neural network obtains faster recognition speed and lower power consumption in MNIST recognition. It provides new insights for the complex application of convolutional neural networks.

Keywords: Convolutional neural network; Memristor; Network pruning; Quantization weight.

© The Author(s), under exclusive licence to Springer Nature B.V. 2023. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.