A Synaptic Pruning-Based Spiking Neural Network for Hand-Written Digits Classification

Faramarz Faghihi; Hany Alashwal; Ahmed A Moustafa

doi:10.3389/frai.2022.680165

A Synaptic Pruning-Based Spiking Neural Network for Hand-Written Digits Classification

Front Artif Intell. 2022 Feb 24:5:680165. doi: 10.3389/frai.2022.680165. eCollection 2022.

Authors

Faramarz Faghihi¹, Hany Alashwal², Ahmed A Moustafa^{3

4}

Affiliations

¹ Machine Listening Lab, University of Bremen, Bremen, Germany.
² College of Information Technology, United Arab Emirates University, Al Ain, United Arab Emirates.
³ School of Psychology, Faculty of Society and Design, Bond University, Gold Coast, QLD, Australia.
⁴ Department of Human Anatomy and Physiology, The Faculty of Health Sciences, University of Johannesburg, Johannesburg, South Africa.

Abstract

A spiking neural network model inspired by synaptic pruning is developed and trained to extract features of hand-written digits. The network is composed of three spiking neural layers and one output neuron whose firing rate is used for classification. The model detects and collects the geometric features of the images from the Modified National Institute of Standards and Technology database (MNIST). In this work, a novel learning rule is developed to train the network to detect features of different digit classes. For this purpose, randomly initialized synaptic weights between the first and second layers are updated using average firing rates of pre- and postsynaptic neurons. Then, using a neuroscience-inspired mechanism named, "synaptic pruning" and its predefined threshold values, some of the synapses are deleted. Hence, these sparse matrices named, "information channels" are constructed so that they show highly specific patterns for each digit class as connection matrices between the first and second layers. The "information channels" are used in the test phase to assign a digit class to each test image. In addition, the role of feed-back inhibition as well as the connectivity rates of the second and third neural layers are studied. Similar to the abilities of the humans to learn from small training trials, the developed spiking neural network needs a very small dataset for training, compared to the conventional deep learning methods that have shown a very good performance on the MNIST dataset. This work introduces a new class of brain-inspired spiking neural networks to extract the features of complex data images.

Keywords: MNIST database; back-propagation; deep spiking neural network; feature detection; synaptic pruning.