q-Neurons: Neuron Activations Based on Stochastic Jackson's Derivative Operators

IEEE Trans Neural Netw Learn Syst. 2021 Jun;32(6):2782-2789. doi: 10.1109/TNNLS.2020.3005167. Epub 2021 Jun 2.

Abstract

We propose a new generic type of artificial neurons called q -neurons. A q -neuron is a stochastic neuron with its activation function relying on Jackson's discrete q -derivative for a stochastic parameter q . We show how to generalize neural network architectures with q -neurons and demonstrate the scalability and ease of implementation of q -neurons into legacy deep learning frameworks. We report experimental results that consistently improve performance over state-of-the-art standard activation functions, both on training and test loss functions.