Online stabilization of block-diagonal recurrent neural networks

S C Sivakumar; W Robertson; W J Phillips

doi:10.1109/72.737503

Online stabilization of block-diagonal recurrent neural networks

IEEE Trans Neural Netw. 1999;10(1):167-75. doi: 10.1109/72.737503.

Authors

S C Sivakumar¹, W Robertson, W J Phillips

Affiliation

¹ Department of Electrical Engineering, Dalhousie University-DalTech, Halifax, Canada.

PMID: 18252513
DOI: 10.1109/72.737503

Abstract

This paper deals with a discrete-time recurrent neural network (DTRNN) with a block-diagonal feedback weight matrix, called the block-diagonal recurrent neural network (BDRNN), that allows a simplified approach to online training and to address network and training stability issues. The structure of the BDRNN is exploited to modify the conventional backpropagation through time (BPTT) algorithm. to reduce its storage requirement by a numerically stable method of recomputing the network state variables. The network and training stability is addressed by exploiting the BDRNN structure to directly monitor and maintain stability during weight updates by developing a functional measure of system stability that augments the cost function being minimized. Simulation results are presented to demonstrate the performance of the BDRNN architecture, its training algorithm, and the stabilization method.