Functional loops: Monitoring functional organization of deep neural networks using algebraic topology

Abstract

Various topological methods have emerged in recent years to investigate the inner workings of deep neural networks (DNNs) based on the structural and weight information. However, their effectiveness is restricted due to the stratified structure and volatile weight information. In this study, we explore the relationship between functional organizations and network performance using algebraic topology. Our results indicate that functional loops reveal functional interaction patterns of multiple neurons in DNNs. We also propose functional persistence as a measure of functional complexity and develop an early stopping criterion that achieves competitive results without requiring a validation set.

Keywords: Deep neural network; Functional network; Topological data analysis.