DNNBrain: A Unifying Toolbox for Mapping Deep Neural Networks and Brains

Xiayu Chen; Ming Zhou; Zhengxin Gong; Wei Xu; Xingyu Liu; Taicheng Huang; Zonglei Zhen; Jia Liu

doi:10.3389/fncom.2020.580632

DNNBrain: A Unifying Toolbox for Mapping Deep Neural Networks and Brains

Front Comput Neurosci. 2020 Nov 30:14:580632. doi: 10.3389/fncom.2020.580632. eCollection 2020.

Authors

Xiayu Chen¹, Ming Zhou¹, Zhengxin Gong¹, Wei Xu¹, Xingyu Liu¹, Taicheng Huang², Zonglei Zhen¹, Jia Liu¹

Affiliations

¹ Beijing Key Laboratory of Applied Experimental Psychology, Faculty of Psychology, Beijing Normal University, Beijing, China.
² State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing, China.

Abstract

Deep neural networks (DNNs) have attained human-level performance on dozens of challenging tasks via an end-to-end deep learning strategy. Deep learning allows data representations that have multiple levels of abstraction; however, it does not explicitly provide any insights into the internal operations of DNNs. Deep learning's success is appealing to neuroscientists not only as a method for applying DNNs to model biological neural systems but also as a means of adopting concepts and methods from cognitive neuroscience to understand the internal representations of DNNs. Although general deep learning frameworks, such as PyTorch and TensorFlow, could be used to allow such cross-disciplinary investigations, the use of these frameworks typically requires high-level programming expertise and comprehensive mathematical knowledge. A toolbox specifically designed as a mechanism for cognitive neuroscientists to map both DNNs and brains is urgently needed. Here, we present DNNBrain, a Python-based toolbox designed for exploring the internal representations of DNNs as well as brains. Through the integration of DNN software packages and well-established brain imaging tools, DNNBrain provides application programming and command line interfaces for a variety of research scenarios. These include extracting DNN activation, probing and visualizing DNN representations, and mapping DNN representations onto the brain. We expect that our toolbox will accelerate scientific research by both applying DNNs to model biological neural systems and utilizing paradigms of cognitive neuroscience to unveil the black box of DNNs.

Keywords: brain imaging; deep neural network; feature visualization; neural encoding and decoding; neural representation; representational similarity analysis (RSA).