How does the brain represent the semantic content of an image?

Abstract

Using deep neural networks (DNNs) as models to explore the biological brain is controversial, which is mainly due to the impenetrability of DNNs. Inspired by neural style transfer, we circumvented this problem by using deep features that were given a clear meaning-the representation of the semantic content of an image. Using encoding models and the representational similarity analysis, we quantitatively showed that the deep features which represented the semantic content of an image mainly predicted the activity of voxels in the early visual areas (V1, V2, and V3) and these features were essentially depictive but also propositional. This result is in line with the core viewpoint of the grounded cognition to some extent, which suggested that the representation of information in our brain is essentially depictive and can implement symbolic functions naturally.

Keywords: Deep neural networks; Early visual areas; Encoding models; Ground cognition; Neural style transfer; Representational similarity analysis.