Deep convolutional networks do not classify based on global object shape

PLoS Comput Biol. 2018 Dec 7;14(12):e1006613. doi: 10.1371/journal.pcbi.1006613. eCollection 2018 Dec.

Abstract

Deep convolutional networks (DCNNs) are achieving previously unseen performance in object classification, raising questions about whether DCNNs operate similarly to human vision. In biological vision, shape is arguably the most important cue for recognition. We tested the role of shape information in DCNNs trained to recognize objects. In Experiment 1, we presented a trained DCNN with object silhouettes that preserved overall shape but were filled with surface texture taken from other objects. Shape cues appeared to play some role in the classification of artifacts, but little or none for animals. In Experiments 2-4, DCNNs showed no ability to classify glass figurines or outlines but correctly classified some silhouettes. Aspects of these results led us to hypothesize that DCNNs do not distinguish object's bounding contours from other edges, and that DCNNs access some local shape features, but not global shape. In Experiment 5, we tested this hypothesis with displays that preserved local features but disrupted global shape, and vice versa. With disrupted global shape, which reduced human accuracy to 28%, DCNNs gave the same classification labels as with ordinary shapes. Conversely, local contour changes eliminated accurate DCNN classification but caused no difficulty for human observers. These results provide evidence that DCNNs have access to some local shape information in the form of local edge relations, but they have no access to global object shapes.

Publication types

  • Research Support, U.S. Gov't, Non-P.H.S.

MeSH terms

  • Animals
  • Computational Biology
  • Deep Learning
  • Form Perception*
  • Humans
  • Neural Networks, Computer*
  • Pattern Recognition, Automated / statistics & numerical data*
  • Pattern Recognition, Visual
  • Photic Stimulation

Grants and funding

This research was funded by the National Science Foundation Research Traineeship for ModEling and uNdersTanding human behaviOR (MENTOR) DGE-1829071 (http://www.math.ucla.edu/~bertozzi/NRT/index.html) to NB and the Advancing Theory and Application in Perceptual and Adaptive Learning to Improve Community College Mathematics NSF Grant ECR-1644916 (https://www.nsf.gov/awardsearch/showAward?AWD_ID=1644916&HistoricalAwards=false) to PJK. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.