Dynamic Electrode-to-Image (DETI) mapping reveals the human brain's spatiotemporal code of visual information

PLoS Comput Biol. 2021 Sep 27;17(9):e1009456. doi: 10.1371/journal.pcbi.1009456. eCollection 2021 Sep.

Abstract

A number of neuroimaging techniques have been employed to understand how visual information is transformed along the visual pathway. Although each technique has spatial and temporal limitations, they can each provide important insights into the visual code. While the BOLD signal of fMRI can be quite informative, the visual code is not static and this can be obscured by fMRI's poor temporal resolution. In this study, we leveraged the high temporal resolution of EEG to develop an encoding technique based on the distribution of responses generated by a population of real-world scenes. This approach maps neural signals to each pixel within a given image and reveals location-specific transformations of the visual code, providing a spatiotemporal signature for the image at each electrode. Our analyses of the mapping results revealed that scenes undergo a series of nonuniform transformations that prioritize different spatial frequencies at different regions of scenes over time. This mapping technique offers a potential avenue for future studies to explore how dynamic feedforward and recurrent processes inform and refine high-level representations of our visual world.

Publication types

  • Research Support, U.S. Gov't, Non-P.H.S.

MeSH terms

  • Adolescent
  • Brain Mapping / instrumentation
  • Brain Mapping / methods*
  • Brain Mapping / statistics & numerical data
  • Computational Biology
  • Electrodes
  • Electroencephalography / instrumentation
  • Electroencephalography / statistics & numerical data*
  • Female
  • Functional Neuroimaging / statistics & numerical data
  • Humans
  • Image Processing, Computer-Assisted
  • Magnetic Resonance Imaging / statistics & numerical data
  • Male
  • Photic Stimulation
  • Spatio-Temporal Analysis
  • Visual Cortex / physiology
  • Visual Pathways / physiology*
  • Young Adult

Grants and funding

James S. McDonnell Foundation grant (220020430) to BCH; National Science Foundation grant (1736394) to BCH and MRG. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.