Audiovisualization of real-time neuroimaging data

PLoS One. 2024 Feb 21;19(2):e0297435. doi: 10.1371/journal.pone.0297435. eCollection 2024.

Abstract

Advancements in brain imaging techniques have significantly expanded the size and complexity of real-time neuroimaging and behavioral data. However, identifying patterns, trends and synchronies within these datasets presents a significant computational challenge. Here, we demonstrate an approach that can translate time-varying neuroimaging data into unique audiovisualizations consisting of audible representations of dynamic data merged with simplified, color-coded movies of spatial components and behavioral recordings. Multiple variables can be encoded as different musical instruments, letting the observer differentiate and track multiple dynamic parameters in parallel. This representation enables intuitive assimilation of these datasets for behavioral correlates and spatiotemporal features such as patterns, rhythms and motifs that could be difficult to detect through conventional data interrogation methods. These audiovisual representations provide a novel perception of the organization and patterns of real-time activity in the brain, and offer an intuitive and compelling method for complex data visualization for a wider range of applications.

MeSH terms

  • Brain* / diagnostic imaging
  • Neuroimaging*

Grants and funding

National Institutes of Health grants: RF1MH114276 (EMCH), R01NS063226 (EMCH), R01NS076628 (EMCH), UF1NS108213 (EMCH) and 5U01NS094296 (EMCH), Columbia ROADS grant RG31 (Hillman / Zheng) and the Simons Collaboration on Global Brain 542991 (Abbott).The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.