Effects of Virtual Reality-Based Multimodal Audio-Tactile Cueing in Patients With Spatial Attention Deficits: Pilot Usability Study

Samuel Elia Johannes Knobel; Brigitte Charlotte Kaufmann; Nora Geiser; Stephan Moreno Gerber; René M Müri; Tobias Nef; Thomas Nyffeler; Dario Cazzoli

doi:10.2196/34884

Effects of Virtual Reality-Based Multimodal Audio-Tactile Cueing in Patients With Spatial Attention Deficits: Pilot Usability Study

JMIR Serious Games. 2022 May 25;10(2):e34884. doi: 10.2196/34884.

Authors

Samuel Elia Johannes Knobel¹, Brigitte Charlotte Kaufmann², Nora Geiser³, Stephan Moreno Gerber¹, René M Müri^{1

4

5}, Tobias Nef^{1

5

6}, Thomas Nyffeler^{1

3

4

5}, Dario Cazzoli^{1

3

7}

Affiliations

¹ Gerontechnology & Rehabilitation Group, University of Bern, Bern, Switzerland.
² Sorbonne Université, Institut du Cerveau - Paris Brain Institute (ICM), Inserm, Centre national de la recherche scientifique, Hôpital de la Pitié-Salpêtrière, Paris, France.
³ Neurocenter, Luzerner Kantonsspital, Luzern, Switzerland.
⁴ Perception and Eye Movement Laboratory, Departments of Neurology and BioMedical Research, Inselspital, Bern University Hospital, Bern, Switzerland.
⁵ Department of Neurology, Inselspital, Bern University Hospital, University of Bern, Bern, Switzerland.
⁶ ARTORG Center for Biomedical Engineering Research, University of Bern, Bern, Switzerland.
⁷ Institute of Psychology, University of Bern, Bern, Switzerland.

PMID: 35612894
PMCID: PMC9178455
DOI: 10.2196/34884

Abstract

Background: Virtual reality (VR) devices are increasingly being used in medicine and other areas for a broad spectrum of applications. One of the possible applications of VR involves the creation of an environment manipulated in a way that helps patients with disturbances in the spatial allocation of visual attention (so-called hemispatial neglect). One approach to ameliorate neglect is to apply cross-modal cues (ie, cues in sensory modalities other than the visual one, eg, auditory and tactile) to guide visual attention toward the neglected space. So far, no study has investigated the effects of audio-tactile cues in VR on the spatial deployment of visual attention in neglect patients.

Objective: This pilot study aimed to investigate the feasibility and usability of multimodal (audio-tactile) cueing, as implemented in a 3D VR setting, in patients with neglect, and obtain preliminary results concerning the effects of different types of cues on visual attention allocation compared with noncued conditions.

Methods: Patients were placed in a virtual environment using a head-mounted display (HMD). The inlay of the HMD was equipped to deliver tactile feedback to the forehead. The task was to find and flag appearing birds. The birds could appear at 4 different presentation angles (lateral and paracentral on the left and right sides), and with (auditory, tactile, or audio-tactile cue) or without (no cue) a spatially meaningful cue. The task usability and feasibility, and 2 simple in-task measures (performance and early orientation) were assessed in 12 right-hemispheric stroke patients with neglect (5 with and 7 without additional somatosensory impairment).

Results: The new VR setup showed high usability (mean score 10.2, SD 1.85; maximum score 12) and no relevant side effects (mean score 0.833, SD 0.834; maximum score 21). A repeated measures ANOVA on task performance data, with presentation angle, cue type, and group as factors, revealed a significant main effect of cue type (F_30,3=9.863; P<.001) and a significant 3-way interaction (F_90,9=2.057; P=.04). Post-hoc analyses revealed that among patients without somatosensory impairment, any cue led to better performance compared with no cue, for targets on the left side, and audio-tactile cues did not seem to have additive effects. Among patients with somatosensory impairment, performance was better with both auditory and audio-tactile cueing than with no cue, at every presentation angle; conversely, tactile cueing alone had no significant effect at any presentation angle. Analysis of early orientation data showed that any type of cue triggered better orientation in both groups for lateral presentation angles, possibly reflecting an early alerting effect.

Conclusions: Overall, audio-tactile cueing seems to be a promising method to guide patient attention. For instance, in the future, it could be used as an add-on method that supports attentional orientation during established therapeutic approaches.

Keywords: bird search task; neglect, multimodal cueing; search task; stroke; virtual reality.

©Samuel Elia Johannes Knobel, Brigitte Charlotte Kaufmann, Nora Geiser, Stephan Moreno Gerber, René M Müri, Tobias Nef, Thomas Nyffeler, Dario Cazzoli. Originally published in JMIR Serious Games (https://games.jmir.org), 25.05.2022.