Crash Prediction Using Deep Learning in a Disorienting Spaceflight Analog Balancing Task

Yonglin Wang; Jie Tang; Vivekanand Pandey Vimal; James R Lackner; Paul DiZio; Pengyu Hong

doi:10.3389/fphys.2022.806357

Crash Prediction Using Deep Learning in a Disorienting Spaceflight Analog Balancing Task

Front Physiol. 2022 Jan 28:13:806357. doi: 10.3389/fphys.2022.806357. eCollection 2022.

Authors

Yonglin Wang¹, Jie Tang¹, Vivekanand Pandey Vimal^{2

3}, James R Lackner^{2

3

4}, Paul DiZio^{2

3

4}, Pengyu Hong^{1

3}

Affiliations

¹ Computer Science Department, Brandeis University, Waltham, MA, United States.
² Ashton Graybiel Spatial Orientation Laboratory, Brandeis University, Waltham, MA, United States.
³ Volen Center for Complex Systems, Brandeis University, Waltham, MA, United States.
⁴ Psychology Department, Brandeis University, Waltham, MA, United States.

Abstract

Were astronauts forced to land on the surface of Mars using manual control of their vehicle, they would not have familiar gravitational cues because Mars' gravity is only 0.38 g. They could become susceptible to spatial disorientation, potentially causing mission ending crashes. In our earlier studies, we secured blindfolded participants into a Multi-Axis Rotation System (MARS) device that was programmed to behave like an inverted pendulum. Participants used a joystick to stabilize around the balance point. We created a spaceflight analog condition by having participants dynamically balance in the horizontal roll plane, where they did not tilt relative to the gravitational vertical and therefore could not use gravitational cues to determine their position. We found 90% of participants in our spaceflight analog condition reported spatial disorientation and all of them showed it in their data. There was a high rate of crashing into boundaries that were set at ± 60^° from the balance point. Our goal was to see whether we could use deep learning to predict the occurrence of crashes before they happened. We used stacked gated recurrent units (GRU) to predict crash events 800 ms in advance with an AUC (area under the curve) value of 99%. When we prioritized reducing false negatives we found it resulted in more false positives. We found that false negatives occurred when participants made destabilizing joystick deflections that rapidly moved the MARS away from the balance point. These unpredictable destabilizing joystick deflections, which occurred in the duration of time after the input data, are likely a result of spatial disorientation. If our model could work in real time, we calculated that immediate human action would result in the prevention of 80.7% of crashes, however, if we accounted for human reaction times (∼400 ms), only 30.3% of crashes could be prevented, suggesting that one solution could be an AI taking temporary control of the spacecraft during these moments.

Keywords: crash prediction; deep learning—artificial neural network (DL-ANN); dynamic balance; spaceflight analog; spatial disorientation (SD); vehicle control; vestibular.