A systematic exploration of reservoir computing for forecasting complex spatiotemporal dynamics

Jason A Platt; Stephen G Penny; Timothy A Smith; Tse-Chun Chen; Henry D I Abarbanel

doi:10.1016/j.neunet.2022.06.025

A systematic exploration of reservoir computing for forecasting complex spatiotemporal dynamics

Neural Netw. 2022 Sep:153:530-552. doi: 10.1016/j.neunet.2022.06.025. Epub 2022 Jun 30.

Authors

Jason A Platt¹, Stephen G Penny², Timothy A Smith², Tse-Chun Chen², Henry D I Abarbanel³

Affiliations

¹ Department of Physics, University of California San Diego, United States of America. Electronic address: 1763platt@gmail.com.
² Cooperative Institute for Research in Environmental Sciences, University of Colorado Boulder, Boulder, CO 80309, United States of America; Physical Sciences Laboratory, National Oceanic and Atmospheric Administration, Boulder, CO 80305, United States of America.
³ Department of Physics, University of California San Diego, United States of America; Marine Physical Laboratory, Scripps Institution of Oceanography, University of California San Diego, 9500 Gilman Drive, La Jolla, CA 92093, USA.

PMID: 35839598
DOI: 10.1016/j.neunet.2022.06.025

Abstract

A reservoir computer (RC) is a type of recurrent neural network architecture with demonstrated success in the prediction of spatiotemporally chaotic dynamical systems. A further advantage of RC is that it reproduces intrinsic dynamical quantities essential for its incorporation into numerical forecasting routines such as the ensemble Kalman filter-used in numerical weather prediction to compensate for sparse and noisy data. We explore here the architecture and design choices for a "best in class" RC for a number of characteristic dynamical systems. Our analysis points to the importance of large scale parameter optimization. We also note in particular the importance of including input bias in the RC design, which has a significant impact on the forecast skill of the trained RC model. In our tests, the use of a nonlinear readout operator does not affect the forecast time or the stability of the forecast. The effects of the reservoir dimension, spinup time, amount of training data, normalization, noise, and the RC time step are also investigated. Finally, we detail how our investigation leads to optimal design choices for a parallel RC scheme applied to the 40 dimensional spatiotemporally chaotic Lorenz 1996 dynamics.

Keywords: Chaotic time series forecasting; Echo-state networks; Machine learning; Nonlinear dynamical systems; Recurrent neural network; Reservoir computing.