Deep learning for Koopman Operator Optimal Control

Abstract

Nonlinear dynamics are ubiquitous in complex systems. Their applications range from robotics to computational neuroscience. In this work, the Koopman framework for globally linearizing nonlinear dynamics is introduced. Under this framework, the nonlinear observable states are lifted into a higher dimensional, linear regime. The challenge is to identify functions that facilitate the coordinate transformation to this raised linear space. This point is tackled using deep learning, where nonlinear dynamics are learned in a model-free manner, i.e., the underlying dynamics are uncovered using data rather than the nonlinear state-space equations. The main contributions include an implementation of the Linearly Recurrent Encoder Network (LREN) that is faster than the existing implementation and is significantly faster than the state-of-the-art deep learning-based approach. Also, a novel architecture termed Deep Encoder with Initial State Parameterization (DENIS) is proposed. By deriving an energy-budget control performance evaluation method, we demonstrate that DENIS also outperforms LREN in control performance. It is also on-par with and sometimes better than the iterative linear quadratic regulator (iLQR), which requires access to the state-space equations. Extensive experiments are done on DENIS to validate its performance. Also, another novel architecture termed Double Encoder for Input Nonaffine systems (DEINA) is described. Additionally, DEINA's potential ability to outperform existing Koopman frameworks for controlling nonaffine input systems is shown. We attribute this to using an auxiliary network to nonlinearly transform the inputs, thereby lifting the strong linear constraints imposed by the traditional Koopman approximation approach. Koopman model predictive control (KMPC) is implemented to verify that our models can also be successfully controlled under this popular approach. Overall, we demonstrate the deep learning-based Koopman framework shows promise for optimally controlling nonlinear dynamics.

Keywords: Deep Encoder with Initial State Parameterization (DENIS); Deep learning; Double Encoder for Input Nonaffine Systems (DEINA); Iterative linear quadratic regulator; Koopman model predictive control (KMPC); Linearly Recurrent Encoder Network (LREN); Optimal control.