Adaptive tracking control for stochastic nonlinear systems with time-varying delays using multi-dimensional Taylor network

Abstract

For stochastic nonlinear systems (SNSs) perturbed by compound uncertainties, the conventional model-based control approaches assume that the evolution behavior of uncertain variables is known. Unfortunately, such approaches are often conservative for most practical scenarios with the slow convergence speed and unsatisfactory anti-interference performance. For this sake, an adaptive control scheme based on deep deterministic policy gradient (DDPG) and multi-dimensional Taylor network (MTN) is proposed here to address the tracking problem for a category of SNSs subject to fast time-varying uncertainties, stochastic disturbance and unknown time-varying delays. The effect of time delay is embedded in the reproducing kernel Hilbert space through the error coordinate transformation. In the framework of DDPG, the MTN-based surrogate is utilized to construct the online network and target network via the temporal-difference method, which promises more desirable real-time performance due to its concise structure than conventional NN-based surrogates. In order to enhance the robustness of the system under fast time-varying uncertainties, a novel persistent excitation (PE) mechanism is designed to ensure that the control policy is appropriately rewarded or punished. Based on the PE condition, weights of MTNs converge exponentially and animate the system to evolve towards the target persistently. The tracking error and closed-loop state signals are proved theoretically to be uniformly ultimately bounded (UUB) via Lyapunov-Krasovskii functional. A numerical simulation from the process industry verifies the effectiveness of the proposed method.

Keywords: Deep deterministic policy gradient; Fast time-varying perturbation; Multi-dimensional Taylor network; Stochastic nonlinear system; Time-varying delay.