Intelligent and strong robust CVS-LVAD control based on soft-actor-critic algorithm

Te Li; Wenbo Cui; Nan Xie; Heng Li; Haibo Liu; Xu Li; Yongqing Wang

doi:10.1016/j.artmed.2022.102308

Intelligent and strong robust CVS-LVAD control based on soft-actor-critic algorithm

Artif Intell Med. 2022 Jun:128:102308. doi: 10.1016/j.artmed.2022.102308. Epub 2022 Apr 22.

Authors

Te Li¹, Wenbo Cui², Nan Xie³, Heng Li⁴, Haibo Liu⁵, Xu Li⁶, Yongqing Wang⁷

Affiliations

¹ Key Laboratory for Precision and Nontraditional Machining Technology of Ministry of Education, Dalian University of Technology, Dalian 116024, China. Electronic address: teli@dlut.edu.cn.
² Graduate School of Mechanical Engineering, Dalian University of Technology, China. Electronic address: 1297691410@mail.dlut.edu.cn.
³ Graduate School of Mechanical Engineering, Dalian University of Technology, China. Electronic address: xiedanan@mail.dlut.edu.cn.
⁴ Graduate School of Mechanical Engineering, Dalian University of Technology, China. Electronic address: Liheng0424@mail.dlut.edu.cn.
⁵ Key Laboratory for Precision and Nontraditional Machining Technology of Ministry of Education, Dalian University of Technology, Dalian 116024, China. Electronic address: hbliu@dlut.edu.cn.
⁶ Key Laboratory for Precision and Nontraditional Machining Technology of Ministry of Education, Dalian University of Technology, Dalian 116024, China. Electronic address: imlixu@dlut.edu.cn.
⁷ Key Laboratory for Precision and Nontraditional Machining Technology of Ministry of Education, Dalian University of Technology, Dalian 116024, China. Electronic address: yqwang@dlut.edu.cn.

PMID: 35534146
DOI: 10.1016/j.artmed.2022.102308

Abstract

Left ventricular assist device (LVAD) is an effective method to treat ventricular failure. According to the physiological conditions of different patients, the device adaptively adjusts its rotation speed to change LVAD output. In this study, a physiological control system for LVAD based on deep reinforcement learning (DRL) is proposed. The system estimates the amount of blood required by LVAD based on a Starling-like method. The DRL controller regulates LVAD to adjust the speed and quickly approach the target value. The changes of vascular resistance, myocardial contractility, and the transition from rest to exercise were simulated, and the single factor and mixed factor experiments were carried out to compare the effects of DRL controller and proportional integral derivative (PID) controller, which controls the system according to the difference between measured variables and expected values. Two metrics are used to illustrate the regulation effect: the sum of absolute error (SAE) and the response time of the two controllers, where SAE is the difference between the estimated required pumped blood flow LVADQ_e and the actual measured blood flow LVADQ_m. The experimental result shows that the SAE of the DRL controller is 47.6% of that of the PID controller, and the response time of the DRL controller is 38.6% of that of the PID controller. This study demonstrates that the LVAD based on the DRL controller can respond more quickly and more effectively to the different physiological needs of a variety of patients than a PID controller.

Keywords: Deep reinforcement learning; Heart failure; Left ventricular assist devices; Physiological control.

Publication types

Research Support, Non-U.S. Gov't

MeSH terms

Algorithms
Computer Simulation
Heart Failure* / therapy
Heart-Assist Devices*
Humans
Models, Cardiovascular