Development of a real-time RGB-D visual feedback-assisted pulmonary rehabilitation system

Wen-Ruei Tang; Wei Su; Jenn-Jier James Lien; Chao-Chun Chang; Yi-Ting Yen; Yau-Lin Tseng

doi:10.1016/j.heliyon.2023.e23704

Development of a real-time RGB-D visual feedback-assisted pulmonary rehabilitation system

Heliyon. 2023 Dec 14;10(1):e23704. doi: 10.1016/j.heliyon.2023.e23704. eCollection 2024 Jan 15.

Authors

Wen-Ruei Tang¹, Wei Su², Jenn-Jier James Lien², Chao-Chun Chang¹, Yi-Ting Yen¹, Yau-Lin Tseng¹

Affiliations

¹ Division of Thoracic Surgery, Department of Surgery, National Cheng Kung University Hospital, Tainan, Taiwan.
² Department of Computer Science and Information Engineering, National Cheng Kung University, Tainan, Taiwan.

Abstract

Background: Following surgery, perioperative pulmonary rehabilitation (PR) is important for patients with early-stage lung cancer. However, current inpatient programs are often limited in time and space, and outpatient settings have access barriers. Therefore, we aimed to develop a background-free, zero-contact thoracoabdominal movement-tracking model that is easily set up and incorporated into a pre-existing PR program or extended to home-based rehabilitation and remote monitoring. We validated its effectiveness in providing preclinical real-time RGB-D (colour-depth camera) visual feedback.

Methods: Twelve healthy volunteers performed deep breathing exercises following audio instruction for three cycles, followed by audio instruction and real-time visual feedback for another three cycles. In the visual feedback system, we used a RealSense™ D415 camera to capture RGB and depth images for human pose-estimation with Google MediaPipe. Target-tracking regions were defined based on the relative position of detected joints. The processed depth information of the tracking regions was visualised on a screen as a motion bar to provide real-time visual feedback of breathing intensity. Pulmonary function was simultaneously recorded using spirometric measurements, and changes in pulmonary volume were derived from respiratory airflow signals.

Results: Our movement-tracking model showed a very strong correlation (r = 0.90 ± 0.05) between thoracic motion signals and spirometric volume, and a strong correlation (r = 0.73 ± 0.22) between abdominal signals and spirometric volume. Displacement of the chest wall was enhanced by RGB-D visual feedback (23 vs 20 mm, P = 0.034), and accompanied by an increased lung volume (2.58 vs 2.30 L, P = 0.003).

Conclusion: We developed an easily implemented thoracoabdominal movement-tracking model and reported the positive impact of real-time RGB-D visual feedback on self-promoted external chest wall expansion, accompanied by increased internal lung volumes. This system can be extended to home-based PR.

Keywords: Depth camera; Noncontact thoracoabdominal movement-tracking; Visual feedback; pulmonary rehabilitation.