A new basic thoracoscopic surgical skill training and assessment system using automatic scoring techniques

Abstract

Purpose: We report a new thoracoscopic surgical skill training and assessment system with automatic scoring techniques, the Huaxi Intelligent Thoracoscopic Skill Training and Assessment (HITSTA) system. We also evaluated the discriminative ability of this system compared to our conventional scoring method at our institution.

Methods: We retrospectively collected training data of thoracic board-certified thoracic surgeons at West China Hospital, Sichuan University from January 1, 2018 to January 1, 2019. Surgeons were assessed by HITSTA system and human examiners simultaneously. Total scores were summed from 3 tasks (grasping with delivery, pattern cutting, and suture with knot). Bland-Altman analysis was used to test agreement of scores made by HITSTA system (automatic scoring) and human examiners (manual scoring). Differentiation ability was also compared between the two scoring methods.

Results: Thirty-nine surgeons were recruited. Scores made by HITSTA system and human examiners were not consistent. For suture with knot, automatic scoring method could detect the score differences between different training status (trained: 26.92 ± 12.04, untrained: 19.85 ± 11.12; p = 0.026) and training duration (< 10 h: 20.67 ± 15.23, ≥ 10 h: 31.92 ± 5.56; p = 0.003). For total scores, automatic scoring approach could discriminate between different training status (trained: 71.90 ± 12.63; untrained: 61.41 ± 13.87; p = 0.016) and training duration (< 10 h: 65.23 ± 15.31; ≥ 10 h 77.23 ± 6.94; p = 0.046).

Conclusion: HITSTA system could discriminate the different levels of thoracoscopic surgical skills better than the traditional manual scoring method. Larger prospective studies are warranted to validate the differentiation ability of HITSTA system.

Keywords: Automatic assessment; Surgical training; Thoracoscopy.