Development and prospective validation of a scoring system for the Basic Endoscopic Skills Training (BEST) box

Ilay Habaz; Silvana Perretta; Allan Okrainec; Anton Svendrovski; Ludovica Guerriero; Fabio Longo; Pietro Mascagni; Ethan Weiss; Louis W C Liu; Lee L Swanstrom; Eran Shlomovitz

doi:10.1007/s00464-020-08152-9

Development and prospective validation of a scoring system for the Basic Endoscopic Skills Training (BEST) box

Surg Endosc. 2021 Dec;35(12):6549-6555. doi: 10.1007/s00464-020-08152-9. Epub 2020 Nov 16.

Authors

Affiliations

¹ Division of General Surgery, University Health Network, University of Toronto, Toronto General Hospital, PMB 1-298, 585 University Ave, Toronto, ON, M5G 2N2, Canada.
² Institute for Image Guided Surgery IHU-Strasbourg, Strasbourg, France.
³ UZIK Consulting Inc, Toronto, ON, Canada.
⁴ Division of Gastroenterology, University Health Network, Toronto, ON, Canada.
⁵ Division of GI/MIS, The Oregon Clinic, Portland, OR, USA.
⁶ Division of General Surgery, University Health Network, University of Toronto, Toronto General Hospital, PMB 1-298, 585 University Ave, Toronto, ON, M5G 2N2, Canada. Eran.shlomovitz@uhn.ca.
⁷ Institute for Image Guided Surgery IHU-Strasbourg, Strasbourg, France. Eran.shlomovitz@uhn.ca.
⁸ Division of Interventional Radiology, University Health Network, Toronto, ON, Canada. Eran.shlomovitz@uhn.ca.

PMID: 33196877
DOI: 10.1007/s00464-020-08152-9

Abstract

Aims: The increased use of endoscopy as a minimally invasive therapeutic technique has created a great demand for endoscopic training. The Basic Endoscopic Skills Training (BEST) box provides a low-cost solution by adapting the Fundamentals of Laparoscopic Surgery (FLS) box for flexible endoscopic simulation. The BEST box consists of six endoscopic tasks with a 5-min time limit per task. This study aims to develop a scoring system for objective evaluation of user performance.

Methods: A total of 165 participants were tested on the BEST box. Participants were divided into two groups: retrospective analysis (n = 100) and prospective analysis (n = 65). From the retrospective group, 55 individuals were also scored on the Global Assessment of Gastrointestinal Endoscopic Skills-Upper Endoscopy (GAGES-UE). Linear regression between user performance on BEST box and GAGES-UE was performed to develop the scoring system. Receiver Operating Characteristic curve was used to determine a threshold score to help users appreciate their endoscopic expertise. Prospective scoring of 65 individuals was then performed using the formula developed (20 experts and 45 trainees).

Results: The minimum and maximum possible scores are 30 and 110, respectively. Retrospective analysis showed that the scoring system was able to distinguish between experts and trainees (p < 0.001), correlated with GAGES-UE (p < 0.001), and had a reliability constant of r = 0.765 (p < 0.001). On prospective testing using the scoring system the expert group received a final average score of 92, whereas the average score for the trainee group was 61 (p < 0.001).

Conclusions: The developed BEST box scoring system correlates with the experience level of the test taker as well as with the GAGES-UE scoring system. The results of this study add further evidence to the validity of the BEST box as an effective, low-cost endoscopic simulator with the scores used by trainees to track their performance level overtime.

Keywords: Endoscopy simulator; FES; FLS; Scoring system; Simulation.

Publication types

Research Support, Non-U.S. Gov't

MeSH terms

Clinical Competence
Computer Simulation
Endoscopy, Gastrointestinal
Humans
Laparoscopy*
Reproducibility of Results
Retrospective Studies
Simulation Training*