Competency in Robotic Surgery: Standard Setting for Robotic Suturing Using Objective Assessment and Expert Evaluation

Taseen F Haque; J Everett Knudsen; Jonathan You; Alvin Hui; Hooman Djaladat; Runzhuo Ma; Steven Cen; Mitchell Goldenberg; Andrew J Hung

doi:10.1016/j.jsurg.2023.12.002

Competency in Robotic Surgery: Standard Setting for Robotic Suturing Using Objective Assessment and Expert Evaluation

J Surg Educ. 2024 Mar;81(3):422-430. doi: 10.1016/j.jsurg.2023.12.002. Epub 2024 Jan 29.

Authors

Taseen F Haque¹, J Everett Knudsen¹, Jonathan You², Alvin Hui², Hooman Djaladat¹, Runzhuo Ma², Steven Cen³, Mitchell Goldenberg¹, Andrew J Hung⁴

Affiliations

¹ Catherine & Joseph Aresty Department of Urology, USC Institute of Urology, University of Southern California, Los Angeles, California.
² Department of Urology, Cedars-Sinai Medical Center, Los Angeles, California.
³ Department Radiology, University of Southern California, Los Angeles, California.
⁴ Department of Urology, Cedars-Sinai Medical Center, Los Angeles, California. Electronic address: andrew.hung@cshs.org.

PMID: 38290967
PMCID: PMC10923136 (available on 2025-03-01)
DOI: 10.1016/j.jsurg.2023.12.002

Abstract

Objective: Surgical skill assessment tools such as the End-to-End Assessment of Suturing Expertise (EASE) can differentiate a surgeon's experience level. In this simulation-based study, we define a competency benchmark for intraoperative robotic suturing using EASE as a validated measure of performance.

Design: Participants conducted a dry-lab vesicourethral anastomosis (VUA) exercise. Videos were each independently scored by 2 trained, blinded reviewers using EASE. Inter-rater reliability was measured with prevalence-adjusted bias-adjusted Kappa (PABAK) using 2 example videos. All videos were reviewed by an expert surgeon, who determined if the suturing skills exhibited were at a competency level expected for residency graduation (pass or fail). The Contrasting Group (CG) method was then used to set a pass/fail score at the intercept of the pass and fail cohorts' EASE score distributions.

Setting: Keck School of Medicine, University of Southern California.

Participants: Twenty-six participants: 8 medical students, 8 junior residents (PGY 1-2), 7 senior residents (PGY 3-5) and 3 attending urologists.

Results: After 1 round of consensus-building, average PABAK across EASE subskills was 0.90 (Range 0.67-1.0). The CG method produced a competency benchmark EASE score of >35/39, with a pass rate of 10/26 (38%); 27% were deemed competent by expert evaluation. False positives and negatives were defined as medical students who passed and attendings who failed the assessment, respectively. This pass/fail score produced no false positives or negatives, and fewer JR than SR were considered competent by both the expert and CG benchmark.

Conclusions: Using an absolute standard setting method, competency scores were set to identify trainees who could competently execute a standardized dry-lab robotic suturing exercise. This standard can be used for high stakes decisions regarding a trainee's technical readiness for independent practice. Future work includes validation of this standard in the clinical environment through correlation with clinical outcomes.

Keywords: benchmarking; clinical competence; education; robotic surgical procedures.

MeSH terms

Clinical Competence
Humans
Internship and Residency*
Reproducibility of Results
Robotic Surgical Procedures* / education
Robotics*
Surgeons*

Grants and funding

R01 CA251579/CA/NCI NIH HHS/United States