Predicting Surgical Resident Performance With Situational Judgment Tests

Aimee K Gardner; Paula Costa

doi:10.1097/ACM.0000000000005680

Predicting Surgical Resident Performance With Situational Judgment Tests

Acad Med. 2024 Feb 27. doi: 10.1097/ACM.0000000000005680. Online ahead of print.

Authors

Aimee K Gardner¹, Paula Costa²

Affiliations

¹ A.K. Gardner is associate dean for faculty development, University of Colorado School of Medicine, Aurora, Colorado, and founder, SurgWise Consulting, Houston, Texas.
² P. Costa is associate, SurgWise Consulting, Houston, Texas.

PMID: 38412475
DOI: 10.1097/ACM.0000000000005680

Abstract

Purpose: Situational judgment tests (SJTs) have been proposed as an efficient, effective, and equitable approach to residency program applicant selection. This study examined how SJTs can predict milestone performance during early residency.

Method: General surgery residency program applicants during 3 selection cycles (2018-2019, 2019-2020, 2020-2021) completed SJTs. Accreditation Council for Graduate Medical Education milestone performance data from selected applicants were collected in March and April 2019, 2020, and 2021 and from residents in March 2020, August 2020, March 2021, September 2021, and March 2022. Descriptive statistics and correlations were computed and analysis of variance tests performed to examine differences among 4 SJT performance groups: green, top 10% to 25%; yellow, next 25% to 50%; red, bottom 50%; and unknown, did not complete the SJT.

Results: Data were collected for 70 residents from 7 surgery residency programs. Differences were found for patient care (F3,189 = 3.19, P = .03), medical knowledge (F3,176 = 3.22, P = .02), practice-based learning and improvement (F3,189 = 3.18, P = .04), professionalism (F3,189 = 3.82, P = .01), interpersonal and communication skills (F3,190 = 3.35, P = .02), and overall milestone score (F3,189 = 3.44, P = .02). The green group performed better on patient care, medical knowledge, practice-based learning and improvement, professionalism, and overall milestone score. The yellow group performed better than the red group on professionalism and overall milestone score, better than the green group on interpersonal and communication skills, and better than the unknown group on all but practice-based learning and improvement. The red group outperformed the unknown group on all but professionalism and outperformed the green group on medical knowledge.

Conclusions: Situational judgment tests demonstrate promise for assessing important noncognitive attributes in residency applicants and align with national efforts to review candidates more holistically and minimize potential biases.