Examination of Skill Acquisition and Grader Bias in a Distal Radius Fracture Fixation Model

Abstract

Objectives: Primary: Assess the ability of faculty graders to predict the objectively measured strength of distal radius fracture fixation. Secondary: Compare resident skill variation and retention related to other knowable training data.

Design: Residents were allowed 60 minutes to stabilize a standardized distal radius fracture using an assigned fixed-angle volar plate. Faculty observed and subjectively graded the residents without providing real-time feedback. Objective biomechanical evaluation (construct strength and stiffness) was compared to subjective grades. Resident-specific characteristics (sex, PGY, and ACGME case log) were also used to compare the objective data.

Setting: A simulated operating room in our laboratory.

Participants: Post-graduate year 2, 3, 4, and 5 orthopedic residents.

Results: Primary: Faculty were not successful at predicting objectively measured fixation, and their subjective scoring suggests confirmation bias as PGY increased. Secondary: Resident year-in-training alone did not predict objective measures (p = 0.53), but was predictive of subjective scores (p < 0.001). Skills learned were not always retained, as 29% of residents objectively failed subsequent to passing. Notably, resident-reported case-specific experience alone was inversely correlated with objective fixation strength.

Conclusions: This testing model enabled the collection of objective and subjective resident skill scores. Faculty graders did not routinely predict objective measures, and their subjective assessment appears biased related to PGY. Also, in vivo case volume alone does not predict objective results. Familiar faculty teaching consistency, and resident grading by external faculty unfamiliar with tested residents, might alter these results.

Keywords: Medical Knowledge; Patient Care; Practice-Based Learning and Improvement; bias; competence; fracture; resident; skill assessment; training.