Transition to Residency: National Study of Factors Contributing to Variability in Learner Milestones Ratings in Emergency Medicine and Family Medicine

Yoon Soo Park; Michael S Ryan; Sean O Hogan; Katherine Berg; Adam Eickmeyer; Tonya L Fancher; Jeanne Farnan; Luan Lawson; Laurah Turner; Marjorie Westervelt; Eric Holmboe; Sally A Santen; Medical School Outcomes Milestones Study Group

doi:10.1097/ACM.0000000000005366

Transition to Residency: National Study of Factors Contributing to Variability in Learner Milestones Ratings in Emergency Medicine and Family Medicine

Acad Med. 2023 Nov 1;98(11S):S123-S132. doi: 10.1097/ACM.0000000000005366. Epub 2023 Aug 3.

Affiliations

¹ Y.S. Park is head, Department of Medical Education, and The Ilene B. Harris Endowed Professor, University of Illinois College of Medicine, Chicago, Illinois; ORCID: https://orcid.org/0000-0001-8583-4335.
² M.S. Ryan is associate dean for assessment, evaluation, research, and innovation, and professor of pediatrics, University of Virginia School of Medicine, Charlottesville, Virginia; ORCID: https://orcid.org/0000-0003-3266-9289.
³ S.O. Hogan is director of outcomes research and evaluation, Accreditation Council for Graduate Medical Education, Chicago, Illinois; ORCID: https://orcid.org/0009-0008-9006-1857.
⁴ K. Berg is associate dean of assessment, director, Rector Clinical Skills and Simulation Center, and professor of medicine, Sydney Kimmel Medical College, Philadelphia, Pennsylvania.
⁵ A. Eickmeyer is director of medical school education, University of Chicago Pritzker School of Medicine, Chicago, Illinois, and a PhD candidate, Maastricht University School of Health Professions Education, Maastricht, the Netherlands.
⁶ T.L. Fancher is associate dean for workforce innovation and education quality improvement and professor of medicine, University of California, Davis, School of Medicine, Sacramento, California.
⁷ J. Farnan is associate dean for undergraduate medical education and professor of medicine, University of Chicago Pritzker School of Medicine, Chicago, Illinois.
⁸ L. Lawson is senior associate dean of medical education and student affairs and professor of emergency medicine, Virginia Commonwealth University, Richmond, Virginia.
⁹ L. Turner is assistant dean for evaluation and assessment and assistant professor of medical education, The University of Cincinnati College of Medicine, Cincinnati, Ohio.
¹⁰ M. Westervelt is director of educational assessment, scholarship, improvement, and innovation, Office of Medical Education, University of California, Davis, School of Medicine, Sacramento, California.
¹¹ E. Holmboe is chief, research, milestones development and evaluation officer, Accreditation Council for Graduate Medical Education, Chicago, Illinois.
¹² S.A. Santen is senior associate dean, Virginia Commonwealth University, Richmond, Virginia, and professor, emergency medicine and medical education, University of Cincinnati, Cincinnati, Ohio; ORCID: https://orcid.org/0000-0002-8327-8002.

PMID: 37983405
DOI: 10.1097/ACM.0000000000005366

Abstract

Purpose: The developmental trajectory of learning during residency may be attributed to multiple factors, including variation in individual trainee performance, program-level factors, graduating medical school effects, and the learning environment. Understanding the relationship between medical school and learner performance during residency is important in prioritizing undergraduate curricular strategies and educational approaches for effective transition to residency and postgraduate training. This study explores factors contributing to longitudinal and developmental variability in resident Milestones ratings, focusing on variability due to graduating medical school, training program, and learners using national cohort data from emergency medicine (EM) and family medicine (FM).

Method: Data from programs with residents entering training in July 2016 were used (EM: n=1,645 residents, 178 residency programs; FM: n=3,997 residents, 487 residency programs). Descriptive statistics were used to examine data trends. Cross-classified mixed-effects regression were used to decompose variance components in Milestones ratings.

Results: During postgraduate year (PGY)-1, graduating medical school accounted for 5% and 6% of the variability in Milestones ratings, decreasing to 2% and 5% by PGY-3 for EM and FM, respectively. Residency program accounted for substantial variability during PGY-1 (EM=70%, FM=53%) but decreased during PGY-3 (EM=62%, FM=44%), with greater variability across training period in patient care (PC), medical knowledge (MK), and systems-based practice (SBP). Learner variance increased significantly between PGY-1 (EM=23%, FM=34%) and PGY-3 (EM=34%, FM=44%), with greater variability in practice-based learning and improvement (PBLI), professionalism (PROF), and interpersonal communication skills (ICS).

Conclusions: The greatest variance in Milestone ratings can be attributed to the residency program and to a lesser degree, learners, and medical school. The dynamic impact of program-level factors on learners shifts during the first year and across the duration of residency training, highlighting the influence of curricular, instructional, and programmatic factors on resident performance throughout residency.

MeSH terms

Clinical Competence
Education, Medical, Graduate
Educational Measurement
Emergency Medicine* / education
Family Practice / education
Humans
Internship and Residency*