The Usage of Mock Oral Examinations for Program Improvement

Abstract

Objective: Mock oral examinations are often used to prepare residents for the American Board of Surgery certifying examination. Another potential use of these examinations is to identify programmatic weaknesses. Results from a multi-institutional mock oral examination were evaluated to determine if specific areas of weakness within each of the participating programs could be identified to facilitate program development.

Design: A mock oral examination was administered annually consisting of 3 examination rooms per resident with 3 cases in each room. Case categories included core general surgery and subspecialties and cases were changed yearly. Each case included facets of patient management from history and physical examination, and differential diagnosis to postoperative care and professional behaviors.

Setting: General Surgery programs at 3 academic medical centers-Northwestern University, Rush University, and University of Illinois at Chicago.

Participants: A total of 259 resident examinations of fourth- and fifth-year general surgery residents over a 7-year period.

Results: A total of 2331 individual resident cases were evaluated with an overall case pass rate of 50.2% across all 3 programs. The weakest case category for each program was different (A = vascular 40.0% pass, B = trauma 41.4% pass, and C = breast 30.0% pass). All programs scored above their mean in gastrointestinal and abdominal surgery and below their mean in vascular surgery. Within vascular surgery, the weakest facet of patient management also differed between programs (A = select tests 44.3% pass, B = complications 57.0% pass, and C = history and physical 55.4% pass).

Conclusions: A standardized mock oral examination is able to identify topic areas of relative strength and weakness that differ between programs. These results can be used to define focused areas for improvement within training programs, guide rotation schedules, and improve didactic curricula.

Keywords: Medical Knowledge; Patient Care; academic medical centers; certification; curriculum; program development.