Assessing Individual Competency Differences Between Third- and Fourth-Year Medical Students Using the National Clinical Assessment Tool for Medical Students in the Emergency Department

Abstract

Medical students rotating through emergency departments as part of their clinical education are typically evaluated using an on-shift evaluation tool. The National Clinical Assessment Tool for Medical Students in the Emergency Department (NCAT-EM) is the current standard of evaluation for medical students in the emergency department, regardless of level of training. This study aims to evaluate whether the NCAT-EM can detect differences in skill levels between third-year medical students (MS3s) and fourth-year medical students (MS4s) rotating at a level 1 trauma center and teaching institution. These authors hypothesized that MS4s should outperform MS3s across all assessment domains given their additional training. A total of 930 performance evaluations were gathered for MS3 and MS4 rotating between May 2022 and June 2023. There were 321 evaluations of MS3s and 609 evaluations of MS4s. Across the six assessment domains, MS4s had statistically significant higher performances in two domains - namely emergency recognition and management (fully entrustable: 37.4% vs. 23.8% (p = 0.03)) and communication (fully entrustable: 46.2% vs. 33.6% (p = 0.03)). These findings indicate that the use of the NCAT-EM at this institution reliably differentiated between MS3s and MS4s in these two assessment domains. There were trends suggesting MS4s outperform MS3s in the other four domains, which did not rise to the level of statistical significance, but are consistent with prior validation studies of the NCAT.

Keywords: emergency medicine; medical education; medical student assessment; medical student clerkship; medical student performance; national clinical assessment tool; ncat-em.