The Long and Short of Summative Assessment in Competency Based Medical Education: Time to Raise the Bar?

Abstract

Background: The Competency-based medical education (CBME) has been introduced for MBBS programme in India from 2019. Reorganization of the assessment system is required to meet the challenges imposed by this new framework.

Objectives: An evaluation of the university summative assessments held prior to the introduction of CBME-based curriculum was carried out to analyze the pattern, relevance and distribution of questions.

Methods: Five sets of annual and supplementary summative examination papers from three universities, State (SU), Private (PU) and Medical (MU) were evaluated. The analysis included format i.e. Structured and Modified Essay questions, Short notes, other formats; marks distribution; terminology-based level of cognitive domain; subject-based relevance and topic-based distribution of questions.

Results: A total of 352 questions were analyzed. The maximum number of questions were from the state university (140, 39.7%).The contribution of short notes in the theory papers was 65.8% (PU), 87.1% (SU) and 88.9% (MU).Only the PU had Modified-Essay Questions (10.5%) and Modified short notes(4.7%). Terminology addressing higher cognitive domains was low as the questions assessed mainly the knowledge level (80-96.8%), comprehension 3.1%-6.4%. Only the PU had problem solving questions comprising 11.7% of total questions. Majority of questions had moderate or high relevance and only 2.1%-8.2% were of low-relevance. Inter-university differences in the topics were observed for dermatology, psychiatry, and infections.

Conclusions: The present evaluation of high-stakes assessment in three universities indicates minor differences in the format of questions. Questions were predominantly in the form of short notes, structured essays and modified essays were a rarity; few questions targeted higher cognitive levels although majority had high-moderate relevance.