Peer assessment: Development and delivery of the OSCE

Abstract

Introduction: There is an expectation that healthcare professionals display competence in teaching, assessment and providing feedback. Development begins with formative peer-assisted learning and teaching in the undergraduate environment. Using peers or near-peers (in this case having 1 year more experience than the examination cohort) to provide assessment in summative exams remains unexplored. This study investigates how the use of near-peers compares to marking by academic staff in a summative OSCE.

Materials and methods: BDS4 Peer assessors (PAs) developed an OSCE question and marking schedule. Each PA (n = 3) was paired with an academic staff assessor (ASA) (n = 3). Peer and academic marked the candidates independently. Two years later, the process was repeated on the same cohort of candidates with the PA now 1-year post qualification. Statistical analysis compared the scores awarded by PA during each timeframe and against the marks awarded by the ASA.

Results: During round 1, 28 students (62.2%) were awarded the same score by PA and ASA. On 17 occasions, there was a discrepancy (37.8%). Bias was skewed in favour of PA scoring higher (mean difference of differences -0.0667). During round 2, 27 students (55.1%) were awarded the same score by PA and ASA. On 22 occasions (44.9%), there was a discrepancy. Bias was skewed in favour of ASA scoring higher (mean difference of differences 0.0612).

Discussion: Levels of agreement between PA and ASA are strong. Our results show PA mark more leniently as undergraduates and less leniently at 1-year post graduation.

Conclusions: Peer assessors are able to write OSCE stations, produce marking schemes and effectively assess their near-peers.

Keywords: OSCE; peer assessment; peer-assisted learning.