Written-Based Progress Testing: A Scoping Review

Vincent Dion; Christina St-Onge; Ilona Bartman; Claire Touchie; Debra Pugh

doi:10.1097/ACM.0000000000004507

Written-Based Progress Testing: A Scoping Review

Acad Med. 2022 May 1;97(5):747-757. doi: 10.1097/ACM.0000000000004507. Epub 2022 Apr 27.

Authors

Vincent Dion¹, Christina St-Onge², Ilona Bartman³, Claire Touchie⁴, Debra Pugh⁵

Affiliations

¹ V. Dion is an undergraduate medical education student, Faculty of Medicine and Health Sciences, Université de Sherbrooke, Sherbrooke, Québec, Canada. He was a research assistant to the Paul Grand'Maison de la Société des médecins de l'Université de Sherbrooke research chair in medical education, Sherbrooke, Québec, Canada, at the time this work was completed.
² C. St-Onge is professor, Department of Medicine, Faculty of Medicine and Health Sciences, Université de Sherbrooke, and the Paul Grand'Maison de la Société des médecins de l'Université de Sherbrooke research chair in medical education, Sherbrooke, Québec, Canada; ORCID: https://orcid.org/0000-0001-5313-0456 .
³ I. Bartman is medical education research associate, Medical Council of Canada, Ottawa, Ontario, Canada; ORCID: https://orcid.org/0000-0002-2056-479X .
⁴ C. Touchie is professor of medicine, University of Ottawa, Ottawa, Ontario, Canada. She was chief medical education officer, Medical Council of Canada, Ottawa, Ontario, Canada, at the time this work was completed; ORCID: https://orcid.org/0000-0001-7926-9720 .
⁵ D. Pugh is medical education advisor, Medical Council of Canada, and associate professor, Department of Medicine, University of Ottawa and The Ottawa Hospital, Ottawa, Ontario, Canada; ORCID: https://orcid.org/0000-0003-4076-9669 .

PMID: 34753858
DOI: 10.1097/ACM.0000000000004507

Abstract

Purpose: Progress testing is an increasingly popular form of assessment in which a comprehensive test is administered to learners repeatedly over time. To inform potential users, this scoping review aimed to document barriers, facilitators, and potential outcomes of the use of written progress tests in higher education.

Method: The authors followed Arksey and O'Malley's scoping review methodology to identify and summarize the literature on progress testing. They searched 6 databases (Academic Search Complete, CINAHL, ERIC, Education Source, MEDLINE, and PsycINFO) on 2 occasions (May 22, 2018, and April 21, 2020) and included articles written in English or French and pertaining to written progress tests in higher education. Two authors screened articles for the inclusion criteria (90% agreement), then data extraction was performed by pairs of authors. Using a snowball approach, the authors also screened additional articles identified from the included reference lists. They completed a thematic analysis through an iterative process.

Results: A total of 104 articles were included. The majority of progress tests used a multiple-choice and/or true-or-false question format (95, 91.3%) and were administered 4 times a year (38, 36.5%). The most documented source of validity evidence was internal consistency (38, 36.5%). Four major themes were identified: (1) barriers and challenges to the implementation of progress testing (e.g., need for additional resources); (2) established collaboration as a facilitator of progress testing implementation; (3) factors that increase the acceptance of progress testing (e.g., formative use); and (4) outcomes and consequences of progress test use (e.g., progress testing contributes to an increase in knowledge).

Conclusions: Progress testing appears to have a positive impact on learning, and there is significant validity evidence to support its use. Although progress testing is resource- and time-intensive, strategies such as collaboration with other institutions may facilitate its use.

Publication types

Review
Research Support, Non-U.S. Gov't

MeSH terms

Delivery of Health Care*
Humans
Knowledge*