Comparison of three evidence-based practice learning assessment methods in dental curricula

Asim A Al-Ansari; Maha M A El Tantawi

Comparison of three evidence-based practice learning assessment methods in dental curricula

J Dent Educ. 2015 Feb;79(2):146-56.

Authors

Asim A Al-Ansari¹, Maha M A El Tantawi²

Affiliations

¹ Dr. Al-Ansari is Assistant Professor, Department of Preventive Dental Sciences and Vice Dean for Academic Affairs, College of Dentistry, University of Dammam, Saudi Arabia; Dr. El Tantawi is Associate Professor, Department of Preventive Dental Sciences, College of Dentistry, University of Dammam, Saudi Arabia.
² Dr. Al-Ansari is Assistant Professor, Department of Preventive Dental Sciences and Vice Dean for Academic Affairs, College of Dentistry, University of Dammam, Saudi Arabia; Dr. El Tantawi is Associate Professor, Department of Preventive Dental Sciences, College of Dentistry, University of Dammam, Saudi Arabia. maha_tantawy@hotmail.com.

PMID: 25640619

Abstract

Incorporating evidence-based practice (EBP) training in dental curricula is now an accreditation requirement for dental schools, but questions remain about the most effective ways to assess learning outcomes. The purpose of this study was to evaluate and compare three assessment methods for EBP training and to assess their relation to students' overall course grades. Participants in the study were dental students from two classes who received training in appraising randomized controlled trials (RCTs) and systematic reviews in 2013 at the University of Dammam, Saudi Arabia. Repeated measures analysis of variance was used to compare students' scores on appraisal assignments, scores on multiple-choice question (MCQ) exams in which EBP concepts were applied to clinical scenarios, and scores for self-reported efficacy in appraisal. Regression analysis was used to assess the relationship among the three assessment methods, gender, program level, and overall grade. The instructors had acceptable reliability in scoring the assignments (overall intraclass correlation coefficient=0.60). The MCQ exams had acceptable discrimination indices although their reliability was less satisfactory (Cronbach's alpha=0.46). Statistically significant differences were observed among the three methods with MCQ exams having the lowest overall scores. Variation in the overall course grades was explained by scores on the appraisal assignment and MCQ exams (partial eta-squared=0.52 and 0.24, respectively), whereas score on the self-efficacy questionnaire was not significantly associated with overall grade. The results suggest that self-reported efficacy is not a valid method to assess dental students' RCT appraisal skills, whereas instructor-graded appraisal assignments explained a greater portion of variation in grade and had inherent validity and acceptable consistency and MCQ exams had good construct validity but low internal consistency.

Keywords: Saudi Arabia; assessment; dental education; educational measurement; evidence-based dentistry; evidence-based practice.

Publication types

Comparative Study

MeSH terms

Curriculum
Education, Dental*
Educational Measurement / methods*
Educational Measurement / statistics & numerical data
Evidence-Based Dentistry / education*
Female
Humans
Learning
Male
Randomized Controlled Trials as Topic
Reproducibility of Results
Review Literature as Topic
Self Efficacy
Sex Factors
Teaching / methods