Background: "One-minute preceptor" (OMP) is a well-established educational technique; however, primary literature on OMP lacks a tool to assess behavioral change after delivery of curricula.Primary aim of this pilot study was to design a checklist for direct observation of teachers using OMP on general medicine rounds and obtain inter-rater reliability evidence for the checklist.
Methods: This study pilots an internally designed 6-item checklist to assess change in directly observed behavior. We describe the process of developing the checklist and training the observers. We calculated a percent agreement and Cohen's kappa to assess inter-rater reliability.
Results: Raters had a high percent agreement ranging from 0.8 to 0.9 for each step of OMP. Cohen's kappa ranged from 0.49 to 0.77 for the five OMP steps. The highest kappa obtained was for getting a commitment (κ = 0.77) step, whereas the lowest agreement was for correcting mistakes (κ = 0.49).
Conclusion: We showed a percent agreement ≥0.8 and moderate agreement based on Cohen's kappa with most steps of OMP on our checklist. A reliable OMP checklist is an important step in further improving the assessment and feedback of resident teaching skills on general medicine wards.
Copyright © 2022 The Alliance for Continuing Education in the Health Professions, the Association for Hospital Medical Education, and the Society for Academic Continuing Medical Education.