Mindfulness-Based Teaching Competency Assessment: Comparing Self-Assessment With Expert Evaluation in the Veterans Administration - Compassionate Awareness Learning Module Program

J Greg Serpa; Stephen R Shamblen; Kathy Atwood; Aree Sangpukdee; Alison Whitehead; Christiane Wolf

doi:10.1177/27536130231197654

Mindfulness-Based Teaching Competency Assessment: Comparing Self-Assessment With Expert Evaluation in the Veterans Administration - Compassionate Awareness Learning Module Program

Glob Adv Integr Med Health. 2023 Sep 7:12:27536130231197654. doi: 10.1177/27536130231197654. eCollection 2023 Jan-Dec.

Authors

J Greg Serpa^{1

2}, Stephen R Shamblen³, Kathy Atwood³, Aree Sangpukdee³, Alison Whitehead¹, Christiane Wolf¹

Affiliations

¹ Department of Veterans Affairs, Office of Patient Centered Care and Cultural Transformation, Washington, DC, USA.
² Department of Veterans Affairs, Greater Los Angeles Healthcare System, Los Angeles, CA, USA.
³ Pacific Institute for Research and Evaluation, Louisville, KY, USA.

Abstract

Background: Meditation, including Mindfulness-Based Interventions (MBI), is a required Complementary and Integrative Health intervention at the US Department of Veterans Affairs (VA). Training VA clinicians to provide MBI at scale must address fidelity concerns and the assessment of clinician competency.

Objective: The psychometric properties of the Mindfulness-Based Intervention: Teaching Assessment Criteria (MBI:TAC), a widely used tool for assessing facilitator competence, continue to be explored. To support the dissemination of MBI, the utility of using the MBI:TAC for self-assessment for clinicians in a national training program was evaluated.

Methods: In a training cohort of VA clinicians (n = 39), participant self evaluations on 2 domains of the MBI:TAC are compared to the competency scores of 2 expert evaluators as based on the observations of a 10-minute exercise. Additionally, the inter-rater reliability between the 2 experts was explored.

Results: Intraclass Correlation for the 2 expert evaluators for Guiding Practice was significant (ρ = .83, P = .003), but was not significant for Embodiment of Mindfulness (ρ = .34, P = .186). Self-evaluation scores were not significantly correlated to expert rater scores such that participants rate their level of competence higher than expert scores.

Conclusion: The MBI:TAC, while an essential tool in teacher training, may not produce accurate scores when used for self-assessment. Instruction from a senior teacher is needed for accurate scoring. Interrater reliability may be improved with enhanced domain operationalization and training. Implications for MBI training are explored.

Keywords: mindfulness; veterans; whole health.