Validity of and interrater agreement on the LINNEAUS Euro-PC medication safety incident classification system in primary care in Poland

Abstract

Introduction: Medication safety incidents occur in all health care sectors and cause considerable morbidity and mortality, with 8.5% of all related incidents reported estimated to occur in primary care. A common incident classification system could facilitate collective learning from the analysis of medication-related errors and improve patient safety

Objective: The objective of this study was to assess the validity of a new classification system of medication safety incidents in primary care in Poland.

Methods: Analysis of data from a descriptive, cross-sectional, self-reported survey on the Learning from International Networks about Errors and Understanding Safety in Primary Care (LINNEAUS Euro-PC) medication safety incident classification for primary care with assessment of 10 case-based clinical scenarios done by doctors and pharmacists form community-based family medicine clinics and pharmacies in Lodz.

Main outcome measures: The percentages of overall agreement on judgements and a fixed-marginal multirater kappa (κ) coefficient as statistical measures of interrater agreement for categorical items.

Results: The overall agreement levels were: category 1 - 86.3%; category 2 - 85.6%; category 3 - 72.1%; category 4 - 71.8%; and category 5 - 70.4%. The interrater agreement between the 15 evaluators varied as follows: category 1 fixed-marginal κ = 0.144; category 5 fixed-marginal κ = 0.565; category 3 fixed-marginal κ = 0.607; category 4 fixed-marginal κ = 0.634; and category 2 fixed-marginal κ = 0.807.

Conclusions: This is the first known study on levels of agreement on the perception of medication safety incidents and assessment of the validity of a related classification system in primary health care in Poland. Interrater agreement in this study was surprisingly high, but still leaves room for improvement.

Keywords: clinical safety; medical error.