How well do surveys on adherence to pandemic policies assess actual behaviour: Measurement properties of the Dutch COVID-19 adherence to prevention advice survey (CAPAS)

Carlijn Bussemakers; Mart van Dijk; Alexandra L Dima; Marijn de Bruin

doi:10.1016/j.socscimed.2023.116395

How well do surveys on adherence to pandemic policies assess actual behaviour: Measurement properties of the Dutch COVID-19 adherence to prevention advice survey (CAPAS)

Soc Sci Med. 2023 Dec:339:116395. doi: 10.1016/j.socscimed.2023.116395. Epub 2023 Nov 8.

Authors

Carlijn Bussemakers¹, Mart van Dijk², Alexandra L Dima³, Marijn de Bruin⁴

Affiliations

¹ Institute of Health Sciences, IQ Healthcare, Radboud University Medical Center, the Netherlands. Electronic address: carlijn.bussemakers@radboudumc.nl.
² Corona Behavioural Unit, National Institute for Public Health and The Environment (RIVM), the Netherlands.
³ Research and Development Unit, Institut de Recerca Sant Joan de Déu, Sant Boi de Llobregat, Barcelona, Spain.
⁴ Institute of Health Sciences, IQ Healthcare, Radboud University Medical Center, the Netherlands; Corona Behavioural Unit, National Institute for Public Health and The Environment (RIVM), the Netherlands.

PMID: 37956618
DOI: 10.1016/j.socscimed.2023.116395

Abstract

Background: Survey data on adherence to COVID-19 prevention measures have often been used to inform policy makers and public health professionals. Although behavioural survey data are often considered to suffer from biases, there is a lack of studies critically examining the validity, reliability and responsiveness of population-survey data on behaviour throughout the COVID-19 pandemic.

Aim: We studied the measurement properties of the COVID-19 Adherence to Prevention Advice Survey (CAPAS), a novel questionnaire implemented in a repeated cross-sectional (i.e., 'Trend') Study and a Cohort Study in the Netherlands during the COVID-19 pandemic.

Methods: The CAPAS is a novel questionnaire developed in March 2020, with the aim to assess social activity and adherence to COVID-19 prevention measures. Items were formulated to minimise social desirability and aid memory retrieval. Based on the COSMIN framework, we selected the most suitable test for each behavioural question. We investigated criterion validity of vaccination, testing behaviour and mobility by comparing (aggregate) trends of self-reported behaviour to trends in objective data. Responsiveness of mobility and ventilation behaviour was assessed by studying whether self-reported behaviour changed following contextual (e.g., policy) changes. Test-retest reliability of hygienic behaviour, wearing face masks, ventilation behaviour and social distancing was examined during a period in which the context was stable.

Results: Overall, aggregate trends in self-reported behaviour closely corresponded to trends in external objective data. Self-reported behaviours were responsive to contextual changes and test-retest reliabilities were adequate. For infrequent behaviours reliability improved when measures were dichotomised. We were able to examine national representativeness for vaccination, which suggested a modest overestimation of on average 3.7%.

Conclusions: This study supports the suitability of using carefully designed, self-reported surveys (and the CAPAS specifically) to study changes in protective behaviours in a dynamic context.

Keywords: COVID-19; Population survey; Reliability; Responsiveness; Self-reported behaviour; Validity.

MeSH terms

COVID-19* / epidemiology
COVID-19* / prevention & control
Cohort Studies
Cross-Sectional Studies
Humans
Pandemics / prevention & control
Reproducibility of Results
Self Report
Surveys and Questionnaires