Measuring disability: An agreement study between two disability measures

Abstract

Background: The Washington Group Short Set (WGSS) is increasingly being included in surveys worldwide to improve the quality and comparability of disability data within and between nations. However, compared to commonly employed binary impairment measures, the WGSS appears to have a relatively high threshold for disability indication. Empirical evidence is required to quantify this potential difference and its impact.

Objective: To determine the agreement between a binary disability question (BDQ) and the WGSS measure administered concurrently in a large representative survey of adults. Two WGSS indication scenarios were considered: one using the recommended threshold to indicate disability (WGSS1); and another using a lower threshold (WGSS2).

Methods: A cross-sectional agreement study nested within the 2018 Canterbury Wellbeing Survey of randomly selected adults aged ≥18 years resident in the greater Christchurch region.

Results: From 2807 valid observations, 493 (17.6%), 259 (9.1%), and 822 (28.7%) participants were identified as having a disability by the BDQ, WGSS1 and WGSS2 measures, respectively. While concordance was high between BDQ and WGSS1 measures (85.3%), agreement was only fair (κ = 0.37) and discordance was significantly asymmetrical (p < 0.001). Participants were more likely to be indicated as 'disabled' using the BDQ but not indicated using the WGSS1 than vice versa.

Conclusions: Different WGSS thresholds produce widely varying disability estimates, and the recommended WGSS1 resulted in an importantly reduced disability prevalence compared to a binary impairment measure when administered concurrently. This has profound implications for inclusivity and policy for people living with disability.

Keywords: Disability definition; Disability measurement; New Zealand Disability; Washington group.