The reliability of psychological instruments in community samples: A cautionary note

J Health Psychol. 2016 Sep;21(9):2033-41. doi: 10.1177/1359105315569859. Epub 2015 Feb 20.

Abstract

Although researchers norm and validate measures of psychological constructs largely on educated samples, they often use these instruments more broadly, assuming generalizability. We examined whether the assumption of generalizability is warranted. We administered three commonly used psychological measures-the Behavioral Activation/Behavioral Inhibition Scale, the Regulatory Focus Questionnaire, and the Need for Cognition Scale-to a community sample (N = 332) with limited education. For the three instruments, five of seven scales/subscales had unsatisfactory reliabilities. Internal consistency was lowest among participants with less education. The results suggest that instruments normed on educated samples may not generalize to uneducated samples.

Keywords: cognitive ability; community psychology; individual differences; reliability; sampling.

Publication types

  • Research Support, N.I.H., Extramural

MeSH terms

  • Adolescent
  • Adult
  • Aged
  • Educational Status
  • Female
  • Humans
  • Male
  • Middle Aged
  • Psychometrics / statistics & numerical data*
  • Reproducibility of Results
  • Residence Characteristics*
  • Sampling Studies
  • Surveys and Questionnaires*
  • Young Adult