A Tale of Two Diverse Qualtrics Samples: Information for Online Survey Researchers

Cancer Epidemiol Biomarkers Prev. 2020 Apr;29(4):731-735. doi: 10.1158/1055-9965.EPI-19-0846. Epub 2020 Feb 17.

Abstract

Background: There is often a lack of transparency in research using online panels related to recruitment methods and sample derivation. The purpose of this study was to describe the recruitment and participation of respondents from two disparate surveys derived from the same online research panel using quota sampling.

Methods: A commercial survey sampling and administration company, Qualtrics, was contracted to recruit participants and implement two Internet-based surveys. The first survey targeted adults aged 50 to 75 years and used sampling quotas to obtain diversity with respect to household income and race/ethnicity. The second focused on women aged 18 to 49 years and utilized quota sampling to achieve a geographically balanced sample.

Results: A racially and economically diverse sample of older adults (n = 419) and a geographically diverse sample of younger women (n = 530) were acquired relatively quickly (within 12 and 4 days, respectively). With exception of the highest income level, quotas were implemented as requested. Recruitment of older adults took longer (vs. younger female adults). Although survey completion rates were reasonable in both studies, there were inconsistencies in the proportion of incomplete survey responses and quality fails.

Conclusions: Cancer researchers, and researchers in general, should consider ways to leverage the use of online panels for future studies. To optimize novel and innovative strategies, researchers should proactively ask questions about panels and carefully consider the strengths and drawbacks of online survey features including quota sampling and forced response.

Impact: Results provide practical insights for cancer researchers developing future online surveys and recruitment protocols.See all articles in this CEBP Focus section, "Modernizing Population Science."

Publication types

  • Research Support, N.I.H., Extramural
  • Research Support, Non-U.S. Gov't

MeSH terms

  • Adolescent
  • Adult
  • Age Factors
  • Aged
  • Colorectal Neoplasms / diagnosis
  • Colorectal Neoplasms / epidemiology*
  • Colorectal Neoplasms / prevention & control
  • Data Accuracy
  • Female
  • Humans
  • Internet / statistics & numerical data
  • Male
  • Mass Screening / statistics & numerical data
  • Middle Aged
  • Patient Acceptance of Health Care / statistics & numerical data
  • Patient Participation / statistics & numerical data*
  • Patient Selection*
  • Risk Assessment / methods
  • Risk Assessment / statistics & numerical data
  • Sample Size
  • Sex Factors
  • Surveys and Questionnaires / statistics & numerical data*
  • Viral Vaccines / administration & dosage
  • Viral Vaccines / immunology
  • Young Adult
  • Zika Virus / immunology
  • Zika Virus Infection / prevention & control*
  • Zika Virus Infection / virology

Substances

  • Viral Vaccines