Shifting of Cognitive Assessments Between Face-to-Face and Telephone Administration: Measurement Considerations

Jason R Smith; Laura E Gibbons; Paul K Crane; Dan M Mungas; M Maria Glymour; Jennifer J Manly; Laura B Zahodne; Elizabeth Rose Mayeda; Richard N Jones; Alden L Gross

doi:10.1093/geronb/gbac135

Shifting of Cognitive Assessments Between Face-to-Face and Telephone Administration: Measurement Considerations

J Gerontol B Psychol Sci Soc Sci. 2023 Feb 19;78(2):191-200. doi: 10.1093/geronb/gbac135.

Authors

Affiliations

¹ Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, USA.
² General Internal Medicine, University of Washington, Seattle, Washington, USA.
³ Department of Neurology, University of California, Davis, California, USA.
⁴ Department of Epidemiology and Biostatistics, University of California, San Francisco, California, USA.
⁵ Taub Institute for Research on Alzheimer's Disease and the Aging Brain, Columbia University, New York, New York, USA.
⁶ Department of Psychology, University of Michigan, Ann Arbor, Michigan, USA.
⁷ Department of Epidemiology, University of California, Los Angeles Fielding School of Public Health, Los Angeles, California, USA.
⁸ Department of Psychiatry and Human Behavior, Warren Alpert Medical School, Brown University, Providence, Rhode Island, USA.
⁹ Department of Mental Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, USA.

Abstract

Objectives: Telephone-administered cognitive assessments are a cost-effective and sometimes necessary alternative to face-to-face assessments. There is limited information in large studies concerning mode effects, or differences in cognition attributable to the assessment method, as a potential measurement threat. We evaluated mode effects on cognitive scores using a population-based sample of community-living older adults.

Methods: We used data from participants aged 65-79 in the 2014 Health and Retirement Study for whom the interview mode was randomized (n = 6,825). We assessed mode differences in test means, whether mode modifies associations of cognition with criterion variables, and formal measurement invariance testing.

Results: Relative to face-to-face assessment, telephone assessment was associated with higher scores for memory and calculation (0.06 to 0.013 standard deviations [SD]) and lower scores for nonmemory items (-0.09 to -0.01 SD). Cognition was significantly differentially related to instrumental activities of daily living difficulty depending on assessment mode. Measurement invariance testing identified evidence of mode differences in certain tests as a function of mode: adjusting for underlying cognition, the largest mode differences in memory and attention: immediate noun recall, delayed word recall, and serial-7s scores were higher given telephone administration.

Discussion: Differences by mode of administration are apparent in cognitive measurement in older adults, albeit to a small degree in our study, and most pronounced for tests of memory and attention. The importance of accounting for mode differences ultimately depends on one's research question and study sample: not all associations may be affected by mode differences, and such modification may only be apparent among those with lower cognitive functioning.

Keywords: Cognition; Mode effects; Psychometrics; Surveys; Telephone.

Publication types

Research Support, N.I.H., Extramural

MeSH terms

Activities of Daily Living
Aged
Cognition
Cognition Disorders* / psychology
Humans
Surveys and Questionnaires
Telephone

Abstract

Publication types

MeSH terms

Grants and funding