Introduction: Research focusing on cognitive aging and dementia is a global endeavor. However, cross-national differences in cognition are embedded in other sociocultural differences, precluding direct comparisons of test scores. Such comparisons can be facilitated by co-calibration using item response theory (IRT). The goal of this study was to explore, using simulation, the necessary conditions for accurate harmonization of cognitive data.
Method: Neuropsychological test scores from the US Health and Retirement Study (HRS) and the Mexican Health and Aging Study (MHAS) were subjected to IRT analysis to estimate item parameters and sample means and standard deviations. These estimates were used to generate simulated item response patterns under 10 scenarios that adjusted the quality and quantity of linking items used in harmonization. IRT-derived factor scores were compared to the known population values to assess bias, efficiency, accuracy, and reliability of the harmonized data.
Results: The current configuration of HRS and MHAS data was not suitable for harmonization, as poor linking item quality led to large bias in both cohorts. Scenarios with more numerous and higher quality linking items led to less biased and more accurate harmonization.
Discussion: Linking items must possess low measurement error across the range of latent ability for co-calibration to be successful.
Highlights: We developed a statistical simulation platform to evaluate the degree to which cross-sample harmonization accuracy varies as a function of the quality and quantity of linking items.Two large studies of aging-one in Mexico and one in the United States-use three common items to measure cognition.These three common items have weak correspondence with the ability being measured and are all low in difficulty.Harmonized scores derived from the three common linking items will provide biased and inaccurate estimates of cognitive ability.Harmonization accuracy is greatest when linking items vary in difficulty and are strongly related to the ability being measured.
Keywords: bias; cognition; computer simulation; global health; item response theory.
© 2023 The Authors. Alzheimer's & Dementia: Diagnosis, Assessment & Disease Monitoring published by Wiley Periodicals, LLC on behalf of Alzheimer's Association.