Evaluating Equivalence Testing Methods for Measurement Invariance

Multivariate Behav Res. 2020 Mar-Apr;55(2):312-328. doi: 10.1080/00273171.2019.1633617. Epub 2019 Aug 7.

Abstract

Measurement Invariance (MI) is often concluded from a nonsignificant chi-square difference test. Researchers have also proposed using change in goodness-of-fit indices ([Formula: see text]GOFs) instead. Both of these commonly used methods for testing MI have important limitations. To combat these issues, To combat these issues, it was proposed using an equivalence test (EQ) to replace the chi-square difference test commonly used to test MI. Due to concerns with the EQ's power, and adjusted version (EQ-A) was created, but provides little evaluation of either procedure. The current study evaluated the Type I error and power of both the EQ and EQ-A, and compared their performance to that of the traditional chi-square difference test and [Formula: see text]GOFs. The EQ was the only procedure that maintained empirical error rates below the nominal alpha level. Results also highlight that the EQ requires larger sample sizes than traditional difference-based approaches or using equivalence bounds based on larger than conventional RMSEA values (e.g., > .05) to ensure adequate power rates. We do not recommend the proposed adjustment (EQ-A) over the EQ.

Keywords: Equivalence tests; confirmatory factor analysis; measurement invariance; model comparison; structural equation modeling.

MeSH terms

  • Behavioral Research / methods*
  • Chi-Square Distribution
  • Computer Simulation
  • Data Interpretation, Statistical*
  • Humans
  • Models, Statistical*