Using a Generalized Logistic Regression Method to Detect Differential Item Functioning With Multiple Groups in Cognitive Diagnostic Tests

Xiaojian Sun; Shimeng Wang; Lei Guo; Tao Xin; Naiqing Song

doi:10.1177/01466216231174559

Using a Generalized Logistic Regression Method to Detect Differential Item Functioning With Multiple Groups in Cognitive Diagnostic Tests

Appl Psychol Meas. 2023 Jun;47(4):328-346. doi: 10.1177/01466216231174559. Epub 2023 May 13.

Authors

Xiaojian Sun^{1

2}, Shimeng Wang³, Lei Guo⁴, Tao Xin⁵, Naiqing Song^{1

2}

Affiliations

¹ School of Mathematics and Statistics, Southwest University, Chongqing, China.
² Southwest University Branch, Collaborative Innovation Center of Assessment for Basic Education Quality, Chongqing, China.
³ High School Affiliated to Southwest University, Chongqing, China.
⁴ Faculty of Psychology, Southwest University, Chongqing, China.
⁵ Collaborative Innovation Center of Assessment for Basic Education Quality, Beijing Normal University, Beijing, China.

Abstract

Items with the presence of differential item functioning (DIF) will compromise the validity and fairness of a test. Studies have investigated the DIF effect in the context of cognitive diagnostic assessment (CDA), and some DIF detection methods have been proposed. Most of these methods are mainly designed to perform the presence of DIF between two groups; however, empirical situations may contain more than two groups. To date, only a handful of studies have detected the DIF effect with multiple groups in the CDA context. This study uses the generalized logistic regression (GLR) method to detect DIF items by using the estimated attribute profile as matching criteria. A simulation study is conducted to examine the performance of the two GLR methods, GLR-based Wald test (GLR-Wald) and GLR-based likelihood ratio test (GLR-LRT), in detecting the DIF items, the results based on the ordinary Wald test are also reported. Results show that (1) both GLR-Wald and GLR-LRT have more reasonable performance in controlling Type I error rates than the ordinary Wald test in most conditions; (2) the GLR method also produces higher empirical rejection rates than the ordinary Wald test in most conditions; and (3) using the estimated attribute profile as the matching criteria can produce similar Type I error rates and empirical rejection rates for GLR-Wald and GLR-LRT. A real data example is also analyzed to illustrate the application of these DIF detection methods in multiple groups.

Keywords: cognitive diagnostic assessment; differential item functioning; generalized logistic regression; multiple groups.