Assessment of treatment response in psychotherapies can be undermined by lack of longitudinal measurement invariance (LMI) in symptom self-report inventories, by measurement error, and/or by wrong model assumptions. To understand and compare these threats to validity of outcome assessment in psychotherapy research, we studied LMI, sum scores, and Davidian Curve Item Response Theory models in a naturalistic guided internet psychotherapy treatment register of 2,218 generalized anxiety disorder (GAD) patients and 3,922 depressive disorder (DD) patients (aged ≥16 years). Symptoms were repeatedly assessed by Generalized Anxiety Disorder Assessment-7 (GAD-7) or Beck Depression Inventory. The symptom self-reports adhered to LMI under equivalence testing, suggesting sum scores are reasonable proxies for disorder status. However, the standard LMI assumption of normally distributed latent factors did not hold and inflated treatment response estimates by 0.2 to 0.3 standard deviation units compared with sum scores. Further methodological research on non-normally distributed latent constructs holds promise in advancing LMI and mental health assessment.
Keywords: equivalence testing; internet-delivered cognitive behavioral therapy; longitudinal measurement invariance; mood disorders; response shifts; unipolar trait theory.