How strong should my anchor be for estimating group and individual level meaningful change? A simulation study assessing anchor correlation strength and the impact of sample size, distribution of change scores and methodology on establishing a true meaningful change threshold

Abstract

Purpose: Treatment benefit as assessed using clinical outcome assessments (COAs), is a key endpoint in many clinical trials at both the individual and group level. Anchor-based methods can aid interpretation of COA change scores beyond statistical significance, and help derive a meaningful change threshold (MCT). However, evidence-based guidance on the selection of appropriately related anchors is lacking.

Methods: A simulation was conducted which varied sample size, change score variability and anchor correlation strength to assess the impact of these variables on recovering the simulated MCT for interpreting individual and group-level results. To assess MCTs derived at the individual-level (i.e. responder definitions; RDs), Receiver Operating Characteristic (ROC) curves and Predictive Modelling (PM) analyses were conducted. To assess MCTs for interpreting change at the group-level, the mean change method was conducted.

Results: Sample sizes, change score variability and magnitude of anchor correlation affected accuracy of the estimated MCT. For individual-level RDs, ROC curves were less accurate than PM methods at recovering the true MCT. For both methods, smaller samples led to higher variability in the returned MCT, but higher variability still using ROC. Anchors with weaker correlations with COA change scores had increased variability in the estimated MCT. An anchor correlation of around 0.50-0.60 identified a true MCT cut-point under certain conditions using ROC. However, anchor correlations as low as 0.30 were appropriate when using PM under certain conditions. For interpreting group-level results, the MCT derived using the mean change method was consistently underestimated regardless of the anchor correlation.

Conclusion: Sample size and change score variability influence the necessary anchor correlation strength when recovering individual-level RDs. Often, this needs to be higher than the commonly accepted threshold of 0.30. Stronger correlations than 0.30 are required when using the mean change method. Results can assist researchers selecting and assessing the quality of anchors.

Keywords: Anchor correlation; Anchor relationship; Meaningful change threshold; Minimal important change.