The difference between statistical significance and clinical relevance. The case of minimal important change, non-inferiority trials, and smallest worthwhile effect

Abstract

Clinical relevance and statistical significance are different concepts, linked via the sample size calculation. Threshold values for detecting a minimal important change over time are frequently (mis)interpreted as a threshold for the clinical relevance of a difference between groups. The magnitude of a difference between groups that is considered clinically relevant directly impacts the sample size calculation, and thereby the statistical significance in clinical study outcomes. Especially in non-inferiority trials the threshold for clinical relevance, i.e. the predefined margin for non-inferiority, is a crucial choice. A truly inferior treatment will be accepted as non-inferior when this margin is chosen too large. The magnitude of a clinically relevant difference between groups should be carefully considered, by determining the smallest effect for each specific study that is considered worthwhile. This means taking into account the (dis)advantages of both study interventions in terms of benefits, harms, costs, and potential side effects. This article clarifies common sources of confusion, illustrates the implications for clinical research with an example and provides specific suggestions to improve the design and interpretation of clinical research.

Keywords: Clinical relevance; Minimal important change; Minimal important difference; Non-inferiority; Smallest worthwhile effect; Statistical significance.