Purpose: To better understand the European Society for Medical Oncology-Magnitude of Clinical Benefit Scale version 1.1 (ESMO-MCBS v1.1) and the ASCO Value Framework Net Health Benefit score version 2 (ASCO-NHB v2), ESMO and ASCO collaborated to evaluate the concordance between the frameworks when used to assess clinical benefit attributable to new therapies.
Methods: The 102 randomized controlled trials in the noncurative setting already evaluated in the field testing of ESMO-MCBS v1.1 were scored using ASCO-NHB v2 by its developers. Measures of agreement between the frameworks were calculated and receiver operating characteristic curves used to define thresholds for the ASCO-NHB v2 corresponding to ESMO-MCBS v1.1 categories. Studies with discordant scoring were identified and evaluated to understand the reasons for discordance.
Results: The correlation of the 102 pairs of scores for studies in the noncurative setting is estimated to be 0.68 (Spearman's rank correlation coefficient; overall survival, 0.71; progression-free survival, 0.67). Receiver operating characteristic curves identified thresholds for ASCO-NHB v2 for facilitating comparisons with ESMO-MCBS v1.1 categories. After applying pragmatic threshold scores of 40 or less (ASCO-NHB v2) and 2 or less (ESMO-MCBS v1.1) for low benefit and 45 or greater (ASCO-NHB v2) and 4 to 5 (ESMO-MCBS v1.1) for substantial benefit, 37 discordant studies were identified. Major factors that contributed to discordance were different approaches to evaluation of relative and absolute gain for overall survival and progression-free survival, crediting tail of the curve gains, and assessing toxicity.
Conclusion: The agreement between the frameworks was higher than observed in other studies that sought to compare them. The factors that contributed to discordant scores suggest potential approaches to improve convergence between the scales.