A review of NICE appraisals of pharmaceuticals 2000-2016 found variation in establishing comparative clinical effectiveness

Abstract

Objective: To identify and assess the methods for estimating comparative clinical effectiveness for novel pharmaceutical products licensed on the basis of nonrandomized controlled trial (non-RCT) data and to evaluate the corresponding National Institute for Health and Care Excellence (NICE) recommendations.

Methods: Our identification strategy was twofold. First, we reviewed all NICE appraisals between 2010 and 2016 and identified technologies where comparative clinical effectiveness estimates were calculated using non-RCT data. Second, we checked if NICE appraisals completed from 2000 to 2010 had included pharmaceuticals that were granted European Medicines Agency marketing authorization without RCT data between 1999 and 2014. Information was extracted on the method used to establish comparative clinical effectiveness as well as the corresponding NICE recommendations. We also collected information on the rationale for utilizing non-RCT data in NICE appraisals.

Results: Of 489 individual pharmaceutical technologies assessed by NICE, 22 (4%) used non-RCT data to estimate comparative clinical effectiveness. Methods for establishing external controls in such studies varied: 13 (59%) used published trials, 6 (27%) used observational data, 2 (9%) used expert opinion, and 1 (5%) used a responder vs nonresponder analysis. Only 5 (23%) used a regression model to adjust for covariates. We did not observe a notable difference in the proportion of pharmaceutical technologies that received a positive recommendation from NICE whether the decision was based on RCT or non-RCT data (83% vs 86%).

Conclusions: To date, a small number of appraisals by NICE based on non-RCT data did not result in substantially different treatment decisions. The majority of the technologies appraised on the basis of non-RCT data either received a positive recommendation or a positive recommendation with restrictions. The methods used to calculate comparative clinical effectiveness estimates varied, highlighting the need to establish clear guidance.

Keywords: Drug approval; National Institute of Health and Care Excellence; Nonrandomized studies; Quality of evidence; Single-arm trials; Uncontrolled trial.