The Role of Noncomparative Evidence in Health Technology Assessment Decisions

Abstract

Background: Many health technology assessment (HTA) agencies express a preference for randomized controlled trial evidence when appraising health technologies; nevertheless, it is not always feasible or ethical to conduct such comparative trials.

Objectives: To assess the role of noncomparative evidence in HTA decision making.

Methods: The Web sites of the National Institute for Health and Care Excellence (NICE) in the United Kingdom, the Canadian Agency for Drugs and Technologies in Health (CADTH) in Canada, and the Institute for Quality and Efficiency in Health Care (Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen [IQWiG]) in Germany were searched for single HTA reports (published between January 2010 and December 2015). The product, indication, outcome, and clinical evidence presented (comparative/noncomparative) were double-extracted, with any discrepancies reconciled. A noncomparative study was defined as any study not presenting results against another treatment (including placebo or best supportive care), regardless of phase or setting, including dose-ranging studies.

Results: A total of 549 appraisals were extracted. Noncomparative evidence was considered in 38% (45 of 118) of NICE submissions, 13% (34 of 262) of CADTH submissions, and 12% (20 of 169) of IQWiG submissions. Evidence submissions based exclusively on noncomparative evidence were presented in only 4% (5 of 118) of NICE appraisals, 6% (16 of 262) of CADTH appraisals, and 4% (6 of 169) of IQWiG appraisals. Most drugs appraised solely on the basis of noncomparative evidence were indicated for cancer or hepatitis C. Positive outcome rates (encompassing recommended/restricted/added-benefit decisions) for submissions presenting only noncomparative evidence were similar to overall recommendation rates for CADTH (69% vs. 68%, respectively), but were numerically lower for NICE (60% vs. 84%, respectively) and IQWiG (17% vs. 38%, respectively) (P > 0.05 for all).

Conclusions: Noncomparative studies can be viewed as acceptable clinical evidence by HTA agencies when these study designs are justifiable and when treatment effect can be convincingly demonstrated, but their use is currently limited.

Keywords: clinical effectiveness; decision making; evidence-based medicine; health technology assessment.