[How robust are studies of currently permanently included digital health applications (DiGA)? Methodological quality of studies demonstrating positive health care effects of DiGA]

Abstract

Introduction: Since September 2020 digital health applications (DiGA) can be prescribed by physicians and psychotherapists and are reimbursed within the German Statutory Health Insurance (SHI) system for the first time worldwide. For full reimbursement, the manufacturers have to provide evidence based on scientific studies that the DiGA can provide 'positive health care effects'. This study aims to analyze and evaluate the methodological quality of efficacy studies of DiGA in the categories 'Nervensystem' and 'Psyche' of the DiGA register that are permanently accepted.

Methods: The methodological quality was assessed using the revised Cochrane risk-of-bias tool for randomized trials (RoB 2). The risk of bias was assessed for the primary endpoint of each study according to an intention-to-treat analysis.

Results: Six DiGA were assessed for their methodological quality. Randomized controlled trials were conducted for all 6 DiGA that showed a high risk of bias, which was, in particular, due to a lack of blinding of the studies. In addition, drop-outs were significantly higher in the intervention group than in the control group in most studies. For most of the DiGA no published study protocol was available in advance so an analysis of a potential selective choice of the evaluation methodology was not possible.

Discussion: For reasons of transparency, verifiability, and comprehensibility of the study results, registration in a study registry and, more importantly, the publication of study protocols should be mandatory before the start of the studies. In addition, studies should be blinded by comparing the DiGA with a 'sham application' to reduce the high risk of bias. Differences in the drop-out rates of the investigated studies could indicate a lack of efficacy of the treatment in the intervention group, (technical) problems in the application of the DiGA, or a lack of motivation of the participants.

Conclusion: The interim results 18 months after the introduction of DiGA in the German SHI system show that the studies on the evidence of the benefits of DiGA have a high potential for bias in certain areas. However, it should be positively emphasized that the manufacturers submitted randomized controlled trials to prove the medical benefit of the DiGAs investigated.

Keywords: DiGA; Digital health applications; Digitale Gesundheitsanwendungen; Gesundheitstechnologie-Bewertung; Health Technology Assessment; Health technology assessment; Methodological quality; Methodologische Qualität; Risk of bias; Verzerrungsrisiko.