Development and design validation of a novel network meta-analysis presentation tool for multiple outcomes: a qualitative descriptive study

Mark R Phillips; Behnam Sadeghirad; Jason W Busse; Romina Brignardello-Petersen; Carlos A Cuello-Garcia; Fernando Kenji Nampo; Yu Jia Guo; Sofia Bzovsky; Raveendhara R Bannuru; Lehana Thabane; Mohit Bhandari; Gordon H Guyatt

doi:10.1136/bmjopen-2021-056400

Development and design validation of a novel network meta-analysis presentation tool for multiple outcomes: a qualitative descriptive study

BMJ Open. 2022 Jun 10;12(6):e056400. doi: 10.1136/bmjopen-2021-056400.

Authors

Affiliations

¹ Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada phillimr@mcmaster.ca.
² Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada.
³ Anesthesia, McMaster University, Hamilton, Ontario, Canada.
⁴ Department of Latin-American Institute of Life and Nature science, Federal University of Latin-American Integration, Foz do Iguacu, Brazil.
⁵ Health Sciences, McMaster University, Hamilton, Ontario, Canada.
⁶ Department of Surgery - Division of Orthopaedics, McMaster University, Hamilton, Ontario, Canada.
⁷ Center for Treatment Comparison and Integrative Analysis, Tufts Medical Center, Boston, Massachusetts, USA.
⁸ Biostatistics Unit, St. Joseph's Healthcare, Hamilton, Ontario, Canada.
⁹ Division of Orthopaedic Surgery, Mcmaster University, Hamilton, Ontario, Canada.

Abstract

Objective: The Grades of Recommendations, Assessment, Development and Evaluation working group recently developed an innovative approach to interpreting results from network meta-analyses (NMA) through minimally and partially contextualised methods; however, the optimal method for presenting results for multiple outcomes using this approach remains uncertain. We; therefore, developed and iteratively modified a presentation method that effectively summarises NMA results of multiple outcomes for clinicians using this new interpretation approach.

Design: Qualitative descriptive study.

Setting: A steering group of seven individuals with experience in NMA and design validation studies developed two colour-coded presentation formats for evaluation. Through an iterative process, we assessed the validity of both formats to maximise their clarity and ease of interpretation.

Participants: 26 participants including 20 clinicians who routinely provide patient care, 3 research staff/research methodologists and 3 residents.

Main outcome measures: Two team members used qualitative content analysis to independently analyse transcripts of all interviews. The steering group reviewed the analyses and responded with serial modifications of the presentation format.

Results: To ensure that readers could easily discern the benefits and safety of each included treatment across all assessed outcomes, participants primarily focused on simple information presentations, with intuitive organisational decisions and colour coding. Feedback ultimately resulted in two presentation versions, each preferred by a substantial group of participants, and development of a legend to facilitate interpretation.

Conclusion: Iterative design validation facilitated the development of two novel formats for presenting minimally or partially contextualised NMA results for multiple outcomes. These presentation approaches appeal to audiences that include clinicians with limited familiarity with NMAs.

Keywords: EDUCATION & TRAINING (see Medical Education & Training); QUALITATIVE RESEARCH; STATISTICS & RESEARCH METHODS.

Publication types

Meta-Analysis

MeSH terms

Humans
Network Meta-Analysis
Qualitative Research
Research Design*