Objectives: To describe methods used to engage stakeholders to prioritize future research needs from Comparative Effectiveness Reviews.
Data Sources: There were three complementary phases in this project: Phase 1: Literature Scan; Phases 2 and 3: Interviews with key informants (KIs) and Evidence-based Practice Centers (EPCs) with experience engaging stakeholders.
Methods: Phase 1: We searched and reviewed the literature from inception until August/September 2010 to identify methods used to engage stakeholders in identifying and prioritizing research. Phase 2: Between June 16 and July 9, 2010, KIs were interviewed about their experiences engaging stakeholders and the processes used to set research priorities. Investigators analyzed interview transcripts to identify common themes. Phase 3: The experiences of EPCs in engaging stakeholders to identify research gaps and prioritize future research needs were evaluated by: (1) review of EPC protocols and (2) semistructured interviews with EPC directors and staff.
Results: Phase 1: Fifty-six studies were identified. Important considerations in stakeholder engagement included the need for consistent terminology, the intended purpose for engagement, the explicit identification of stakeholder groups, and the distribution of stakeholders. Studies frequently used a mixed-methods approach for research prioritization, combining in-person venues with quantitative prioritization processes such as voting or Delphi. Phase 2: We conducted 13 interviews. KIs used e-mail, conference calls, focus groups, and the modified Delphi technique to engage stakeholders. Processes for prioritization ranged from no formal process to a structured process that consisted of multiple rounds of voting/ranking. Phase 3: Eight EPCs were interviewed. Group conference calls were the most common approach of stakeholder engagement, along with e-mail or Web-based prioritization. EPCs routinely identified three main challenges, including timing, restriction on number of stakeholders, and limited availability of Federal stakeholders.
Conclusions: Important considerations regarding stakeholder engagement to prioritize research include use of:
Consistent terminology and definitions throughout the process.
In-person methods for brainstorming, identifying topics, clarifying issues, and eliciting a deeper understanding.
Quantitative methods for prioritizing research.