Objectives: To develop dissemination summaries based on AHRQ EPC evidence reviews, and to evaluate the summaries, and their components, for usability among health systems decisionmakers.
Methods: For each of two reviews, we designed a three-page summary with the intent that the first page could potentially be a stand-alone one-page summary. Summaries included various report elements (e.g., key questions, methods, eligibility criteria, analytic framework, summary tables, forest plots) and varied in layout and design (e.g., text only vs. use of images). We conducted ‘think aloud’ telephone interviews with six decisionmakers from Strategic Clinical Networks, which are clinician-led teams that develop and implement evidence-informed health improvement strategies within a province-wide fully integrated health system. We analyzed the data thematically and translated themes into recommendations.
Results: Interviews yielded information on content, layout and design, and usability. With respect to content, themes included purpose, methods, summary data, and overall message. Recommendations regarding content were to: include a purpose statement and define the intended audience; discuss the methods briefly; when possible, provide data to support the findings (e.g., summary estimates with confidence intervals and strength of evidence, use standard well-organized tables and figures); state the overall message on the first page (include key findings, knowledge gaps, and take home message(s)). For layout and design, themes included readability, visual appeal, figures, tables, and credibility. Recommendations were to: use brief titles and subheadings; avoid acronyms; organize content logically; balance text with images and white space; use standard figures to present findings and when feasible, use figures instead of tables; use standard tables to present findings and when feasible, use tables instead of text; include the organization and program logos; and employ professional design choices. Themes regarding usability included language, length, and content. Recommendations were to: use plain language consistently; keep the summary to three pages or less; provide a link to the full text; organize the findings for maximum use; and consider usability in greyscale.
Conclusions: Decisionmakers preferred a three-page summary that presented key messages, details on results, and strength of evidence; detailed methods and contextual information were less important. Decisionmakers preferred summaries with use of meaningful images, graphs, figures, and tables to convey information. Decisionmakers thought the summaries would be foundational for other knowledge translation products such as clinical pathways.