Use of guidelines, checklists, frameworks, and recommendations in behavioral intervention preliminary studies and associations with reporting comprehensiveness: a scoping bibliometric review

Christopher D Pfledderer; Lauren von Klinggraeff; Sarah Burkart; Alexsandra da Silva Bandeira; Bridget Armstrong; R Glenn Weaver; Elizabeth L Adams; Michael W Beets

doi:10.1186/s40814-023-01389-w

Use of guidelines, checklists, frameworks, and recommendations in behavioral intervention preliminary studies and associations with reporting comprehensiveness: a scoping bibliometric review

Pilot Feasibility Stud. 2023 Sep 13;9(1):161. doi: 10.1186/s40814-023-01389-w.

Authors

Christopher D Pfledderer¹, Lauren von Klinggraeff², Sarah Burkart², Alexsandra da Silva Bandeira², Bridget Armstrong², R Glenn Weaver², Elizabeth L Adams², Michael W Beets²

Affiliations

¹ Department of Health Promotion and Behavioral Sciences, The University of Texas Health Science Center at Houston, School of Public Health Austin Campus, Austin, TX, 78701, USA. christopher.d.pfledderer@uth.tmc.edu.
² Arnold School of Public Health, University of South Carolina, 921 Assembly Street, Columbia, SC, 29205, USA.

Abstract

Background: Guidelines, checklists, frameworks, and recommendations (GCFRs) related to preliminary studies serve as essential resources to assist behavioral intervention researchers in reporting findings from preliminary studies, but their impact on preliminary study reporting comprehensiveness is unknown. The purpose of this study was to conduct a scoping bibliometric review of recently published preliminary behavioral-focused intervention studies to (1) examine the prevalence of GCFR usage and (2) determine the associations between GCFR usage and reporting feasibility-related characteristics.

Methods: A systematic search was conducted for preliminary studies of behavioral-focused interventions published between 2018 and 2020. Studies were limited to the top 25 journals publishing behavioral-focused interventions, text mined to identify usage of GCFRs, and categorized as either not citing GCFRs or citing ≥ 2 GCFRs (Citers). A random sample of non-Citers was text mined to identify studies which cited other preliminary studies that cited GCFRs (Indirect Citers) and those that did not (Never Citers). The presence/absence of feasibility-related characteristics was compared between Citers, Indirect Citers, and Never Citers via univariate logistic regression.

Results: Studies (n = 4143) were identified, and 1316 were text mined to identify GCFR usage (n = 167 Citers). A random sample of 200 studies not citing a GCFR were selected and categorized into Indirect Citers (n = 71) and Never Citers (n = 129). Compared to Never Citers, Citers had higher odds of reporting retention, acceptability, adverse events, compliance, cost, data collection feasibility, and treatment fidelity (OR_range = 2.62-14.15, p < 0.005). Citers also had higher odds of mentioning feasibility in purpose statements, providing progression criteria, framing feasibility as the primary outcome, and mentioning feasibility in conclusions (OR_range = 6.31-17.04, p < 0.005) and lower odds of mentioning efficacy in purpose statements, testing for efficacy, mentioning efficacy in conclusions, and suggesting future testing (ORrange = 0.13-0.54, p < 0.05). Indirect Citers had higher odds of reporting acceptability and treatment fidelity (OR_range = 2.12-2.39, p < 0.05) but lower odds of testing for efficacy (OR = 0.36, p < 0.05) compared to Never Citers.

Conclusion: The citation of GCFRs is associated with greater reporting of feasibility-related characteristics in preliminary studies of behavioral-focused interventions. Researchers are encouraged to use and cite literature that provides guidance on design, implementation, analysis, and reporting to improve the comprehensiveness of reporting for preliminary studies.

Keywords: Feasibility; Framework; Pilot; Preliminary studies; Reporting quality; Review.

Publication types

Review

Abstract

Publication types

Grants and funding