Designing and evaluating complex educational research to meet federal grant requirement: A demonstration of a funded large-scale protocol

Abstract

Federal funding agencies have invested significant resources supporting evidence-based, innovative approaches that address education problems and incorporate rigorous design and evaluation, particularly through randomized controlled trials (RCTs), the gold standard for yielding causal inference in scientific research. In this study, we introduced factors (i.e., evaluation design, attrition, outcome measures, analytic approach, and fidelity of implementation) that are often times required in the Federal Notice for application by the U.S. Department of Education, with an emphasis on What Works Clearinghouse (WWC) standards. We further presented a federally funded research protocol with a multi-year, clustered RCT design to determine the impact of an instructional intervention on students' academic performance in high-needs schools. In the protocol, we elaborated on how our research design, evaluation plan, power analysis, as well as confirmatory research questions and analytical approaches were aligned with the grant requirement and WWC standards. We intend to provide a road map to meeting WWC standards and to increase the likelihood of successful grant applications.

Keywords: Clustered longitudinal design; Federal grants; Meeting WWC standards with/without reservations; Project evaluation; Randomized controlled trial (RCT); What Works Clearinghouse (WWC) standards.