Purpose: To be effective, evidence-based programs should be delivered as prescribed. This suggests that adaptations that deviate from intervention goals may limit a program's effectiveness. This study examines the impact that number and quality of adaptations have on substance use outcomes.
Design: We examined 306 video recordings of teachers delivering 'All Stars', a middle school drug prevention program. Multiple observers coded each recording, noting the number and type of adaptation each teacher made. Each adaptation was given a valence rating. Adaptations that were deleterious to program goals received negative valence ratings; positive ratings were given for adaptations that were likely to facilitate achievement of program goals; neutral ratings were given to adaptations that were expected to have neither a positive nor negative impact on program goals.
Findings: All teachers made adaptations. Teachers were consistent across time in the types of adaptations they made, suggesting each teacher has a personalized style of adapting. Those who made few adaptations, and whose average adaptation was rated as being positive had a higher percentage of students who remained non-drug users. In contrast, teachers who made many adaptations, whether their average valence rating was positive, neutral or negative, failed to have as many students remain non-drug users. Measures of fidelity, including quality of delivery and teacher understanding were related to valence of adaptations, with better performance related to making positive adaptations.
Practical implications: Through training and supervision, teachers should be guided and encouraged to follow programs directions, making few adaptations and ensuring that adaptations that are made advance the goals of intervention. Programs should define acceptable and unacceptable ways they may be adapted.
Value: This study provides significant evidence about the challenges that face disseminated evidence-based programs.
Keywords: adaptation; dissemination; evaluation; fidelity; prevention; program; project; substance abuse.