A cost-effectiveness analysis of four approaches to universal screening for reading risk in upper elementary and middle school

Abstract

Research on universal screening in reading has primarily focused on the psychometric adequacy of screening procedures without critically considering costs and value. Educators in upper elementary and middle school have access to a great deal of extant student achievement data, which makes the evaluation of the costs associated with collecting new data for screening purposes paramount. We conducted a retrospective analysis of four approaches to reading screening using cost-effectiveness analysis. Universal screening (i.e., aimswebPlus reading) and statewide reading test data were collected in a midsize school district in Texas for 19,417 students in Grades 4-8. We analyzed the total cost, the classification accuracy based on local cut-scores derived using receiver operating characteristic analysis, and the cost-effectiveness of each approach. Full implementation of the fall administration of aimswebPlus reading cost the district $55,199 in upper elementary and $76,832 in middle school. Both the use of prior-year state achievement test data alone (which represented no additional costs for screening) and the multivariate model (i.e., prior-year state achievement test data plus aimswebPlus) met typical recommendations for classification accuracy. A multiple-gate model that used prior-year state test data and fall aimswebPlus reading on a low-achieving subset of students produced the lowest cost-effectiveness ratio at $156 per additional student accurately classified in upper elementary and $181 in middle school, but this model also led to the highest number of false negatives. The use of aimswebPlus alone resulted in the highest costs and lowest classification accuracy. The results suggest that the use of prior-year statewide achievement test data alone in Grades 4-8 is an efficient approach to universal screening for reading risk that may allow schools to shift resources from screening to other educational priorities.

Keywords: Academic screening; Cost-effectiveness; Curriculum-based measure; Economic evaluation; Universal screening.