Using ranked survey data in education research: Methods and applications

Abstract

Researchers and practitioners in education and school psychology regularly use ranked data to drive student- and systems-level decision-making. These types of data can be derived from assessments of individual preferences among researchers and practitioners, assessments of preferences among stakeholders including parents and children, and rankings of students on academic and social-emotional competency. However, the analysis of ranked data in education and psychology has typically been limited to simple approaches such as the examination of mean ranks assigned to items. This paper unifies a collection of classical methodologies, as well as proposes new techniques, for analyzing ranked data used across disciplines and applies the methods to data generated in school psychological research. The scope of the paper is to serve as a roadmap for researchers in education and school psychology who seek to more fully leverage information contained in ranked data. These methodologies include descriptive analyses, visualizations, tests of uniformity, cluster analyses, and predictive models. We demonstrate these techniques on the survey data of Fefer, DeMagistris, and Shuttleton (2016) and illustrate how using a broader set of tools can yield improved insights by researchers and practitioners.

Keywords: Cluster analysis; Ranked data; Ranked data visualization; Survey data; Tests of uniformity.