Semi-automated Rasch analysis using in-plus-out-of-questionnaire log likelihood

Abstract

Rasch analysis is a popular statistical tool for developing and validating instruments that aim to measure human performance, attitudes and perceptions. Despite the availability of various software packages, constructing a good instrument based on Rasch analysis is still considered to be a complex, labour-intensive task, requiring human expertise and rather subjective judgements along the way. In this paper we propose a semi-automated method for Rasch analysis based on first principles that reduces the need for human input. To this end, we introduce a novel criterion, called in-plus-out-of-questionnaire log likelihood (IPOQ-LL). On artificial data sets, we confirm that optimization of IPOQ-LL leads to the desired behaviour in the case of multi-dimensional and inhomogeneous surveys. On three publicly available real-world data sets, our method leads to instruments that are, for all practical purposes, indistinguishable from those obtained by Rasch analysis experts through a manual procedure.

Keywords: generalized partial credit model; penalized JMLE; rasch model; semi-automated rasch analysis.