Brief measures are important in psychology research because they reduce participant burden. Researchers can select items from longer measures either to build a short-form or to administer items conditional on a participant's previous responses. Researchers who carry out these item selection strategies either focus on estimating a precise score on the measure (typically carried out in a psychometric approach) or on predicting the score on the measure (possibly taking a machine learning approach). However, it is unclear how scores from the psychometric and machine learning approaches compare to each other. In this paper, the following four statistical approaches to select items are reviewed and illustrated: item response theory to build static short-forms, computerized adaptive testing, the genetic algorithm, and regression trees. Theoretical strengths and weaknesses between these four statistical approaches are discussed, and the overlap between the areas of psychometrics and machine learning is considered.
Keywords: Item response theory; machine learning; short-forms; tailored test.