Statistical approaches to identifying lapses in psychometric response data

Abstract

Psychometric curve fits relate physical stimuli to an observer's performance. In experiments an observer may "lapse" and respond with a random guess, which may negatively impact (e.g., bias) the psychometric fit parameters. A lapse-rate model has been popularized by Wichmann and Hill, which reduces the impact of lapses on other estimated parameters by adding a parameter to model the lapse rate. Since lapses are discrete events, we developed a discrete lapse theory and tested a "lapse identification" algorithm to identify individual outlier trials (i.e., potential lapses) based upon an approximate statistical criterion and discard these trials. Specifically, we focused on stimuli sampled using an adaptive staircase for a one-interval, direction-recognition task (i.e., psychometric function ranging from 0 to 1 and the spread of the curve corresponds to the threshold, which is often a parameter of interest for many fitted psychometric functions). Through simulations, we found that as the lapse rate increased the threshold became substantially overestimated, consistent with earlier analyses. While the lapse-rate model reduced the overestimation of threshold with many lapses, with lower lapse rates it yielded substantial threshold underestimation, though less so when fitting many (e.g., 1,000) trials. In comparison, the lapse-identification algorithm yielded accurate threshold estimates across a wide range of lapse rates (from 0 to 5%), which is critical since the lapse rate is seldom known. We further demonstrate the performance of the lapse-identification algorithm to be suitable for a variety of experimental conditions and conclude with some considerations of its use. In particular, we suggest using the lapse-identification algorithm unless the experiment has many trials (e.g., >500) or if somehow the lapse rate is known to be high (e.g., ≥5%), for which the lapse-rate model approaches remain preferred.

Keywords: Bayesian inference and parameter estimation; Judgment and decision making; Signal detection theory.