Assessing Rationality in Discrete Choice Experiments in Health: An Investigation into the Use of Dominance Tests

Abstract

Background: Dominance tests are often applied to test for the rationality in the choice behavior of participants in discrete choice experiments (DCEs).

Objectives: To examine how dominance tests have been implemented in recent DCE applications in health and discuss their theoretical and empirical interpretation.

Methods: Health-related DCEs published in 2015 were reviewed for the inclusion of tests on choice behavior. For studies that implemented a dominance test, information on application and interpretation of the test was extracted. Authors were contacted for test choice sets and observed proportions of subjects who chose the dominated option. Coefficients corresponding to the choice set were extracted to estimate the expected probability of choosing the dominated option with a logistic model and compared with the observed proportion. The theoretical range of expected probabilities of possible dominance tests was calculated.

Results: Of 112 health-related DCEs, 49% included at least one test for choice behavior; 28 studies (25%) included a dominance test. The proportion of subjects in each study who chose the dominated option ranged from 0% to 21%. In 46% of the studies, the dominance test led to the exclusion of participants. In the 15 choice sets that were analyzed, 2 had larger proportions of participants choosing the dominated option than expected (P < 0.05).

Conclusions: Although dominance tests are frequently applied in DCEs, there is no consensus on how to account for them in data analysis and interpretation. Comparison of expected and observed proportions of participants failing the test might be indicative of DCE quality.

Keywords: discrete choice experiment; internal validity; preference; random utility theory.