Objective: Survival extrapolation using a single, best-fit model ignores 2 sources of model uncertainty: uncertainty in the true underlying distribution and uncertainty about the stability of the model parameters over time. Bayesian model averaging (BMA) has been used to account for the former, but it can also account for the latter. We investigated BMA using a published comparison of the Charnley and Spectron hip prostheses using the original 8-year follow-up registry data.
Methods: A wide variety of alternative distributions were fitted. Two additional distributions were used to address uncertainty about parameter stability: optimistic and skeptical. The optimistic (skeptical) model represented the model distribution with the highest (lowest) estimated probabilities of survival but reestimated using, as prior information, the most optimistic (skeptical) parameter estimated for intermediate follow-up periods. Distributions were then averaged assuming the same posterior probabilities for the optimistic, skeptical, and noninformative models. Cost-effectiveness was compared using both the original 8-year and extended 16-year follow-up data.
Results: We found that all models obtained similar revision-free years during the observed period. In contrast, there was variability over the decision time horizon. Over the observed period, we detected considerable uncertainty in the shape parameter for Spectron. After BMA, Spectron was cost-effective at a threshold of £20,000 with 93% probability, whereas the best-fit model was 100%; by contrast, with a 16-year follow-up, it was 0%.
Conclusions: This case study casts doubt on the ability of the single best-fit model selected by information criteria statistics to adequately capture model uncertainty. Under this scenario, BMA weighted by posterior probabilities better addressed model uncertainty. However, there is still value in regularly updating health economic models, even where decision uncertainty is low.
Keywords: Bayesian statistical methods; Markov models; acceptability curves; cost-effectiveness analysis; decision analysis; econometric methods; provider decision making; survival analysis.