The paradigm shift towards precision medicine reignited interest in determining whether there are differential treatment effects in subgroups of trial participants. Intrinsic to this problem is that any assessment of a differential treatment effect is predicated on being able to estimate the treatment response accurately while satisfying constraints of balancing the risk of overlooking an important subgroup with the potential to make a decision based on a false discovery. While shrinkage models have been widely used to improve accuracy of subgroup parameter estimates by leveraging the relationship between them, there is still a possibility that it can lead to excessively conservative or anti-conservative results. This can possibly be due to the use of the normal distribution as prior, which forces outlying subjects to have their means over-shrunk towards the population mean, and the data from such subjects may be excessively influential in estimation of both the overall mean response and the mean response for each subgroup, or a model misspecification due to unaccounted variation or clustering. To address this issue, we investigate the use of nonparametric Bayes, particularly Dirichlet process priors, to create a flexible shrinkage model. This model represents uncertainty in the prior distribution for the overall response while accommodating heterogeneity among individual subgroups. We simulated data to compare estimates when there is no differential subgroup effect and when there is a differential subgroup effect. In either of these scenarios, the flexible shrinkage model does not force estimates to shrink excessively when similarity of treatment effects is not supported but still retains the attractiveness of improved precision given by the narrower credible intervals. We also applied the same method to a dataset based on trials conducted for an antimicrobial therapy on several related indications.
Keywords: Dirichlet process prior; exchangeability; heterogeneity; model selection; shrinkage; subgroups.