Purpose: Factorial designs may be proposed to test extra questions within a clinical trial. A common approach to sample size and analysis for factorial trials assumes no statistical interactions and does not adjust for multiple testing. This investigation considered the trade-off between potential gains from testing more questions with fewer patients versus how often a factorial trial might arrive at an incorrect conclusion.
Methods: A simulation study of a 2 x 2 design (observation v chemotherapy v radiation therapy v the combination) was performed under various conditions, including effect of one, both, or neither treatment and absence or presence of statistical interaction (effect of one treatment differed according to the presence of the other). Three analysis approaches were investigated, one assuming no interaction, a second testing first for interaction, and the third testing for interaction as well as adjusting for multiple testing. The approaches were compared with respect to the probability of selecting the correct treatment arm.
Results: No one approach was superior. Testing for interaction was beneficial in some settings but detrimental in others. Under some scenarios, the factorial design improved efficiency, but under others, all three approaches resulted in poor probability of selecting the correct treatment arm at the end of the trial.
Conclusion: Extra efficiency is possible, but it is difficult to predict when favorable conditions exist. If a factorial design is used, potential efficiency gains should be weighed against potential loss of power to arrive at the correct conclusion under possible scenarios of interest.