General Pairwise Comparison (GPC) statistics, such as the net benefit and the win ratio, have been applied in clinical trial data analysis and design. In the literature, inferential methods based on re-sampling, asymptotic or exact methods have been proposed for these GPC statistics, but they have not been compared to each other. In this paper, the small sample bias of the variance estimation, Type I error control and 95% confidence interval coverage of the GPC inferential methods are evaluated using simulations. The exact permutation and bootstrap tests perform best in all evaluated aspects for the net benefit, while the exact bootstrap test performs best for the win ratio.
Keywords: General Pairwise Comparisons; bootstrap; net benefit; permutation; statistical inference; win Ratio.