Sample-based preclinical drug efficacy studies compare frequency (proportion) or incidences of successes within respective samples of test and control groups. The word success in principle refers to a protected (e.g., due to vaccination), recovered, or surviving animal, depending on the particular experiment. We introduce here a modified t-test for two independent groups, aimed at statistical analysis of the difference between frequencies of successes in sample based preclinical studies. The test is applicable whenever the study is based on repeating replicate experiments, as required by certain procedures such as validation. Such experiments are based on constant drug dose and performed under identical conditions and protocol. The proposed test combines the computational rules of t-test for two independent groups and analysis of variance. In the initial steps, incidences are transformed to proportions, and variance between proportions in samples of the j(th) group (s(p(j))(2)), is then transformed into theoretical weighted variance within the i(th) repetition (sample) of the j(th) group (s(i,j)(2)). The variance of proportions in samples of the size of the whole group (SE(j)(2)) is then calculated. The t-statistic is computed according to the rules of t-test for two independent groups. Significance is calculated using (N(1) - 1) + (N(2) - 1) degrees of freedom, where N(j) denotes the total number of animals in the j(th) group. The proposed model offers an important advantage over incidence or proportion distribution models, such as chi-square or normal approximation of binomial distribution, respectively, because it considers variance between replicate experiments. It moreover offers important flexibility by limiting the requirement for identical sample sizes only to samples within the control or test group. A difference between groups in sample sizes, number of samples, or both, preventing application of block designs or the standard formats of t-test, may still exist. Theoretical considerations and working examples are provided.
Lay abstract: Sample based preclinical drug efficacy studies compare frequency (proportion) or incidences of successful results (e.g., protected, recovered, or surviving animals, depending on the particular experiment) within respective samples of the test and the control groups. Certain procedures, such as validation, require replicate experiments that are identical in all controllable factors, such as drug dose, sample sizes within each group, general experimental conditions, etc. Still, the control sample size is not required to be identical to that of the respective test sample size. In such cases, t-test or block designs are not applicable for statistical analyses. Moreover, incidence or frequency distribution models, such as chi-square or normal approximation of binomial distribution, respectively, which are performed on pooled data of the examined groups, ignore variance between experiments and thereby result in impaired validity of the statistical inference. We propose here a modified t-test that limits the requirement for identical sample sizes to only within each group. This aim is achieved by combining the computational rules of t-test together with analysis of variance. The proposed t-test allows the incorporating of variance between experiments into frequency or incidence assessments. We recommend using the proposed modified t-test as a complementary test to incidence distribution models.