Introduction: Testing for toxicities is an important activity in drug development. In an ideal world the tests applied would be definitive. In reality this is seldom the case. There are two types of power associated with a test. A test's discriminatory power is characterized by its sensitivity and specificity and tells the investigator the probability of obtaining a test positive in the presence (sensitivity) or a test negative in the absence (specificity) of the toxicity. A test's discriminatory power is an attribute of the test itself. The investigator is, however, more interested in a test's predictive power, which is the probability that the toxicity is present or absent in a novel molecule given the test result. A test's predictive power is a consequence of the test's discriminatory power and the context of its application. Unlike its discriminatory power, the predictive power of a test is not 'fixed' and varies with testing context. This means that tests and test context must be taken together to enable an investigator to achieve their desired predictive power. Our intent is to illustrate a broadly applicable approach to testing schemes designed to maximize a test's positive or negative predictive power. Rather than hypothetical tests and toxicities, we use as examples tests available for the prediction of a substance's liability to cause the cardiac arrhythmia torsade de pointes.
Methods: Owing to intense focus over the last two decades, the discriminatory powers of a number of tests for predicting a torsade de pointes liability are publicly available. Having randomly chosen an initial test (random although plausible as an early screening assessment), the inter-relationship between the prevalence of torsadogenic liability and the discriminatory power of potential follow-on tests were explored in a probability framework, based on Bayes Theorem, to show how testing schemes could be developed based on odds and likelihood ratios. Uncertainty around the prevalence of torsade liability and the discriminatory power of a test were addressed by varying these values and examining their impact on the test's predictive power.
Results: Overall, the analysis demonstrates that tests can be strategically combined to reach a desired level of predictive power. This is generally more easily achieved for negative predictive power given a low prevalence of the toxicity under scrutiny. For this work, we used a base prevalence of 10% for a substance to carry a tordsadogenic liability. Given uncertainty around a test's discriminatory power, a probabilistic rather than deterministic approach was recommended. Such an approach necessarily requires the investigator to define distributions around test characteristics as well as their desired probability of attaining a given predictive power.
Conclusions: The proposed approach is easily implemented deterministically since values of the discriminatory power of the tests are readily and publicly available. The probabilistic implementation is also easily implemented, but requires that the uncertainty around the test performance and prevalence, and the targets for probability of attaining the desired predictive value all be made explicit rather than remain implicit as is often the case in 'integrated risk assessment' or 'totality of evidence' presentations. This general approach could form a basis for testing and decision-making that can be communicated and discussed in a consistent manner between scientists as well as between sponsors and regulators.
Keywords: Bayesian; Predictive value; Probability; Toxicity testing.
Copyright © 2020. Published by Elsevier Inc.