Validation of Decision-enabling Tools: Showing That the Model is Useful

Abstract

The rapidly increasing cost to develop new drugs calls for new tools that efficiently enable the demonstration of the safety and effectiveness of a new drug. When validating such a decision-enabling tool, a traditional approach is typically to apply the tool on a positive control, known to be effective, and ascertain that a statistically significant effect is obtained. We argue, however, that the validation study should be designed to show that the tool provides a variability that is small in relation to the treatment effect, which means that the tool has the capacity of providing decision-enabling results in small-sample studies in routine use.

We give details on the relevant test to perform in the validation of a decision-enabling tool and use the development of a human pharmacological model, aimed at studying neuropathic pain in 2 × 2 crossover trials, as a motivating example. We also develop power and sample size calculations, and illustrate the implications on sample size needed for a validation study. Results show that to obtain pertinent evidence that the decision-enabling tool is useful, that is, to reject the relevant null hypothesis, a substantially increased sample size would often be needed in the validation study, as compared to traditional approaches.