Comparisons of statistical methods in analyzing clinical data through systematically using historical data

Abstract

With increasing requirements, from both regulatory and scientific community, for pre-specification of details of all analyses prior to unblinding of data in clinical trials, it is critical that one selects the most appropriate statistical model. Selecting a model based on assumption checking either inflates type I error or compromises the statistical power. Previous research is mainly focused on comparing various analysis models through either simulation or case studies. Simulation does provide a flexible way to compare models but requires assumptions on models for generating simulation data. On the other hand, although results based on case studies are close to the real situations, it is difficult to draw a definite conclusion due to lack of replication. These two approaches ignore the fact that for most variables, large amount of data may be available from historical studies. We propose a procedure that systematically utilizes the historical data, evaluates various models of interest, and provides a powerful choice for model pre-specification for subsequent studies. Based on the comparisons on the generated data from a historical data base, one can pre-specify the particular model for the purpose of controlling the type I error and power of prospective studies, or ease of interpretation when they all have similar performance.