Monitoring futility in a two-by-two factorial design: The SPS3 experience

Abstract

Background

For studies with two-by-two factorial designs, the complexity of determining an appropriate futility analysis plan is increased as compared to studies where patients are randomized to one treatment. Issues that must be addressed include the possibility of a significant interaction and the need to determine how to proceed given evidence of futility in one arm. Suggested approaches include a two-stage plan, which first assesses futility of the interaction term and proceeds to examine the main effects, given sufficient evidence that no interaction is present, and variations on one-stage plans, which assume the trial will not be stopped for futility in the interaction.

Purpose

To discuss different approaches to monitoring futility in two-by-two factorial clinical trials and compare their properties.

Methods

We utilized a simulation study, designed to mimic the Secondary Prevention of Small Subcortical Strokes (SPS3) Study, to determine which approach to monitoring futility in two-by-two factorial studies had the most desirable statistical properties.

Results

We found that in most scenarios typical of clinical trials, monitoring futility in each arm simultaneously was superior to or as good as monitoring the interaction and then assessing futility in each arm only when the interaction was deemed futile. Monitoring each arm simultaneously lead to early stopping more often when no treatment effect was present, and lower average sample numbers (ASNs). The exception to this was the unlikely case when a qualitative interaction was present.

Limitations

We assumed that one-sided tests were to be performed, and only assessed some of the possible methods for monitoring futility under the study design.

Conclusions

Futility monitoring in two-by-two factorial studies should proceed by assessing each arm simultaneously, rather than monitoring the interaction first. If sizeable interactions are anticipated, study design, rather than study monitoring, should account for this.