Response-adaptive randomization with imperfect intermediate endpoints

Abstract

Background:

Response-adaptive randomization is controversial even in the best circumstances when based on a quickly determined primary outcome. In disease settings in which the primary outcome requires long follow-up, an intermediate endpoint may be chosen to update randomization allocations. The aim of our study is to evaluate the impact of response-adaptive randomization applied to an imperfect intermediate endpoint. We use tuberculosis trials as the motivating example.

Methods:

We simulated a response-adaptive randomization design, adapting randomization allocations using an imperfect intermediate endpoint, in a superiority trial of two experimental regimens and one control arm. The primary study outcome was treatment success after 73 weeks from randomization; the intermediate endpoint was culture conversion at 8 weeks. We compared different sensitivity (Se) and specificity (Spe) scenarios for the intermediate endpoint, while varying the true treatment efficacy. We evaluated the performance of response-adaptive randomization to achieve its primary goal of allocating more participants to the better arm and the impact of time-trends on type I error rate.

Results:

Even in an ideal state of perfect accuracy (i.e. intermediate endpoint with Se = 100% and Spe = 100%), response-adaptive randomization did not always live up to its main purpose of allocating more patients to the better arm. Lower accuracy of the intermediate endpoint leads to greater divergence from the goal of more allocations to the better arm. The larger the difference in treatment efficacy between the arms, the more striking the impact of an intermediate endpoint with poor diagnostic accuracy. Time-trends inflate the type I error rate, and while stratified tests can correct this, they do so at the cost of a power loss. Allocating more patients to the worst arm increases power for comparisons with this arm but reduces power for comparisons of the best arm to control.

Conclusion:

Given the objective of evaluating several new therapeutic regimens in a timely manner, response-adaptive randomization is tempting. However, it requires at least reliance on highly accurate intermediate endpoints, which are still no guarantee of response-adaptive randomization’s trustworthiness.