Assessing Methods for Adjusting Estimates of Treatment Effectiveness for Patient Nonadherence in the Context of Time-to-Event Outcomes and Health Technology Assessment: A Simulation Study

Abstract

Purpose

We aim to assess the performance of methods for adjusting estimates of treatment effectiveness for patient nonadherence in the context of health technology assessment using simulation methods.

Methods

We simulated trial datasets with nonadherence, prognostic characteristics, and a time-to-event outcome. The simulated scenarios were based on a trial investigating immunosuppressive treatments for improving graft survival in patients who had had a kidney transplant. The primary estimand was the difference in restricted mean survival times in all patients had there been no nonadherence. We compared generalized methods (g-methods; marginal structural model with inverse probability of censoring weighting [IPCW], structural nested failure time model [SNFTM] with g-estimation) and simple methods (intention-to-treat [ITT] analysis, per-protocol [PP] analysis) in 90 scenarios each with 1,900 simulations. The methods’ performance was primarily assessed according to bias.

Results

In implementation nonadherence scenarios, the average percentage bias was 20% (ranging from 7% to 37%) for IPCW, 20% (8%–38%) for SNFTM, 20% (8%–38%) for PP, and 40% (20%–75%) for ITT. In persistence nonadherence scenarios, the average percentage bias was 26% (9%–36%) for IPCW, 26% (14%–39%) for SNFTM, 26% (14%–36%) for PP, and 47% (16%–72%) for ITT. In initiation nonadherence scenarios, the percentage bias ranged from −29% to 110% for IPCW, −34% to 108% for SNFTM, −32% to 102% for PP, and between −18% and 200% for ITT.

Conclusion

In this study, g-methods and PP produced more accurate estimates of the treatment effect adjusted for nonadherence than the ITT analysis did. However, considerable bias remained in some scenarios.

Highlights

• Randomized controlled trials are usually analyzed using the intention-to-treat (ITT) principle, which produces a valid estimate of effectiveness relating to the underlying trial, but when patient adherence to medications in the real world is known to differ from that observed in the trial, such estimates are likely to result in a biased representation of real-world effectiveness and cost-effectiveness.

• Our simulation study demonstrates that generalized methods (g-methods; IPCW, SNFTM) and per-protocol analysis provide more accurate estimates of the treatment effect than the ITT analysis does, when adjustment for nonadherence is required; however, even with these adjustment methods, considerable bias may remain in some scenarios.

• When real-world adherence is expected to differ from adherence observed in a trial, adjustment methods should be used to provide estimates of real-world effectiveness.