Real-World Effectiveness: Addressing Selection Bias in the Well Elderly 2 Study Strengthens Effect Estimates

Abstract

Rationale: Intention-to-treat (ITT) analyses in randomized trials may underestimate effects due to poor adherence or follow-up loss. Objectives: To demonstrate methods to address internal validity threats in ITT using the Well Elderly 2 study as a case example. Methodology: This secondary analysis used 4-year data from 460 older adults. The original trial, by a separate team, included a crossover intervention with 6-month follow-ups. We addressed follow-up loss using inverse probability of censoring weights to report an adjusted ITT. Results: ITT showed improvement in 10 of 17 outcomes; inverse probability of censoring weight (IPCW)-adjusted ITT in 13. In SF-36v2, ITT improved mental (B = 0.54, 99% confidence interval (CI) [0.05, 0.60]) but not physical (B = 0.71, 99% CI [−0.21, 0.08]); IPCW-adjusted ITT improved both mental (B = 0.77, 99% CI [0.34, 1.19]) and physical composites (B = 2.28, 99% CI [0.14, 0.71]). Conclusion: IPCW-adjusted ITT showed stronger effects but may be biased, while ITT was more conservative yet prone to misclassification; reporting both provides a clearer interpretation.

Plain Language Summary

How Occupational Therapy Lifestyle Interventions Improve Health for Older Adults: Findings from a Re-analysis of the Well Elderly 2 Study

Our findings highlight the benefits of occupational therapy-based lifestyle interventions for community-dwelling older adults. To inform clinical practice, we applied and compared two analytical approaches: one based on the study arms participants were originally assigned to, and the other adjusting for differences between participants who remained in the study versus those lost to follow-up. By using a method to account for participants who dropped out of the study, we reanalyzed the effectiveness of the Well Elderly 2 study, resulting in a greater number of statistically significant findings. The adjusted approach demonstrated greater effectiveness, helping to address the clinically relevant question of how results may differ when accounting for loss to follow-up. This study supports the significance of lifestyle interventions in occupational therapy and provides meaningful insight into their impact on older adults’ functions and activities, thereby suggesting future directions for intervention design.