Challenges encountered in the conduct of Optimal Health: A patient-centered comparative effectiveness study of interventions for adults with serious mental illness

Abstract

Background:

The aim of patient-centered comparative effectiveness research is to conduct stakeholder-driven investigations that identify which interventions are most effective for which patients under specific circumstances. Conducting this research in real-world settings comes with unique experiences and challenges. We provide the study design, challenges confronted, and the solutions we devised for Optimal Health, a stakeholder-informed patient-centered comparative effectiveness study focused on the needs of seriously mentally ill individuals receiving case management services in community mental health centers across Pennsylvania.

Methods:

Optimal Health, supported by the Patient-Centered Outcomes Research Institute, is a cluster-randomized trial of two evidence-based interventions for improving health and wellness across 11 provider sites. Participants were followed for 18–24 months, with repeated measurements of self-reported health status and activation in care and administrative measurements of primary and specialty health service utilization. Health-related quality of life, engagement in care, and service utilization are to be compared via random effects mixed models. Stakeholders were, and continue to be, engaged via focus groups, interviews, and stakeholder advisory board meetings. A learning collaborative model was used to support shared learning and implementation fidelity across provider sites.

Results:

From 1 November 2013 through 15 July 2014, we recruited 1229 adults with serious mental illness, representing 85.1% of those eligible for study participation. Of these, 713 are in the Provider-Supported arm of the study and 516 in Patient Self-Directed Care. Across five data collection time points, we retained 86% and 83% of the participants in the Provider-Supported and Self-Directed arms, respectively.

Lessons learned:

Lessons learned relate to estimation of the size of our study population, the value of multiple data sources, and intervention training and implementation. The use of historical claims data can lead to an overestimation of eligible participants and, subsequently, a reduced study sample and an imbalance between intervention arms. Disruptions in continuity of care in real-world settings can pose challenges to on-site self-report data collection, although the inclusion of multiple data sources in study design can improve data completeness. Geographic dispersion of rural provider sites and staff turnover can lead to training and intervention fidelity challenges that can be overcome with the use of a “train-the-trainer” model, “wellness champions,” and the use of a Learning Collaborative approach. Stakeholder engagement in mitigating these challenges proved to be critical to study progress.

Conclusion:

Conducting real-world patient-centered comparative effectiveness research in healthcare systems that care for seriously mentally ill persons is an important yet challenging undertaking, one which requires flexibility in identifying potential adaptations within all major study phases. Advice from a wide range of stakeholders is critical in development of successful strategies.