Stop,Listen,and Learn: Using Mixed Methods to Add Value to Clinical Trials*

Abstract

This commentary discusses the concept of value-based or value-focused health care as a rationale for researchers to incorporate mixed methods study designs a priori into clinical trials evaluating traditional, complementary, alternative, and integrative medicine (TCAIM). Along with assessing patient outcomes, information about patients’ experiences and preferences are needed to determine the value of an intervention. Incorporating a mixed-methods approach can improve the quality of clinical trials and provide important information about the potential value of the intervention.

Value is the latest buzzword driving health care decisions about service delivery, funding, and policy. It has emerged in response to the rising demand for health care, concurrent resource constraints, and an increasing number of efficacious and cost-effective interventions. Demonstrating evidence of safety, efficacy, and cost-effectiveness is no longer enough in conventional medicine. The intervention, investigation, or service must also demonstrate value. These same requirements will also apply to traditional, complementary, alternative, and integrative medicine (TCAIM).

Despite an exponential growth of clinical trials evaluating the efficacy of TCAIM interventions, translating these results into mainstream clinical practice guidelines, service delivery, and policy is proving to be a lot more challenging. ¹ While the reasons for this are complex and multifactorial, incorporating mixed-methods study designs a priori into programs of TCAIM research that include clinical trials may not only improve the quality of the clinical trial but also provide important information about the potential value of the TCAIM intervention.

What Is Value Health Care?

Value in health care can be thought of as the costs associated with providing quality care that is safe, effective and appropriate. The concept is challenging however, because various stakeholders (patients, practitioners, providers, insurers, policy makers, and even countries) often have different definitions, viewpoints, and priorities about what they think is important. ²

As part of a national survey exploring stakeholders views, the University of Utah suggested the following definition; value is the product of the quality of care plus the patient experience at a given cost. This can be expressed as Value = Quality * Service / Cost. ³

What Do Patients Value?

While there is a plethora of research reporting the frequency and reasons for TCAIM use, much of these data are not being linked to the specific TCAIM interventions that are being evaluated in clinical trials. Furthermore, according to Downey et al, ⁴ clinical trials often fail to evaluate what patients value:

You never ask how important it is to me to receive this service…

I so much look forward to it…I’m amazed you don’t ask me this question. It should be the featured question.

Along with measuring patient outcomes—be they objective or subjective—the results of clinical trials can therefore be strengthened by reporting the patients’ experiences. This may include collecting data on patient satisfaction, tolerability of side effects, acceptability of risks, logistics around accessing or using an intervention, and other factors such as a person’s preferences, beliefs and values that influence health care choices.

The Role of Clinical Trials

Randomized controlled trials play a key role in evaluating the outcome of interventions. They can be undertaken in a variety of environments, from those in more “controlled” environments, where trials are often designed to assess efficacy (so called explanatory trials), to those undertaken in community settings (often called pragmatic trials ^5,6 ) where there is a focus on effectiveness. There is not, as sometimes assumed, a dichotomous choice between efficacy and effectiveness but rather trials exist on a continuum between explanatory and effectiveness. ⁷ Trials all along this continuum can be considered randomized controlled trials providing they meet certain characteristics. ⁸ Trials that tend to sit on the pragmatic end of the continuum are common in TCAIM. ^{9 –11}

Using Mixed Methods to Improve the Quality of Clinical Trials

Mixed methods can be used to collect essential background information for clinical trial design. ^{12 –14} Mixed-methods research refers to the integration of methods, often that collect both quantitative and qualitative data, either sequentially or concurrently, as part of a single program of inquiry. ¹⁵ The role of mixed methods in health services research, or as part of the process for developing or validating patient reported outcome measures, is commonplace. ^{12 –14} Increasingly, the role of mixed methods as an integral part of a clinical trial is being recognized, ¹⁴ with TCAIM researchers often leading the way. ^{16 –21}

TCAIM are often complex interventions ²² consisting of multiple “characteristic” components that are likely to contribute to the therapeutic outcome. ²³ Furthermore, outcomes that participants may value highly may be quite different from those that researchers think they will. ²⁰ It is common, however, for trialists to focus on the most obvious “active” component, such as needle insertion in acupuncture, and ignore or minimize other characteristic components such as holistic, patient-centered care. Indeed, many patients using TCAIM in the community report wide-ranging changes in their health that may fall outside the narrower range of condition specific outcomes that are commonly evaluated in clinical trials and are important to patients. ²⁴

Another issue for trial design is that most TCAIM interventions, in contrast to pharmaceutical interventions, lack early phase studies ²⁵ and therefore information on a suitable “dose” may be lacking. ²⁶ This has been a significant issue in acupuncture research, where clinicians report that research design does not reflect contemporary clinical practice ²⁷ due to marked differences between frequency of treatment in clinical trials and in clinical practice for example. ²⁶ Using a mixed-methods approach when designing and pilot testing clinical trials can promote input from a diverse group of practitioners and other experts on trial design, ensuring that the intervention has sufficient model validity ²⁸ or clinical relevance, including a suitable “dose” and incorporation of other important clinical components. ²³ Failure to do this can mean that only a facsimile of the intervention is being examined and any results may be irrelevant to how that intervention is practiced or used in the community. ²⁹

Using Mixed Methods to Evaluate the Costs

Trialists are continuingly being encouraged to include economic evaluations, such as measuring the cost-effectiveness of a TCAIM intervention. Along with potentially high out-of-pocket costs to patients, there are health service resource implications that must be considered. However, the resources required to conduct a robust economic analysis are often prohibitive for researchers or are difficult to justify if it is a pilot study. Including patient-reported outcome measures (PROMs) that use algorithms to calculate quality-adjusted life years (QALYs) are relatively easy to include ³⁰ and measures such as the SF-36/12 (36-/12-item Short Form Health Survey) or the AQoL (Assessment of Quality of Life) can be used for both health-related quality of life and for QALYs. ³¹ Some chronic conditions have disease-specific cost of illness measures ³² that are suitable to be included as part of a TCAIM clinical trial. ³³ Even if this is not possible, at the very least, by incorporating mixed methods, the cost of providing/accessing the intervention(s) can be estimated and reported.

Using Mixed Methods in Clinical Research to Demonstrate Value

Rather than waiting (or hoping) for a second round of funding once the intervention is proven to work, along with clinical outcomes, research that might otherwise only consist of a randomized controlled trial can benefit from incorporating mixed methods to also provide relevant information about the potential value of the intervention. An a priori mixed-method study design can facilitate the integration of quantitative clinical and economic data with qualitative data about patient’s experiences, preferences and values. ^16,17 Qualitative data can augment the quantitative results by providing richness and context to the results that cannot be captured by numbers alone. ^{18 –20} Using a mixed-method approach that combines quantitative and qualitative data collection may further help uncover context, outcomes, and experiences that are important to patients and may otherwise be missed. ²⁴ Mixed methods can also be used to evaluate practical aspects from the practitioners’ perspectives about providing or recommending a TCAIM intervention, along with organizational barriers and facilitators to implementing the intervention. ^20,21

Conclusion

Designing and conducting high-quality clinical trials is all consuming. It is tempting to think that demonstrating safety and efficacy, as opposed to value, is the only priority. Yet patient experiences are increasingly becoming important when making value-based health care decisions. Taking the time to stop, listen, and learn from patients along with practitioners, service providers and policy makers may make all the difference when it comes to translating the results of TCAIM clinical trials research into practice and policy.