Rethinking the Endpoints of mHealth Intervention Research in Diabetes Care

Abstract

Mobile health (mHealth) is an emerging branch of medicine that utilizes technology to impact health care. With increasing access to smartphones and mobile devices, there has been an increase in both the availability and use of mHealth resources, including the use of text messages, apps, and web portals. In diabetes, there is particular interest as technology has long been a mainstay of management through glucometers, insulin pumps and continuous glucose monitors, which are increasingly interconnected. Although there is high demand for mHealth interventions in diabetes, there is no clear consensus on the best way to monitor these interventions. While randomized controlled trials are considered the gold standard for most medical research, we propose that alternative trial designs may be more applicable in the ever-changing mHealth landscape.

With the rapid advance of technology, mobile health (mHealth) has emerged as a prominent branch of eHealth, defined as the use of information and technologies for health. ¹ These technologies include web pages, online services, electronic medical records, and patient portals. mHealth strives to accommodate the busy, on-the-go American lifestyle with portable, convenient IT tools such as text and multimedia messaging, mobile applications, device add-ons, wearable devices, Bluetooth-enabled glucometers, and secure mobile web portals.^2,3 Providers and patients currently utilize mHealth technologies to support diabetes and hypertension management, smoking cessation, weight loss, increased physical activity, and sexually transmitted disease prevention and treatment. ²

Studies on general mHealth usage have reported increasing use of mobile devices in recent years. Chou et al analyzed National Health Interview Survey data on mHealth usage among adults ages 18 and over from 2009 to 2013. In this analysis, mHealth was defined in a relatively limited fashion as scheduling appointments, communicating with health care providers, and refilling prescriptions. Over that five-year period, mHealth usage increased by nearly 4%. ⁴ In line with this, data from the Pew Center reported an increase in smartphone usage from 83% to 92% of Americans over that same time period. ⁵ Despite this increased adoption of mobile technology, US patients with chronic diseases have lower rates of overall internet access compared to their healthy peers, even when demographic factors are controlled (62% vs 81%, respectively); even lower rates are specifically seen in patients with diabetes (50%). ⁶ Interestingly, those with chronic diseases were more likely to blog or participate in online discussions, ⁶ thus suggesting that mHealth interventions could overcome limited internet access among safety-net populations by incentivizing mHealth usage with blog and discussion components.

Despite the broad applicability and exciting potential of these emerging technologies, the quality, capability, reliability, security and sheer diversity of choices raise many important, yet unanswered, questions. The consumer currently has immense responsibility and little guidance from the scientific and health care communities in selecting tools and resources that are credible, accurate, and most appropriate to their condition. ⁷ Even if scientific credibility is established, mHealth technology must also be accessible, user-friendly, secure, and based on the most recent standards of medical care.

As of 2013, consumers could download more than 15,000 health-related mobile applications. ² Despite their ubiquity, many mHealth applications go unused for a variety of reasons. On the most basic level, consumers, and health care providers by extension, are often simply unaware that they exist. ⁷ Some common questions that patients often direct toward their clinical practitioners are: What is the best app to use? Do you know of any good apps that will “sync” with the technology that I am already using? Will it make things easier? In a survey reported by Krebs and Duncan, 58% of smartphone users in the United States had downloaded a health-related app, but 46% no longer used a health app they had downloaded, showing a high rate of mHealth abandonment. ⁸ In addition, mHealth adoption and effectiveness are limited by multiple technical complexities, inadequate marketing to consumers, and lack of a clear understanding of the determinants that allow consumers to derive ongoing benefit from this technology.

Current State of mHealth in Diabetes

Given the enormous worldwide burden of chronic disease, mHealth implementation studies have focused on diabetes and other chronic conditions, and more specifically on patient self-management strategies for these conditions. Diabetes naturally lends itself toward technology usage, with patients routinely using glucometers, continuous glucose monitoring (CGM) devices, and insulin pumps. These devices are becoming increasingly sophisticated and interconnected. The mHealth opportunities in diabetes are extensive, and multiple players are involved in the development and advancement of connectivity in diabetes care. With more than 1000 diabetes-related mobile apps currently available for download, the high level of both supply and demand is obvious. However, much is still unknown about the best ways to incorporate and rigorously evaluate this technology to achieve maximal benefit in diabetes care. ⁹

One of the major advantages of app-based diabetes management, and by extension app-based research, is that data can be accessed remotely. Remote access to individual data introduces the opportunity for patients living in rural areas and those with limited resources to connect with their clinicians to receive excellent individualized care, akin to those able to directly see their provider. ¹⁰ Could we foresee a world in which face-to-face clinic visits become near obsolete for certain chronic conditions in favor of remote access to care?

Current Strategies in mHealth Research in Diabetes

The current gold standard for intervention research studies is the randomized controlled trial (RCT). ¹¹ RCTs are considered to be the ideal research method to evaluate new pharmaceutical agents, since they reduce unfounded causality and bias. Taking the lead from this idealized evidence-based research milieu, most diabetes technology intervention studies have similarly employed the RCT as the “highest-quality evidence available.” ¹⁰ The many advantages of RCTs are well-established. However, as a result of some of these well-recognized strengths, we propose the suggestion that RCTs may not be the optimal method to assess mHealth interventions.

The end points of an RCT are essentially fixed at its onset, and revisions to a particular protocol are avoided as much as possible during the intervention period of a trial, even as new evidence and advances in technology develop. This begs the question: how does an RCT conclusively evaluate mHealth technology when the technology may well be obsolete by the conclusion of the trial? By way of example, early insulin pump intervention studies were performed in an era preceding the availability of rapid acting analog insulins. Modest benefits in favor of insulin pump therapy were reported, but how is this relevant today with all of the changes to the standard of diabetes care since then?

A large sample size is desirable to ensure study results are appropriately powered and generalizable. Most RCTs of sufficient size are multiyear, multisite endeavors of great cost. Given the incredible pace of technological advancement, the pump and sensor devices currently under investigation in RCTs will most likely be publicly reported and available for widespread use right at the time when the technology is no longer on the “cutting edge” and the next generation of technology is in development.

The careful evaluation of mHealth applications is even more critical given the proliferation and multitude of available options. If mHealth is to truly be integrated and accepted into the health care system, it must be relevant, safe and unequivocally medically beneficial to patients and clinicians. Answering these questions may not be best achieved by an RCT, as RCTs are not conducive to the ever-changing technological landscape. Most RCT participants in recent artificial pancreas diabetes technology studies are also not typical “real-world” patients in terms of sociodemographic or economic variables. ¹²

In their 2013 report on the current and future role of health care apps, the IMS Institute for Healthcare Informatics reported that although observational studies are more easily performed, they are insufficient to show an effect on outcomes, and RCTs are necessary to provide evidence-based recommendations. ¹³ Without RCTs to show health outcome benefits, the IMS Institute suggested it would be unlikely that there will be widespread adoption of apps by either providers or patients. In line with this report, a search of clinicaltrials.gov for “diabetes” and “mHealth” reveals 70 studies, 27 of which are actively recruiting, soon to be recruiting, and available for expanded access. ¹⁴ Of these 27 open studies, 22 are RCTs. The study duration for these studies ranges from three months to five years, with the majority being between six and 12 months.

Prior RCTs of mHealth diabetes care interventions have recruited patients over as little as three months, which is likely fast enough to ensure the relevance of the intervention, though it may limit the sample size and statistical power. In addition, any effect that is observed within a short follow up time could be attributable to the novelty of using a new app, and not the effect of the app itself. RCTs performed over longer time periods eliminate the risk of the novelty period, but at the possible expense of the intervention no longer being relevant or being replaced by its next iteration. In addition, RCTs are often limited in physical reach; patients who are economically or geographically isolated may be excluded, and thus may not have the opportunity to benefit from many of the mHealth interventions. In unpublished data from our urban safety-net institution, where more than 70% of patients receive insurance coverage through Medicare or Medicaid, over 90% of patients with diabetes and obesity reported having smartphones. Although historically this group is underrepresented in RCTs, they may benefit from mHealth interventions in different ways, as mHealth tools may result in increased access to care, when often this access is curtailed by competing sociodemographic complexities.

Proposed Alternative Study Designs

The demand for robust mHealth in diabetes care for both patients and clinicians is readily apparent, but to succeed, researchers must consider shifting from a focus on RCTs to other trial designs that may be more appropriate for mHealth interventions and allow for simultaneous analysis of multiple outcomes. New methodologies, such as Continuous Evaluation of Evolving Behavioral Intervention Technologies (CEEBIT), monitor the efficacy of an intervention over time and through progressive versions, which will allow for longer term study of an app while accounting for the rapid changes that are expected in technology. ¹⁵ Frameworks such as this are particularly desirable for tracking outcomes from glucometer, CGM, and insulin pump technologies, as they are increasingly integrated with mHealth platforms.

For shorter-term outcomes, cohort studies may best suggest associations between interventions and relevant clinical outcomes. Although this study design does not allow for causality attribution, strong associations between an app and improved endpoints may provide temporal evidence that validate routine use of specific mHealth interventions. This is particularly attractive as observational data from patient experience can be more readily analyzed, thus allowing for more rapid incorporation of an app into routine care.

If the focus in research remains on clinical trials, then perhaps pragmatic clinical trials are more appropriate than traditional RCTs. This trial design innately allows researchers to evaluate the effectiveness of interventions in the real-life setting. From our experience working at a safety-net health care institution, pragmatic trials are invaluable, particularly due to the inconsistent access to internet and mobile resources among our resource-limited patient population and allowance for more generalizability to a broad population. From a similar perspective, response-adaptive randomization in clinical trials may be a way to keep traditional RCTs more relevant, as patients would preferentially be randomized to beneficial interventions over the course of a study based on interim analyses.

Rather than waiting years for the results of an RCT proving that an intervention definitively leads to improvement in an outcome of interest, mHealth interventions can be promoted almost immediately by considering alternative study designs. Even if we are not able to clearly attribute direct responsibility for improvement in the outcome of interest to the mHealth intervention, we may conclude that it was, at the minimum, a meaningful contributor to an overall outcome. While it is unlikely that RCTs will soon be widely abandoned as the preferred and most common clinical trial design for mHealth in diabetes, a willingness to consider other methodologies for rapid adoption of mHealth strategies is necessary to assure clinicians and patients, alike, that mHealth interventions can be directly incorporated into routine medical care with positive effect on outcomes.