Analysis of Healthy Coping Feedback Messages from Diabetes Mobile Apps: Validation Against an Evidence-Based Framework

Abstract

Background:

In this study, we focused on Healthy Coping, a key principle of ADCES7 Self-Care Behaviors^® (ADCES7^®) that enables people with diabetes to achieve health goals for self-care. We aimed to validate Healthy Coping-related feedback messages from diabetes mobile apps against the framework based on behavioral change theories.

Methods:

We searched apps using the search terms: “diabetes,” “blood sugar,” “glucose,” and “mood” from iTunes and Google Play stores. We entered a range of values on 3 Healthy Coping domains: (1) diabetes-related measures including blood glucose, blood pressure, HbA1c, weight, (2) physical exercise/activity, and (3) mood to generate feedback messages. We used a framework by adopting validated behavioral change theory-based models to evaluate the feedback messages against 3 dimensions of timing, intention, and content (feedback purpose and feedback response). The feedback purposes in this study were categorized into 7 purposes; warning, suggestion, self-monitoring, acknowledging, reinforcement, goal setting, and behavior contract.

Results:

We identified 1,749 apps from which 156 diabetes mobile apps were eligible and generated 473 feedback messages. The majority of generated feedback messages were related to blood sugar measurement. Only feedback messages on blood sugar under diabetes-related measures and mood domains encompassed all 7 feedback purposes under the content dimension.

Conclusions:

Many feedback messages neither supported Healthy Coping domains nor followed the behavioral theory-based framework. It is important that feedback messages be structured around the dimensions of the behavioral theory-based framework to promote behavior change. Furthermore, our framework had the generalizability that can be used in other clinical areas.

Introduction

There were 463 million people with diabetes (8.3%) worldwide in 2019, and without sufficient and effective strategies to address this problem, the predicted number will increase to 578 million (9.2%) by 2030 and 700 million (9.6%) by 2045. ¹ Hence, the American Diabetes Association (ADA) has recommended that all individuals with diabetes should receive diabetes self-management education and support (DSMES) throughout the treatment process—at diagnosis, during an annual assessment by a health care provider, when complications arise, and during transitions in care. ² DSMES can reduce admission and readmission rates,^3-6 reduce complications,^7-12 decrease costs,^4,13 improve glycemic control,^4,8,14-16 and improve quality of life.^8,17-19 The purpose of this study is to evaluate, based on DSMES guidelines, Healthy Coping related feedback messages from diabetes mobile apps.

Healthy Coping

The Association of Diabetes Care & Education Specialists (ADCES) has developed patient centered DSME guidelines for people with diabetes, named “ADCES7 Self-Care Behaviors^® (ADCES7^®).” ADCES7^® has 7 principles, which are Healthy Eating, Being Active, Monitoring, Taking Medication, Problem Solving, Reducing Risks, and Healthy Coping. ²⁰ In this study, we focus on Healthy Coping, which is defined as “a positive attitude toward diabetes and self-management, positive relationships with others, and quality of life,” ²¹ because the ability to cope is an initial step to achieve health goals and self-care. Healthy Coping will help increase motivation to keep diabetes in control. ²²

Behavioral Change Theory and m-Health Intervention

Behavioral change theories have played a key role in feedback message design, because a characteristic of a feedback message is to allow people to reflect on their progress and to remind them of their goals. Cho et al ²³ performed a systematic review for applied behavioral change theories to m-health intervention for health promotion in low- and middle-income countries. From a total of 14 included studies, only 5 studies^24-28 used behavior change theories: others used behavior learning theory, ²⁹ health belief model, ³⁰ integrated theory of behavior change, ³¹ social cognitive theory, ³² and transtheoretical model. ³³ This study suggested that the findings of theory-based research were more effective than those from studies that did not apply a theory.^34-36

Mobile technologies can be used to promote behavioral change in people with diabetes.^37,38 Diabetes self-management apps have a variety of functionalities including diaries, blood sugar records, calculating and recording insulin dosage, reminders, carbohydrate counting, and monitoring of physical activities. ³⁹ Some mobile apps have a real-time response to a user, such as just-in-time feedback, ⁴⁰ tailored messaging, ³⁷ and motivational messaging and personalized coaching, ⁴¹ which support behavior changes.

Hawkins et al ⁴² proposed the main concepts of tailoring strategies for the message to achieve specified goals, which are personalization, feedback, and content-matching. Personalization means the message should include specific information of each individual, for example, name, age, gender, and race, to increase engagement. Feedback, as mentioned above, helps provide the individual’s behavioral and psychological conditions during the intervention. Last, content-matching refers to the content of the message that matches personal needs and goals. By encompassing these 3 components, the intervention can help and support patients to change their behavior.

Applied DSMES with m-Health

Greenwood et al reviewed 25 studies that utilized technology for integrating DSMES. The primary outcome of studies was HbA1c (22/25), and improvement in HbA1c was a range from −0.1% to −0.8%. The majority of technology was mobile phones (19/25) and most of them (18/19) had feedback intervention features. Based on ADCES7^®, most studies described Healthy Eating (19/25), Being Active (17/25), and Monitoring (17/25). On the other hand, Healthy Coping strategies were less frequently addressed (6/25). The authors suggested the concepts of Problem Solving, Reducing Risk and Healthy Coping need to be integrated with the technology-enabled diabetes self-management core design. ⁴³ Specifically, our pilot study suggests that Healthy Coping (10%) is one of the least explored DSME self-care principles. ⁴⁴

The objective of this present study was to evaluate feedback messages from diabetes mobile apps regarding Healthy Coping principle against our proposed framework based on the model of motivational messages. ⁴¹ Our framework includes dimensions of timing, intention, content (feedback purpose, response). ⁴⁵ Our findings will help to generate an idea for further research on mobile app development integrating motivational feedback messages.

Methods

Study Design

To understand Healthy Coping feedback messages from diabetes mobile apps, we identified apps from iTunes and Google Play stores and analyzed their feedback messages. We categorized feedback messages into 3 domains of Healthy Coping principles, (1) diabetes-related measures including blood glucose, blood pressure, HbA1c, weight, (2) physical exercise/activity, and (3) mood. We developed a framework by employing 2 validated models to analyze the feedback messages. We adopted a model by op den Akker et al ⁴¹ based on Goal Setting Theory ⁴⁶ and various concepts from the field of health behavior change, and a model by Strong et al ⁴⁵ based on Bandura’s Social Cognitive Theory ⁴⁷ and also Goal Setting Theory. ⁴⁶ Descriptive analysis was performed to determine the frequency of feedback messages for each category across diabetes measurements, physical exercise/activity, and mood. The association between variables and Healthy Coping categories was assessed using Chi-square. This study was approved by the University of Missouri Institutional Review Board.

Data Collection Strategy

We searched 2 major app stores, iTunes and Google Play, from August to October 2019 using the search terms: “diabetes,” “blood sugar,” “glucose,” and “mood” to capture a wide range of diabetes apps focusing on Healthy Coping principles. After removing duplicates, we excluded apps if they met the following criteria: (1) not designed for people with diabetes, (2) not related to diabetes self-management, (3) not written in English, (4) only providing access to reference material, (5) not functional at the time of the study, (6) not providing features related to diabetes-related measures, physical exercise/activity, and mood, and (7) not free.

We installed the eligible apps to the tablets, including iPad (4th generation), iPhone 6, Nexus 7, and Google Pixel 3 XL. We entered a range of values into the apps to generate feedback messages regarding 3 domains of Healthy Coping. We entered diabetes-related measures based on recommendations by the ADA, ⁴⁸ American Heart Association ⁴⁹ and National Heart, Lung, and Blood Institute. ⁵⁰ Regarding the target of blood glucose levels, we used 80 to 130 mg/dL before a meal, and less than 180 mg/dL 1 to 2 hours after a meal. HbA1c that is less than 5.7% is considered as normal, less than 6.5% is considered as prediabetes, and equal or higher than 6.5% is considered as diabetes. Blood pressure less than 90/60 mmHg is considered as low blood pressure, less than 120/80 mmHg is within the normal range, and higher than 140/90 mmHg is considered as high blood pressure. Body mass index (BMI) that is equal or less 18.5 is considered as underweight, between 18.5 and 24.9 is considered as normal weight, between 25 and 29.9 is considered as overweight, and equal or higher than 30 is considered as obesity.

We collected key information from selected diabetes mobile apps including application name, Uniform Resource Locator (URL), app store, input values, and feedback messages. According to the standard recommendations above, we entered values of (1) blood glucose: 60 mg/dL, 130 mg/dL, 200 mg/dL, (2) blood pressure: 85/55 mmHg, 120/80 mmHg, 180/120 mmHg, (3) HbA1c: 5%, 6%, 10%, (4) height: 5.6 feet, weight:100 lbs., 150 lbs., 200 lbs., (5) physical exercise: 0, 30, 60 minutes of walking for 5 days, and (6) mood: happy or sad.

Feedback Message Evaluation Framework and Analysis Methods

The framework development

We reviewed the literature to identify the most suitable model that we could use for evaluating feedback messages. There were a few studies that involved message evaluation or feedback message development, such as a model of motivational messages proposed by op den Akker et al, ⁴¹ a model by Strong et al, ⁴⁵ a practical framework for designing just-in-time feedback proposed by Schembre et al, ⁴⁰ and a mobile messaging platform evaluation matrix by Iribarren et al. ⁵¹ However, due to the lack of a framework to evaluate feedback messages from diabetes mobile apps based on DSMES, our framework (Figure 1) was developed based on 2 validated models: the model of motivational messages by op den Akker et al ⁴¹ and a model by Strong et al. ⁴⁵ We adopted op den Akker et al’s ⁴¹ model because this model provides a complete framework of motivational messages based on behavioral change theories. The model of motivational messages can be the backbone of our framework to evaluate feedback messages. However, op den Akker et al’s model focuses on the general motivational messages; thus, we combined a model by Strong et al, ⁴⁵ which is more specific to feedback messages to complete our framework.

Figure 1.

The framework for evaluating feedback message. The dimensions of feedback message consist timing, intention, and content. The content of feedback messages includes feedback purpose and feedback response. Based on behavioral change theories, we categorized the feedback purposes into 7 purposes: warning, suggestion, self-monitoring, acknowledging, reinforcement, goal setting, and behavior contract.

The backbone of the framework

The first model (Figure 2) we used is a motivational message model proposed by op den Akker et al. ⁴¹ The model of motivational messages provides the backbone of the framework, which are timing, intention, and content, and representation. The “timing” of a motivational message is divided into 2 groups: system-initiated and user-initiated. The system-initiated means the system will send the message to the user without user action. The user-initiated means the user requests the information from the system. The “intention” refers to the motivational message intention that provides an encouraging physical activity message (tell the user to increase activity), neutral comment (tell the user to maintain activity) or discouraging physical activity message (tell the user to reduce activity). The “content” of the motivational message includes feedback, argument, and follow-up types. The feedback is a “statement regarding the user’s current activity performance.” The argument is “to provide a reason as to why the user should increase (or decrease) his physical activity.” Last, the follow-up is related to previous activity. The follow-up also includes suggestion and reinforcement. Because our study focused on only “feedback” messages, we excluded “argument” and “follow-up” types of motivational messages from our framework. Due to the lack of variety of message representation, we excluded the representation domain from our classification.

Figure 2.

The backbone of the framework for evaluating feedback message. The dimensions of feedback message consist timing, intention, and content. The timing includes system-initiated and user-initiated. The intention includes encourage, neutral, and discourage. The content of feedback messages includes feedback purpose and feedback response.

In conclusion, our framework has 3 dimensions: timing, intention, and content. The timing has 2 subdimensions: system-initiated and use-initiated. The intention has 3 subdimensions: encouraging, neutral comment, and discouraging. Last, we developed subdimensions of “content” based on the second model, a model by Strong et al, which are feedback purpose and feedback response.

The feedback purpose evaluation framework

The second model (Figure 3) we used is a model by Strong et al, which is based on Bandura’s Social Cognitive Theory ⁴⁷ and Goal Setting Theory. ⁴⁶ The model captures 4 dimensions of feedback: feedback domain, feedback purpose, feedback goal type, and feedback frequency for diabetes self-care smartphone apps. Our study adopted 3 key dimensions: the feedback domain, the feedback purpose, and feedback frequency to evaluate feedback messages from diabetes apps.

Figure 3.

The feedback purpose evaluation framework. The feedback purposes were categorized into 7 purposes: warning, suggestion, self-monitoring, acknowledging, reinforcement, goal setting, and behavior contract.

In our study, we replaced the feedback domains of the model by Strong et al with Healthy Coping components, which are diabetes control measures, physical exercise/activity, and moods. We classified the feedback purpose into 7 groups, which differs from the Strong et al model to cover all purposes of feedback messages. Our feedback purpose includes warning, suggestion, self-monitoring, acknowledging, reinforcement, goal setting, and behavior contract. The feedback frequency was changed to the feedback response in our study because Strong et al app generates user feedback at 3 frequencies: immediately, daily, and weekly. In contrast, our study focuses on real-time and delayed responses. We did not include the feedback goal type in our study because the feedback goal type is designed for users to set values and measurement frequency of goals in the app. Instead, we used the feedback goal type in Strong et al with self-monitoring and goal setting in our feedback purpose subdimension.

The final model of framework

In summary, our final feedback message evaluation framework (Figure 1) is composed of 3 domains and 3 dimensions. The 3 domains were adapted following Healthy Coping principles, which are (1) diabetes-related measures including blood glucose, blood pressure, HbA1c, weight, (2) physical exercise/activity, and (3) mood. The 3 dimensions are timing, intention, content. The content dimension has 2 subdimensions of feedback purpose and feedback response. For the feedback purpose, there are 7 types of purpose: warning, suggestion, self-monitoring, acknowledging, reinforcement, goal setting, and behavior contract (Table 1). The feedback response is divided into real-time and delayed responses. Real-time messages provide instant information when users input data, while delayed messages provide information at a later time after users input data.

Table 1.

Definitions of Feedback Purpose Components.

Feedback purpose	Description
Warning	Something that makes you understand there is a possible danger or problem, especially one in the future. 52
Suggestion	An idea, plan, or action that is suggested or the act of suggesting it. 53
Self-monitoring	A method used in behavioral management in which individuals keep a record of their behavior, especially in connection with efforts to change or control the self. 54
Acknowledging	To accept, admit, or recognize something, or the truth or existence of something. 55
Reinforcement	A consequence that follows an operant response that increase (or attempts to increase) the likelihood of that response occurring in the future. 56
Goal setting	Individuals are more likely to change the higher the specificity and difficulty of a goal; taking into account e.g. the importance of the goal, levels of self-efficacy, feedback, and task complexity. 57
Behavioral contract	A behavioral contract is a means of scheduling the exchange of positive reinforcements among 2 or more persons. 58

Analysis methods

After we collected all feedback messages from each domain, we classified and analyzed each message by timing, intention, and content. Statistical analysis was performed with RStudio (Version 1.1.463). We determined the characteristics of Healthy Coping feedback messages from selected diabetes mobile apps by descriptive analysis. The associations between 3 feedback messages dimensions and the Healthy Coping domains were calculated by Chi-square or Fisher’s exact tests with a 95% confidence interval (CI).

Results

Identification of Apps

Our search identified 1,749 apps from iTunes and Google Play stores (Figure 4). After removing the duplicate apps, the remaining 1,193 apps were screened via a multilevel review process, and 242 eligible apps were included in our study. Of those 242 eligible apps, 156 apps (66 from iTunes and 90 from Google Play) generated feedback messages for review in the study.

Figure 4.

Flow diagram for healthy coping related diabetes mobile applications found in 2 app stores (iTunes and Google Play).

Feedback Messages Collection and Analysis

We collected 473 feedback messages from 156 apps. We analyzed the feedback messages following domains, timing, intention, content aspects, including feedback purpose and feedback response. Regarding the diabetes-related measures domain, the majority of messages were related to blood sugar (219, 46.3%) and followed by mood domain (128, 27.0%). On the other hand, feedback messages related to blood pressure measurement were the least frequently generated (20, 4.2%).

Table 2 shows the examples of how we analyzed feedback messages by the proposed framework. A total of 473 feedback messages were analyzed and characteristics of Healthy Coping feedback messages are presented in Table 3. Regarding the timing dimension, most of the feedback messages in each domain were user-initiated. However, half of the messages in the mood domain were system-initiated (66, 51.6%) without statistical significance difference between the means of least square of counts of feedback messages across Healthy Coping domains by the feedback messages characteristics (P = .72).

Table 2.

Example of Feedback Messages Analysis.

Feedback messages	Domain	Timing	Intention	Feedback purposes	Feedback response
1. Your current blood glucose level is relatively low. A common approach to improve hypoglycemia is eating some small snacks containing 15 grams of carbohydrates (such as half a cup of juice, a cup of skim milk, 2 coffee sugar or glucose tablets 2-5 tablet, etc.) and 15 minutes later test for blood glucose. If you need emergency medical assistance, call 911 immediately.	Diabetes-related measures – blood glucose	User-initiated	Encourage	Warning, suggestion, self-monitoring, acknowledging	Real-time
2. Your estimated A1C level is 5%. Great job, keep up the good work! See your updated A1C insight!	Diabetes-related measures – HbA1c	User-initiated	Neutral	Self-monitoring, Acknowledging	Real-time
3. DON’T GIVE UP! OK, so you missed your exercise goal last week. Remember, exercise is an important part of managing your blood glucose. Everyday activities like walking your dog for 30 minutes can count too. Always talk to your doctor before starting or changing an exercise plan.	Physical exercise/activity	User-initiated	Encourage	Suggestion, self-monitoring, acknowledging, goal setting, behavioral contract	Real-time

Table 3.

Characteristics of Healthy Coping Related Feedback Messages (N = 473).

	Diabetes-related measures												Physical exercise/activity (n = 36)			Mood (n = 128)
	Blood glucose (n = 219)			Blood pressure (n = 20)			HbA1c (n = 24)			Weight (n = 46)
	n	%	P value	n	%	P value	n	%	P value	n	%	P value	n	%	P value	n	%	P value
Timing
System-initiated	45	20.5	<.001	5	25.0	.03	6	25.0	.01	7	15.2	<.001	8	22.2	<.001	66	51.6	.72
User-initiated	174	79.5		15	75.0		18	75.0		39	84.8		28	77.8		62	48.4
Intention
Encourage	39	17.8	<.001	6	30.0	<.001	3	12.5	<.001	6	13.0	<.001	5	13.9%	<.001	51	39.8	<.001
Neutral	175	79.9		14	70.0		21	87.5		40	87.0		31	86.1%		77	60.2
Discourage	2	0.9		0	0.0		0	0.0		0	0.0		0	0.0%		0	0.0
Content
Feedback purpose
Warning	34	15.5	<.001	3	15.0	<.001	0	0.0	<.001	0	0.0	<.001	0	0.0%	<.001	2	1.6	<.001
Suggestion	21	9.6		3	15.0		0	0.0		3	6.5		3	8.3%		8	6.3
Self-monitoring	219	100.0		15	75.0		24	100.0		46	100.0		36	100.0%		128	100.0
Acknowledging	172	78.5		15	75.0		18	75.0		39	84.8		28	77.8%		62	48.4
Reinforcement	3	1.4		0	0.0		0	0.0		0	0.0		1	2.8%		4	3.1
Goal setting	20	9.1		3	15.0		0	0.0		6	13.0		3	8.3%		4	3.1
Behavioral contract	1	0.5		0	0.0		0	0.0		3	6.5		3	8.3%		3	2.3
Feedback response
Real-time	176	80.4	<.001	15	75.0	.03	18	75.0	.01	39	84.8	<.001	30	81.1	<.001	62	48.4	.72
Delayed	43	19.6		5	25.0		6	25.0		7	15.2		7	18.9		66	51.6

Regarding the intention dimension, most of the feedback messages across all domains had neutral intentions, which means the user is doing well and ask to continue the activity. For example, a message states, “Your estimated A1C level is 5%. Great job, keep up the good work! See your updated A1C insight!” Regarding the content, most of the feedback message purposes across all domains were self-monitoring (75-100%) and acknowledging (48.4-84.8%).

Regarding the purpose under content dimension, all the feedback messages for blood sugar under the diabetes-related measures domain were about self-monitoring (219, 100%) followed by acknowledging (172, 78.5%), warning (34, 15.5%), suggestions (21, 9.6%), goal setting (20, 9.1%), reinforcement (3, 1.4%), and behavioral contact (1, 0.5%) with statistical significance (P < .001). Only blood sugar under diabetes-related measures and mood domains encompassed all 7 purposes. On the other hand, there were no messages generated for warning, suggestion, reinforcement, goal setting, and behavioral contract in HbA1c of diabetes-related measures domain. Regarding the response under the content dimension, more than 75% of feedback messages among all Healthy Coping domains were real-time responses, except mood where 51.6% of messages were delayed responses without statistical significance (P = .72). Overall, the outcome of the analysis reveals that there is a lack of structured feedback messages across Healthy Coping domains.

Discussion

Healthy Coping is an instrumental diabetes self-management principle. Healthy Coping supports monitoring of diabetes-related measures, physical exercise/activity, and mood of people with diabetes. Although the Healthy Coping principle significantly improves diabetes control (22, 24), Greenwood et al ⁴³ found that only 24% of utilized technology for integrating DSMES mentioned Healthy Coping. Even though more than 318,000 health apps were in the market worldwide, there was a lack of study in Healthy Coping feedback messages from diabetes mobile apps. Hence, we aimed to understand Healthy Coping related diabetes apps and evaluate feedback messages based on the behavioral change theories. Our study evaluated a total of 473 feedback messages generated from 156 diabetes mobile apps regarding the Healthy Coping principle. Because there was no framework for evaluating Healthy Coping feedback messages from diabetes mobile apps, we adopted 2 validated models as we designed feedback message evaluation framework: the motivational message model by op den Akker et al ⁴¹ and the behavior theory-based model by Strong et al. ⁴⁵ For our feedback messages evaluation framework, we analyzed feedback messages across Healthy Coping domains by timing, intention, and content dimensions (purpose and response). We found that there were unbalanced numbers of feedback messages across 3 Healthy Coping domains, and also 3 dimensions of timing, intention, and content (feedback purpose and feedback response).

In this study, we designed our framework for evaluating Healthy Coping feedback messages from diabetes mobile apps. However, it can be extended to evaluate all 7 principles of ADCES7^®, which are Healthy Eating, Being Active, Monitoring, Taking Medication, Problem Solving, Reducing Risks, and Healthy Coping by adding the feedback domains. Furthermore, our framework had the generalizability that can be used to evaluate feedback messages in other clinical areas or other mobile apps because the backbone of the model is stable. Future studies can change the feedback domain, which depends on the goal of individual study.

Conclusion

This study suggests that current feedback messages in diabetes apps regarding the Healthy Coping principle were not theory-based. Moreover, there was an unbalanced distribution of feedback messages across healthy coping domains and feedback dimensions. For example, blood glucose data entry in the diabetes-related measures domain generated approximately half of the entire feedback messages. In contrast, feedback messages generated from blood pressure, an important measure in the diabetes-related measures domain, were very few. Diabetes mobile apps should apply behavioral change theories as feedback messages are developed. We presented a feedback message evaluation framework by adopting the model of motivation messages by op den Akker et al ⁴¹ and the theory-based model by Strong et al. ⁴⁵ There was an unbalanced distribution of feedback messages across dimensions of timing, intention, and content (feedback purpose and feedback response). Diabetes mobile apps also should include more principles of DSMES, such as Healthy Eating, Being Active, Monitoring, Taking Medication, Problem Solving, Reducing Risks, and Healthy Coping to provide an effective DSMES and promote behavior change. We believe our framework for evaluating Healthy Coping feedback messages from diabetes mobile apps is an initial step to study feedback messages for other principles of ADCES7^® to enhance DSMES effectiveness for people with diabetes using diabetes mobile apps. Our study will also further framework development and validation against other DSMES guidelines. Furthermore, our framework can be used to evaluate feedback messages of mobile apps in other clinical areas.