Abstract
Background
Program engagement is positively associated with improved outcomes in mobile health (mHealth) interventions, but little is known about which factors may increase or decrease the likelihood of participants viewing program messages. This study examined the association between daily behavioral goal achievement and likelihood of reading daily messages, and if this relationship is moderated by baseline depressive symptoms in an mHealth weight loss intervention.
Methods
Data come from a 12-week microrandomized pilot mHealth weight management trial that tested the effects of daily messages on behavioral goals among 52 young adults (78.8% female, 61.5% white, ages 21–35). Conditional growth curve modeling was used to regress message viewing indicators onto the number of daily behavioral goals that participants had not met at the time of message receipt, with testing for moderation by depressive symptoms and controlling for covariates clustered within participants over time.
Results
For each additional goal not met at the time of message receipt, participants were 34.8% less likely to read any message sent (p < 0.0001), and this relationship did not appear to be related to depressive symptoms (p = 0.1).
Conclusions
Participants may tend to avoid reading program messages when they know they are not meeting goals in a program, possibly due to motivated information avoidance to prevent negative emotional reactions from anticipated negative feedback messages. Future interventions may want to consider ways to contact participants who may be struggling in programs and also avoiding viewing standard message pushes in order to reduce the risk of disengagement.
Introduction
Living with overweight and obesity is a major risk factor for chronic disease development including several types of cancers, all-cause mortality, and has been associated with lower quality mental health statuses.1–3 Digital behavior change interventions (DBCIs) deliver behavioral intervention content through web-based (eHealth) and/or mobile-based (mHealth) interfaces and have been applied for decades as effective, scalable alternatives to in-person programs and elicit positive behavior change to benefit downstream health outcomes more effectively than usual care.4–8 However, DBCIs are often plagued by rapidly declining engagement, and few predictors of this decline have been identified besides user burden and overall lower success in programs.9,10
One possible barrier to program engagement could be that participants may actively avoid reading program messages when they believe or know they are not doing well in a program. Webb and colleagues termed this phenomenon the “ostrich problem” to reference participants metaphorically burying their heads in the sand. 11 While relevant theories are not actively identified in Webb et al.'s narrative review, several studies described concepts such as how users presented with goal progress that is better than expected will likely experience positive affect, while those confronted with lower progress than desired can expect negative affect, which is congruent with literature for goal setting theory (GST), attribution theory (AT), and the cognitive theory of depression (CTD).11–15 Briefly, GST and AT posit that when individuals are confronted with feedback on goal attainment status, they will likely experience positive or negative affect/emotion following feedback on goal success or failure, respectively. 12 To conserve mental resources, individuals tend to rely on heuristics and interpret these events in similar ways, becoming what is known as their attributional style. 16 Those with negative information processing biases according to CTD are likely to form pessimistic attributional styles—focusing on negative information and interpreting causes of failure as something within themselves that they cannot control or change.12,15,17–20 These pessimistic attributional styles are associated with increased risk for developing depression, can elicit strong negative affective/emotional reactions, and possibly erode self-efficacy over time.12,17,18 This “ostrich problem” is likely a maladaptive coping mechanism of motivated information avoidance done to protect oneself from the negative affect/emotions experienced by being confronted with goal failure, which may be exacerbated among those with pessimistic attributional styles. If participants in a DBCI assume that incoming messages will likely contain goal-discrepant (failing) information, they may choose to not read them, which lowers their program engagement and could increase their risk of complete disengagement from the DBCI program over time. 21
Participants susceptible to depression or other mental health issues may find it easier to join digital rather than in-person programs, as they typically have lower conceptual barriers to enter and are more likely to avoid perceived risk of mental health and/or obesity stigma from others. 22 However, given that depression is known to negatively influence motivation and cognitive control, these participants may also likely struggle in these interventions.23,24 Since the Covid-19 pandemic, rates of overweight and obesity have sharply increased, along with rates of anxiety and depression.25,26 The American Psychological Association reports a >300% increase in US adults reporting depression and anxiety symptoms between 2019 and 2021, and 5.9–7.5% of US adults reported depressive symptoms in 2019 compared to 20.2–31.1% between 2020 and 2021.27–29 For these reasons, it may be prudent to examine if levels of depressive symptoms affect the participant experience in DBCIs that send frequent, tailored, automated messages.
This study is a secondary analysis of data from a completed pilot microrandomized trial (MRT) of just-in-time adaptive intervention (JITAI) messaging among young adults living with overweight or obesity. Briefly, JITAIs are advanced DBCIs that use algorithms to determine when to send messages (or not) at ideal times during participant behavioral states or contexts, when messages will be more likely to be read and evoke meaningful change in behaviors.30–32 MRTs can inform JITAI development by continuously randomizing the probability of any given message being sent (or not) according to known probabilities, collecting covariate data at each decision point whether a message was sent or not, which may then be used to assess a user's proximal behavioral outcomes during times they did or did not receive a message.30,33–37 As an example, the HeartSteps microrandomized JITAI showed that sending brief activity suggestions messages at key times significantly increased proximal step counts in the following 30 min compared to times messages were randomized to not be sent. 34 MRT analyses typically focus on behavioral outcomes, and are not often applied to assess time-varying impacts on future user engagement patterns with the programs in question, which might then contribute to changes in proximal and distal behavioral outcomes.38,39
The objective of this secondary analysis is to evaluate the likelihood of participants viewing program messages as a function of the number of daily behavioral goals currently unmet at the time of message receipt. There are two guiding hypotheses: (1) The likelihood of participants reading push messages will decrease as the number of daily goals they have not currently met increases. (2) This negative relationship will be exacerbated by depressive symptoms, such that participants reporting higher baseline depressive symptomatology will be significantly less likely to read program messages for each additional goal they have not met at the time of message receipt than participants with lower symptomatology.
Materials and methods
Study design and participants
Data for this secondary analysis come from the Nudge pilot trial (clinicaltrials.gov identifier NCT05625061), a 12-week MRT studying the effects of various types of intervention messages on the achievement of daily weight-related behavioral goals for weight loss. This study was approved by the University of North Carolina Institutional Review Board (#16-0775), and informed consent was collected prior to enrollment and data collection. Research staff guided participants through an online consent form via telephone, and afterwards, interested participants completed and submitted the online informed consent forms.
Nudge enrolled 53 young adults between ages 18–35 with a body mass index (BMI) between 25 and 40 kg/m2, self-reporting < 150 min weekly moderate-to-vigorous physical activity, who owned an iPhone (Apple, Cupertino, CA, USA), and had not been pregnant within the past 6 months. Data collection occurred from February to September 2019. Participants downloaded the Nudge study smartphone app, received a wireless scale and activity tracker (Fitbit, San Francisco, CA, USA), and were assigned three daily behavioral goals to promote weight loss: a daily weighing goal, a daily dietary goal, and a daily physical activity goal.35,40
The Nudge MRT decision points occurred at four different times per day (early morning = 7:00 a.m., late morning = 10:00–12:00 p.m., afternoon = 2:00–4:00 p.m., evening = 7:00–9:00 p.m.). At each decision point, the Nudge algorithm would assess participant eligibility for seven different message types based on program decision rules, select one of these messages at random, and then microrandomize with a 50:50 probability on whether to send the message or not. 35 Each message focused on a single behavior, and program decision rules ensured no more than one message was delivered for each of the three daily goal behaviors (e.g. weighing, diet, and physical activity) in a single day. Users would receive a push notification that a new message was available in the Nudge app when received, but message content was not viewable from participants’ lock or home screens. Rather, messages were displayed in the Nudge app on their corresponding behavior page(s) until midnight before disappearing, enabling accurate measurement of message views. Covariate data related to each participant's behavioral status were collected at each of the four daily decision points, amounting to N = 16,425 observations clustered across 4,368 person days for the n = 52 participants, permitting detailed analysis for the time-varying proximal engagement outcomes. A screenshot of the pilot app, including an example of one of the seven types of daily messages, is shown in Figure 1. For further details on the Nudge pilot intervention, see Valle, Nezami, and Tate (2020). 35

Nudge pilot MRT app screenshot.
Measures
The dichotomous dependent variable (DV) in all models is viewing (vs not viewing) received program messages. A message was considered viewed if a participant opened the behavior page on which a given message was displayed between the time of message receipt and when it expired at midnight that same day. This serves as a manifest indicator for the latent construct of proximal program engagement, as validated in previous studies.38,41,42
The primary independent variable (IV) was the number of daily behavioral goals not met at the time of message receipt. This was calculated at each decision point when a message was sent, dummy coding whether each participant had achieved their daily weighing, physical activity, and/or dietary goals (meeting the dietary goal required logging at least two meals and remaining at or under their daily dietary limit), summing these dummy codes, and then reverse-scoring the values ranging from 0–3; such that if a user had met all three goals, they would score 0 “goals not met,” and if they had achieved two goals, they would score 1 “goal not met,” and so on.
Depressive symptoms were measured using responses to the validated Centers for Epidemiology-Depression (CES-D) scale, administered online as part of a baseline study assessment package. 43 The CES-D is a non-diagnostic scale with scores ranging from 0 to 60, with higher values indicating greater depressive symptomatology, and scores ≥16 indicating a risk for clinical depression. 43 Age, gender, and race are applied as time-invarying sociodemographic covariates. Time is indicated via program day. All analyses were conducted using R statistical software. 44
Statistical analysis
The longitudinal associations between the number of behavioral goals not met and the odds of viewing program messages were assessed via multilevel growth models using maximum likelihood estimation with logit transformation to accommodate binary parameter outcomes. 45 All models were specified at two levels: Level one specified repeating measures nested within each individual over time, including viewing a pushed message, program day, and the number of goals not achieved at the time of message push. Level two specified person-level differences, including CES-D scores, age, sex, and race. All covariates were meaningfully centered to disentangle within- and between-person effects, as well as to facilitate moderation analysis between mean-centered CES-D scores and goals not currently met.46,47 As this analysis only focuses on instances where messages are received, all null MRT decision points (i.e., when messages were randomized to not be sent) are excluded from the sample, leaving an effective sample size of n = 6210 messages across 4368 person days. At all stages, the goodness of fit was determined via holistic examination of model fit statistics including AIC and BIC, as well as comparative analysis using likelihood ratio tests (LRTs) where applicable. 48
Unconditional growth models were tested for random slopes and intercepts during model building to accommodate variation in participant starting levels and rates of change over time. Additionally, linear and quadratic functions of time were tested to determine the best-fitting unconditional growth model prior to the inclusion of covariates for conditional growth modeling.
The first conditional growth model included only IVs directly related to the guiding hypotheses (i.e., program day, number of goals not met at message receipt, mean-centered baseline CES-D scores, and an interaction term of CES-D scores and number of goals not met). Next, a second model was specified containing the IVs from the previous model as well as sociodemographic control covariates. These models were tested using an LRT to determine if the model with additional control covariates explained significantly more variance in the DV than the first model.
Results
Participant characteristics
The 52 participants were on average 29.6 (SD = 3.8) years old, with a mean BMI of 31.9 kg/m2 (SD = 4.4). Approximately 78.8% of participants self-identified as female, and 61.5% (n = 32) self-identified as white. The sample was highly educated, as most participants (84.6%; n = 44) had a college degree. Participants on average had low CES-D scores, with an average baseline score of 9.27 (SD = 7.4), and a median score of 6. Reported scores ranged from 0 to 35, with approximately 10 participants (19.2%) reporting scores ≥ 16. Full baseline demographic characteristics are displayed in Table 1. One participant withdrew during the first week of the program due to becoming pregnant, so all summary statistics and analytical models are restricted to n = 52 participants with full data available.
Participant characteristics of Nudge sample (n = 52).
SD: standard deviation; POC: people of color; BMI: body mass index; CES-D: Centers for Epidemiology Scale-Depression.
Three-month BMI includes only n = 51 participants with complete values.
Likelihood of viewing program messages
The best-fitting unconditional growth model with a logit link measuring the likelihood of viewing pushed messages used a random intercept and linear random effects over time. Full results are displayed in Table 2. The LRT of models applying fixed or random slopes for time were both significant compared to the null model. However, holistic fit statistics and the intraclass correlation (ICC) were higher in the random slope model, indicating that it more clearly differentiated person-level differences, also confirmed via visual examination of the data, so random slopes are applied as the basis for conditional growth curve modeling. Table results are displayed in log-odds.
Unconditional growth model (n = 52).
SE: standard error; SD: standard deviation; CI: confidence interval; AIC: Akaike information criterion; BIC: Bayesian information criterion.
Results are displayed in log-odds.
Comparing the conditional growth models, the log likelihood values of both models were almost identical (approximately −2762), rendering the LRT to be nonsignificant with a p-value of 0.95. As a result, the more parsimonious conditional growth model is used for hypothesis testing in this study, detailed in Table 3. Table results are displayed in log-odds.
Conditional growth model (n = 52).
SE: standard error; SD: standard deviation; CI: confidence interval; mc: mean-centered; AIC: Akaike information criterion; BIC: Bayesian information criterion; CES-D: Centers for Epidemiology Scale-Depression; [CES-D*goals]: interaction term to probe for moderation.
Results from this model indicate that users are less likely to view incoming pushed messages as a function of the number of goals they have not met, controlling for the length of time they have been enrolled in the intervention. Exponentiating results of the goals not met variable, for each additional goal not met at the time of message push, users have 0.531 times the odds of viewing any received message (p < 0.0001), controlling for covariates. Restated in terms of probability, for each one-unit increase in the number of goals not currently met at a decision point where a message is delivered, participants are 34.8% less likely to view that message (p < 0.0001), controlling for covariates. Exponentiating results from program day (time), for each additional day in the Nudge study, users have approximately 0.977 times the odds of viewing any pushed messages (p < 0.0001), controlling for other covariates. However, this model provides no evidence to support an independent nor moderating influence of baseline CES-D scores on the likelihood of viewing program messages.
Discussion
This study examines one potential mechanism for reduced engagement in an mHealth intervention whereby the fewer goals participants have met, the less likely they are to view any program messages sent. These findings provide some evidence to support Webb et al.'s “ostrich problem” of motivated information avoidance when participants are presumably aware they have not met their goals in an intervention. 11 While this study supports the first hypothesis that the likelihood of viewing messages is negatively associated with the number of goals not met at the time of message receipt, this study did not support the second hypothesis that this relationship was moderated by baseline depressive symptoms. However, this is not unexpected, as the Nudge pilot study recruited a small number of participants who also reported a fairly low average CES-D score of 9.27 and median score of 6, well below the recognized threshold of 16 to indicate risk of clinical depression.43,49 As research indicates that information avoidance tendencies are correlated with anxiety and depressive symptoms, and depressive symptoms are also likely associated with reduced engagement, this factor still warrants consideration in future analyses with larger sample sizes with increased variability in participant depressive symptoms.11,21,38,50,51 This is particularly relevant since recent research found that during the Covid-19 pandemic, those reporting increased anxiety and/or depression experienced greater weight gains than those with lower reported symptoms, which could indicate an increased need for mHealth weight management interventions.21,25,26
Previous studies have found that users were more likely to read messages and engage with DBCIs when they are meeting their goals and generally succeeding in the program, but few studies, if any, have specifically examined factors negatively influencing engagement.30,35 In a systematic review of qualitative consumer perspective studies, Lyzwinski et al. noted that participants in three studies expressed concern that insensitive or negative messages may create guilt or fear of failing in the DBCI when not meeting their goals. 52
It will be fruitful for future DBCI and JITAI research to investigate opportune methods to connect with participants who may be struggling with a behavior change program, require more assistance than what is currently being provided, and may be at risk of lapse and disengagement. Valle, Nezami, and Tate noted that merely pushing a message to participants may not be sufficient to prevent sustained lapses in program engagement or behavioral goal attainment. 35 Indeed, if a participant is already on a downward slope of not meeting their goals and is unlikely to view any messages sent to redress this issue, they could easily slide into a lapse or full disengagement from the DBCI if they habituate to program messages and ignore or silence app notification settings from their device.
One option may be changing contact methods for users at different states of program success and/or engagement in an attempt to break through this observed reticence to open typical push messages. Varying contact methods such as reaching out via email and text messages, arranging for phone or video calls with intervention coaches, or possibly building in moments of silence by ceasing notifications for a time may disrupt this avoidance-to-lapse pattern and re-invest participants in the program. As an example, a meta-analysis by Schippers et al. found that DBCIs with a mixture of delivery modes (e.g. email, phone calls, and/or interpersonal meetings in addition to mHealth communication) showed greater effect sizes than those only using a single mode of contact. 53 Further, Unick et al. found that providing early treatment non-responders with additional counseling sessions, phone calls, and other resources was associated with almost twice the weight loss compared to early non-responders randomized not to receive such assistance at 12 weeks. 54 While it is true that adding interpersonal coaching components will likely increase the cost and may impact the scalability of these types of interventions, this research indicates that notifications alone may not be sufficient to reach struggling participants, and it would be informative for future DBCI research to investigate the possible influence(s) of alternate contact methods.
Additionally, if participants are less likely to view or read any messages sent while in a goal-discrepant state, it will be important to ensure the contents of any messages they do read will be motivating and not discouraging. Motivated information avoidance can occur to protect oneself from negative emotions upon receiving information anticipated to be negative. 21 Reading goal-discrepant feedback messages can cause feelings of disappointment or guilt according to attribution theory and confirm those anticipations, leading users to continue assuming feedback will be negative if they know they aren’t doing well in a program and exacerbate these avoidant tendencies.12,21 Lastly, it might also be useful to change push notification formatting in these circumstances. While it is helpful to send push notifications containing phrases like “A new message is available in the app” to preserve privacy and enable view tracking from participants opening the app, an alternative option could be to send positive notifications of push messages with language like “Congratulations, you’ve reached a new milestone!” during times of struggle to counter possible negative anticipations fueling their avoidance to view messages.
A core aspect of JITAI development involves building in moments of silence, that is, not messaging participants when they may be less receptive, and focusing on times when they are more receptive to program messages to increase the likelihood of messages contributing to desirable proximal responses and reduce risk of intervention fatigue.32,34 Results from this analysis indicate that participants in this weight management DBCI were likely less receptive to viewing program messages in times when they were not meeting multiple behavioral goals, compared to times when they were meeting more of them. It may be helpful for future studies to test hypotheses related to this finding to determine if there are effective strategies to reach individuals while in this goal-discrepant state. Some possibilities might be including positive language in the app notification message to counter expectations of negative content in incoming messages, or to follow a more just-in-time approach and delay sending until after a behavioral goal has been met, or goal progress indicative of participant effort being measured, and participants might be more receptive to positive or reinforcing information. Ideally, sending messages when these participants are in a more receptive state will increase the likelihood of content being viewed and read and potentially influencing future behaviors.
While this study did not find a significant influence of depressive symptoms in this sample, it is important to note that this is but a single factor, and there may well be other participant characteristics measurable at baseline that could meaningfully influence these observed tendencies. For example, intrinsic motivation for using an mHealth app has been positively associated with user engagement.42,55 It is not difficult to imagine that users who show low intrinsic motivation to use an app might be quicker to demonstrate more avoidant tendencies and lower engagement with a DBCI they know they are struggling with; although to our knowledge, this relationship has not been explicitly examined within the literature.
Limitations and strengths
This study has several limitations: First, the Nudge study was a pilot intervention that recruited a small number of participants and is not necessarily powered for secondary analyses. Thus, results should be interpreted with some caution, and replication of analyses using larger, more generalizable samples would be beneficial. Second, participants in the Nudge pilot had fairly low CES-D scores overall; thus, potential moderating influences of depressive symptoms could not be fully explored. Third, in this analysis, we were only able to measure if participants opened the app and viewed messages, but could not determine the extent they were read after being opened. Fourth, the conditional growth models are unable to control for several time-varying factors: (1) It is unable to control for the number of goal-discrepant messages received and/or viewed while a user is in their goal-discrepant state—only that the more goals they are not meeting, the less likely they are to view any message sent. (2) It is unable to control for unmeasured contextual factors which may influence user viewing behaviors such as forgetting one's phone. These factors are anticipated to contribute to the overall variance of the sample and make it more difficult to detect statistically meaningful signal; however, the high significance value for the primary analysis of p < 0.0001 indicates this is unlikely to be an issue.
Strengths of this analysis include its reliance on objective usage indicators of participant program engagement. The calculation of the DV should also secure temporality as well as statistical association (e.g. since the number of goals not met would influence the message sent and likelihood of viewing, but not vice-versa in the current format). Additionally, the microrandomized aspect of Nudge messages means participants were unlikely to habituate to intervention messaging schedules, increasing the plausibility of causal arguments. Lastly, the brief 3-month duration of the Nudge study could also act as a potential safeguard against declining participant engagement compared to longer studies with 6- or 12-month durations. 38
Conclusion
This analysis shows that the fewer goals participants have met, the less likely they are to read any message sent to them within a weight management DBCI. This follows some theoretically informed mechanisms, although this relationship did not appear to be influenced by baseline depressive symptoms as hypothesized. Future research should consider testing alternate contact methods and strategies to contact users who may be struggling to meet behavioral goals and not viewing program messages and are at risk of lapses in engagement or full disengagement from the program. Careful consideration of behavioral and psychological predictors can potentially guide these efforts with greater chances of success to best help struggling participants who may need more assistance than others.
Footnotes
Acknowledgements
The authors would like to thank Nisha G. O'Shea for additional statistical method guidance for this analysis.
Contributorship
LH authored the first draft of the manuscript, implemented co-authors’ feedback, and conceived the secondary study design. BN, CV, DT: supervision, manuscript review, and methodology.
Data availability
De-identified data from this specific analysis may be made available by request to the corresponding author.
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Ethical approval
The parent study was approved by the University of North Carolina Institutional Review Board (#16-0775), and informed consent was collected prior to enrollment and data collection. This secondary analysis was exempt from ethical approval as all data were de-identified such that the identities of human subjects could not be readily ascertained directly or through identifiers linked to the subjects.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by a grant from the T32 Cancer Health Disparities Training Grant from the National Cancer Institute of the National Institutes of Health (grant number T32CA128582).
Guarantor
LH
