Abstract
Text messages are widely used to deliver intervention content; however, sending more intensive messages may not always improve behavioral outcomes. This study investigated whether message frequency was associated with daily physical activity, either by itself or in interaction with message content relevance. Healthy but insufficiently active young adults (aged 18–29 years) wore Fitbit activity trackers and received text messages for 180 days. Message frequencies varied daily at random, and messages were sent from three content libraries (40% Move More, 40% Sit Less, 20% Inspirational Quotes). Contrary to expectations, the results revealed a null association between total daily text message frequency and physical activity, both for daily step counts and moderate-to-vigorous physical activity (MVPA) duration. Additional analyses revealed that the daily frequency of messages with relevant content (i.e. Move More, Sit Less) was not associated with physical activity, but the daily frequency of messages with irrelevant content (i.e. Inspirational Quotes) was negatively associated with physical activity. We concluded that the effectiveness of text messages in promoting physical activity is impacted by the combination of content relevance and frequency, with frequent irrelevant messages potentially decreasing activity levels. This study suggests that irrelevant message frequency can negatively impact physical activity, highlighting the risks of delivering irrelevant content in digital health interventions.
Keywords
Physical activity in young adulthood can reduce the risk for cardiovascular disease in later life.1,2 However, less than half of all adults attain the recommended aerobic physical activity level. 3 Young adults have widely adopted mobile and wearable technologies that can be used to deliver digital messages to prompt or motivate physical activity. 4 Although it might be tempting to think that increasing message frequency can increase intervention effects, sending more frequent messages to promote physical activity may be counterproductive if the messages disrupt the recipient's ongoing activities and create burden. For our purposes, the optimal daily message frequency (i.e. message budget) is the frequency associated with the greatest level of physical activity, but that frequency is unknown. This study examined the effects of digital message prompt frequency on daily physical activity.
Digital tools are widely used to promote physical activity, with a particular emphasis on asynchronous, unidirectional digital messaging interventions.5,6 Text messages are a common implementation of this approach, but methods such as app-based notifications are also utilized.5,7,8 The potential of digital message interventions has been recognized for their extensive reach, effectiveness, and cost-effectiveness.5,9–12
Digital behavior change interventions are more likely to be effective when users engage with the intervention content. Cole-Lewis et al. 13 differentiated between ‘small e’ engagement, which describes user interactions with the user interface and behavior change techniques, and ‘big E’ engagement, which describes subsequent enactment of one or more targeted health behaviors. Although ‘small e’ engagement is often presumed to be a necessary condition for ‘big E’ engagement, the limited research on this association has yielded mixed results. Park et al. 11 reported that higher message frequency and user engagement (‘small e’) were associated with enhanced health behavior outcomes (‘big E’). On the other hand, a meta-analysis revealed that text messaging-based health promotion interventions significantly enhance health outcomes, noting that interventions employing a decreasing frequency of messages throughout their duration were more effective than those with a constant message frequency. 14 Habituation and message fatigue may be responsible for these effects. 15 Thus, ‘small e’ engagement may not always be monotonically related to ‘big E’ engagement. 16 We hypothesized that sending too few messages may not expose participants to a strong enough intervention dose and leads to less physical activity but sending too many messages may be burdensome and also leads to less physical activity.
Factors like message content can also influence participant engagement. Content that is perceived as relevant to goal pursuit may be more effective than the less relevant content at prompting ‘big E’ engagement in health behavior change because it engages participants’ attention and motivates subsequent behavior change.17,18 In a physical activity intervention, some messages are clearly relevant because they focus on moving more. Messages focused on distinct, relevant behaviors, such as sitting less, may be perceived as slightly less relevant. Messages lacking any behavioral relevance may just be disruptive. To the best of our knowledge, the effects of ‘small e’ message engagements, specifically how content relevance influences daily physical activity, have not been thoroughly examined. Our goal is to examine how various types of content engagements within digital messages affect the physical activity levels of young adults.
To deepen our understanding of the effects of message frequency and relevance on physical activity change, this research involved a secondary data analysis from the Random AIM trial (NCT03907683). The Random AIM trial delivered zero to six messages/day from one of three content libraries: Move More (high explicit relevance to physical activity promotion), Sit Less (moderate explicit relevance to physical activity promotion), and Inspirational Quotes (no explicit relevance to physical activity promotion). Message frequencies for each participant were determined randomly each day, and messages were drawn from a randomly selected content library with pre-specified randomization probabilities (40% Move More, 40% Sit Less, 20% Inspirational Quotes). This study was a secondary analysis to test the hypotheses that daily physical activity would demonstrate (a) positive associations with the frequency of messages containing relevant behavior change content (i.e. Move More and Sit Less messages), and (b) negative or no associations with the frequency of messages containing irrelevant content (i.e. Inspirational Quotes).
Methods
Participants
The sample comprised overtly healthy but insufficiently active young adults. Recruitment was conducted using a multi-channel approach including flyers, a web-based tool (Studyfinder.psu.edu), and word of mouth. Participants were eligible if they were 18–29 years old, fluent in English, free of visual impairment that would interfere with smartphone use and owned a smartphone (iPhone iOS v10 or higher; Android 7 or higher). Participants were excluded if they reported engaging in 90+ minutes/week of moderate or greater intensity physical activity, recorded more than 150 minutes of moderate or greater intensity physical activity during a 1-week ambulatory monitoring period, reported engaging in organized programs with required physical activity, reported any contraindications to physical activity on the Physical Activity Readiness Questionnaire, 19 required an assistive device for mobility, had any condition that limited or prevented participation in moderate-intensity physical activity, were pregnant or planning to become pregnant in the next 6 months, or had a prior diagnosis of cancer, cardiovascular disease, diabetes, or metabolic syndrome.
Measures
The Fitbit Versa or Versa Lite watch, equipped with a three-axis accelerometer and an optical heart rate monitor, was used to track daily physical activity in the study. Participants gave permission for their data to be shared with Fitabase, where researchers accessed minute-level step and heart rate data for each participant upon completion of the study. Any minutes showing zero steps and no heart rate were coded as missing data. 20 Days with more than 10 hours of missing data were treated as missing (n = 2341, 16%). Minutes with at least 100 steps were labeled as moderate-to-vigorous physical activity (MVPA), based on Tudor-Locke et al. 21 Total daily step counts and the duration of MVPA were summed up for every 24-hour span.
Study design
The Random AIM (Adaptive Intervention Messaging) study was conducted at The Pennsylvania State University. It aimed to model responses to text message interventions promoting physical activity among inactive young adults aged 18–29 for six months, residing or working within a 90-mile radius of Centre County, Pennsylvania. This research did not base its target sample size on inferential statistical tests but rather on preliminary work using switch models from control systems engineering to model time-varying responses, aiming for a sample size of 80 participants.
Procedure
The protocol for the Random AIM clinical trial (NCT03907683) was approved by the Institutional Review Board at The Pennsylvania State University. Data collection occurred from April 2019 to July 2020. Prospective participants completed a telephone screening questionnaire. Provisionally eligible participants were scheduled for a training session and a one-week ambulatory monitoring period during which they wore an ActiGraph GT3X+ device on their dominant-side hip. Participants returned the device to the lab and upon review of the ambulatory physical activity data, eligible participants were invited to enroll in the clinical trial. The intervention phase began with a training session where written informed consent was attained. A research team member helped participants to download Fitbit and custom Random AIM mobile applications for their smartphones to track physical activity and deliver intervention messages, respectively. The researcher reviewed print educational materials about physical activity guidelines, exercise safety and the use of the Fitbit Versa smartwatch. The researcher asked participants to identify 10-hour daily windows on weekdays and weekends when they would be willing to receive messages and entered those start and end times on the backend server. Participants were asked to wear the device for a minimum of 10 hours daily that overlapped with their messaging availability window. A test message was sent during this initial session, and participants were trained how to acknowledge message receipt by clicking on the notification when it appeared on their device.
Digital messages were drawn at random from three message libraries with fixed selection probabilities (40% Move More, 40% Sit Less, 20% Inspirational Quotes). Table 1 presents two sample messages from each content library. The Move More and Sit Less messages were written to engage validated self-regulatory processes for increasing physical activity and decreasing sedentary behavior, respectively (e.g. goal setting, planning, social support, outcome expectations). Inspirational Quote messages were selected to be interesting and not to activate concepts related to physical activity or sedentary behavior. The backend server recorded the timestamp when each message was sent and when acknowledgments were received. Messages that were unacknowledged within 30 minutes disappeared from the notification tray automatically. The backend server also delivered up to one “reminder to sync” message daily if physical activity data were not received from the Fitbit device during the availability window.
Sample messages from each content library.
For six months, participants wore the Versa activity monitor and received zero to six messages daily. Message frequencies varied daily within-person with equal probabilities for each frequency. To incentivize engagement throughout the study, participants were compensated for wearing the Fitbit and engaging with the message. For months 1–2, participants received $40 for daily Fitbit Versa use, monthly check-ins, and message allowances, plus a $0.25 bonus for each timely acknowledged message, totaling up to $85. Months 3–4 had the same compensation structure. In Months 5–6, participants got $55 for their involvement, and with message bonuses, they could earn up to $100. As a bonus for completing all portions of the study, participants kept the Fitbit Versa. Additional details about the trial are available elsewhere.20,22–27
Data processing pipeline
The initial data pre-processing step involved merging the minute-level Fitbit heart rate and step count files. Minutes were labeled as valid if they either (a) recorded a step count above zero or (b) displayed a heart rate reading alongside a zero-step count. Minutes with at least 100 steps were categorized as moderate-to-vigorous physical activity. 21 Next, we pre-processed message data by eliminating test and “reminder to sync” messages. Each message was labeled based on its content (Move More, Sit Less, or Inspirational Quotes). Message data and physical activity data were then aligned by timestamps and merged.
Using the combined minute-level activity and messages datafile, we aggregated data into daily summaries for each participant. Variables in this file included daily totals of valid minutes, step counts, durations of moderate-to-vigorous physical activity (using data from days with at least 600 minutes of Fitbit wear time), and frequencies of messages from each library. We created a second dataset for sensitivity analyses with daily totals during the messaging window (i.e. excluding activity outside of the messaging windows).
Next, we calculated a daily variable to represent the time of year when data were collected in relation to distance from the summer and winter solstices. These dates have the longest and shortest daylight periods each year, respectively, and daylight duration is a consistent covariate of physical activity. 28 This variable was based on a cosine function that ranged from −1 (winter solstice) to +1 (summer solstice) and provided a statistical control for physical activity differences as a function of daylight duration and seasonal variations. The COVID-19 pandemic was declared during the study so we created a binary variable to indicate whether data were collected before (0) or after (1) the pandemic declaration on 12 March 2020.
Statistical analysis
Days were nested within individuals in this dataset so multilevel models were estimated to account for dependencies in observations. We used the lmer function from the lme4 package 29 in R 30 to model relations between daily steps, intervention messages, and other predictors using linear mixed-effects models. Each of the three models incorporated predictors such as sample mean-centered age, sex, person-mean centered study day, pandemic status, weekday-weekend designation and the daily photoperiod. Three separate models were estimated using different sets of message data. In the primary model, daily steps were regressed on the frequency of acknowledged messages. Additional analyses were conducted using the frequency of received and sent messages.
To address the zero-inflation of MVPA outcomes, a hurdle gamma multilevel model was implemented using the glmmTMB function from the glmmTMB package 31 in R. The analytic approach used for daily steps was applied here with three models (one for each type of message) and the same set of predictor variables. To handle excess zeros in the MVPA data, zero-inflation was incorporated in the models, with the zero-inflation component reflecting the main effects. All models were defined using the ziGamma family and a log link function, signifying a zero-inflated gamma distribution.
Results
A total of 82 participants enrolled, and two participants dropped out, leaving a total of 80 participants in the analytic dataset. Their mean age was 23.9 ± 2.8 years. Table 2 summarizes demographic characteristics of the sample. Most participants were female (66%), White (56%), Not Hispanic or Latino (96%) and overweight or obese (64%). Table 3 summarizes descriptive statistics for daily message and Fitbit data. The mean time difference between timestamps for acknowledged and received messages across all 28,360 observations was 5.95 minutes (SD = 21.25).
Demographic characteristics of participants (n = 80).
Descriptive statistics for daily physical activity and messages.
The first set of multilevel models tested associations between daily steps and overall message frequencies. Table 4 presents coefficients from three multilevel models of daily steps regressed on the overall frequencies of acknowledged, received and sent messages (left, middle and right columns). Overall message frequency was not associated with daily steps in any of these models. These models adjusted for the decrease in daily steps following the pandemic declaration, greater step counts on weekdays than weekends, and variability in steps as a function of the amount of daylight.
Multilevel model coefficients regressing daily step counts on overall message frequencies across all content libraries.
CI: Confidence Interval.
The second set of multilevel models tested associations between daily steps and the frequency of messages from high relevance (Move More, Sit Less) and low relevance (Inspirational Quotes) content libraries. Table 5 presents coefficients from three multilevel models of daily step counts regressed on the frequency of messages from each content library. In the models using acknowledged and received messages (left and center columns), daily step counts were negatively associated with the frequency of irrelevant content (effects ranged from 82 to 104 fewer steps for every additional message with irrelevant content) but not associated with the frequency of relevant content. In the model using sent messages (right columns), daily step counts were not associated with the frequency of messages in any content libraries.
Multilevel model coefficients regressing daily step counts on content library-specific message frequencies.
The final set of multilevel models tested associations between daily MVPA and the frequency of messages from each of the three content libraries. Due to the zero-inflated distribution of daily MVPA durations, hurdle gamma models were estimated to quantify associations with both the odds of engaging in any MVPA on a given day and the duration of MVPA on days when participants engaged in MVPA. Table 6 reports results from the model of acknowledged messages. Neither the odds of engaging in MVPA nor the duration of daily MVPA were associated with the frequency of acknowledged messages from any content library. Daily MVPA was more frequent after the pandemic declaration and on weekends. Daily MVPA duration was longer for younger participants, before the pandemic declaration, on weekdays, and on days with more daylight hours. Conclusions were identical across models of acknowledged, received and sent messages. Results from the models of received and sent messages are available in a supplementary file.
Multilevel (hurdle gamma) model coefficients regressing daily moderate-to-vigorous physical activity (MVPA) frequency and duration on the frequency of acknowledged messages from three content libraries.
SE: standard error.
Discussion
This study investigated relations between experimentally manipulated message frequency and daily physical activity. Results indicated that neither overall message frequency nor the frequency of messages with behavior-relevant content was associated with daily physical activity. The daily frequency of messages with irrelevant content was associated with less physical activity. This study makes three contributions to literature.
First, our investigation into the impact of message frequency on physical activity levels among young adults offers new insights into digital health communication. Contrary to the positive and negative associations reported by Park et al. 11 and Head et al., 14 respectively, our results indicated that an increase in message frequency did not necessarily enhance physical activity levels. Although high-frequency messaging may contribute to habituation or message fatigue that reduces impact, 15 a meta-analysis by Armanasco et al. 9 revealed that infrequent (less than weekly) messages were ineffective and sending several messages either weekly or daily was associated with beneficial intervention outcomes. Those effects were not statistically heterogeneous so fixed message frequencies (i.e. budgets for sending text messages) appear to be viable for future work with static messaging algorithms. With adaptive messaging algorithms based on control systems, it is possible that optimal messaging budgets will vary—some people may benefit from more or less frequent messages each day than others. Although our multilevel modeling did not specifically investigate individual differences associated with message frequency effects on daily activity levels, this question represents a promising avenue for future exploration. Delving into this aspect could provide valuable insights into identifying which individuals respond more favorably to more intensive interventions compared to less intensive ones, furthering our understanding of personalized health behavior change strategies.
Second, our research highlighted how stimulating engagement with messages that are irrelevant to physical activity can adversely impact daily step counts. One implication of this finding is that using the same channel to send irrelevant messages may compromise message receptivity and physical activity. This observation is critical in the context of commercial app-based interventions, where a single channel often serves dual roles—disseminating both behavior change messages and unrelated content such as updates, membership news, or product promotions. Such a practice may inadvertently diminish intervention effects. Therefore, from a design perspective, it may be advisable to employ separate communication channels for behavior change messages and ancillary information.
This study also offers important insights into the dosing strategies for digital message interventions in behavior change programs. Our analysis shows that participants received 82.6% (SD = 36.8) of the messages sent and acknowledged 68.6% (SD = 38.8%) of them. This high rate of message receipt and acknowledgment suggests that messages have a high reading rate. Dosing considerations can be made based on the desired frequency of activating goal pursuit with no more than 50% inflation to compensate for messages that are expected to be unread. Additionally, experimental dose-finding research is recommended to establish the optimal number of messages required for different populations and behavior change goals.
Strengths of this study included the within-person experimental design, rigorous eligibility screening, use of device-based measures, and the 6-month monitoring period. This study also had limitations. First, the narrow age range of participants, which was confined to individuals aged 18–29 limits the generalizability of conclusions to other age groups. Additionally, this study exclusively focused on message content based on social cognitive theory, without explicitly attempting to engage other theoretical targets for promoting physical activity. This intervention focused on physical activity so conclusions may not generalize to messaging interventions targeting sedentary behavior or other health behaviors. This study did not directly assess the effects of using separate communication channels for behavior change content versus non-relevant information. Given the observed influence of message relevance, future research could beneficially explore the potential advantages of segregating communication channels to enhance the efficacy of digital health interventions. This study was also limited by using static content libraries. It is not clear if conclusions will generalize to libraries with content tailored to user characteristics and that question should be investigated in future research. Finally, our study focuses on within-person associations, without the inclusion of a control group that would allow us to draw conclusions about the average effect of the intervention. Future research would benefit from incorporating a control group, which would enable a more robust comparison and provide clearer insights into the effectiveness of the intervention.
Conclusion
This study offers valuable insights into the complexities of using digital communication strategies. This study examined how text messages impact physical activity in young adults. Sending more messages overall did not impact either daily step counts or moderate-to-vigorous physical activity duration. However, sending more messages with seemingly irrelevant content (e.g. Inspirational Quotes) was associated with less physical activity, indicating that content relevance plays a moderating role on frequency effects. This finding suggests that non-physical activity messages sent through the same channel (i.e. text messages) can reduce daily step counts, highlighting the benefit of separate channels for behavior change messages in app-based interventions. As the digital landscape continues to evolve, this study highlights the need for ongoing research to understand better and optimize the use of technology in promoting health behaviors among young adults.
Supplemental Material
sj-docx-1-dhj-10.1177_20552076241255656 - Supplemental material for Assessing the impact of message relevance and frequency on physical activity change: A secondary data analysis from the random AIM trial
Supplemental material, sj-docx-1-dhj-10.1177_20552076241255656 for Assessing the impact of message relevance and frequency on physical activity change: A secondary data analysis from the random AIM trial by Jingchuan Wu, Deborah Brunke-Reese, Constantino M Lagoa and David E Conroy in DIGITAL HEALTH
Footnotes
Contributorship
All authors satisfy the criteria for authorship.
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Ethical approval
This research was approved by the Institutional Review Board at The Pennsylvania State University (STUDY00009455). All participants provided written informed consent to participate in the study.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Research reported in this publication was supported by the National Heart, Lung, and Blood Institute of the National Institutes of Health under Award Number R01HL142732. JW was supported on a training grant from the National Institute of Aging of the National Institutes of Health under Award Number T32 AG049676. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Guarantor
DEC.
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
