Abstract
Objective
Goal setting is an effective strategy to promote physical activity. Commercial apps that tackle physical activity often include goal setting; however, it is unknown whether the implementation of the goal-setting components is congruent with the theory. This study evaluated the quality of goal setting in popular free and paid physical activity apps by assessing the presence of effective goal-setting components.
Methods
A six-item scale was developed based on the goal-setting literature and used for coding each app for the presence/absence of goal-setting components (i.e. specificity, difficulty, action planning, timeframe, goal evaluation and goal re-evaluation). Cohen’s Kappa was used to evaluate inter-rater reliability for each scale item. The number of goal-setting components included in the 40 apps was calculated and the difference between free and paid apps was assessed.
Results
All scale items achieved satisfactory inter-rater reliability except ‘goal evaluation’. The most frequently included goal-setting components in popular physical activity apps were ‘goal specificity’ (95% of the apps) and ‘goal timeframe’ (67.5%). Conversely, only 47.5% and 25% of the apps implemented ‘action planning’ and ‘goal difficulty’, respectively, and none included ‘goal re-evaluation’. No differences emerged between free and paid apps.
Conclusions
The quality of the goal-setting strategy in popular physical activity apps could be improved by introducing components scarcely implemented to date. In particular, tailoring the goal difficulty to the users’ ability level and re-evaluating the goals based on achievements should be implemented to increase the quality of goal setting.
Keywords
Introduction
Mobile applications (apps) represent one of the widespread and accessible
PA apps available in the app stores are rarely evidence based or evaluated using high standard research methods, such as randomised controlled trials.3–6 In addition, randomised controlled trials can be slow and resource-heavy, 7 hence they may not be feasible considering the scale of the health app market. In the absence of high-quality evidence of effectiveness, previous studies attempted to assess the quality of the apps by focusing on adherence to theory, most commonly the presence of behaviour change theory, operationalised as the inclusion of behaviour change techniques (BCTs). 8 Results evidenced that the most common BCTs included in PA apps were self-regulatory BCTs, such as goal setting, self-monitoring, and feedback on behaviour.9–12 Self-regulatory BCTs have been shown to be associated with effectiveness of interventions targeting PA.13,14 Among this group of BCTs, the role played by goal setting is particularly crucial in promoting PA behaviour change.15,16 While feedback on behaviour and self-monitoring provide objective behavioural parameters, goal setting implies a prospective and proactive decision about the future standard (i.e. target behaviour) at the basis of the self-regulatory process. The relevance of goal setting in behaviour change has been acknowledged by prominent theories, as highlighted by Michie and colleagues, 17 and supported by meta-analytic evidence. 18
Even though previous reviews showed that a goal-setting BCT is often implemented in PA apps,
8
the quality with which such a BCT was delivered remains unclear. While assessing the presence of BCTs allows us to specify the minimum content of
Goal-setting theory23–25 proposed specific goal components that should be addressed when setting behavioural goals. These mainly relate to the specificity, difficulty and timing of the goal being set. For instance, in addition of being specific and measurable, goals should be adequate and tailored to the individual skills (i.e. realistic but challenging goals), and be defined in terms of timeframe. Furthermore, the effect of goal setting on exercise was proposed to be moderated by additional components, namely action planning, goal evaluation and goal re-evaluation. Once a personal goal has been set, the presence of such components allows adjustment of the level and direction of the effort, thus increasing the potential effectiveness of the goal-setting strategy. Goal-setting components have been widely proposed to impact on the likelihood of effectiveness of behaviour change interventions.25,26 Furthermore, previous research syntheses evaluating the effectiveness of goal-setting components on PA behaviour change interventions found that multi-component goal-setting interventions are generally effective in promoting PA behaviour.15,16 Therefore, goal-setting components can be considered as a theoretical and evidence-based guide for assessing the quality of PA interventions that implement goal-setting strategies.15,16,25 To date, however, no studies have assessed the quality of the goal setting in popular PA apps, defined as the presence of goal-setting components.
The aim of this study was to evaluate the quality of goal setting in popular publicly available PA apps by assessing whether and how goal-setting components were implemented in popular PA apps. The specific objectives were to: (1) describe and quantify the presence of evidence-based components of goal setting in PA apps, and (2) assess the presence of goal-setting components in free versus paid apps.
Methods
Study design
The present study was a review and content analysis of the implementation of goal-setting components in PA apps.
App screening process
The sample was identified from 400 highly ranked free and paid apps from the ‘Health & Fitness category’ in iTunes and Google Play stores to represent the apps that were most visible to the user. This method of assessing popularity has been used in other studies assessing apps.27–29
The inclusion criteria were:
Behaviour targeted is PA. The app targets adults. The app had stand-alone functionality (i.e. does not need to be linked with peripheral devices, such as pedometers; and does not require paid membership to access the app).
Exclusion criteria:
Primary aim of the app is focused on multiple behaviours or behavioural outcomes (e.g. apps targeting both PA and diet), as it would be difficult to compare apps targeting multiple health behaviours. The app focuses on increasing PA as part of management of a long-term condition, as it may not be comparable to apps targeting PA levels in healthy individuals. Apps sold in a pack (i.e. ‘bundle’) were excluded, as it is impossible to extract the popularity ranking from a pack of multiple apps. These apps are very often positioned within the ranks separately.
Sample identification
The target sample for the present study was 40 PA apps. The sample was obtained from a previous review and content analysis of the most popular PA apps (for details see Michie et al.
8
), which randomly selected and assessed 65 apps for the presence of BCTs. This previous review investigated top-ranked free and paid apps from iTunes and Google Play. Identifying the sample of apps for the current study from the previous review ensured time efficiency and that the most popular PA apps were assessed. As the apps were similar in terms of their primary function, it was deemed that 40 apps would suffice to characterise the popular apps from the market. Five app types/categories were identified in terms of their primary function: workouts (
The summary of the app sample with app types/categories in presented in Table 1. Apps that met the inclusion criteria were identified until a total
The summary of the app sample with app types/categories.
Measures
Goal setting theory24,25 and its application to exercise and sport contexts30,31 were used as theoretical references for (i) identifying the components of appropriate goal-setting strategies to support PA, and (ii) specifying the evaluation criteria for assessing the quality of goal setting in PA apps. Furthermore, recent reviews of goal-setting strategies in behaviour change interventions targeting PA, diet, and smoking cessation served as references for refining the evaluation criteria 32 and defining the scoring of the resulting scale. 33
The following components identified in the literature were used in the current study to assess the quality of goal setting in PA apps:
Specificity: Specific goals have been shown to lead to higher levels of performance than no goals or general ‘do your best’ goals. Goals should be measurable and defined in behavioural terms.25,30 Difficulty: even though goal difficulty is associated with performance, goals should be commensurate to the individuals’ ability level to ensure that goal achievement constitutes a challenging but doable task at the same time.
25
If goals are too difficult they can lead to failure and undermine self-efficacy beliefs, while goals that are too easy can lead to mediocre performance or demotivation.
26
For these reasons, an early assessment of the individual skills is required to define the most appropriate goals. Timing/timeframe: Research in sport contexts has constantly evidenced the importance of setting both short- and long-term goals.
31
Long-term goals provide the direction and final destination for individuals, while short-term goals are important to provide feedback on progress towards the long-term goal. When both long- and short-term goals are not present, goal-setting research in exercise
16
and primary care
34
has shown that interventions targeting short goals alone are more effective than interventions that target long-term goals exclusively. However, the definition of the goal-setting timeframe in exercise and PA promotion is somewhat vague. In a sport context, long-term goals mainly refer to seasonal or Olympic goals, whereas in interventions targeting PA the length of interventions is extremely heterogeneous, varying from a couple of weeks to 1 year.
16
In the exercise and PA promotion context, there are no clear and well-defined time boundaries that differentiate short-term from long-term goals. Based on the timeframe of goal setting in current PA apps, for the purpose of the present study we considered daily goals as short-term, and weekly or farther goals as long-term goals. Action planning: Goals should be supported by a plan that specifies the modalities in which the goal will be achieved. By specifying when, where, and how to act, action planning helps individuals to implement their intentions and initiate the goal faster than those who do not form action plans.
35
For this reason, it is important to make specific plans that describe how the goal will be achieved. Goal evaluation: Performance should be constantly evaluated, highlighting individual progress towards the goal. The evaluation process allows individuals to adjust their behaviour (i.e. effort and strategies) to match what the goal requires or to redefine goals based on failures or successes.
25
Goal re-evaluation: Based on goal evaluation outputs, goals should be constantly redefined in order to set new ones that are commensurate to individuals’ achievements and progresses. When individuals constantly exceed their set goals or, conversely, repeatedly do not reach them, it might be the case to review behaviour goals (i.e. redefine more difficult or easier goals).25,30 The goal evaluation and goal re-evaluation constitute an iterative loop that helps individuals to progress over time and sustain their motivation and commitment by providing achievement experiences and preventing failure.
Based on these components, a scale was developed to assess the degree to which such components are implemented in PA apps (see Table 2).
Evaluation criteria and scoring for each goal-setting component.
Procedures
Two researchers (DB, PB) downloaded the 40 apps identified and familiarised themselves with the app functionalities, focusing on features related to goal setting. Ten apps were user-tested and independently assessed for the presence/absence of each goal-setting component, after which the researchers met to compare results and discuss discrepancies. Consequently, the assessment tool was refined, and the remaining 30 apps downloaded and independently evaluated. Disagreements were resolved through discussion between DB and PB following the completion of app coding.
Data analysis
Inter-rater reliability was assessed on all the 40 apps using Cohen’s Kappa with two raters.
36
Cohen’s Kappa cut-off values considered for the study are described in McHugh.
37
For the purpose of the present study we decided to consider a less restrictive cut-off value of .50 and solving potential disagreement at a later stage. We anticipated coding discrepancies due to the novelty of the coding instrument and because the assessment of the implementation of goal-setting components was inferred from app user-interfaces rather than from text descriptions. A score was assigned to each specific app on the basis of the agreed components. Quantitative data were analysed using frequencies and percentages of the scores of each goal-setting component. Differences in terms of overall quality of goal setting between free and paid PA apps were tested with
Results
Results indicated levels of agreement varying from none to almost perfect (see Table 3). The item goal evaluation was removed from the scale since the level of agreement between raters was below .50.
Cohen’s K for the items representing the quality of goal setting in physical activity.
*
As shown in Table 4, 95% of the apps (38/40) set goals that were specific and measurable while just 10 apps out of 40 (25%) tailored the difficulty of the goal to the user level of expertise. Timing is one of the most common goal-setting components in PA apps: 27 apps (67.5%) specify or ask the user to specify the timeframe for achieving the goal. Specifically, 23 apps (57.5%) included a goal timeframe that comprehends both proximal and distal goals, one app (2.5%) adopted a proximal timeframe alone, and three apps (7.5%) included a distal timeframe alone. Furthermore, 47.5% of the apps (19/40) provided a strategy to enable users to make specific plans to achieve the goal. The goal re-evaluation component was absent in the sample of PA apps assessed.
Frequencies of the presence of each goal-setting component in PA apps.
Presence of goal-setting components and score for each app are shown in Table 5. The
Goal-setting components and overall score for each of the apps and mean score for free and paid apps (scale 0–5.5 points).
Note: all apps are grouped by price and ranked by goal-setting score;
Type abbreviations = W: workout app; S: step counter; T :tracking of movement; RP: running programmes;
Goal-setting component abbreviations = S: specificity; D: difficulty; Tpd: proximal and distal timing; Tp: proximal timing; Td: distal timing; AP: action planning.
Discussion
Principal findings
This study evaluated the quality of goal setting in publicly available popular PA apps by assessing whether goal-setting components were implemented in those PA apps. Overall, popular PA apps lack theory-based goal-setting components that are key for the efficacy of the goal-setting strategy. Specifically, results showed that 38 apps (95%) set specific and measurable goals, allowing each user to evaluate whether the goal has been achieved or not. However, the implementation of the remaining goal-setting components was limited, and goal re-evaluation was absent from app features. There were no differences in the score of goal-setting components between free and paid apps.
Remarkably, only 10/40 apps (25%) tailored the goal difficulty to the users’ experience and ability level, highlighting opportunities to enhance the behaviour change potential of PA apps. Individuals are motivated and committed with their goal if the goal is achievable but challenging at the same time,26,38 and it is known that tailoring intervention content positively influences engagement with digital interventions. 39
Nearly 70% of the apps (
Furthermore, 47.5% of the apps (19/40) provided features that enabled action planning. This is similar to previous studies that assessed the presence of BCTs in PA apps.9,12 It is important that specific plans are set to facilitate the effectiveness of goal-setting strategies and, consequently, to promote goal attainment. 35 In agreement, recent qualitative research found that tailored action planning features are deemed important by users for promoting engagement with commercial PA apps. 40 Therefore, commercial PA apps could benefit from the implementation of action planning features, for instance by acknowledging potential facilitators and barriers of goal achievement (e.g. weather, weekdays/weekends). 41
Behaviour change strategies are not effective under all conditions, and their translation onto usable intervention elements, such as app features, should consider their parameters for effectiveness. 42 This study pinpointed clear opportunities for improvement of the goal-setting strategy in commercial PA apps. Previous findings showed significant positive effects on PA when goal modifications occur on the basis of individual achievements, 16 probably because individuals, as suggested by Bandura, 38 are more likely to be confident and stay motivated if goals are re-evaluated and adjusted as a function of achievement. Goal setting is an iterative process of continuous cycles of evaluation and goal re-definition, rather than a static decisional point. 30 In support, digital interventions assigning PA goals that adapt to individual achievements (e.g. level of PA over time) were more effective in promoting PA than interventions where goals were static. 20
Adapting app content to the user’s progress has also been hypothesised as an effective strategy to improve engagement with digital interventions targeting health-related behaviours. 43 However, in the current study, app features implementing goal setting were largely fixed and unresponsive to changes in the user’s performance. Specifically, none of the apps provided the opportunity to re-evaluate the goal, even though digital technology enabling goal re-evaluation based on achievement has been developed for some time. 44 Recent advanced methodologies in the field of digital interventions, such as Just in Time Adaptive Interventions, 45 seem particularly suited for developing PA apps that leverage time-varying information (e.g. the user’s progress towards the goal) to implement dynamic goal-setting strategies.
Strengths
This study followed a systematic method of sample identification and assessment. First, the sample was identified through screening of the 400 most popular apps, likely representative of apps that many individuals have downloaded and used. Second, the sample was assessed independently by two reviewers by user-testing the apps, rather than basing the app evaluation on text descriptions found in the app stores.
Limitations
Although apps were left running on the smartphones to assess any features related to goal setting that were not evident at all times, it is possible that some of the features were missed. Second, even though a Kappa below .60 is suggested to indicate inadequate agreement, 37 we used a less restrictive cut-off value of .50 as we envisaged some discrepancies due to the novelty of the assessment tool and because of the implementation of goal-setting component was inferred from graphical details in the apps rather than from a scientific description of behaviour change interventions. However, this is the first attempt to characterise goal-setting components in popular apps; discrepancies between raters were discussed, and the instrument was revised and improved accordingly. Third, the ‘goal evaluation’ goal-setting component did not achieve satisfactory agreement between raters and was excluded from the scoring form. As implemented in current PA apps, ‘goal evaluation’ design prevented a clear distinction from ‘feedback on behaviour’. The distinctive characteristics of these two components were based on small graphical differences, which were not interpreted similarly by raters. Finally, the generalisability of our findings warrant caution as it is possible that some features of goal-setting components in current PA apps were not captured in our study sample. However, we user-tested the most popular and commonly downloaded PA apps in iTunes and Google Play, which ensures we assessed a sample of apps that many individuals are using, and this therefore increases the relevance of our study.
Implications
First, research is needed to assess the quality of delivery of theory-based constructs through app features. Second, there is a need to understand how users utilise the PA apps and how particular components of the goal setting can be adapted to the user to facilitate PA behaviour.
Developers of digital interventions should consider the inclusion of the full range of components of the goal setting to increase the behaviour change potential of the PA apps. Tailoring the goals to user’s needs and re-evaluating the goals based on the performance should be utilised in the PA apps. The assessment criteria used in this study can be used as a guidance to develop the goal-setting features of apps.
Conclusions
This study showed that the quality of the goal-setting strategy in PA apps could be improved. Some of the components comprising the most effective goal-setting strategy to increase PA have been utilised infrequently, or not utilised at all, in publicly available popular PA apps. Apps enable convenient features that facilitate monitoring of goal attainment, such as automatic record of PA sessions and convenient logging of behaviour. However, there is a vast opportunity for improvement of goal-setting features, specifically by tailoring the goal setting to the user and adapting it according to user performance to increase the likely efficacy of PA apps.
Footnotes
Contributorship
DB, PB, AD and PS conceived the study. DB, with support from PS, developed the measures for the study. PB extracted the sample of physical activity apps to be assessed. DB and PB coded the apps, analysed the data, and drafted the first version of the manuscript. All authors reviewed and edited the manuscript.
Conflict of interest
The authors declared no conflicting interests.
Ethical approval
Ethical approval was not needed for this study.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: PB is a PhD student funded by the Medical Research Council.
Guarantor
DB and PB
Peer review
This manuscript was reviewed by two reviewers who remained anonymous.
