Abstract
Background
Empirical research indicates that impulsive processes that operate below conscious monitoring can undermine peoples’ attempts to change behaviour patterns, especially those that have become habitual. This may, therefore, be a serious challenge for those trying to lose weight. A novel smartphone app-based intervention (ImpulsePal) offers practical strategies to manage impulsive urges to facilitate reductions in the consumption of energy-dense processed food and overeating.
Aim
This process evaluation of ImpulsePal aimed to explore what was delivered/received and used, mechanisms of action, and potential contextual factors impacting intervention engagement and outcomes.
Methods
A mixed-methods process evaluation, with composite analysis of the quantitative (app usage statistics) and qualitative data (semi-structured interviews), was conducted alongside a feasibility randomised controlled trial with individuals with a body mass index of at least 25 kg/m2 who wanted to lose weight.
Results
Of 58 participants receiving ImpulsePal, 56 had successfully shared app usage statistics, and 36 (62%) were interviewed. Although usage statistics indicated reductions in the use of some features, interviews indicated that participants were still using app-recommended strategies without requiring them to open ImpulsPal. Overall, interviews highlighted that participants valued having access to in-the-moment support, felt more aware of their own eating behaviour and influences on it, and felt an increased ability to avoid and reconceptualise, in-the-moment temptations.
Conclusion
This process evaluation offers support for a logic model suggesting that impulse management, using ImpulsePal, can promote healthier eating among those motivated to lose weight. It also highlights the necessity of using multimodal methods to explore the delivery and use of digital interventions.
Keywords
“It helps me to stay on the right path, rather than give in”: Mixed-method process evaluation of the ImpulsePal app-based intervention for weight management.
Introduction
Behavioural or lifestyle-based weight loss interventions aim to support individuals to change eating patterns in order to achieve an energy deficit. Such interventions mainly focus on deliberative precursors of behaviour, such as trying to change beliefs and attitudes to bolster intentions, planning and problem-solving. 1 However, even when people are motivated to change, this does not always result in changes in the target behaviour and even when they do, they are often not maintained over an extended period of time.2–4 The discrepancy between intention and behaviour led to the development of dual-process models (e.g.,5–7) which posit that behaviour is regulated by two qualitatively distinct, yet interacting systems. While reflective processes take a conscious and rational approach to behavioural decisions, by taking into account available information and following slow, deliberate thinking when there is time and motivation to do so, these dual-process models clarify that much of our behaviour is not consciously monitored. 8 Implicit processes are triggered by an associative network where cues are evaluated automatically based on their emotional and motivational significance. If this leads to repeated behavioural routines in particular contexts, habitual behavioural patterns with high automaticity develop. Although these impulsive processes guiding behaviour can operate outside of conscious awareness, the reflective system can always become aware of and potentially override these processes.
It has been increasingly recognised that there is a need for techniques to help people to manage or modify such implicit processes to better achieve health outcomes9,10 resulting in the proliferation of the development and investigation of such techniques.11,12 There are various ways in which these techniques may support individuals to manage impulsive processes and facilitate dietary change. 12 For example, reflective, cognition-change, techniques facilitate people's capacity to engage reflective processes to actively override or interrupt impulsive processes as described by van Beurden et al. 12 One such technique is the use of Implementation Intentions, such as if-then plans, which involves the identification of a cue known to trigger an unwanted behaviour, and consciously resolving to take a particular action when it is encountered. Although this process requires conscious deliberative planning in advance when the targeted cue is encountered, the planned response may be initiated automatically. 13 The implementation intention may also include a response that involves engaging the reflective system (e.g., “then I will think of my goals for healthy eating.”), bringing the behavioural decision into conscious awareness when the cue is encountered. Impulse-focused techniques (e.g., approach/avoidance training) on the other hand, aim to directly alter the impulsive processes that generally lead to unhealthy food choices. Among impulse-focused techniques, the food Go/No-Go tasks have demonstrated the most reliable effects on changing food intake both in and outside of the lab.14–16 An increasing number of studies investigating impulse-focused techniques are being conducted in real-world settings, using more diverse samples (e.g.,17–20), but these studies have not evaluated multi-technique interventions. In addition, there are no in-depth investigations of how impulse management techniques to change eating behaviour are perceived and used by users within their own day-to-day lives.
ImpulsePal is an app-based intervention developed to support dietary change 21 trough targeting the impulsive processes by using both evidence-based impulse-focused and reflective techniques. 12 There are very few other app-based interventions that focus on impulsive processes to change eating behaviour and these only incorporate a single technique. Food Trainer 22 utilises inhibition training operationalised as the Go/No-go task and there is a smartphone app-based cognitive bias modification intervention operationalised as approach-avoidance training task. 17 However, to our knowledge, Food Trainer is the only publicly available app (https://www.exeter.ac.uk/research/foodt/). The feasibility randomised controlled trial of ImpulsePal indicated that participants using ImpulsePal lost around 1 kg more weight (over one month) than those in a control group receiving usual care alone and that minor refinements to the intervention improved the fidelity of intervention delivery. 23
Purpose of the study
We aimed to conduct a pragmatic mixed-method 24 process evaluation alongside the feasibility trial to provide valuable insights into the feasibility of implementation, possible mechanisms of action, and the possible impact of delivery context on intended outcomes, to help refine the programme theory, optimise the intervention, and inform its future full-scale evaluation.25–27
The research questions for the feasibility stage process evaluation of ImpulsePal were as follows:
What were participants’ perceptions of using the ImpulsePal app? To what extent were components of the ImpulsePal intervention understood and used by participants (receipt and enactment)? What factors (e.g., features of the app, change processes) facilitated changes in behaviour (potential mediators and mechanisms of action)? What contextual factors affect engagement with ImpulsePal and behaviour change patterns (potential moderators)? What particular advice in relation to impulse management and lifestyle behaviour change worked well, or not?
Methods
Study design (QUANT + QUAL) and setting
We used a pragmatic mixed-methods approach 24 with two cycles of intervention delivery and data collection at 1-month and 3-month (from baseline). This iterative process of intervention development and evaluation is akin to action-research described by Whitehead et al. 28 and encompasses feedback-loops that consist of (a) systematic data collection, (b) linking data back to the design, (c) taking action by changing/refining the intervention, and (d) exploring the result of the changes made by collecting and analysing more data. The study was conducted in the South West of England and contributed to the doctoral training programme for the lead author. The project was approved by the UK NHS National Research Ethics Services Committee South West – Exeter (Ref: 15/SW/0181). Participant recruitment ran between September 2015 and March 2016 for Cycle 1 and October 2016 and April 2017 for Cycle 2. The feasibility study results are reported elsewhere. 23
Participants
Our participants were sampled from the participants randomised to the intervention group (n = 58) in the feasibility randomised controlled trial. 23 For the qualitative component, an opportunistic sample (any of the participants in the trial who were randomised to the intervention group who were willing to be interviewed at their one-month follow-up appointment) of 36 participants took part in semi-structured interviews and a purposive sample 29 of 22 interviews were initially transcribed. Participants were sampled to give diversity in age and body mass index (BMI) by applying a sampling framework comprising three BMI categories (25–30, 30–35, and 35+) by three age categories (<40, 40–55, and 50+), and filling as many cells within the framework as possible. The purposive sample had a BMI of 33.3 kg/m2 (SD = 4.8) and an average age of 50.0 years (SD = 13.3). Other participant characteristics are provided in Table 1. The participants included in the pool of transcripts were similar to the remainder of the participants in the intervention group in terms of BMI, weight change, and changes in snacking behaviour, but included a higher percentage of females and were slightly older.
Intervention sample characteristics and weight and eating behaviour change at 1 month follow-up.
Data available at 1-month follow-up for n = 26.
Data available at 1-month follow-up for n = 25.
Intervention
ImpulsePal is a theory- and evidence-based self-delivered weight management intervention accessible as an app on Android-based devices. Its systematic development 21 followed the Intervention Mapping protocol. 30 For an overview of the Intervention Mapping approach see the work of Abraham and Denford. 31 The development process and intervention content of ImpulsePal are described in detail elsewhere.21 Briefly, ImpulsePal offers its users access to impulse-focused and reflective techniques to manage impulses towards unhealthy eating/snacking. The core elements of the intervention are (a) Brain Training (Inhibitory Control Training using the Go/No Go task); (b) Urge-Surfing, a mindfulness-based strategy teaching individuals to see a craving to be like a wave that builds up overtime but eventually subsides; (c) Visual dynamic noise when the individual presses an emergency button, preventing them from engaging in craving imagery; (d) ‘My plan’ documenting Implementation Intentions in the form of if-then plans to link situational cues to the activation of alternative responses; (e) Danger Zones, providing the individual with self-selected location-based goal reminders, bringing the decisions into conscious awareness. Figure 1 presents the logic model (visual representation of the programme theory) of ImpulsePal, highlights the elements explored in this feasibility stage process evaluation, and whether quantitative and/or qualitative data was collected to do so. In Cycle 1 of intervention delivery and data collection, participants were provided with ImpulsePal v1 and in Cycle 2 new participants were provided with ImpulsePal v2 which resulted from the data-driven refinements made to the intervention (reported below).
Elements of the logic model explored in the feasibility phase process evaluation are highlighted in blue. aApp usage statistics. bSemi-structured interviews.
Data collection procedures
App usage statistics (QUANT)
Participants’ use of the ImpulsePal app was objectively recorded in the app and automatically sent to the ImpulsePal database hosted on a secure University of Exeter server whenever the smartphone was connected to WiFi. Each time a participant opened the app, their unique identifier code, with a Unix timestamp, name of screen accessed (See Table 2) were logged. The data were downloaded from the database at the end of each participant's study participation (after the 3-month assessment visit).
Names of the separate screens in the ImpulsePal app for which data was collected.
Semi-structured interviews (QUAL)
Individual semi-structured face-to-face interviews were conducted with intervention group participants who were available and willing to be interviewed at the one-month study visit and lasted between 16 and 40 min (M = 24.9 min; SD = 8.4 min). The topic guide was developed to explore elements of the logic model. More specifically, the topic guide explored participants’ experiences of using the ImpulsePal app, elicited facilitators of and barriers to engaging with the app and the various strategies within app, and perceptions of intervention mechanisms and effects. The interviews were conducted by the lead author who has had training and previous experience in qualitative interviewing and analysis32,33 in a comfortable, private meeting room at the University of Exeter Medical School. No relationship between the interviewer and the participants in the analysis existed prior to the feasibility study, however, the interviewer had been able to build a rapport with all participants during the recruitment process and baseline study visits. To improve transparency, trustworthiness and rigour of study findings, the interviewer employed respondent validation (member checking) during the interview to seek clarification or validation from participants ensuring accuracy in interpretation of the data. The purposively sampled interviews were transcribed (as described above). Transcripts were checked for accuracy by listening to the recording again, anonymised by removing any identifiable data (such as names and place names) and imported into NVivo 10 to help organise the data.
Analyses
A composite analysis of the data was conducted whereby the data of both quantitative and qualitative components were analysed in isolation but the findings are integrated for discussion. 24 Integration of the findings is reported in the Discussion section highlighting where the findings from the analysis streams support each other, differ, or provide further insight.
For this process evaluation, descriptive statistics were used to summarise app usage in terms of the number of times participants viewed specific screens. The paper reporting the feasibility study 23 presents the total time spent using ImpulsePal and the number of separate days ImpulsePal is accessed during the first month, during first three months, and following the first month for continuing users only.
All 36 interviews were used immediately and prior to the formal analysis reported here to identify key areas for improvement in terms of navigation, textual content, and barriers to engagement. Notes taken on these by the interviewer directly informed changes to be implemented for version 2 of ImpulsePal, but also provided context to the more in-depth thematic analysis of a purposive sample of 22 interviews reported here. The interview transcripts of the purposive sample were analysed using inductive thematic analysis 34 which involved (a) familiarising with the data by reading the transcripts and listening to the interview recordings, (b) coding the content of the interviews into concepts and continuously updating the coding manual to reflect the ongoing analysis, (c) using constant comparison to check coding was consistent throughout the analysis, and updating definitions regularly to better reflect the data, and (d) merging codes which reflected similar aspects into higher level themes. The lead author conducted the coding and met with other members of the team (CG and JS) frequently during the process of analysis to discuss and refine: (a) codes and categories, (b) interpretation of key interview excerpts, and (c) potential new lines of enquiry. Once all transcripts had been coded, the lead author listened to the remaining interview recordings and compared to the developed codebook to check if these interviews would bring anything to the qualitative analysis not already covered by the transcribed data. Following discussion with two other team members (CG and JS) the decision was made that these did not require to be transcribed or coded as there were no novel categories or themes that would be added through this process. Final themes and illustrative quotes for reporting were agreed by the second author (CG).
Results
App-usage data
An overview of the app-usage analysis is provided in Table 3. Out of the 58 intervention group participants 56 participants (97%) had usage data available. Table 3 reports the number of participants accessing the separate components and the total, range, and average (median due to skewed data) number of screen views. During participants’ first month of using the ImpulsePal app (See Table 3), the brain training game, the if-then plans, and the emergency button were the most accessed key components in Cycle 1. In Cycle 2 these were urge-surfing, if-then plans, and the emergency button. In Cycle 1, 22 of the 26 participants (85%) used the Brain Training game as intended (i.e., two uses or more). In Cycle 2, only one participant (3%) did not use the Brain Training game, five used it only once (17%), and the remaining 24 (80%) used it as intended. The remaining components do not have a specification regarding frequency of use. Following the first month, continued access was observed for 70% of the 56 participants. The number of participants using the Brain Training game and the Emergency Button dropped the least compared to the other key components, but views of these dropped more in Cycle 2 than in Cycle 1.
Impulsepal app use up to three months as screen views for participants accessing the main menu, motivations, or stats screens or the five key components in cycle 1 (N = 26) and cycle 2 (N = 30).
The stacked polar plots in Figures 2 and 3 separate this data and show individual usage of the key components (i.e., number of times key screens had been accessed for each individual) during the first month of use. These illustrate the proportionate use of each of the components relative to the individual's total ImpulsePal usage, again showing that the Brain training and If-then planning sections were accessed most, relative to the other components. It also clearly highlights a number of extensive (super) users in both Cycle 1 and 2, relative to the other participants in the sample.
Individual usage of ImpulsePal version 1 and key components – cycle 1*
Individual usage of ImpulsePal version 2 and key components – cycle 2*
Qualitative analysis
Findings are presented under three overarching themes that link to the process evaluation objectives: “Perceptions and use of ImpulsePal” (use of the intervention), “Impact of ImpulsePal” (potential mechanisms of action), and “Factors influencing engagement with ImpulsePal and outcomes” (contextual factors). Illustrative quotes are presented below using pseudonyms, gender, age, BMI, and weight change. The quotes were selected to illustrate the themes, subthemes, or categories referred to in the sections below and are edited to remove speech hesitancies where this did not alter the meaning of the quote.
Perceptions and use of ImpulsePal
There were two clear sub-themes in how the participants talked about their views and use of ImpulsePal. Talk focused on the usability and acceptability of the intervention as a whole, as something that is delivered via an app, but also went more in-depth by focussing on the specific components of the intervention.
Usability and acceptability of the app
Most participants using either version of ImpulsePal reported finding the app easy to use and navigate, although one felt that the user interface did not flow well and made interacting with specific components difficult, hindering ongoing engagement. However, being easy to use did not necessarily mean correct use of the intervention. Some participants using ImpulsePalv1 highlighted having not read or having misread instructions, affecting receipt and understanding of the intervention (See Figure 1) resulting in using it incorrectly and in some cases discontinuing use altogether. there were some instructions, but … just trying to think, no I don't … For me I got into using it, but that could well be. Because I sort of, if it says you know, next page or something like that, you know [taps table] go straight on and see what happens and then I got to the point where I don't know what happened, so don't I use it now. (V1 Matthew, M, 69 yrs, 38 kg/m2, +0.4 kg).
Participants’ views about ImpulsePal varied; most participants described the app as “a useful tool” (V2, Rachel, F, 31 yrs, 44.3 kg/m2, +0.5 kg) that “helps to understand and deal with why an individual eats the way that they do” (V1 Anna, F, 34 yrs, 39.9 kg/m2, −3.2 kg). However, one participant noted that the app was not meeting expectations in providing immediate answers to deal with in-the-moment issues. It didn’t do anything for me which said “you’ve just been offered a bunch of birthday cake. You’re not going to have any. There's a really good reason why not”. And it's just not that. (V2, Lewis, M, 43 yrs, 26.1 kg/m2, −2 kg)
Specific impulse management strategies
Much of the data focused on the use of the specific components of ImpulsePal and the strategies learnt. Most participants reported that they liked the Brain Training game and that this was the component that they used most. Some participants reported using particular techniques without accessing the app to suit their needs (e.g., if-then planning and urge surfing). Below we report such feedback within identified themes.
Brain training
Participants’ views about the brain training game were fairly consistent. In Cycle 1 many enjoyed the gameplay element, although not everyone understood the game, and most felt it became boring and tedious over time. You know, quite enjoyed the process. But it was quite long time to play it. And I think my motivation to want to keep playing did start to die off after if I’m honest. (V1, Theo, M, 30 yrs, 28 kg/m2, +0.25 kg)
Participants liked to be reminded to do their brain training which facilitated engagement with this technique. I think that's going to get into that habit, which is why I like it comes up with this little thing saying: "don't forget to do your brain training" and it's like: "oh yeah, I'll do my brain training!" (V1 Lisa, F, 46 yrs, 31.3 kg/m2, −2.5 kg)
Changes to the brain training component were made in ImpulsePalV2 (See Table 4). Despite the changes, the repetitive nature of the task remained a barrier to ongoing use. One of the things that interest me about the app was the brain training, but I just found it boring after a while … well it's just the same thing. You know, I mean, I changed it, I adapted it to biscuits and cheese, and this is my little weakness, but then using it, it's the same thing … (V2, Robert, M, 50 yrs, 30.5 kg/m2, −2 kg)
Data-driven refinements implemented from ImpulsePalv1 to ImpulsePalv2.
If-then planning
All participants reported having used the if-then component to set initial plans. Although most found it useful to create if-then plans and look at suggested plans, one participant reported that it required too much effort. And I liked the bespoke, you know, you could adapt your responses to what you found challenging yourself. (V2, Robert, M, 50 yrs, 30.5 kg/m2, −2 kg)
Participants in Cycle 1 highlighted issues with creating plans for infrequently encountered situations, which required having to fall back on memory. So I haven’t been in a restaurant where I might say to a friend, “Oh look, this is the menu, what could I choose? You know I’m on the diet, what do you think would be best?” I mean, I think that's a good method, but I haven’t sort of had to use that. (V1, Kathryn, F, 71 yrs, 33.9 kg/m2, −1.8 kg)
Participants in Cycle 2 were given the option of setting reminders for their if-then plans, but only few had made use of them. Some reported that they would have liked to have set reminders, indicating that they had not noticed this function. These are the two main reminders I got…. because sometimes, you know, you don’t think but then you keep getting reminders on your phone, it beeps or vibrates, whatever you choose the setting, and then you stop and think about it. It's quite nice, so yeah. (V2, Hollie, F, 33 yrs, 31.6 kg/m2, +2.8 kg)
Urge surfing
Those who used urge surfing were very positive about the technique, quickly learnt how to use it, no longer needed their phone to be able to apply it, and adapted the technique to fit their lifestyle. Well it just, I really really like the Urge Surfing!… never would it have crossed my mind to actually sit there and think about the urges to eat, usually you do anything to sort of distract yourself from it, which is great if you're in a situation when you can distract yourself from it …..but that's [Urge Surfing] actually good because you can use that in any situation and you can … sitting there and focusing on it, I don't know why it works, but it does seem to really help, yeah, it's strange, but, yeah, it's good I like that, it's very good. (V1 Anna, F, 34 yrs, 39.9 kg/m2, −3.2 kg)
In ImpulsePalV2 an audio-guide for urge surfing was added (See Table 4). This was used by some, but others felt the text-based steps were enough. I think surfing the wave one, I used to spend a bit more time on, where I go on with sort of headphones, had a bit of time away somewhere. (V2, Harry, Male, 40 yrs, 30 kg/m2, −2.5 kg).
Danger zones
In Cycle 1 only a few reported using the Danger Zones, and very few used it successfully. Most reported that this function was not easy to use, and some did not understand what it was for or how to use it. There were also reports of technological issues and frustrations with the notifications going off without being in the selected location or at the right time. More use as intended was reported in Cycle 2, although some participants noted that they did not feel the need to use this function. In fact I sort of moved away from the brain training and carried on with the danger zones considering I’d get it somewhere I think, so that my little messages, you know, steer clear of the pasties, step away from the fridge that sort of thing, it helps a great deal, yeah (V2, Toby, M, 47 yrs, 36.3 kg/m2, −0.5 kg) I’ve looked at it and I’ve thought about it. Should I put the supermarket? But because I felt that I’ve had the willpower not to succumb to treats and things, so what we used to see as treat, so I haven’t, no. (V2, Louise, F, 65 yrs, 29.8 kg/m2, −0.6 kg)
Emergency button
This was considered to be a useful function to fall back on at critical times “if I’m really feeling, you know, I might fail.” (V2, Georgina, F, 50, 34.4, −2.0) and for some served as a constant reminder with its prominent position on the phone's home screen. However, when participants moved the location of the button on their phone it was easily forgotten. It's just brilliant to have, I think it's a novelty having the emergency button to be able … and to know that you’ve got that there because it is an instantaneous reminder of what you … of what you’re trying to do, I suppose, for me anyway. Because it's on my … the first page.” (V2 Georgina, F, 65 yrs, 29.8 kg/m2, −2.0 kg) And having the emergency button on the phone, very useful. The first two weeks I was using it quite a bit. The next couple of weeks it was just, I didn't think about using it. I think I moved it to a different screen on my phone. (V2, Harry, M, 40 yrs, 30.1 kg/m2, −2.5 kg)
Some participants also reported that they liked receiving the question about their “emergency button event” asking them to rate what the strength of their craving, urge, or impulse was and whether they were successful, partially successful, or not at not acting upon that craving. In particular the style and supportive phrasing of the feedback message which was prompted by their answer. I like the little messages it comes out with, they're kind of encouraging like, y'know, you've resisted that one and it's gonna get easier, and it's like "oh is it?" 'cause I actually literally thought that was it, I just thought it was gonna be continuous struggle but it's like no. and then it comes, so the message it comes up with, I think they're written really nicely, they're quite encouraging and reassuring, so that when you've hit the emergency button and then it asks you afterwards how you did and you haven't done very well, then it does make it sort of clear that that's ok you just, y'know, carry on (V1, Lisa, 46 yrs, 31.3 kg/m2, −2.5 kg)
Some participants noted not needing to use the emergency button, because they did not feel tempted or were using other strategies straight away to deal with situations. Yeah, the first couple of weeks but I haven’t used it much at all since then. And I thought about why and maybe it's because it's the most important. The first thing I did in the plan as I say was to remove things in the house and, so they’re not there … But I haven’t needed to use the button. Whether it's because it's in my mind that I’ve got this thing I could do it, but I don’t need to because I’m not tempted. (V2, Louise, F, 65 yrs, 29.8 kg/m2, −0.6 kg)
Impact of ImpulsePal
Users reported ImpulsePal impacting them in a variety of ways which map onto the logic model presented in Figure 1, including (a) increased awareness of influences on eating behaviour and in-the-moment decisions around food choices, (b) fewer strong cravings or temptations, (c) strengthened feelings of control or ability to resist temptations, and (d) changes in eating behaviour. the only time … this sort of thing would, you know, demolish a bag of crisps or something, you just sort of get that out and it's by the time you finished with it, it's taking it [Craving] away and I can say I’ve actually moved off of that now and because I feel that I can and I know how to manage my cravings now and whenever I go into a shop now or that, you know, what are the ones that's on my list, it actually takes away … it makes me think about what I buy rather than just impulse buying snacks and things. (V2, Toby, M, 47 yrs, 36.3 kg/m2, −0.5 kg)
Participants reported being more aware of their in-the-moment food choices and eating behaviour as well as learning and knowing more about the factors that influence these, including identifying and engaging in strategies to avoid, or manage these situations. it helps me to stay on the right path, rather than give in so, yeah, and also avoiding those situations as well, y'know, if you recognise, where you're more likely, I'd never really thought about it. Where I was more likely, and when I was more likely to overeat, or to eat the wrong foods. (V1, Anna, F, 34 yrs, 39.9 kg/m2, −3.2 kg)
Some participants reported noticing reduced frequency of habitual craving or desire to snack, or a reduction in the strength of cravings. But something is definitely making me not have the craving or the automatic because I can’t believe it's willpower alone. It's … there's got to be a little switch must have gone the right way in my brain. Because I just I don’t need it and I’m a completely different person to the person I was when I was here that day. (Georgina, F, 50 yrs, 34.4 kg/m2, −2.0 kg)
In terms of impacts on behaviour, participants also reported making healthier food choices, reducing unhealthy snacking, making changes in portion sizes, and actively engaging in strategies offered in the if-then plan section, or self-identified strategies, to manage tempting situations. I’ve set the danger zone to anywhere around the [location mentioned] which basically everything is. And to kind of remind me until half past nine which is when obviously I should be willing to work. And you know I have been a couple of times where I kind of overridden it and got some breakfast but it's been a … the healthiest option I could possibly find if I sort of overslept and just need something. (V2 Rachel, F, 31 yrs, 44.3 kg/m2, −0.5 kg) I brought my coffee down with me [to work coffee and cake morning] so I had something to hold as well so I wasn’t tempted to grab something because my hands are free. So I think I am more aware. (V2, Rachel, F, 31 yrs, 44.3 kg/m2, +0.5 kg)
Participants also reported increased feelings of control or ability to resist temptations or cravings when they arose. I did find also, that often when I had an impulse to eat something I didn't use the app, but I just thought: "I've got that app, I can go and use the app!" Which worked better if my phone was with me, than when it did when I was in the kitchen and the phone was upstairs. Like: "ah it's upstairs, I can't use it." Which is stupid 'cause … I'm in the kitchen … I'm not actually using it, I'm thinking about it and it's having the effect. (V1, Lisa, 46 yrs, 31.3 kg/m2, −2.5 kg)
Factors affecting engagement with ImpulsePal
A number of sub-themes related to an overarching theme of factors influencing engagement with the ImpulsePal app, were identified as described below.
General digital device use
Most participants mentioned being drawn to the study because it involved testing a novel weight management app. That it was an app on the phone and so it was instantly available to you. You know, you could be in any location and it wasn’t a book you were supposed to read at home. And so that appealed to me. (V1, Kathryn, 71 yrs, 33.9 kg/m2, −1.8 kg)
Some mentioned that they did not always carry their phone with them, were not regular app users, or did not want to use their phone to help manage their eating in social situations, which is consistent with the logic model (See Figure 1). I wouldn't habitually walk around the supermarket with my phone in my hand um, so I'm not sure it would really work for me unless I have to change my habitual use of the phone quite significantly …(V1 Matthew, M, 69 yrs, 38 kg/m2, +0.4 kg).
However, other participants highlighted that they no longer needed access to the app at all times once they had learnt to implement specific strategies. I don't need to go on to it to remember how to do the Urge Surfing, I can, I remember how to do it, so I can sit there and I can just do it and people will be around me talking and I'm sat there and y'know, listening intently, but at the same time sort of thinking about my Urge Surfing. (V1 Anna, F, 34 yrs, 39.9 kg/m2, −3.2 kg)
A few participants used the app on an Android-based tablet or on a partner's smartphone. Although they were able to access the app and learn some of the strategies, because they were not using these devices in the same way they would their own smartphone, this affected its in-the-moment use. One participant reported changing phones and in trying to access the app using the new phone was frustrated with having to start all over again. Their experience with other apps meant they had expected data to be automatically carried over. all my other apps on my phone just sort of … it let me login or they’re already done, whereas that one, you know, I think it's almost a week maybe where I wasn't using that because I haven't logged in. And I find it as well a really frustrating thing, was like I had to do it all again, like all my plans are gone and everything are gone. (V2, Megan, F, 35 yrs, 39.5 kg/m2, −0.3 kg)
Intention or motivation to change eating behaviour
Although participants were recruited into the study based on a desire or motivation to lose weight, some reported that their weight management efforts were not focussed on changes in eating behaviour, or that they did not experience temptations or struggles with their eating. The part of the app that I think you're supposed to use when you're feeling you want to break the regime that you should be on.… hasn't really felt entirely relevant. 'Cause my weight loss programme is more about increased physical exercise. (V1 Matthew, M, 69 yrs, 38 kg/m2, +0.4 kg).
Those wanting to lose weight and wanted to change their eating behaviour, reported situational changes in intentions which affected engagement with the intervention. For example, for some participants the first month of using ImpulsePal coincided with the Christmas period. Yeah, and I did do that a couple of times, but, again sort of Christmas it was just, well I was resigned that I was gonna, y'know, eat what I want and what ever, so, within reason (V1, Alice, F, 53 yrs, 35.5 kg/m2, 0 kg)
Some also reported losing or struggling to maintain motivation over time and others that they would have liked to be able to record their weight to maintain motivation. The thing that actually motivates me more is Google Fit simply because I can put in my weight and I can see how it's going up or I can see it going down. (V2, Lewis, M, 43 yrs, 26.1 kg/m2, −2 kg)
Although ImpulsePal was designed to support people in managing impulsive processes, some participants did report that the succesful impact ImpulsePal was having on their ability to resist something, resulted at times in active avoidance of engaging with the app or its strategies. if you want to eat something and you know it's not good for you … so you've got this app that you can go to, that will help … but then you really really want to eat it, so maybe I'm not going to go to that app, because I want to eat it and that's going to stop me. (V1 Lisa, F, 46 yrs, 31.3 kg/m2, −2.5 kg)
Competing commitments/activities
Some participants mentioned that other commitments or activities took up time and focus that could be spent on engaging with the intervention. because I was always watching TV or doing something else, I didn’t want to sort of sit there and like half-heartedly do it ….So I didn’t do it rather than doing like a half attempt at it, if that makes sense. (V1, Theo, M, 30 yrs, 28.0 kg/m2, +0.25 kg)
One participant highlighted that there are many apps available (not necessarily health apps) that capture attention and could be prioritised for use over ImpulsePal. You’re up against Candy Crush, you know, which is my sort of new thing, I’m totally addicted to it. I also got a fishing app that I’ve come across and I just started fishing and then I’m like I don’t really like fishing. But I’m doing this on an app. And I’m much more challenged with these apps. (V2, Robert, Male, 50 yrs, 30.5 kg/m2, −2 kg)
Refinements to ImpulsePal between v1 and v2
The findings from the quantitative app usage data and qualitative interviews informed refinements to the intervention, to further improve appropriate receipt and understanding of, and engagement with the intervention. The Cycle 1 data-driven refinements resulted in ImpulsePalv2. The changes are presented in Table 4.
Discussion
This paper describes a mixed-method process evaluation exploring (a) participants’ use and perceptions of, and experiences with, the specific impulse management components to help identify necessary refinements, and (b) potential mechanisms of action and contextual factors influencing engagement and behaviour change. The most used components of ImpulsePal were the brain training game (inhibition training), the if-then plans (Implementation Intentions), and the emergency button (in-the-moment support following visual dynamic noise). However, this is not surprising since these components require access to the app more so than the urge-surfing component (mindfulness-based strategy) or the danger zones (goal primes). The usage analysis demonstrated good intervention fidelity in terms of delivery and the majority of the participants used the Brain Training component as intended. The qualitative analysis, however, highlighted some misunderstanding of (or missed) instructions and misuse in ImpulsePalv1.This was no longer a problem in ImpulsePalv2, suggesting improvements in receipt and enactment had been achieved. In the qualitative analysis, participants reported finding ImpulsePal easy to use and its content acceptable. As found in other studies investigating digital behaviour change interventions, participants particularly liked being reminded to use their strategies.33,35,36 The qualitative analysis also supports the quantitative usage analysis, in suggesting that the brain training game, if-then plans, and emergency button are the most “accessed” components. Although usage (metrics) of digital interventions have been associated with outcomes37–39 there was no association between ImpulsePal app use (as measured by time spent using the app) and weight loss (See van Beurden and colleagues 23 ). However, the qualitative analysis indicates that urge-surfing was considered easy to use, adaptable to individual lifestyles, and did not require access to the phone and could therefore be (and was) used on-the-go and in situations where individuals could not access their phone. Moreover, some participants highlighted no longer needing to access the app because they either no longer felt tempted over time, or where temptations did arise, felt they were in control and capable of resisting temptations, or were actively using in-the-moment strategies such as urge-surfing, or alternative responses in their if-then plans (sufficient mastery of the techniques). This highlights the challenge in conducting dose-response analyses using usage statistics, like time spent using the app or frequency of app access.
Our qualitative analysis supports our logic model Figure 1. It shows that for some, ImpulsePal facilitates reductions in perceived frequency and strength of impulses (e.g., cravings, temptation), which in turn impacted changes in eating behaviour. It also shows that (situational) intention and motivation are still a big influencing factor in that it impacts whether or not an individual decides to override the impulsive process when the in-the-moment decision regarding eating behaviour has come into conscious awareness, but also impacts whether an individual engages with the intervention in the first place. It is also clear that, as proposed in the logic model, personal factors (e.g., motivation, habitual device use), programme features (e.g., enjoyability, accessibility, ease of use, perceived relevance, etc), and environmental /social factors (e.g., competing commitment) influence engagement with the intervention itself. Through the identification of challenges in the fidelity of delivery, receipt, and enactment, and what factors influence these and how, this process evaluation also enabled us to make data-driven refinements to improve engagement.
Most people liked the fact that the intervention is delivered via an app, being able to choose from a variety of support tools and enjoyed the gameplay element of brain training. However, many felt that the brain training game was repetitive and boring after some time. This perception remained after refinements had been made to improve continued engagement which included personalisation of the trainable food items and reducing the duration of a training session. Gamification (i.e., the use of game elements such as points and difficulty levels) 40 ; is increasingly used in digital interventions and has been shown to be effective in increasing engagement with the intervention. 41 However, the brain training component consists of an inhibition training task (i.e., food specific go/no-go task) which has been shown to be effective in reducing snack intake 18 and weight loss 18 and there is work suggesting that gamification of inhibition training may undo the effects possible. Although giving the user what they want may enhance engagement, there is a potential risk that the use of the task would no longer positively affect behaviour change and health outcomes. This could, potentially be detrimental to effective engagement. 42
Strengths and limitations
This process evaluation of ImpulsePal has some key strengths. Firstly, by adopting a pragmatic mixed-methods approach, we were able to gather much richer data than would have been the case if a single method had been used. The value of a mixed-methods approach is highlighted in this study. For example, although limited use was identified in the quantitative analysis, the qualitative analysis suggested that this does not necessarily reflect a disengagement from the intervention. Others have also highlighted this disconnect between usage statistics and effective intervention engagement. 42 Therefore, usage data should not be seen as a comprehensive measure of engagement in isolation. Secondly, the iterative cycles of intervention delivery and evaluation enabled further intervention development in close collaboration with its intended users.
Some limitations need to be acknowledged. Firstly, there is potential for missing quantitative usage data. The data suggest that some participants did not see the log-in page, or the welcome screen, although their data show they were able to access the other components. As the log-in page and the welcome screen are the first screens the user is guided through, it is not possible that these have been missed. Therefore, this clearly indicates that some quantitative usage data are missing likely resulting from an error in data transfer. It is unclear how much more app usage data could be missing. Second, the qualitative sample consisted of a slightly higher percentage than the overall intervention group. Although purposive sampling was used, more women than men were willing to take part in the interviews. Third, the interviewer is also the researcher who conducted the feasibility trial assessments and voiced the audio-guided urge surfing. It is possible that a social desirability effect may have occurred where participants may have felt they wanted to provide a positive account of their experiences with the intervention. 43 However, a large purposive sample was included in the qualitative analysis to capture a variety of experiences and there are positive as well as negative accounts regarding the use of ImpulsePal. Finally, this process evaluation conducted mixed-method data collection, with analysis of the qualitative and quantitative components in isolation only for their findings to be discussed in relation to each other. Conducting an integrated mixed-methods analysis (e.g., through visual joint displays 44 ) in a future full-scale trial, may be able to provide a more comprehensive assessment of the logic model.
Conclusion
In conclusion, the use of impulse management techniques delivered via a smartphone app is acceptable to people trying to lose weight. Some strategies are accessed more frequently and consistently (e.g., brain training, if-then planning) than others (e.g., urge-surfing), but it is clear that lack of app use does not necessarily mean the strategies are not being used. Furthermore, interviews with participants highlight mechanisms (i.e., reduction in craving strength and frequency) by which ImpulsePal may have had an impact on eating behaviour and contextual factors which (i.e., motivation) impact engagement with the intervention and whether changes in behaviour are achieved or not, which seem to be in support of the logic model.
Footnotes
Acknowledgements
The authors would particularly like to thank Prof Frederick Verbruggen and Dr Jenny Lloyd, and the Patient and Public advisory group and expert consultation group, for their constructive advice on the intervention development and evaluation plans, and Dan Marshall and Diana Onu from Psynovigo Ltd for the programming of ImpulsePal and their invaluable insight in the collection and analysis of app-usage statistics. The authors would also like to thank Prof Lucy Yardley and Prof Sarah Dean, who critically and constructively examined the doctoral thesis towards which the work reported here contributed. The views expressed in this publication are those of the authors and not necessarily of the University of Exeter.
Contributorship
The work reported here formed part of SvB's doctoral work. SvB developed the protocol, conducted the data collection, and analyses. CG, JS, and NS made major contributions to the protocol. CG contributed to the qualitative analyses. SvB wrote the manuscript and all authors read, and critically reviewed the manuscript, contributed to structure and writing, and approved the final manuscript.
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Ethical approval
The process evaluation reported in this paper was conducted as part of a feasibility randomised controlled trial which was approved by the UK National Health Service (NHS) National Research Ethics Services Committee South West – Exeter (Ref: 15/SW/0181).
Funding
The authors received the following financial support for the research, authorship, and/or publication of this article: This process evaluation was funded through a University of Exeter Medical School PhD Scholarship held by SvB. The funding body had no role in the design of the development study, collection, analysis, and interpretation of the evidence base, the design of the intervention, or the writing of the manuscript. CA's and CG's contracts of permanent academic employment were supported in part by the UK National Institute for Health Research (NIHR) through the Collaboration for Leadership in Applied Health Research and Care in the South West Peninsula (PenCLAHRC) and a Career Development Fellowship (CDF-2012-05-259) respectively during which CA and CG supervised SvB's PhD work. The time for writing this manuscript was funded as part of the Economic and Social Research Council Postdoctoral Fellowship held by SvB (2020).
Guarantor
SvB
Informed consent
Participants in this research provided written informed consent to take part in the study and for anonymised data to be reported in dissemination materials including this publication.
