Protocol of a mixed-methods evaluation of Perfect Fit: A personalized mHealth intervention with a virtual coach to promote smoking cessation and physical activity in adults

Abstract

Objective

Adopting healthy behavior is vital for preventing chronic diseases. Mobile health (mHealth) interventions utilizing virtual coaches (i.e., artificial intelligence conversational agents) can offer scalable and cost-effective solutions. Additionally, targeting multiple unhealthy behaviors, like low physical activity and smoking, simultaneously seems beneficial. We developed Perfect Fit, an mHealth intervention with a virtual coach providing personalized feedback to simultaneously promote smoking cessation and physical activity. Through innovative methods (e.g., sensor technology) and iterative development involving end-users, we strive to overcome challenges encountered by mHealth interventions, such as shortage of evidence-based interventions and insufficient personalization. This paper outlines the content of Perfect Fit and the protocol for evaluating its feasibility, acceptability, and preliminary effectiveness, the role of participant characteristics, and the study's feasibility.

Methods

A single-arm, mixed-method, real-world evaluation study will be conducted in the Netherlands. We aim to recruit 100 adult daily smokers intending to quit within 6 weeks. The personalized intervention will last approximately 16 weeks. Primary outcomes include Perfect Fit's feasibility and acceptability. Secondary outcomes are preliminary effectiveness and study feasibility, and we will measure participant characteristics. Quantitative data will be collected through questionnaires administered at baseline, post-intervention and 2, 6, and 12 months post-intervention. Qualitative data will be gathered via semi-structured interviews post-intervention. Data analysis will involve descriptive analyses, generalized linear mixed models (quantitative) and the Framework Approach (qualitative), integrating quantitative and qualitative data during interpretation.

Conclusions

This study will provide novel insight into the potential of interventions like Perfect Fit, as a multiple health behavior change strategy. Findings will inform further intervention development and help identify methods to foster feasibility and acceptability. Successful mHealth interventions with virtual coaches will prevent chronic diseases and promote public health.

Keywords

mHealth mobile health virtual coach coaching activity trackers smoking low physical activity mixed-methods evaluation

Introduction

Unhealthy behaviors like low physical activity (PA) and tobacco smoking are significant preventable risk factors for chronic diseases, such as cardiovascular diseases and various types of cancer, and premature death.¹ Recent studies indicate that globally, smoking accounts for 20.2% of male deaths and 5.8% of female deaths,² while physical inactivity accounts for 7.2% of deaths.³ Hence, reducing tobacco use⁴ and increasing PA⁵ are among the most important strategies to reduce chronic disease mortality.¹ Although adopting healthy behavior is a well-known strategy to prevent chronic diseases, approximately 22% of the world's adult population still smoke tobacco, and over 25% are insufficiently active.^5,6 Consequently, the prevalence of chronic diseases is still rising, and chronic diseases currently lead to 17 million premature deaths each year globally.^1,7 Traditional face-to-face interventions have proven to be successful in health promotion,^7–9 but often face challenges like low user engagement,¹⁰ logistic and time constraints, cost-effectiveness problems⁹ and low accessibility.^8,11 Additionally, the global shortage of healthcare personnel severely restricts the capacity for face-to-face interventions.¹² Hence, there is a discrepancy between the high global burden of chronic diseases and the available support to help people adopt and maintain healthy behavior.^7,11,12 This is worrisome since health behavior change is often more successful when receiving support.^8,13 Therefore, optimizing the delivery of effective health behavior change interventions is vital.

eHealth has the potential to increase accessibility, reach, adaptability^11,12 and cost-effectiveness^11,14 of health behavior change interventions. eHealth can be defined as health services and information delivered or enhanced through the Internet and related technologies.¹⁵ eHealth interventions have demonstrated effectiveness comparable to traditional face-to-face interventions¹¹ and can be deployed with lower, or no, involvement of health professionals, making them an interesting approach to improve healthcare efficiency.^12,16 Specifically, health services and information supported by mobile devices, known as mHealth interventions, are increasingly being used due to their potential to exchange health-related information and provide support in a fast, scalable and cost-effective manner. Given the rapid increase in the number of people with mobile phones, also in low- and middle-income countries, mHealth interventions could potentially also reach individuals who have limited access to healthcare.¹⁷

Literature shows that adherence to online health behavior change interventions can be enhanced when combined with human support, as it can create social support, feedback and accountability.^11,18 This support can also be in the form of online support from a virtual coach, provided it employs strategies to enhance the user's relationship with the virtual coach.¹¹ Virtual coaches are artificial intelligence conversational agents that can mimic human conversations using text, spoken language or both.^10,14,19 Combining mHealth interventions with the support of a virtual coach is suggested to increase user engagement and effectiveness while still having the advantages of mobile-delivered interventions.¹¹ Two previous reviews have highlighted the potential effectiveness of virtual coach interventions in promoting PA¹⁴ and smoking cessation.¹⁰ Furthermore, the review of He et al.¹⁰ provides initial evidence supporting positive user experiences and high engagement. These promising findings, coupled with the capability of virtual coaches to be always available for multiple individuals simultaneously,^10,14 make virtual coach interventions an interesting health promotion strategy.

Many mHealth behavior change interventions which employ virtual coaches target a single health behavior.^10,14 Research suggests, however, that targeting multiple health behaviors simultaneously can be beneficial, as unhealthy behaviors, such as low PA and smoking, often co-occur.^7,20 Additionally, successful change in one health behavior can enhance self-efficacy and motivation for changing another.²⁰ Studies have highlighted the benefits of addressing both low PA and smoking in a single intervention. For example, an exercise intervention in smokers has demonstrated that engaging in PA decreases negative affect and nicotine cravings, thereby increasing the chances of successful smoking cessation.²¹ Conversely, an observational study showed that quitting smoking can improve fitness, thereby facilitating PA.²² As PA promotion and smoking cessation also reduce the risks for many chronic diseases, simultaneously targeting these health behaviors in a multi behavior change intervention may be efficient.^22–24 Moreover, research suggests that aligning behavioral goals with a higher-order conceptual goal can facilitate goal attainment. For instance, aspiring to become a healthy person, can aid in achieving specific behavioral goals such as increasing health-promoting behaviors (e.g., PA) and discontinuing health-compromising behaviors (e.g., smoking).²⁵ Hence, an intervention providing guidance on common behavior change principles, along with specific strategies for applying these principles to both PA and smoking cessation, could be an efficient approach for individuals facing multiple health behavior risks.^26,27

The interdisciplinary consortium ‘Perfect Fit’ has developed an mHealth intervention with a virtual coach providing personalized, real-time, text-based and visual feedback to simultaneously promote smoking cessation and PA in adults. Despite the great potential of mHealth interventions with a virtual coach, previous reviews have also highlighted challenges such as lack of evidence- and theory-based interventions,^10,12 insufficient tailoring and personalization,^10,14 the need for users to have a certain level of (health) literacy and digital skills^12,17 and low long-term user engagement.¹⁰ In the development of Perfect Fit, we have aimed to address these challenges.

First, Perfect Fit integrates behavior change taxonomies,^e.g.,²⁸ the Relapse Prevention Model²⁹ and theories of identity change (e.g., to become a physically active person or non-smoker),^30–32 components from existing interventions (e.g., StopCoach)³³ and findings from previous (empirical) studies by our research group.^e.g.,^34–39 Theoretical foundations increase the likelihood of interventions being successful and help to understand potential mechanisms of behavior change.^10,12 Second, Perfect Fit is personalized and tailored in various ways. For instance, the virtual coach provides real-time and personalized feedback using sensor technology (i.e., smartwatch data). Moreover, the coach is both system- and user-initiated and allows a combination of fixed and free-text answers. Users can flexibly adjust intervention aspects such as content, timing, and duration based on their needs. This personalization not only makes the coach more human-like but can also enhance users’ sense of ownership and autonomy, potentially increasing effectiveness and user engagement.^10–12 Third, Perfect Fit is designed to maximize accessibility, especially for users with lower (health) literacy or limited digital skills. To achieve this, end-users were involved in the early stages of development and prototype testing with iterative feedback was employed. Furthermore, experts, including specialists in tailoring research to populations with lower socio-economic position (SEP), were consulted. This collaborative process has led to the inclusion of intervention components such as animated educational videos and the use of easy-to-understand language (i.e., B1-level Dutch), ensuring inclusivity and user-friendliness.^12,17 Fourth, by incorporating the aforementioned strategies and innovative techniques, we aim to increase long-term engagement. Moreover, relational aspects, such as empathy and the use of emoticons, are integrated into the virtual coach to strengthen the user-coach relationship, potentially increasing long-term engagement.^10,11,40 A detailed description of the development of Perfect Fit and lessons learned is reported elsewhere.⁴¹

In this paper, we describe the content of Perfect Fit and outline the study protocol for evaluating the feasibility, acceptability and preliminary effectiveness of the intervention and the virtual coach using a single-arm, concurrent mixed-methods design. Additionally, associations between individual participant characteristics, feasibility, acceptability and preliminary effectiveness parameters will be examined. Finally, the feasibility of the research study will be evaluated. Reporting on the feasibility and acceptability of mHealth interventions is currently limited.^14,17 Although evidence of effectiveness is a basic requirement for implementation, effectiveness and user experience are intricately related.^10,42 With this study, we aim to gain more insight into whether Perfect Fit can overcome the challenges that mHealth interventions typically face. Developing feasible and effective mHealth interventions for promoting PA and smoking cessation is crucial to alleviate the burden on healthcare systems and professionals, reduce the impact of chronic conditions and ultimately promote public health.

Methods

Design

A single-arm, concurrent mixed-method, real-world evaluation study with a pretest-posttest design will be conducted in the Netherlands. The intervention duration is personalized, but is expected to be on average 16 weeks. The total study duration per participant is 16 months (i.e., 68 weeks). The quantitative part consists of self-report questionnaires on feasibility, acceptability and preliminary effects of Perfect Fit, usage data, objective level of PA (measured with a smartwatch), and research procedures monitored with a screening and inclusion logfile. The questionnaires will be administered at baseline (week 0 [T0]), post-intervention (week 16 [T1]), and at three follow-up time points (2 months [T2], 6 months [T3] and 12 months [T4] after the expected end of the intervention). The qualitative part consists of semi-structured individual interviews, conducted post-intervention, and will complement the quantitative results. These interviews will focus on preliminary effectiveness, feasibility (e.g., usage), acceptability (e.g., relationship with virtual coach), study feasibility, and requirements for implementation.

The study was approved by the Medical Research Ethics Committee Leiden The Hague Delft (N23.045 METC-LDD) on July 17, 2023, and was registered at ClinicalTrials.gov (NCT06095999) in October 2023. The study protocol follows the SPIRIT 2013 guidelines,⁴³ the SPIRIT-outcomes 2022 extension⁴⁴ and the CONSORT- EHEALTH checklist (V1.6.1).⁴⁵

Participants

To be eligible to participate participants must meet the following inclusion criteria (based on self-report): 1) age ≥ 18 years, 2) current daily smoker (cigarette, cigar, or rolling tobacco), 3) having the intention to quit smoking between now and 6 weeks, 4) being able to walk pain free, 5) being able to understand and read Dutch (at least B1 level), and 6) having a smartphone with an Internet connection. To examine Perfect Fit in high-risk individuals, at least 50% of participants should have a ≥ 10% risk of cardiovascular diseases according to the Dutch general practice guideline for cardiovascular risk management.⁴⁶ Specifically, this will be estimated based on being female and 55 + years old or male and 50 + years old. Finally, at least 75% of participants should live in the region of Leiden (i.e., the region between Leiden, Amsterdam and Utrecht in the Netherlands).

Exclusion criteria are: 1) Being involved in smoking cessation treatment at the start of the intervention, 2) having undergone major lower extremity surgery in the past year (to prevent giving PA advice that contradicts medical guidance), 3) taking antipsychotics or having a serious psychiatric illness (e.g., schizophrenia/psychosis, bipolar disorder, major depression), and 4) being pregnant. Participants are not excluded based on PA levels, as even active individuals can benefit from increasing activity, and daily smokers can still significantly benefit from quitting, regardless of their activity level.

Sample size

Previously conducted comparable (mixed-methods) single-arm studies, investigating the feasibility, acceptability and/or preliminary effectiveness of an mHealth or eHealth intervention, reported sample sizes between 30 and 56.^47–50 This aligns with recommendations for sample sizes of usability/acceptability studies.⁵¹ It is known that mHealth intervention studies often have high attrition rates⁵² and previously conducted comparable studies indeed report attrition rates between 7% and 30%.^47–50 Therefore, we aim to include 100 participants in our study, allowing for a dropout rate of 44%.

For the individual semi-structured interviews with participants, we aim to recruit a heterogeneous subsample considering differences in individual characteristics (e.g., gender, age), (health) literacy, adherence to Perfect Fit, and success of health behavior change. We expect to include around 10–15 participants to achieve data saturation.⁵³

Setting and recruitment

The study will take place in the Netherlands and will primarily be conducted online (e.g., introduction meeting via video calling, online self-report questionnaires). Participants will be recruited through a combination of online and offline strategies. Online recruitment will involve social media posts and advertisements (Facebook and Linkedin), news articles on the Perfect Fit website and newsletters of affiliated organizations (e.g., Trimbos-institute and healthcare insurer Zorg & Zekerheid). Offline recruitment will be facilitated through a news item in a local newspaper and by enlisting the help of Perfect Fit consortium partners and the Perfect Fit advisory board, who will distribute recruitment flyers within their network. Additionally, news items will be featured in magazines for healthcare professionals, asking them to inform individuals about the study. Finally, participants from a prior project study who had expressed interest in follow-up research will be invited to participate.

Recruitment started in August 2023 and ended when the intended sample size was reached, on February 12, 2024. Data collection (i.e., first inclusion) started in September 2023 and is expected to end (i.e., final follow-up questionnaire completed) in May 2025.

Intervention

Perfect Fit is the result of interdisciplinary collaboration and is an evidence-based, personalized program in which users receive support from a virtual coach to help them quit smoking and become more physically active. The development, intervention content, design and implementation of Perfect Fit, as well as lessons learned, are described elsewhere.⁴¹

Perfect Fit spans approximately 16 weeks, guiding participants through three phases with the virtual coach, ‘Sam’. In the preparation phase, participants engage in activities to prepare for smoking cessation and increasing PA, supported by the coach through chat conversations and informative animation videos on topics such as medication and nicotine replacement therapy, and goal setting. During the execution phase, participants focus on quitting smoking and increasing PA, supported by the coach through various components, such as activities to facilitate behavior change and weekly reflections. In the closing phase, participants engage in a final chat conversation to review their progress and results, and develop a relapse prevention plan. A comprehensive overview of Perfect Fit components and features, including their aims and the evidence supporting them, is provided in Appendix A. While some components are the same for all users, the personalized intervention allows users to choose optional components and decide whether to consult external links provided. Users are free to use additional PA or smoking cessation aids (e.g., medication, websites, booklets) and can adjust the timing of components to their preference.

The virtual coaching system

The Perfect Fit virtual coaching system runs on a cloud system and the user can interact with it through an existing smartphone app equipped with chat functionality (i.e., the NiceDay app used for remote therapy^a), which is available for both Android and iOS systems. The coaching system allows both system- and user-initiated chat conversations, and a combination of constrained (i.e., selecting pre-programmed responses) and unconstrained (i.e., free-reponse format) user input. The virtual coach is represented by the face of an animated robot (i.e., non-embodied) named ‘Coach Sam’ and offers text-based support enriched with emoticons, images, graphs and links to educational videos and external sources (see Appendix B for screenshots). The coach utilizes artificial intelligence techniques and input from users (e.g., user states) and employs persuasive (e.g., goal-setting) and relational strategies (e.g., being empathic). Additionally, sensor data (i.e., number of steps) measured by the smartwatch is integrated into the personalized PA coaching provided by the coach. A description of the technical architecture of the open-source text-based coaching system, which integrates big-data science, sensor technology, and personalized real-time feedback, can be found elsewhere.⁵⁴ Finally, to promote reusability and accessibility, the source code of the Perfect Fit system^b is publicly available.⁵⁵

Procedure

Interested individuals can register via the study website^c or by sending an e-mail to the researchers. Subsequently, these individuals receive a digital information letter and a link to the online screening questions (using secured software Castor EDC^®) via e-mail. Eligible individuals are asked to sign the online informed consent form, which is a requirement for study participation. Participants will be informed that they can withdraw from the study at any time for any reason if they wish to do so.

A timeline for participants is shown in Figure 1. After providing informed consent, a researcher will call the participant to explain the onboarding procedure and schedule an online onboarding meeting. Next, the participant will receive a smartwatch (Garmin Forerunner 55) plus information materials and instructions. The materials will include: 1) a video explaining the study, the intervention and the smartwatch (e.g., rationale, privacy, global overview of the intervention, how to use the smartwatch), 2) an installation booklet with instructions for downloading the necessary apps and connecting with the virtual coach, and 3) an information booklet with a summary of important instructions and information, including tips on how to communicate with the virtual coach. The information booklet can also be used as an exercise book, as it provides space for notes. The link to the video will be sent via e-mail, and the booklets and smartwatch will be sent via postal service. After receiving all the materials, the participant will have the onboarding meeting with a researcher via video call or telephone. During this meeting, the researcher will verify if the participant has successfully installed the apps, understands how to wear the smartwatch, and has connected with the virtual coach. Any remaining questions will be addressed during this meeting. The researchers will offer assistance to participants via e-mail or telephone when necessary (e.g., in case of technical problems). Some optional short activities and educational videos from the intervention will also be accessible through the study website should participants wish to revisit them post-intervention.

Figure 1.

Participant timeline.

The link to the online baseline questionnaire (using software Castor EDC^®) will be sent in the week of the onboarding meeting (T0), and the link to the post-intervention questionnaire will be sent 16 weeks later (i.e., expected end of the intervention, T1). Invitations for online follow-up questionnaires will be sent via e-mail 2 months (T2), 6 months (T3) and 12 months (T4) after the expected end of the intervention. The researchers will monitor questionnaire completion and send reminders via e-mail and phone calls as needed. Additionally, the Castor EDC^® feature requiring participants to answer every question will be used to prevent missed responses. As an incentive for completing the study, participants may keep the smartwatch, if they complete at least 80% of the questionnaires. No additional monetary compensation is provided for completing Perfect Fit and the questionnaires.

A subsample of the participants will be invited for a semi-structured individual interview via e-mail (see Appendix C for the interview protocol) at T1, but only if they provided their informed consent at the beginning of the study to be contacted for this purpose. The interviews will last 45–60 min and will be mainly conducted online, or via telephone or face-to-face if the participant strongly prefers this. The interviews will be conducted by two Medicine master's students trained in interviewing, of which one will take notes during the interview. Before the start of the interview, verbal informed consent from the participant will be obtained and audio-recorded. Participants will receive a gift voucher of 25 euros for the interview, along with the option to keep the smartwatch.

Outcomes

An overview of the instruments used to assess the study outcomes at each time point is presented in Table 1. Primary outcomes are the intervention's feasibility, acceptability, and usability, and the acceptability and usability of the virtual coach. Secondary outcomes are preliminary effectiveness and study feasibility. A more detailed description of the different study outcomes can be found in Appendix D.

Table 1.

Overview of outcomes, instruments, data collection methods and assessment time points.

	Method	Time points
		T0	Int.	T1	T2	T3	T4
		Wk 0	Wk 1–16	Wk 16	Wk 24	Wk 40	Wk 64
Primary outcomes
Intervention's feasibility
Usage data on completed intervention components	L
Frequency contact coach (1 item)	Q			X
Intervention's acceptability
Satisfaction with the intervention (2 items)	Q			X
Intervention's usability
SUS (10 items)⁵⁶	Q			X
Acceptability and usability coach
Acceptability and usability of the coach (7 items)^57,58	Q			X
Qualitative interview data on primary outcome measures	I			X
Secondary outcomes
Preliminary effectiveness
Smoking abstinence (2–3 items)	Q			X	X	X	X
Level of PA: GSLTPAQ (3 items)^59,60	Q	X		X	X	X	X
Non-smoker self-identity (7 items)	Q	X		X	X	X	X
PA self-identity: adapted EIQ (9 items)^61,62	Q	X		X	X	X	X
Smoking abstinence self-efficacy (1 item)	Q	X		X	X	X	X
PA self-efficacy (1 item)	Q	X		X	X	X	X
Objective level of PA: number of steps (smartwatch)	S
Study feasibility
Recruitment, response and consent rate^a	R
Recruitment strategies (1 item)	Q	X
Users’ adherence^b	R
Intervention completion (1 item)	Q			X	X	X	X
Qualitative interview data on secondary outcome measures	I			X
Other variables
Participant characteristics, including eHLQ (30 items)^63,64 ^c	Q	X
Intention to quit smoking (1 item)	Q	X
Smoking behavior and nicotine dependence: FTND (6 items)⁶⁵	Q	X
Intention to become more physically active (1 item)	Q	X
Use of additional aids/support for smoking cessation (1 item)	Q			X	X	X	X
Use of additional aids/support for PA (1 item)	Q			X	X	X	X
Self-reported (re)lapses during intervention after quit date	L
Smoking e-cigarettes (1–3 items)	Q	X		X	X	X	X
Self-reported level of occupational PA (1 item)	Q	X		X	X	X	X

Recruitment, response and consent rate will be assessed from the start of the recruitment phase until the inclusion of the final participant.

Users’ adherence will be assessed from the moment informed consent is obtained until T4.

A license for the use of the eHealth Literacy Questionnaire was obtained from the copyright holders.

T0 = baseline; Int. = During intervention; T1 = Post-intervention; T2 = 2-month follow-up; T3 = 6-month follow-up; T4 = 12-month follow-up; Wk = Week.

L = log data of coaching system; Q = self-reported questionnaire; I = semi-structured individual interview; S = smartwatch; R = researchers using a participant screening and inclusion logfile.

SUS = System Usability Scale; PA = Physical activity; GSLTPAQ = Godin-Shephard Leisure-Time Physical Activity Questionnaire; EIQ = Exercise Identity Questionnaire; eHLQ = eHealth Literacy Questionnaire; FTND = Fagerström Test of Nicotine Dependence.

Data analyses

Quantitative and qualitative data will be analysed independently and integrated at the point of interpretation.

Quantitative analyses

Descriptive analyses will be performed in SPSS (SPSS 25.0 statistics for Windows, IBM, Armonk, New York) and generalized linear mixed models in RStudio (RStudio Version 2023.12.1 + 402, PBC, Boston, MA, http://www.rstudio.com/). Outliers and assumptions will be checked before data analyses are conducted. A p-value of < .05 will be considered statistically significant.

Descriptive analyses (e.g., M ± SD, N, percentages) will be conducted to summarize the participant characteristics (e.g., age, eHealth literacy) and the characteristics related to PA (e.g., intention to become more physically active) and smoking (e.g., physical nicotine dependence and intention to quit). Descriptive analyses will also be conducted to describe quantitative data concerning the interventions’ feasibility, acceptability and usability, the acceptability and usability of the virtual coach, and the study feasibility.

To examine whether potential changes over time in preliminary effectiveness outcomes are significant compared to baseline, we will use generalized linear mixed models for ordinal data (abstinence self-efficacy, PA self-efficacy, non-smoker self-identity, PA self-identity), continuous data (number of steps) and binary data (prolonged smoking abstinence and level of PA). For each outcome variable, a separate model will be run. The outcome variable will serve as the dependent variable, with time (i.e., T0-T4) as a fixed effect and subject as a random effect. This will allow us to examine whether there is a significant change over time in these preliminary effectiveness outcomes and to examine whether the change over time differs per participant. Generalized mixed models were chosen because they can handle missing values implicitly while retaining maximum power, and these analyses can deal with unequal time intervals between measurements.

We also intend to perform exploratory subgroup analyses, depending on the power for these analyses. This could provide insight into, for instance, the relation between SEP and the preliminary effectiveness of Perfect Fit. To do so, the individual characteristic, in this case, SEP, will be included as a covariate in the generalized mixed model (i.e., allowing us to identify whether SEP impacts change over time in preliminary effectiveness outcomes). Data will be pseudonymized, processed confidentially, and securely stored on an internal drive. Access to data will be restricted to study personnel. The data will be stored for 15 years.

Qualitative analyses

All individual semi-structured interviews will be audio-recorded, anonymized, and transcribed verbatim for subsequent analyses. Qualitative data analysis will be conducted following the principles of the Framework Approach^66,67 and using software ATLAS.ti (ATLAS.ti Version 23.2.3.27778, 2023, ATLAS.ti Scientific Software Development GmbH, https://atlasti.com/). The Framework Approach is a structured and transparent qualitative analysis method that uses matrices to organize and analyze data by case (i.e., participant) and by code. It is chosen for its ease of use, particularly when multiple (quantitative) researchers are involved, and because it combines both inductive and deductive analysis.^66,67 The first transcripts will be independently coded by the two master's students who conduct the interviews and MV, resulting in an initial coding scheme. Subsequently, the students will proceed to code the remaining transcripts and discuss any adjustments to the coding scheme with the author team. Data interpretation and theme generation will be conducted by MV, in collaboration with the author team. During data analyses, the interviews will be securely stored on an internal drive. Upon completion of data analysis, the audio records will be deleted.

Discussion

This paper presents the novel mHealth intervention called Perfect Fit and outlines the protocol for a single-arm, concurrent mixed-method, real-world evaluation of Perfect Fit. Perfect Fit utilizes a virtual coach providing personalized, real-time, text-based, and visual feedback to promote smoking cessation and PA in adults. The main aim of this study is to examine the feasibility and acceptability of Perfect Fit. Additionally, the preliminary effectiveness of Perfect Fit, and associations between individual characteristics and parameters of feasibility, acceptability and/or preliminary effectiveness will be explored. Finally, the overall feasibility of the study will be evaluated. The combination of quantitative and qualitative methods will enable us to contextualize the quantitative data, leading to a deeper understanding of our findings.

Although mHealth interventions with virtual coaches are a promising health promotion strategy with many benefits (e.g., scalable and cost-effective),^14,17 they also encounter challenges.^10,14 With Perfect Fit, we aim to address challenges such as the scarcity of evidence- and theory-based interventions,^10,12 insufficient tailoring and personalization,^10,14 the need for users to have a certain level of (health) literacy and digital skills^12,17 and low long-term user engagement.¹⁰ Through an iterative development process involving an interdisciplinary team and end-users and by making use of innovative techniques (e.g., sensor technology), tailoring and personalization, we strive to overcome common challenges faced by mHealth interventions. Moreover, to our knowledge, integrating both a health-compromising behavior, like smoking, and a health-promoting behavior, like PA, into a single mHealth intervention is novel.

Our primary aim is to investigate the feasibility and acceptability of Perfect Fit as a whole and the virtual coach itself. Considering the interplay between user characteristics and technology is essential for ensuring the long-term use and implementation of mHealth interventions.¹⁶ By gaining insights into user experiences and needs, including how users perceive the engagement with Perfect Fit and their relationship with the virtual coach, we aim to contribute valuable knowledge for the development of new mHealth interventions with long-term engagement. We also hope to gain first insights into criteria for future implementation strategies, such as determining whether mHealth interventions with virtual coaches are better suited as stand-alone or an add-on intervention (e.g., to pharmacological treatment), and for whom they are most effective. Furthermore, we investigate the feasibility of the study itself, aiming to provide insights that can inform the design of future studies investigating mHealth interventions. This is particularly relevant given the high attrition rates observed in these studies.⁵²

Another aim is to investigate the preliminary effectiveness of Perfect Fit, in which we simultaneously target PA enhancement and smoking cessation. Investigating the preliminary effectiveness of Perfect Fit will provide valuable insights into the effectiveness of targeting multiple health behaviors simultaneously.^7,20 Specifically, these findings will shed light on the effectiveness of integrating PA enhancement and smoking cessation within a single intervention. Additionally, in this study, PA levels and smoking abstinence will be monitored up to 12 months post-intervention. Health behavior change is a dynamic process, which is particularly evident in smoking cessation,⁶⁸ and long-term behavior change can be difficult to achieve.^9,10 Despite this, many studies on mHealth interventions for smoking cessation lack long follow-up periods.¹⁰ However, research shows that short-term smoking abstinence immediately following an intervention does not reliably predict long-term abstinence.^68,69 By incorporating a 12-month follow-up period, we aim to gain insight into the potential of Perfect Fit to facilitate long-term health behavior change.

Exploratory subgroup analyses will also indicate whether the intervention is more feasible, acceptable or effective for certain populations (e.g., individuals with low eHealth literacy or SEP), guiding further development. mHealth interventions with virtual coaches have the potential to reach individuals who have limited access to healthcare, but may also unintentionally increase health disparities.^10,16 Therefore, it is vital that mHealth interventions are developed and implemented in a way that meets the characteristics, needs and preferences of these individuals. With this study, we aim to gain insight into the extent to which Perfect Fit meets the needs of individuals with diverse socio-demographic characteristics.

This study also has some limitations. Although we aimed to develop Perfect Fit to be accessible for users with lower (health) literacy and digital skills (e.g., through the use of animation videos and B1-level Dutch), users still need to be able to read text-based chat messages, have basic digital skills, and have access to a smartphone to use Perfect Fit. As a result, the findings of this study may not represent the experiences of individuals who lack these skills or who do not have access to a smartphone. However, participants’ eHealth literacy will be measured at baseline to better understand for which users, in terms of eHealth literacy levels, Perfect Fit is suitable. Another limitation is that Perfect Fit will be investigated as a stand-alone intervention, as we want to assess the feasibility, acceptability, and preliminary effectiveness of the intervention itself. However, it is not yet certain whether this will be its optimal implementation form. For instance, Perfect Fit might be more effective as part of a blended intervention, combined with face-to-face sessions with a healthcare professional. Nonetheless, participants are allowed to use other PA or smoking cessation aids during the study. By assessing the use of these additional aids through questionnaires and post-intervention interviews, we hope to gain preliminary insight into the feasibility and suitability of Perfect Fit as a stand-alone or add-on intervention, as well as the types of interventions it could potentially complement.

Conclusions

To conclude, the burden on healthcare systems and professionals is substantial, demanding new, cost-effective, accessible and efficient strategies.¹² mHealth interventions with virtual coaches have the potential to play a complementary role alongside healthcare professionals, enhancing the public health of diverse populations.^10,14 By combining quantitative and qualitative results on feasibility, acceptability and preliminary effectiveness of Perfect Fit, we hope to gain more insight into whether interventions like Perfect Fit show promise as a multiple health behavior change strategy. The knowledge gained from this study will be used to further develop the intervention, identify improvements and identify how we can promote its usability for different individuals. This knowledge is vital as it will contribute to research on and the development of mHealth interventions with virtual coaches. Further research in this domain could eventually lead to successful interventions to promote public health and thereby reduce the impact of chronic conditions.

Supplemental Material

sj-docx-1-dhj-10.1177_20552076241300020 - Supplemental material for Protocol of a mixed-methods evaluation of Perfect Fit: A personalized mHealth intervention with a virtual coach to promote smoking cessation and physical activity in adults

Supplemental material, sj-docx-1-dhj-10.1177_20552076241300020 for Protocol of a mixed-methods evaluation of Perfect Fit: A personalized mHealth intervention with a virtual coach to promote smoking cessation and physical activity in adults by Milon H. M. van Vliet, Anke Versluis, Niels H. Chavannes, Bouke L. Scheltinga, Nele Albers, Kristell M. Penfornis, Walter Baccinelli, Eline Meijer and in DIGITAL HEALTH

Supplemental Material

sj-docx-2-dhj-10.1177_20552076241300020 - Supplemental material for Protocol of a mixed-methods evaluation of Perfect Fit: A personalized mHealth intervention with a virtual coach to promote smoking cessation and physical activity in adults

Supplemental material, sj-docx-2-dhj-10.1177_20552076241300020 for Protocol of a mixed-methods evaluation of Perfect Fit: A personalized mHealth intervention with a virtual coach to promote smoking cessation and physical activity in adults by Milon H. M. van Vliet, Anke Versluis, Niels H. Chavannes, Bouke L. Scheltinga, Nele Albers, Kristell M. Penfornis, Walter Baccinelli, Eline Meijer and in DIGITAL HEALTH

Supplemental Material

sj-docx-3-dhj-10.1177_20552076241300020 - Supplemental material for Protocol of a mixed-methods evaluation of Perfect Fit: A personalized mHealth intervention with a virtual coach to promote smoking cessation and physical activity in adults

Supplemental material, sj-docx-3-dhj-10.1177_20552076241300020 for Protocol of a mixed-methods evaluation of Perfect Fit: A personalized mHealth intervention with a virtual coach to promote smoking cessation and physical activity in adults by Milon H. M. van Vliet, Anke Versluis, Niels H. Chavannes, Bouke L. Scheltinga, Nele Albers, Kristell M. Penfornis, Walter Baccinelli, Eline Meijer and in DIGITAL HEALTH

Supplemental Material

sj-docx-4-dhj-10.1177_20552076241300020 - Supplemental material for Protocol of a mixed-methods evaluation of Perfect Fit: A personalized mHealth intervention with a virtual coach to promote smoking cessation and physical activity in adults

Supplemental material, sj-docx-4-dhj-10.1177_20552076241300020 for Protocol of a mixed-methods evaluation of Perfect Fit: A personalized mHealth intervention with a virtual coach to promote smoking cessation and physical activity in adults by Milon H. M. van Vliet, Anke Versluis, Niels H. Chavannes, Bouke L. Scheltinga, Nele Albers, Kristell M. Penfornis, Walter Baccinelli, Eline Meijer and in DIGITAL HEALTH

Footnotes

Acknowledgements

The authors would like to thank the advisory board members, members of the Perfect Fit consortium, including collaborating public and private parties, experts who shared their knowledge during expert sessions, and several students for their contributions to the development of the Perfect Fit intervention. The authors would also like to thank Sven van der Burg of the Netherlands eScience Center (Amsterdam, The Netherlands) for his contribution to the intervention software development.

Credit authorship contribution statement

EM led funding acquisition by the Perfect Fit consortium for this study. MHMvV, AV, NHC and EM conceived the study and conceptualized the study design. All authors contributed to the conceptualization and development of the intervention content. BLS, NA, WB and the Perfect Fit consortium developed the software for the intervention. MHMvV drafted the manuscript under the supervision of AV, NHC and EM. All authors critically reviewed and revised the entire manuscript, and read and approved the final manuscript.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Declaration of generative AI and AI-assisted technologies in the writing process

During the preparation of this work the author(s) used ChatGPT (OpenAI, 2023) in order to improve readability and language. After using this tool/service, the author(s) reviewed and edited the content as needed and take(s) full responsibility for the content of the publication.

Ethical approval

The study was approved by the Medical Research Ethics Committee Leiden The Hague Delft (N23.045 METC-LDD) on July 17, 2023.

Funding

This work is part of the multidisciplinary research project Perfect Fit. This research received funding from the Netherlands Organisation for Scientific Research (NWO) program Commit2Data - Big Data & Health (project number 628.011.211). The program was funded by the following parties: NWO, the Netherlands Organisation for Health Research and Development (ZonMw), Hartstichting, Ministry of Health, Welfare and Sport (VWS), Health Holland, and the Netherlands eScience Center. The publication reflects only the authors’ views and the funders are not liable for any use that may be made of the information contained herein.

Guarantor

ORCID iDs

Milon H. M. van Vliet

Walter Baccinelli

Supplemental material

Supplemental material for this article is available online.

Notes

References

World Health Organization. Noncommunicable diseases, https://www.who.int/news-room/fact-sheets/detail/noncommunicable-diseases (2023, accessed 17 April 2024).

GBD 2019 Tobacco Collaborators. Spatial, temporal, and demographic patterns in prevalence of smoking tobacco use and attributable disease burden in 204 countries and territories, 1990-2019: a systematic analysis from the Global Burden of Disease Study 2019. Lancet 2021; 397: 2337–2360.

Katzmarzyk

Friedenreich

Shiroma

, et al. Physical inactivity and non-communicable disease burden in low-income, middle-income and high-income countries. Br J Sports Med 2022; 56: 101–106.

NCD Countdown 2030 collaborators. NCD Countdown 2030: worldwide trends in non-communicable disease mortality and progress towards Sustainable Development Goal target 3.4. Lancet 2018; 392: 1072–1088.

World Health Organization. Physical activity, https://www.who.int/news-room/fact-sheets/detail/physical-activity (2022, accessed 17 April 2024).

World Health Organization. Tobacco, https://www.who.int/news-room/fact-sheets/detail/tobacco (2023, accessed 17 April 2024).

Duan

Shang

Liang

, et al. Effects of eHealth-based multiple health behavior change interventions on physical activity, healthy diet, and weight in people with noncommunicable diseases: systematic review and meta-analysis. J Med Internet Res 2021; 23: e23786.

Taylor

GMJ

Dalili

Semwal

, et al. Internet-based interventions for smoking cessation. Cochrane Database Syst Rev 2017; 2017: CD007078.

Greaves

Sheppard

Abraham

, et al. Systematic review of reviews of intervention components associated with increased effectiveness in dietary and physical activity interventions. BMC Public Health 2011; 11: 19.

10.

Balaji

Wiers

, et al. Effectiveness and acceptability of conversational agents for smoking cessation: a systematic review and meta-analysis. Nicotine Tob Res 2023; 25: 1241–1250.

11.

Santarossa

Kane

Senn

, et al.

Exploring the role of in-person components for online health behavior change interventions: can a digital person-to-person component suffice?

J Med Internet Res 2018; 20: e144.

12.

van der Kleij

RMJJ

Kasteleyn

Meijer

, et al. SERIES: eHealth in primary care. Part 1: concepts, conditions and challenges. Eur J Gen Pract 2019; 25: 179–189.

13.

Jung

Cho

. Promoting physical activity and weight loss with mhealth interventions among workers: systematic review and meta-analysis of randomized controlled trials. Jmir Mhealth Uhealth 2022; 10: e30682.

14.

Zhang

Fang

, et al. A systematic review of artificial intelligence chatbots for promoting physical activity, healthy diet, and weight loss. Int J Behav Nutr Phys Act 2021; 18: 60.

15.

Eysenbach

. What is e-health? J Med Internet Res 2001; 3: 20.

16.

Versluis

van Luenen

Meijer

, et al. SERIES: eHealth in primary care. Part 4: addressing the challenges of implementation. Eur J Gen Pract 2020; 26: 140–145.

17.

Marcolino

Oliveira

JAQ

D'Agostino

, et al. The impact of mHealth interventions: systematic review of systematic reviews. Jmir Mhealth Uhealth 2018; 6: e8873.

18.

Mohr

Cuijpers

Lehman

. Supportive accountability: a model for providing human support to enhance adherence to ehealth interventions. J Med Internet Res 2011; 13: e1602.

19.

Laranjo

Dunn

Tong

, et al. Conversational agents in healthcare: a systematic review. J Am Med Inform Assn 2018; 25: 1248–1258.

20.

Prochaska

. A review of multiple health behavior change interventions for primary prevention. Am J Lifestyle Med 2011; 5: 208–221.

21.

Kunicki

Hallgren

Uebelacker

, et al. Examining the effect of exercise on the relationship between affect and cravings among smokers engaged in cessation treatment. Addict Behav 2022; 125: 107156.

22.

Berkovitch

Kivity

Klempfner

, et al. Time-dependent relation between smoking cessation and improved exercise tolerance in apparently healthy middle-age men and women. Eur J Prev Cardiol 2015; 22: 807–814.

23.

World Health Organization. Cardiovascular diseases (CVDs), https://www.who.int/news-room/fact-sheets/detail/cardiovascular-diseases-(cvds) (2021, accessed 17 April 2024).

24.

Priebe

Atkinson

Faulkner

. Run to quit: an evaluation of a scalable physical activity-based smoking cessation intervention. Ment Health Phys Act 2017; 13: 15–21.

25.

Rhodes

Quinlan

Mistry

. Do other goals influence physical activity? A systematic review examining the relationship between other goals and physical activity behavior. Prev Med 2016; 91: 306–317.

26.

Noar

Chabot

Zimmerman

. Applying health behavior theory to multiple behavior change: considerations and approaches. Prev Med 2008; 46: 275–280.

27.

Prochaska

. Multiple health behavior research represents the future of preventive medicine. Prev Med 2008; 46: 281–285.

28.

Michie

Hyder

Walia

, et al. Development of a taxonomy of behaviour change techniques used in individual behavioural support for smoking cessation. Addict Behav 2011; 36: 315–319.

29.

Marlatt

Donovan

. Relapse prevention: Maintenance strategies in the treatment of addictive behaviors. New York, NY: Guilford press, 2007.

30.

Burke

Stets

. Identity theory. New York, NY: Oxford University Press, 2009.

31.

West

Brown

. Theory of addiction. 2nd ed. Hoboken, New Jersey: Wiley-Blackwell, 2013.

32.

Oyserman

Elmore

Smith

. Self, self-concept, and identity. In: Handbook of self and identity, 2nd ed. New York: The Guilford Press, 2012, pp.69–104.

33.

Meijer

Korst

Oosting

, et al. “At least someone thinks I'm doing well": a real-world evaluation of the quit-smoking app StopCoach for lower socio-economic status smokers. Addict Sci Clin Pract 2021; 16: 1–14.

34.

Albers

Neerincx

Penfornis

, et al. Users’ needs for a digital smoking cessation application and how to address them: a mixed-methods study. PeerJ 2022; 10: e13824.

35.

Albers

Hizli

Scheltinga

, et al. Setting physical activity goals with a virtual coach: vicarious experiences, personalization and acceptance. J Med Syst 2023; 47: 15.

36.

Penfornis

Van Vliet

MHM

Meijer

, et al. Mapping the evidence on identity processes and identity-related interventions in the smoking and physical activity domains: a scoping review protocol. Bmj Open 2022; 12: e058405.

37.

Penfornis

Gebhardt

Rippe

RCA

, et al. My future-self has (not) quit smoking: an experimental study into the effect of a future-self intervention on smoking-related self-identity constructs. Soc Sci Med 2023; 320: 115667.

38.

Scheltinga

Usta

Reenalda

, et al. Estimating vertical ground reaction force during running with 3 inertial measurement units. J Biomed Eng Biosci 2022; 9: 31–38.

39.

Scheltinga

Kok

Buurke

, et al. Estimating 3D ground reaction forces in running using three inertial measurement units. Front Sports Act Living 2023; 5: 1176466.

40.

Bickmore

Schulman

Yin

. Maintaining engagement in long-term interventions with relational agents. Appl Artif Intell 2010; 24: 648–666.

41.

Versluis

Penfornis

van der Burg

, et al. Targeting key risk factors for cardiovascular disease in at-risk individuals: developing a digital, personalized and real-time intervention to facilitate smoking cessation and physical activity. Manuscript under review 2024.

42.

Smyth

Syrek

Reins

, et al. User experience predicts the effectiveness of a gamified recovery app. Prävention und Gesundheitsförderung 2018; 13: 319–326.

43.

Chan

Tetzlaff

Gotzsche

, et al. SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials. Br Med J 2013; 346: e7586.

44.

Butcher

Monsour

Mew

, et al. Guidelines for reporting outcomes in trial protocols: the SPIRIT-outcomes 2022 extension. JAMA 2022; 328: 2345–2356.

45.

Eysenbach

Group

C-E

. CONSORT-EHEALTH: improving and standardizing evaluation reports of Web-based and mobile health interventions. J Med Internet Res 2011; 13: e126.

46.

Nederlands Huisartsen Genootschap. NHG-Richtlijnen Cardiovasculair risicomanagement: Risicocategorieën en streefwaarden bij behandelindicatie en beleid, https://richtlijnen.nhg.org/standaarden/cardiovasculair-risicomanagement (2019, accessed 17 April 2024).

47.

Fortuna

Myers

Ferron

, et al. Assessing a digital peer support self-management intervention for adults with serious mental illness: feasibility, acceptability, and preliminary effectiveness. J Ment Health 2022; 31: 833–841.

48.

Maher

Davis

Curtis

, et al. A physical activity and diet program delivered by artificially intelligent virtual health coach: proof-of-concept study. Jmir Mhealth Uhealth 2020; 8: e17558.

49.

Masaki

Tateno

Kameyama

, et al. Impact of a novel smartphone app (CureApp smoking cessation) on nicotine dependence: prospective single-arm interventional pilot study. Jmir Mhealth Uhealth 2019; 7: e12694.

50.

Whitaker

Jones

Dziewior

, et al. Feasibility, acceptability, and preliminary efficacy of a single-arm, remotely-delivered health coaching intervention to increase physical activity and reduce sedentary behavior during pregnancy. Bmc Pregnancy Childb 2022; 22: 740.

51.

Macefield

. How to specify the participant group size for usability studies: a practitioner's guide. J Usability Stud 2009; 5: 34–45.

52.

Meyerowitz-Katz

Ravi

Arnolda

, et al. Rates of attrition and dropout in app-based interventions for chronic disease: systematic review and meta-analysis. J Med Internet Res 2020; 22: e20283.

53.

Hennink

Kaiser

Marconi

. Code saturation versus meaning saturation: how many interviews are enough? Qual Health Res 2017; 27: 591–608.

54.

Baccinelli

Van Der Burg

Richardson

, et al. Reusable virtual coach for smoking cessation and physical activity coaching. In: Proceedings of the 22nd ACM International Conference on Intelligent Virtual Agents 2022, pp.1–3.

55.

van der Burg

Baccinelli

Richardson

, et al. PerfectFit (v.1.0.0), https://doi.org/10.5281/zenodo.10837093 (2024).

56.

Brooke

. SUS-A quick and dirty usability scale. Usabil Evaluat Ind 1996; 189: 4–7.

57.

Bickmore

Mitchell

Jack

, et al. Response to a relational agent by hospital patients with depressive symptoms. Interact Comput 2010; 22: 289–298.

58.

Provoost

Kleiboer

Ornelas

, et al. Improving adherence to an online intervention for low mood with a virtual coach: study protocol of a pilot randomized controlled trial. Trials 2020; 21: 1–12.

59.

Amireault

Godin

. The godin-shephard leisure-time physical activity questionnaire: validity evidence supporting its use for classifying healthy adults into active and insufficiently active categories. Percept Motor Skill 2015; 120: 604–622.

60.

Godin

. The godin-shephard leisure-time physical activity questionnaire. Health Fitness J Can 2011; 4: 18–22.

61.

Strachan

Brawley

Spink

, et al. Older adults’ physically-active identity: relationships between social cognitions, physical activity and satisfaction with life. Psychol Sport Exerc 2010; 11: 114–121.

62.

Anderson

Cychosz

. Development of an exercise identity scale. Percept Motor Skill 1994; 78: 747–751.

63.

Poot

Meijer

Fokkema

, et al. Translation, cultural adaptation and validity assessment of the Dutch version of the eHealth Literacy Questionnaire: a mixed-method approach. Bmc Public Health 2023; 23: 1006.

64.

Kayser

Karnoe

Furstrand

, et al. A multidimensional tool based on the eHealth literacy framework: development and initial validity testing of the eHealth literacy questionnaire (eHLQ). J Med Internet Res 2018; 20: 36.

65.

Heatherton

Kozlowski

Frecker

, et al. The fagerstrom test for nicotine dependence: a revision of the fagerstrom tolerance questionnaire. Brit J Addict 1991; 86: 1119–1127.

66.

Gale

Heath

Cameron

, et al. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med Res Methodol 2013; 13: –8.

67.

Ritchie

Lewis

Nicholls

, et al. Qualitative research practice: A guide for social science students and researchers. London: Sage Publications Ltd, 2013.

68.

Velicer

Prochaska

. A comparison of four self-report smoking cessation outcome measures. Addict Behav 2004; 29: 51–60.

69.

West

Hajek

Stead

, et al. Outcome criteria in smoking cessation trials: proposal for a common standard. Addiction 2005; 100: 299–303.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.36 MB

0.71 MB

0.03 MB

0.08 MB