Sage Journals: Discover world-class research

Abstract

Introduction

Varenicline is one of the most effective smoking cessation medications; however, non-adherence remains a significant barrier to successful quitting. Conversational agents have the potential to support medication adherence in home and community settings. However, generative AI models pose risks due to hallucinations, making them less reliable for this purpose. Rule-based chatbots provide a more transparent, theory-driven approach to patient support. Thus, we developed ChatV, a rule based chatbot grounded in the Behaviour Change Wheel framework, to enhance varenicline adherence.

Methods

ChatV was developed using a three-step process. First, we identified core determinants of varenicline adherence through a rapid review and qualitative interviews with healthcare providers and patients using the Theoretical Domains Framework. Second, we identified the intervention options through group discussions. Third, we identified intervention components using Behaviour Change Techniques (BCTs) Taxonomy v1. We applied the Acceptability, Practicability, Effectiveness, Affordability, Safety, and Equity (APEASE) criteria to determine the final intervention components.

Results

We identified 11 key domains relevant to behaviour change, including knowledge, beliefs about capabilities and consequences, memory, attention and decision-making processes, reinforcement, intentions, goals, social influences, environmental context and resources, behaviour regulation, and skill. Applying the APEASE criteria, we refined these to nine theoretical domains and identified 21 BCTs as core components of ChatV.

Conclusion

This study demonstrates a structured, theory-informed approach to chatbot development for medication adherence. By integrating evidence-based behaviour change principles with practical considerations, ChatV offers a model for designing rule-based conversational agents in healthcare.

Keywords

Behaviour change wheel behaviour change techniques theoretical domains framework varenicline medication adherence smoking cessation chatbot

Introduction

The Canadian healthcare system is grappling with a multitude of issues including a shortage of healthcare providers (HCPs), financial constraints, and HCP burnout,¹ which have been exacerbated by the COVID-19 pandemic.² Primary care faces additional hurdles, including an aging physician population, declining interest in entering general practice, and rising rates of multi-morbidity amidst rapid population growth and aging.^3,4

In this context, conversational agents, designed to generate responses that mimic human language, emerge as promising tools to address common healthcare challenges seen in primary care settings.⁵ Conversational agents can streamline patient interactions, provide timely access to information, support decision-making processes, and enhance patient engagement.⁶ Their potential to automate routine tasks, offer personalised healthcare recommendations and facilitate remote monitoring holds promise in optimising healthcare delivery and improving outcomes. These capabilities could help solve many of the challenges the healthcare systems around the world are facing, including reducing financial costs,⁷ HCP burnout,⁸ and enhancing patient experiences and outcomes.⁹

However, implementing conversational agents within healthcare settings also poses several challenges. Several researchers have shown that generative artificial intelligence can augment our existing biases.^10–12 Historically disadvantaged groups can be especially negatively affected by bias, leading to less accurate diagnoses and inadequate estimates of the need for medical care.¹⁰ In addition, most generative artificial intelligence systems still operate as a ‘black box’,¹³ where we lack transparency into the mechanisms governing their outputs, limiting our ability to comprehend their decision-making processes fully. This stands in contrast to the U.K. Medical Research Council (MRC) guidelines¹⁴ which advocate for a systematic approach to intervention development and evaluation, placing greater emphasis on the supporting evidence base and the utilisation of theory.

In contrast to generative AI, rule-based chatbots provide a more transparent framework, where the decision-making processes are explicitly coded based on predefined rules and logical structures.¹⁵ This transparency allows researchers and practitioners to understand exactly how the chatbot arrives at its conclusions and guides users through interactions. While this approach may limit the flexibility and adaptability of the chatbot compared to generative AI, it ensures a more predictable and controllable system. Moreover, rule-based chatbots can be more easily aligned with theoretical models and evidence-based practices, as their behaviour can be explicitly designed to follow intervention frameworks.

One area where conversational agents could have an important role is in smoking cessation. Each year, over 46,000 Canadians die due to smoking-related illnesses.¹⁶ Evidence-based treatments are available to aid people who smoke in quitting, with numerous clinical practice guidelines recommending pharmacotherapy and behavioural interventions to those ready to quit.¹⁷ Varenicline is one of the most effective smoking cessation medications, that acts as a partial agonist of the α4β2 nicotinic acetylcholine receptor, reducing nicotine withdrawal symptoms and diminishing the rewarding effects of smoking.¹⁸

Compared to other first-line pharmacotherapies such as nicotine replacement therapy or bupropion, varenicline has demonstrated higher abstinence rates in head-to-head trials and meta-analyses.¹⁹ Despite its proven efficacy, and coverage on public and several private drug formularies, and being the most commonly prescribed smoking cessation medication in the United States,²⁰ poor adherence remains a significant barrier, undermining the potential for successful smoking cessation.^21,22 Common reasons for poor adherence include side effects such as nausea and vivid dreams, lack of ongoing behavioural support, forgetfulness, early relapse to smoking, and limited understanding of the importance of full-course treatment.^22–25 A rule-based chatbot offers a promising solution to this problem given its ability to provide consistent, scalable, and evidence-based behavioural support. By automating interactions that mimic those of human counsellors, a rule-based chatbot can guide users through medication routines, deliver reminders, and offer tailored feedback, all the while adhering closely to structured behavioural intervention frameworks. Its predictable and transparent nature makes it a reliable tool for ensuring fidelity to adherence strategies, while also being more accessible than counselling services to a broader audience.

While studies have shown that conversational agents can increase the engagement of smoking cessation apps²⁶ and a recent scoping review found preliminary evidence of their potential benefit for smoking cessation overall,²⁷ none of the reviewed interventions appeared to be directed specifically towards supporting smoking cessation medication adherence. The review also emphasised the need for researchers to provide more in-depth descriptions of chatbot theoretical underpinnings, and consistency in language and terminology was recommended to facilitate synthesis across studies. To help address these gaps, in this article, we detail the development of ChatV, a theory-informed, rule-based chatbot designed specifically to support adherence to varenicline and informed by the Behaviour Change Wheel (BCW).²⁸

The BCW offers a robust theoretical foundation for designing interventions and has proven successful across diverse settings.^29–33 The BCW consists of the Capability, Opportunity, and Motivation (COM-B) model, which identifies the essential factors for behaviour change; the nine intervention functions, such as education, training, and incentivization that define the types of interventions needed; and seven policy categories, offering high-level strategies for implementing these interventions. In addition, interventions incorporate behaviour change techniques (BCTs), the ‘active ingredients’ specified in the BCTs Taxonomy v1 (BCTTv1), which enable precise content descriptions.^28,34 These BCTs were later converted into BCT Ontology (BCTO) labels³⁵ and allow for improved standardisation, interoperability with other ontologies, and enhanced clarity in digital and automated intervention design. A BCW guide provides theory-based connections between COM-B behaviour influences, intervention types, and BCTs, aiding in the selection of intervention components.²⁸ The BCW aligns with other behavioural frameworks, including the Theoretical Domains Framework (TDF), which builds on COM-B by revealing specific barriers and facilitators to change.³⁶ The TDF synthesises key constructs from multiple behaviour change theories into 14 domains that help identify the underlying determinants of behaviour in a systematic and theory-informed way.³⁶

In addition to its structured approach, the BCW framework incorporates the Acceptability, Practicability, Effectiveness and cost-effectiveness, Affordability, Safety, and Equity (APEASE) criteria to evaluate potential intervention options.³⁷ This ensures that chosen interventions not only address the identified behaviour determinants but also align with real-world considerations such as resource constraints, stakeholder support, and ethical implications. By applying the APEASE criteria, researchers and practitioners can enhance the feasibility and impact of interventions while mitigating unintended consequences.

In our protocol papers,^38,39 we detail our proposed approach to developing and testing a rule-based chatbot, ChatV, designed to improve adherence to varenicline. The literature review, which also helped inform the development, was previously published.⁴⁰ ChatV is a web-based chatbot that can currently be accessed by research study participants with log in credentials provided by research staff members. It can be used 24/7 to document medication adherence and smoking status, and use other functionalities such as asking questions about varenicline and cigarette smoking.

This article focuses on describing the initial steps of adopting a theory-driven approach in the development process and the rationale for selecting BCTs to develop ChatV.

Methods

Study design

This study describes the qualitative interviews and consensus discussion used in the development of the ChatV. It was conducted at the Centre for Addiction and Mental Health and the University of Toronto. We designed ChatV in three steps (Figure 1). The first step focused on identifying and situating the determinants of varenicline adherence and of using digital technology, and the use of digital technology within the COM-B model.²⁸ In the second step, we identified intervention functions using the BCW's structured approach. The intervention options are chosen from a set of nine potential functions (e.g. education, persuasion, incentivization, training, environmental restructuring) based on their relevance to the identified behavioural determinants. The third step involved specifying the active components of the intervention using the BCTTv1, and later converted to BCTO,³⁵ and finalising components we would include in ChatV based on the APEASE criteria.

Figure 1.

Design process.

Across all three steps, we used the same dataset, comprising findings from a rapid systematic review that examined the determinants of varenicline adherence (methodology and results reported elsewhere⁴⁰) and interviews conducted with HCPs and patients, the details of which are presented in this article.

Ethics approval

Ethical approval was granted by The Centre for Addiction and Mental Health (CAMH) Research Ethics Board (REB No. 050/2022).

Step 1 – Understanding the behaviour

Sampling and recruitment of patients and HCPs

The target sample size for the development of ChatV was 20 HCPs and 20 people who had used varenicline (patients). This target was based on prior research indicating that data saturation in qualitative studies is typically achieved with 9–17 interviews.⁴¹ We employed purposeful, non-probabilistic sampling⁴² to ensure a diverse sample. For patients, we aimed for a diverse group of participants across different genders and ages; for HCPs, we prioritised gaining diversity by purposefully sampling by specialty, clinic type, geographic location, and population size. We monitored for data saturation throughout the analytic process. Saturation was determined when no new codes, domains, or themes were identified in consecutive interviews.

Eligibility criteria for patients and HCPs

Patients were eligible if they:

Were currently using varenicline or had taken it within the last six months.

Self-reported difficulty adhering to their prescription.

Were at least 18 years old.

Could read and speak English.

HCPs were eligible if they:

Had seen at least five patients prescribed varenicline within the last two years.

Could read and speak English.

Recruitment methods for patients and HCPs

Both HCPs and patients were recruited through The CAMH Smoking Treatment for Ontario Patients (STOP) Program, an Ontario-wide smoking cessation initiative offering free counselling and nicotine replacement therapy to individuals interested in quitting smoking.⁴³ In addition, patients were recruited through Community Boards and Online Platforms: These included Rainbow Health Ontario, the CAMH Nicotine Dependence Clinic website, and clinics offering smoking cessation services in the Greater Toronto Area. HCPs were also recruited through the monthly STOP Program teleconferences, the Nicotine Dependence Clinic at CAMH, and the Training Enhancement in Applied Counselling and Health Project Listserv, which is a tobacco cessation education programme.⁴⁴

Procedures for patient and HCP interviews

After confirming eligibility, an MSc student researcher (mentored by NM) conducted the informed consent discussion over the phone, explained the study, and answered questions. We emailed the consent forms to all participants, who signed them electronically. One MSc student conducted all patient interviews between October 2022 and January 2023, and a different MSc student conducted all HCP interviews between January and May 2023. Both students explained that the project was funded by the PI (NM) and would form part of their MSc theses, and that we were developing a chatbot where participants’ input would directly shape its design and functionality, reflecting a co-creation approach. For patients, this ensured the chatbot met their needs, while for HCPs, it helped identify features that would make the chatbot easier and more appealing to recommend to their patients. Only the interviewer and the interviewee were present during each session. Both interviewers received training in qualitative interviewing from NM and KM. After each interview, the interviewers kept detailed notes, which were discussed with NM or KM on a weekly basis along with the interview content. Both interview guides probed for determinants of varenicline adherence and usefulness of digital technologies (conversational agents for varenicline adherence in particular), intervention functions, and BCTs that should be included in a digital intervention (interview guides available in Supplemental Material). The interview guide had been pilot tested among research staff before use. Patient interviews lasted between 27 to 66 min, and HCP interviews lasted between 30 to 60 min. Interviews were conducted over the phone, audio-recorded, transcribed verbatim, and cross-checked for accuracy by the student researcher. Transcripts were not returned to participants for comment and/or correction.

Analysis of patient and HCP interview data

Step 1: Identification of determinants of varenicline adherence. We employed framework analysis, a method well-suited for applied health research that allows for systematic categorisation of data into predetermined and emergent themes.⁴⁵ The data was analysed by:

Transcription: A professional transcriptionist transcribed all interviews verbatim, and a researcher cross-referenced the transcripts with audio recordings to ensure accuracy. All transcripts were de-identified and securely stored in CAMH's server.

Familiarisation: Researchers read and re-read the transcripts to immerse themselves in the data and gain an overall understanding.

Coding: Initial coding was conducted by organising data into themes based on the TDF and BCTs, along with any emerging concepts from the interviews.

Developing an analytical framework: An analytical framework was developed by refining the codes and categories based on the initial coding process.

Applying the framework: The framework was systematically applied to all transcripts to ensure consistency in coding.

Charting data into a framework matrix: The data were summarised and organised into a matrix, facilitating comparison of themes across participants.

Interpreting the data: Patterns and relationships between the themes were identified, and findings were synthesised to draw meaningful conclusions.

After the interviews were transcribed and checked for accuracy, they were imported to qualitative analysis software (NVivo V. 12). To ensure the reliability of the coding process, two independent researchers coded 25% of the transcripts. Interrater reliability was assessed using Cohen's kappa, yielding a kappa coefficient of 0.90 and a 99.6% agreement for patient interviews and a kappa coefficient of 0.94 and a 99.7% agreement for HCP interviews. After achieving this high level of agreement, a single researcher coded the remaining transcripts. The transcripts were coded based on the TDF to identify determinants of varenicline adherence. Participants did not provide feedback on the findings.

Step 2 – Identification of intervention options

The framework matrix, compiled from Step 1, was reviewed and discussed by the research team to establish criteria for determining the relevance of each domain to varenicline adherence. The summaries were then assessed to determine whether participants explicitly connected the domain to the target behaviours. Using a consensus-based approach, the multidisciplinary research team (researchers, clinicians, and the tech vendor) selected key theoretical domains for the intervention. Selection decisions were informed by identified barriers and facilitators within the relevant domains and were grounded in the feasibility of addressing these factors within the constraints of digital intervention and the available project resources.

Step 3 – Identification of intervention content and implementation outcomes

The final step involved mapping theoretical domains to BCTs. Three main sources of data were used to identify BCTs:

BCTs mentioned by patients and HCPs during the interviews,

BCTs identified in the rapid systematic review,⁴⁰

A published table of BCTs reliably mapped to the 14-domain version of the TDF.⁴⁶

Two researchers (NM and KM) reviewed all BCTs mapped to the key domains across both sources and reached a consensus on which BCTs to include in the intervention by applying the APEASE criteria. Each behavioural change technique identified was assessed not only for its potential impact on varenicline adherence but also for its feasibility and relevance in the real world:

Affordability ensured that the intervention could be delivered at a reasonable cost without compromising quality.

Practicability focused on how easily the intervention could be implemented within existing healthcare systems.

Effectiveness required evidence that the intervention could lead to meaningful improvements in adherence.

Acceptability addressed whether the intervention would be well-received by users, ensuring buy-in from both patients and HCPs.

Side-effects evaluated any potential negative consequences of the intervention, balancing risks and benefits.

Equity ensured that the intervention was accessible to all populations, preventing disparities in care.

For the purpose of this article, and to ensure consistency with the latest terminology, we converted the identified BCTs to their corresponding BCTO³⁵ labels and Behaviour Change Intervention Ontology (BCIO) identifiers. The BCTO was released after we had initiated our study; therefore, we report both the original BCT labels, used during intervention development, and the updated BCTO labels and numbers. This dual reporting approach enhances transparency, reproducibility, and alignment with evolving standards in behavioural science and digital health research. By adopting the BCTO framework, we contribute to the growing movement towards more precise, computable representations of behaviour change interventions, enabling better cross-study comparisons, meta-analyses, and implementation in digital systems.^35,47

Results

Participant characteristics

We reached 89 of the 139 patients who we attempted to contact to participate in the interview. Of these, 27 declined participation and 62 were screened for eligibility. Thirty-six patients were found to be eligible to participate (reasons for ineligibility usually included not taking varenicline in the last six months or not experiencing issues with adherence), and 26 people provided informed consent, out of which 20 patients completed the interview. Twenty HCPs reached out to participate in the interviews, out of which 19 met eligibility criteria, provided informed consent and completed the interviews. Table 1 provides the socio-demographic characteristics of the patient and HCP participants.

Table 1.

Patients’ and healthcare providers (HCPs)’ socio-demographic characteristics.

Socio-demographic characteristics	N (%)
Patient characteristics
Gender
Man	9 (45%)
Woman	11 (55%)
Age
18–40	4 (20%)
41–60	8 (40%)
61+	8 (40%)
Race
Black	1 (5%)
Métis	1 (5%)
Southeast Asian	1 (5%)
First Nations	1 (5%)
White	16 (80%)
Born in Canada
No	4 (20%)
Yes	16 (80%)
Sexual orientation
LGBTQ+	1 (5%)
Heterosexual	18 (90%)
Do not wish to disclose	1 (5%)
Marital status
Widowed	1 (5%)
Separated/divorced	3 (15%)
Single	6 (30%)
Domestic partnership	10 (50%)
Education
Graduate degree (MSc, MBA, PhD)	2 (10%)
Elementary school	2 (10%)
Technical or vocationschool, apprenticeship training or technical CEGEP	3 (15%)
High school	3 (15%)
Diploma from community college, pre-university CEGEP or non-university	5 (25%)
University degree	5 (25%)
Total family income per capita
>$37,500 (above the poverty line)	8 (40%)
<$37,500 (below the poverty line)	12 (60%)
Prefer not to answer	2 (10%)
HCP characteristics
Profession type
Physician	1 (5%)
Pharmacist	6 (32%)
Nurse	5 (26%)
Respiratory therapist	4 (22%)
Addictions counsellor	1 (5%)
Social worker	1 (5%)
Health educator	1 (5%)
Education
Medical degree (MD)	1 (5%)
Masters	3 (16%)
Bachelors	12 (63%)
Diploma/advanced diploma	3 (16%)
Clinic type
Family health team	12 (63%)
Community health centre	2 (11%)
Nurse practitioner-led clinic	1 (6%)
Addiction agency	4 (21%)
Years providing smoking cessation treatment
1–4	3 (16%)
5–9	2 (11%)
10–14	8 (42%)
15–19	2 (11%)
20+	4 (21%)
Prescribes varenicline
Yes	9 (47%)
No	10 (53%)
Patients treated with varenicline in last 24 months
5–9	4 (21%)
10–19	6 (32%)
20–29	3 (16%)
30+	6 (32%)
Current patients taking varenicline
1–4	11 (58%)
5–9	3 (16%)
10+	5 (26%)

Summary of interviews

Data saturation was reached after 16 patient interviews and 15 HCP interviews; additional interviews were conducted to ensure robustness and confirm saturation.

Analysis of patient interviews identified four overarching themes related to barriers and facilitators to varenicline adherence.

Beliefs about consequences: Between fear and hope. Patients’ beliefs about the potential outcomes of taking varenicline strongly influenced adherence. Concerns about side effects often acted as a barrier, while positive expectations of the medication's efficacy and awareness of the harms of smoking acted as important facilitators.

Social influences: Discouragement and support. Interpersonal relationships were central to adherence behaviours. Social stigma, ridicule, and competing family responsibilities acted as barriers, whereas encouragement and understanding from family, friends, and colleagues served as facilitators.

Environmental context and routines: Competing demands and helpful cues. Environmental demands, such as childcare responsibilities, household disorder, or work stress, often disrupted medication-taking routines. Conversely, environmental structuring, such as embedding varenicline intake into daily routines or placing pills in visible locations, supported adherence.

Memory, decision processes, and self-efficacy: Forgetting versus choosing. Patients clearly distinguished between unintentional non-adherence (forgetting doses during stressful or distracting circumstances) and intentional non-adherence (deliberately skipping doses to continue smoking or due to discouragement). High levels of self-efficacy, motivation, and willpower were described as critical facilitators of consistent adherence.

Analysis of HCP interviews identified six overarching themes that shaped their decision to recommend or not recommend a digital health tool such as a ChatV.

Digital divide and structural barriers. The most frequently cited barrier was patients’ ability to access or use technology, particularly among socioeconomically disadvantaged or rural populations. Broader structural considerations, including organisational policies, cost, language, and ethical concerns, were also highlighted. These contextual factors strongly influenced whether HCPs perceived recommending a chatbot as feasible and equitable.

Balancing benefits and risks in care relationships. HCPs frequently reflected on the potential benefits of recommending digital health tools, such as providing support between appointments and extending patient self-management capacity. At the same time, some expressed concerns about negative consequences, including potential strain on the patient–provider relationship or reputational risks if the tool failed to meet patient expectations.

Social influences: Shaped by patients, colleagues, and leadership. Decisions to recommend digital health tools were influenced by interpersonal dynamics. Positive feedback from patients and endorsements from colleagues or organisational leaders increased willingness to recommend, while negative experiences or lack of peer support discouraged uptake.

Needing clarity and alignment. HCPs emphasised the importance of knowing what information the chatbot would provide and ensuring alignment with their own clinical advice. Awareness of who developed the chatbot (e.g. university, hospital, or commercial company) was also critical, as it influenced perceptions of credibility and ethical integrity.

Professional judgement and patient-centred care. Many HCPs described drawing on their professional skills to determine whether recommending a digital health tool was appropriate for a given patient. This included assessing patient preferences, technological literacy, and mental health status. The ability to tailor recommendations to individual circumstances was seen as a facilitator of adoption.

Hope and hesitation towards AI in healthcare. Providers’ confidence in the capabilities of digital tools varied. While some viewed them as a valuable adjunct to care, others expressed doubts about their ability to adequately respond to patients with complex needs, disabilities, or lower literacy levels. Emotional responses also shaped decision-making: optimism, hope, and curiosity coexisted with scepticism, fear, and hesitation, particularly regarding artificial intelligence in healthcare.

Summary of key determinants

As can be seen in the themes above, various factors were identified as barriers and facilitators to medication adherence as well as to use of digital technologies When we mapped these determinate to the TDF we identified eleven key TDF domains for behaviour change : knowledge, belief about capabilities, belief about consequences, memory, attention and decision process, reinforcement, intentions, goals, social influences, environmental context and resources, behaviour regulation, and skills. A comprehensive list of barriers and facilitators we identified are presented in Table 2, together with illustrative quotes.

Table 2.

Key theoretical domains identified in patient and healthcare provider (HCP) interviews.

COM-B	Theoretical domains	Barriers/facilitators	Illustrative quote from patient interviews	Illustrative quotes from HCP interviews
Capability	Knowledge	Knowing the importance of taking varenicline as prescribed (facilitator – varenicline adherenceIncorrect knowledge of prescription (barrier – varenicline adherence)Knowing digital health provides evidence-based information (facilitator – recommending/using digital health)Not trusting who developed the digital intervention (barrier – using/recommending digital intervention)	‘[Varenicline] works best when there's a constant supply of the drug in my blood system …. Having it go up and down does affect the efficacy of the medication’. (Patient participant 04, 37-year-old man)‘I've got to get [varenicline] into me, otherwise it doesn’t work as well. Taking the one pill a day doesn’t work as well, so I need to take that second pill’. (Patient participant 02, 67-year-old woman)	‘So, they have the first two weeks, they get really confused with refills after the first pack, because there's no refills on the first pack because it's a starter pack. So then they think, oh, well, there's no more refills, I'm done taking them. And it's like no, there's no refills because the next pack is different than this one’. (ID HCP 02, Respiratory educator)‘Well, I’d want to make sure that the information that the chatbot is providing aligns with the information that I would provide. I really just want to be confident that making sure that the technology is appropriate, if – sure that they are getting answers right away. And yeah, I would say as long as it's kind of aligning with the care that I’m providing, I would be confident’. (HCP 03, Pharmacist)
Motivation	Belief about capabilities	Belief that they should be able to quit without using medication (barrier – varenicline adherence)Belief that the digital intervention will not be able to answer questions as well as a person (barrier – using/recommending digital intervention)Belief that digital intervention will provide supports that are currently not available (facilitator – recommending/using digital health)	‘The fact that you have to take medication, that kind of affects me because it makes me feel low as a person. Like, oh my God, like I can’t believe I have to actually take a medication just to quit smoking, and then it just makes me, it just attributes to my self-esteem, my low self-esteem’. (Patient participant 05, 43-year-old woman)‘There's always going to be, and I do truly believe this, there's always going to be one person with one question, or one concern that is not going to get the answer they need because they’re not speaking to a live person’. (Patient participant 12, 43-year-old woman)	‘Also, around using medications, including varenicline or NRT, any form of medication, part of them can feel hesitant to have to do that in order to quit smoking, if they've heard that others have been successful quitting cold turkey, or they think that's the way it should be. That somehow they're flawed by needing to seek more intensive support, whether it just be through counselling treatment and/or medication’. (HCP 07, Respiratory educator)‘I think that there are a lot of benefits for this. I think the main one is really kind of extending the provider. Because providers aren’t available – I think we spoke about the frequency of follow-ups. I mean, I would see a patient if it's – you know, if they're lucky, in a week after they've started, and anywhere between two and four weeks. And a lot of the times, after two, three days that they've started the medication, that's when they stop it, because they have side-effects, or they're not sure what to do. So I think having kind of an extension of the provider to sort of check in with them, they can check in with the chatbot to kind of get some more information, some more knowledge, some more reassurance, I think that could be really helpful for people. Especially because providers aren't available 24/7, and this is. So I think this is kind of the greatest utility, just sort of helping where providers are not available’. (HCP 01, Physician)
Motivation	Belief about consequences	Concerns about medication side effects (beliefs that side effects are worse than medication; won’t work for them; not knowing how to deal with side effects) (barrier – varenicline adherence)Belief that medication will help them quit smoking/be healthier (facilitator – varenicline adherence)	‘I suppose taking any prescription medication, for the most part, is not good’. (Patient participant 01, 56-year-old woman)	‘People have – even though a lot of the mental health warnings have been debunked – are still very concerned about whether they're going to have a psychotic break, they're going to have nightmares. “Can I take it if I've got depression?” You know, all of those kinds of things because, usually, the first news is the news people remember. The information that comes out afterwards is less well-publicised, so they don't realise that maybe these are not any more a valid concern’. (HCP 05, Pharmacist)
Capability	Memory, attention, and decision processes	Unintentionally forgetting to take varenicline (barrier – varenicline adherence)Forgetting if already took medication (barrier – varenicline adherence)	‘Forgetting to bring it with me, or completely forgetting, you know, that I'm supposed to take it at – you know, when I normally do, or even that day. You know, forgetting to take it that day’. (Patient participant 01, 56-year-old woman)‘Questioning whether I'd taken it or not, because there's not really any way to tell with that huge bottle sitting there if I had. So it's – and the older I get, the crappier my memory gets too’. (Patient participant 03, 59-year-old man)	‘I think it's like memory, like forgetfulness, just not being able to, you know – like if they’re not – for one, if people are not taking their medications regularly, as it was at baseline, like if they didn’t have any medications that they’re taking, I think it's easier for them to forget’. (HCP 13, Registered nurse)
Motivation	Reinforcement	Encouragement to take medication through incentives, rewards, or punishment (facilitator)	‘I think positive reinforcement's always helpful. So if it said, oh, you didn’t have any cigarettes today, good going, or something like that, that might be helpful. You know. It's been three days and you haven’t smoked, that's great. I think we all like that, we all like positive reinforcement, even if it's coming from a machine’. (Patient participant 17, 61-year-old woman)‘It's almost like your app needs to guilt people – someone like me, you need to guilt me. Like you know what I mean? Like that type of reinforcement’. (Patient participant 09, 39-year-old man)	‘So any kind of like health benefit, you know, any of that kind of positive encouragement is, I think, good’. (HCP 06, Addiction counsellor)‘Some people really find it reinforcing to see how many days they didn’t smoke, or how much money they saved, or how much they’ve reduced it. It helps them keep on track’. (HCP 07, Respiratory educator)
Motivation	Intentions	Intentionally forgetting to take medication (barrier – varenicline adherence)	‘Sometimes I wouldn’t take [varenicline] because I wanted to smoke too right? … in the morning sometimes I would put [off taking varenicline] … so that I could have morning smokes you know’. (Patient participant 10, 40-year-old woman)	N/A
Motivation	Goals	Having a clear goal to quit smoking (facilitator – varenicline adherence)Motivated to take varenicline as prescribed (facilitator – varenicline adherence)Motivation to use digital technology (facilitator – recommending/using digital health)	‘Because I think overall your main objective is to be able to quit completely, and I think the benefits enable you to accomplish that goal’. (Patient participant 18, 31-year-old woman)‘I'm willing to tolerate a certain amount of side-effects if I'm going to get towards the goal that I want’. (Patient participant 17, 61-year-old woman)	‘I think one is just motivation, so kind of along the way the person starts wanting to quit smoking, but then after a week or two weeks they kind of lose that motivation to quit, and then with it comes stopping to take the medication because their mind is now set on something else’. (HCP 01, Physician)‘I’m not a computer person, and I find them kind of irritating at times, and, you know … I could imagine other people getting irritated by them as well. So it would have to be easy, it would have to be something that, you know, feels helpful to the person, and not just “Oh, my God, like really, you’re tossing me off on some app?”’ (HCP 04, Nurse practitioner)
Opportunity	Environmental context and resources	Impact of cost of medication (barrier – varenicline adherence)Changing routine on medication adherence (barrier – varenicline adherence)	‘If I'm doing like a closing shift or an opening shift, and I'm only on four hours of sleep or something like that, not being able to eat breakfast before I go to work will affect my ability to take it’. (Patient participant 04, 37-year-old man)‘Well, when I flew home to – on the death watch of my grandmother, but I didn’t have the pills (varenicline) with me. And it was much easier just to start smoking again’. (Patient participant 03, 59-year-old man)	‘Number one, the cost. A lot of third-party plans don't cover it, so people are unwilling to pay for it. Even though they're paying for their cigarettes, they're unwilling to pay for the varenicline. So that is the biggest upfront problem is that, even if people want to use it, they don't want to pay for it’. (HCP 05, Pharmacist)
Motivation	Social influences	Need for social support (facilitator - varenicline adherence)	‘I don’t want people to know that I’m taking [varenicline] because I don’t want people laughing at me and thinking “Oh my God, you’re taking medication to quit smoking, that's so stupid”…. my friends… They make fun of me and say that it's stupid that I take it because “why can’t you just quit on your own, why do you need to take a medication for it”’. (Patient participant 05, 43-year-old woman)	‘So some have heard of it or have tried before themselves and they come in optimistically saying, “My neighbour said that they were able to quit smoking with varenicline”’ (HCP 17, Pharmacist)
Motivation	Behavioural regulation	Need for compliance with medication regimen (facilitator – varenicline adherence)	N/A	‘I think that there is more frequent follow-up, and just providing more reassurance, more support, troubleshooting into varenicline, I think that that's something important, that that could help folks feel that they're not alone, they have someone to turn to, to answer their questions, and possibly make them want to continue taking the medication’. (HCP 01, Physician)
Capability	Skills	Unable to take medication due to physical or mental health condition (barrier – varenicline adherence)Having skills to use digital technology (facilitator/barrier – recommending/using digital health)	‘Sometimes I just don’t have the strength because of my challenges to be able to open [the blister pack]. So, if [my wife is] not home, you know, I struggle, and I got to grab scissors and somehow pry this whole thing open just to be able to get my pill out’. (Patient participant 12, 43-year-old man)‘If I remember to take it right away, I'm fine, if I don’t remember to take it … being bedridden, it's not a matter of I wake up, I get up, I go do my thing. I wake up at whatever time I wake up, and then I have to wait for home care to get there’. (Patient participant 03, 59-year-old man)	‘If it's not appropriate for their age and comprehension level and their tech level, or they didn’t have access, good internet access. And again, someone may be struggling with a more severe mental health disorder that – I'd rather speak with them and do more frequent follow-up myself than leaving them to an automated system’. (HCP 09, Nurse practitioner)

Note: COM-B: Capability, Opportunity, and Motivation.

Identification of intervention options (Step 2)

The findings from the rapid systematic review and interviews identified similar domains, with one exception of pessimism, where patient participants reported that their confidence in the effectiveness of varenicline diminished over time.⁴⁰ While both sources highlighted comparable barriers and facilitators to varenicline adherence, the interviews offered richer context through personal and HCP narratives. The research team determined that all domains, except for environmental context and skills, were relevant to include in ChatV. Ultimately, nine key domains were selected as targets for ChatV: knowledge, beliefs about capabilities, beliefs about consequences, memory, attention and decision processes, reinforcement, intentions, goals, social influences, and behaviour regulation.

Identification of BCTS (Step 3)

A total of 24 BCTs were identified from the rapid systematic review⁴⁰ and interviews with patients and HCPs and considered for inclusion in ChatV. These BCTs were systematically evaluated for inclusion, guided by the APEASE criteria. Of the 23 BCTs, 20 met the APEASE criteria and were endorsed for implementation. Two BCTs, problem solving and social support (unspecified), were partially implemented based on participants’ suggestions, while other proposed features were excluded (see Tables 3 and 4 for details). Although most BCTs contained elements that aligned with the inclusion criteria alongside others that did not, the excluded BCTs posed more substantial challenges, ultimately warranting their exclusion.

Table 3.

Behaviour change techniques (BCTs) integrated into ChatV and their alignment with APEASE criteria.

TDF	BCTs/BCTO to address barrier/facilitator	Quotes on how participants thought we could operationalise the BCT	Ways in which BCTs were operationalised in ChatV
Knowledge	Instruction on how to perform the behaviour (4.1)BCTO: instruct how to perform a behaviour BCT (BCIO:007058)BCTO: Promote pharmacological support BCT(BCIO: 007144)Credible source (9.1)BCTO: present information from credibleinfluence BCT (BCIO:007075))	‘So, my prescription that I just got, it's in a bottle, it's not in a package, right? It's just the pills individually. And it just says as directed. So, I looked up online, but then I also called the pharmacy, and I said okay, so as directed, I'm to take one a day, because this is my second step, right? So, you know, it would be easy to go to the chatbot or bot chat, whatever, and just say okay, this is what step I'm on, this is my pills, I've got one – you know, they're one milligram each’. (Patient participant 01, 56-year-old woman)	ChatV to provide information on how and when to take vareniclineChatV is able to answer questions about drug interactionsChatV able to answer questions on what to do about a missed doseChatV to include evidence-based information and mention trustworthy sources of information
Belief about capabilities	Verbal persuasion about capability (15.1)BCTO: persuade about personal capability BCT (BCIO:007137)Self-monitor of behaviour (2.3)BCTO: self-monitor behaviour BCT (BCIO:007024)Self-monitor outcomes of behaviour (2.4)BCTO: self-monitor outcome of behaviour BCT (BCIO:007025)Feedback on behaviour (2.2)BCTO: provide feedback on behaviour BCT (BCIO:007023)Feedback on outcome of behaviour (2.7)BCTO: provide feedback on outcome of behaviour BCT (BCIO:007027)	‘Yes, that would be helpful. Logging in the number of cigarettes and that kind of thing. Because it's accountability and awareness. You know. If you see, oh, this thing is telling me I've had five cigarettes today, or this thing is telling – x number, whatever the number is. You don’t keep track when you smoke. You don’t generally keep track, unless you smoke, let's say, a whole pack a day or something, then you know. But if you don’t smoke that many, you don’t know. And so I could see that as being something helpful’. (Patient participant 17, 61-year-old woman)‘I don’t know about you, but one day blends into the next for me. And everyone else is like that, too, right … So people won’t know, they have busy lives, kids, jobs, stuff going on. Right? So a way of tracking how long they've been taking varenicline for, and how long they've been smoke-free for’. (Healthcare provider (HCP) 11, Other HCP)	ChatV provides encouraging messages about users’ ability to successfully take the medication and reduce/quit smoking (this also falls under rewards and reinforcements).ChatV to enable patients to track medication intakeChatV to enable patients to track cigarettes smokedChatV provides weekly feedback on medication intakeChatV to provide progress updates on smokingChatV to give milestones on smoking cessationChatV to provide a weekly summary of cigarettes smoked
Belief about consequences	Information about health consequences (5.1)BCTO: inform about health consequences BCT (BCIO:00763)Problem-solving (1.2)BCTO: goal strategizing BCT (BCIO:007008)Instruction on how to perform the behaviour (4.1)BCTO: suggest how to perform a behaviour BCT (BCIO:007303)	‘They have all that info… “You haven’t smoked for a – you haven’t smoked for a week, you know, guess what, food tastes better, you reduced your cancer risk, your heart's doing this better”, you know?’. (Patient participant 07, 46-year-old man)	ChatV to provide information about the efficacy of medicationChatV to answer questions about side effects (what to do if feeling nauseous)ChatV to provide strategies to overcome cravingChatV to provide strategies to quit smoking
Memory, attention, and decision processes	Prompts/cues (7.1)BCTO: prompt intended action BCT(BCIO:007080)Problem-solving (1.2)BCTO: goal strategizing BCT(BCIO:007008)Action planning (1.4)BCTO: action planning BCT (BCIO:007010)	‘I would forget what the alarm was for, so it would have to be a text’. (Patient participant 14, 70-year-old woman)‘That would probably be helpful, like, you know, I realised I forgot to take it four hours ago. What do I do now? Do I just skip it and go to the next one, or do I take it right away and then adjust when I'm going to take the one after? How do I get back on track? That would be a good question to include’. Patient participant 20, 45-year-old man‘I would say definitely reminders to take their medications would be very beneficial’. (HCP 03, Pharmacist)	ChatV to provide reminders to take medicationChatV to provide information about missed medicationChatV to provide strategies to take medication regularly (pairing it with an activity that is already part of the daily routine)
Intentions goals	Goal setting behaviour (1.1)BCTO: Goal-setting (behaviour) set behaviour goal BCT(BCIO:007003)Behavioural contract (1.8)BCTO: create behavioural contract BCT(BCIO:007014)Commitment (1.9)BCTO: affirm commitment BCT(BCIO:007015)Goal setting- smoking (1.3)BCTO: set outcome goal BCT(BCIO: 007005)Habit formation (8.3)BCTO: context-specific repetition of behaviour BCT(BCIO:007096)	‘I think that quitting smoking, both including a pharmacological treatment plan like Varenicline, along with when would you like to quit, maybe some suggested goals. You know, we've found that it's more effective if you quit within two weeks of starting the medication. So that being the case, how many cigarettes do you smoke a day? Here's what we suggest in terms of cutting back. You know. Aim for six smokes today instead of, you know, 16, or whatever that looks like for the individual person. That would be helpful’. (Patient participant 20, 45-year-old man)‘Typically with Varenicline I do talk about starting it, and then typically staying on it for two weeks, while it gets into your system, and then choosing a quit date, and then when you quit, when you’ve chosen that quit date, that's the day you, throw away your cigarettes. And so maybe, I guess, if there is something on the app that just sort of says, you know, this is my quit date, then, you know, it's coming up, or whatever, right, like maybe some reminders around my quit date's coming, and, you know, it's in three days, or whatever, you know, how are you doing, do you want to still have that as your quit date? So like I don’t think it's imperative, but I think it helps with that motivation, because I think – and again, it's kind of one of those, you know, sort of like SMART goal, right? Like once you’ve written it down, and kind of committed to it, maybe it's a bit more dynamic. You say, oh, yeah, I said I was going to quit in, you know, two weeks, on the – on this date, and oh, and that date's coming up, so am I ready to really quit?’ (HCP 04, Nurse practitioner)	ChatV allows patients to set personalised goalsChatV encourages participants to establish contracts that help them make commitments to take medication regularly
Reinforcement	Reward (10.10)BCTO: provide positive consequence for behaviour BCT(BCIO:007264)Social reward (10.4)BCTO: provide positive social consequence for behaviour BCT(BCIO:007265)Self-reward (10.9)BCTO: provide positive consequence for behaviour BCT(BCIO:007252)	‘Just a message, you know, saying hey, congratulations, you've done well today, or, you know, keep going and – I guess what would be important is, you know, it telling you the benefits of taking the medication to reach your goal. You know. You haven’t had a cigarette in 24 h, so, you know, this is – already your lungs are starting to get better or increase or – that kind of thing. You know what I mean?’ Patient participant 01, 56-year-old woman‘And then you go down these streaks … “You're on an eight-day streak”. And then they get like little – in the app, they get like little – “You're on an 8-day streak of being smoke-free” or “You get a little sticker for being one week smoke free”, or whatever, at these pivotal times’. (HCP 11, Other HCP)	ChatV provides a reward (trivia) for recording medication intake.Chatbot provides stars/trophies for adheringChatV to share success with friends and research staff while maintaining privacy
Goals	Action planning (1.4)BCTO: action planning BCT (BCIO:007010)Problem-solving (1.2)BCTO: goal strategizing BCT(BCIO:007008)	‘I think it would be more useful if it did things like tell you things you can do instead of having that cigarette’. (Patient participant 02, 67-year-old woman)‘I think the biggest problem with most people quitting smoking is coping strategies for their triggers and barriers to quitting, right? So if there's something that they can go to and say this is really a problem for me, oh, look, I can do that, maybe think of that’. (HCP 09, Nurse practitioner)	ChatV to provide strategies to overcome craving.ChatV to use a harm reduction approach.ChatV to provide strategies to quit smoking
Social influences	Social support (3.1)BCTO: deliver emotional support BCT(BCIO:007041)	‘Well, you know, I think positive reinforcement's always helpful. So if it said, oh, you didn’t have any cigarettes today, good going, or something like that, that might be helpful’. (Patient participant 17, 61-year-old woman)‘I'm not sure what kind of motivational situations are going to happen with this chatbot or if it's strictly just a question-and-answer thing, but “hey” – if it's going to give, like – “hey, it's been three weeks, keep going”, like back and forth, then I think that would be nice’. (HCP 10, Other HCP)	ChatV provides positive reinforcement messages when patients achieve the goals they had set
Behavioural regulation	Action planning (1.4)BCTO: action planning BCT (BCIO:007010)Problem-solving (1.2)BCTO: goal strategizing BCT(BCIO:007008)Self-monitoring (2.3)BCTO: self-monitor behaviour BCT(BCIO:007024)Habit formation (8.3)BCTO: context-specific repetition of behaviour BCT(BCIO:007096)	‘An actual punch-in. Like you could say, “taken” and then you don’t have to worry about it because the app keeps track of the fact you’ve taken it or the bot keeps track of the fact you’ve taken it’. (Patient participant 07, 46-year-old man)‘But I think some way to track their smoking, as well as their varenicline, would be very helpful. So, you know, it could say, like, okay, today is day ten, you are now taking 2 milligrams, you were smoking 40 cigarettes two weeks – ten days ago, and now you're smoking 30, that's great, you've reduced by ten. You know, so it's looking at the progress rather than just the end goal’. (HCP 06, Addictions counsellor)	ChatV to enable patients to track medication intake.ChatV to provide progress updates about medication adherence.ChatV to suggest strategies to manage stressful situationsChatV to enable patients to track cigarettes smokedChatV to provide progress updates on smoking

Note: APEASE: Acceptability, Practicability, Effectiveness, Affordability, Safety, and Equity; TDF: Theoretical Domains Framework; BCTO: Behavior Change Technique Ontology.

Table 4.

BCTs that did not fulfil APEASE criteria.

BCT	Way suggested operationalising BCTs in ChatV	Quotes	APEASE criteria not fulfilled
Monitoring outcome(s) of behaviour by others without feedback (2.1)BCTO: observe outcome of behaviour without feedback BCT (BCIO:007020)	Healthcare providers (HCPs) to have access to patient profiles on ChatV.	‘What might be beneficial to me, would I be able to pull some of those conversations into my computer just to see what issues the patient is having’. (HCP 19, Pharmacist)	Affordability, practicability
Problem-solving (1.2)BCTO: goal strategizing BCT (BCIO:007008)	ChatV to provide information unrelated to varenicline or smoking, but could help (e.g. location of nearby restaurants)	‘If I haven’t taken it, question why, and try to give me solutions that I could practically take. So if I hadn't eaten yet, you know – I don’t know how big of a privacy limit that you'd put on it, but if it knew my location, say hey, here's the closest Timmie's or McDonald's, go, stop, grab a sandwich or a bagel or something, then take it’. (ID Patient participant 04, 37-year-old man)	Affordability, practicability
Social support emotional (3.3)BCTO: deliver emotional support BCT (BCIO:007036)	ChatV to link with other apps, provide meditation, songs	‘Maybe a link to mental health, or if there was even, like if there was a link to an online mental health help’. (Patient participant 07, 46-year-old man)	Affordability
Social support unspecified (3.1)BCTO: advise to seek support BCT (BCIO:007029)	ChatV to have a communal chat board to ask questions to other users and answer their questions.	‘I guess it is a chatbot that answers the questions and, you know, did you get the answer you were looking for. No. Reach out to a community, you know. So, me and five hundred other people are using this app that the chatbot is located in. You can post a question to a communal board where you will not necessarily get a medical professional giving you an answer, but other users who may have experienced your scenario and can answer through there’. (Patient participant 12, 43-year-old man)	Affordability, practicability, safety
Punishment (14.2)BCTO: provide aversive consequence for behaviour BCT (BCIO:007241)	ChatV to include messages that remind users of the consequences of non-adherence, potentially by emphasising the benefits they’re missing out on.	‘Interestingly this medication works fabulously, and then there's the choice to just not adhere to the rules of taking it and finish off quitting smoking. Like the first six to eight weeks are fabulous, and then after that it just drops off. It's almost like your app needs to guilt people – someone like me, you need to guilt me. Like you know what I mean? Like that type of reinforcement. Like the eighties style coach type of – you know what I mean?’ (Patient Participant 09, 39-year-old man)	Acceptability, safety

Note: APEASE: Acceptability, Practicability, Effectiveness, Affordability, Safety, and Equity; BCTO: Behavior Change Technique Ontology.

The excluded BCTs were as follows (the numbers beside each BCT is from the BCTTv1³⁴ and from BCIO³⁵ and will enable others to compare and replicate the intervention):

Monitoring outcome(s) of behaviour by others without feedback (2.1); (BCIO: 007020): This involved providing HCPs access to patient profiles without offering feedback. Concerns were raised regarding this approach's cost, practicability and acceptability, particularly around data privacy and patient autonomy.

Problem solving (1.2) (BCIO: 007008): This proposed providing information unrelated to varenicline or smoking but potentially helpful (e.g. locations of nearby restaurants to enable patients to have food before taking medication). This BCT was excluded due to its limited relevance to the intervention's primary objectives and questions about its effectiveness.

Social support (unspecified) (3.1) (BCIO: 007029): This involved the introduction of a communal chat board for user-to-user interaction. While this approach could foster peer support, it was excluded due to concerns about the practicability and potential for misuse or misinformation.

Social support (emotional) (3.3) (BCIO: 007036): This proposed linking ChatV to unrelated apps for meditation or music to provide emotional support. Affordability and integration challenges were primary reasons for exclusion.

Punishment (14.2) (BCIO: 007241): This involved emphasising guilt to promote adherence. This BCT was excluded due to safety concerns, as it could exacerbate stress or shame, particularly among users facing adherence challenges due to personal or environmental barriers. Such an approach risked reducing motivation, increasing anxiety, and potentially leading to avoidance of ChatV, thereby undermining the intervention's purpose and causing unintended harm.

Table 3 provides a detailed overview of the 21 BCTs that met the APEASE criteria, including their theoretical underpinnings within the TDF and how they were operationalised in ChatV. Table 4 summarises the five BCTs that did not fulfil the APEASE criteria, along with the specific reasons for their exclusion.

Figures 2–5 illustrate ChatV's visual layout, tone of messaging, and how selected BCTs are operationalised within the conversation flow.

Figure 2.

ChatV's menu.

Figure 3.

Example of Commitment (1.9) BCTO: affirm commitment BCT (BCIO:007015).

Figure 4.

Example of Feedback on outcome of behaviour (2.7) BCTO: provide feedback on outcome of behaviour BCT (BCIO:007027).

Figure 5.

Example of Self-monitor outcomes of behaviour (2.4) BCTO: self-monitor outcome of behaviour BCT (BCIO:007025).

Discussion

This study outlines the systematic development of ChatV, a conversational agent designed to enhance adherence to varenicline, guided by the BCW as a theoretical framework. Using a previously published rapid systematic review⁴⁰ and semi-structured interviews with HCPs and patients, we identified 20 BCTs from nine TDFs for potential incorporation into a varenicline-adherence app. Compared to other conversational agents focused on health behaviour change, ChatV incorporates a greater number of BCTs. A recent scoping review found that the average number of BCTs used in ‘lifestyle-change’ conversational agents was nine,⁴⁸ which is considerably lower than what we included in ChatV. Some of the BCTs in ChatV, such as problem-solving, goal setting, and self-monitoring, align with those commonly identified in the scoping review.⁴⁸

While there were many similarities in the preferences expressed by patients and HCPs regarding features (or BCTs) for ChatV, we identified a few differences. Patients emphasised the importance of immediate, accessible, and personalised support, whereas HCPs prioritised evidence-based, transparent features that align with clinical guidelines. In addition, one patient participant mentioned the BCT of punishment as a desirable feature in the chatbot. However, the HCPs did not report a similar BCT, nor does the literature report such features. Indeed, based on the APEASE criteria, this feature was excluded due to the acceptability and safety for other patients.

A strength of this study is its deliberate inclusion of input from individuals who face social and structural disadvantage. Many, participants represented groups that are often underrepresented in digital health design, including women, people with low income, and low formal education.^49,50 This demographic diversity enhanced the relevance, acceptability, and potential equity impact of ChatV, particularly in tailoring behavioural strategies to users at higher risk of tobacco-related harms who are often excluded from clinical and digital innovation efforts.⁵¹

To be able to provide tailored evidence-based support to users, we programmed ChatV from December 2023 to May 2024 based on the data collected to answer user questions, aligning with instructions on how to perform the behaviour and problem-solving. In order to build trust, we include the source of the evidence. This feature provides tailored, evidence-based information addressing user doubts about taking varenicline and managing side effects. For instance, if a user is unsure how to handle a missed dose or manage a specific side effect, ChatV delivers actionable advice informed by the product monograph, clinical guidelines, and user input. According to self-determination theory,⁵² addressing user questions in this way enhances autonomy by helping individuals feel informed and confident in their decisions.

Several BCTs incorporated into ChatV aim to reduce cognitive load, which, according to Cognitive Load Theory,⁵³ enhances decision-making by optimising working memory capacity. ChatV supports users in developing clear, actionable strategies for medication adherence, for example, taking their medication after breakfast with a glass of water (action planning). These strategies are further reinforced through text message reminders (prompts and cues), promoting habit formation and reducing the memory demands associated with taking varenicline. This approach also aligns with habit theory,⁵⁴ which posits that pairing a new behaviour with a stable, recurring context reduces cognitive demands and increases behavioural consistency. Self-monitoring features, such as tracking medication intake and cigarette consumption, provide structured feedback, transforming abstract goals into tangible, accessible progress indicators and thereby lowering mental effort. In addition, ‘instructions on how to perform a behaviour’ reduce cognitive load by providing clear, step-by-step guidance on taking varenicline, managing missed doses, and understanding medication effects, eliminating the need for users to research or interpret complex instructions.

Both patients and HCPs emphasised the importance of providing positive reinforcement to users, whether through rewards (e.g. stars, streaks, and accomplishments), encouraging messages, or feedback on progress. This emphasis on positive reinforcement aligns with a growing body of evidence from other digital behaviour change interventions, which frequently incorporate reward systems and positive feedback as effective mechanisms to drive adherence and sustain behaviour change.^48,55 Research highlights that such strategies not only enhance user engagement but also create a sense of intrinsic motivation and satisfaction, further reinforcing the desired behaviours.^48,55 This emphasis on positive reinforcement aligns with findings from other digital interventions, which frequently incorporate gamification elements and positive feedback as effective strategies for sustaining behaviour change.^56,57

Another core focus of ChatV is building self-efficacy and promoting self-management. Techniques such as goal setting, action planning, and problem-solving were incorporated to empower users to take ownership of their adherence journey. Based on patient interviews, personalised goal-setting features were designed to integrate adherence behaviours into users’ daily routines while addressing their unique challenges. For example, users can set specific goals, such as taking their medication at the same time each day, and receive tailored prompts to help them achieve these objectives. Evidence suggests that self-management strategies are particularly effective in fostering adherence, especially in the context of smoking cessation and chronic disease management.⁵⁸ The personalisation implemented in ChatV reflects the successful use of self-management techniques in digital interventions, as documented in systematic reviews.

Several participants expressed a desire for ChatV to provide social support. While we could only implement a limited number of the requested features, we included options such as encouraging users to share their progress and behavioural contracts with friends, family, or research staff, to harness the accountability and motivation derived from social networks, which are well-documented as key factors in sustaining long-term behaviour change.⁵⁹ Some suggested features, such as communal chat boards, were excluded. Although these could have facilitated peer-to-peer support, they were considered impractical and potentially unsafe. Concerns about long-term sustainability, risks such as cyberbullying, misinformation, and inadequate moderation posed significant barriers to their inclusion.

ChatV differs in several important ways from existing smoking cessation and medication adherence chatbots. First, whereas most chatbots have been developed without an explicit theoretical foundation,^27,60 ChatV was intentionally designed using the BCW, ensuring theory-driven, evidence-based content and structure. Compared to existing smoking cessation chatbots, which often focus broadly on motivation or behaviour change without addressing medication adherence specifically,²⁶ ChatV was uniquely designed to support adherence to varenicline, one of the most effective smoking cessation pharmacotherapy, by targeting known barriers and facilitators of adherence identified through interviews with both patients and HCPs. When compared to medication adherence chatbots, ChatV is also distinctive. Many such tools rely primarily on reminders and basic educational prompts,⁶¹ whereas ChatV integrates a broader range of BCTs.

This study contributes to the growing body of literature demonstrating how the TDF and the U.K. MRC's guidance can be applied to the development of digital health interventions, with a novel focus on their application in the context of a rule-based chatbot. In doing so, it addresses key gaps identified in a recent scoping review,²⁷ which highlighted the lack of detailed descriptions of chatbot functionality, delivery mode, and theoretical underpinnings in existing smoking cessation interventions, as well as the need for greater consistency in terminology. Importantly, our work documents not just what we built, but how and why, offering a replicable model for others. In addition to showcasing theoretical applicability, we address practical considerations encountered during the BCT selection and implementation process. The development team was highly conscious of time constraints, ensuring that BCTs were both evidence-based and feasible to operationalise within the technical and resource limitations of a rule-based chatbot. Decisions regarding which BCTs to include were informed not only by their theoretical alignment with identified barriers but also by their practicality in a time-limited development cycle, for example, developing features such as secure access for HCPs to review the chatbot data or a communal board for patients to communicate with each other would have taken a longer period of time. This approach reflects the balancing act between rigour and pragmatism, emphasising the importance of prioritising techniques that meet the APEASE criteria while maintaining realistic expectations for implementation in a digital health tool.

By detailing these considerations and challenges, this study offers insights into the nuanced process of translating behavioural science frameworks into actionable components for rule based conversational agents. As detailed in our protocol manuscript,³⁹ we are currently conducting a feasibility study to evaluate whether ChatV is used, whether it is used as intended, and whether it warrants testing in a randomised controlled trial.

Several limitations should be acknowledged in the design of ChatV. While we aimed to make the process of choosing BCTs as transparent and reproducible as possible, we encountered several challenges.

First, using the APEASE criteria to select BCTs required subjective judgement in evaluating techniques across multiple dimensions. For instance, assessing the ‘acceptability’ of a technique can vary significantly based on individual user experiences or cultural contexts. Similarly, what is considered ‘practicable’ for a chatbot may depend heavily on the available technical infrastructure. These subjective interpretations introduce variability in how BCTs are selected and implemented within a digital intervention.

Second, many BCTs overlap conceptually, making it difficult to distinguish between them and select the most appropriate one for a given scenario. For example, techniques related to ‘goal setting’ and ‘action planning’ often share similar features but may be classified separately depending on their emphasis. This overlap can complicate efforts to maintain consistency and clarity in intervention design. To mitigate these issues, in this article we also report on the updated BCTO,³⁵ which offers more granular and clearly defined categories, improving precision, standardisation, and interoperability in the classification and application of BCTs within digital interventions.

Third, we did not have a working prototype of ChatV, which may have made it harder for participants to fully envision how specific techniques would function within the chatbot. Without direct interaction with a prototype, participants had to rely on descriptions and hypothetical scenarios, which could have influenced their judgements about the feasibility and usability of different BCTs. To address this, we conducted a Wizard of Oz (WoZ) study, where human operators simulated the chatbot's responses to assess how users interacted with the intervention. The findings from the WoZ will be described in a separate manuscript.

The use of a systematic approach to ChatV development is feasible and should be followed before testing the chatbot in a feasibility trial. Despite the limitations, the MRC framework helps locate the stages of the project, and the COM-B model helped make decisions about the inclusion or exclusion of features in the chatbot. It remains to be proved that such efforts translate into better patient outcomes at scale.

Conclusion

This study describes the systematic, theory-informed development of ChatV, a rule-based conversational agent designed to improve adherence to varenicline, a highly effective smoking cessation pharmacotherapy. We selected a diverse set of BCTs based on input from patients and HCPs and applied the APEASE criteria to ensure each BCT was evidence-based, feasible, equitable, and acceptable for implementation within a digital intervention.

To enhance transparency, reproducibility, and alignment with emerging standards, we reported both the original BCT labels and their corresponding BCTO labels and BCIO identifiers. This dual reporting ensures our work remains accessible and relevant to both current and future behaviour change researchers and digital health developers.

Importantly, ChatV incorporates more BCTs than most existing health behaviour chatbots and addresses several documented gaps in the literature, including limited use of theory, poor reporting of chatbot functionality, and lack of consistency in terminology. By integrating theoretical frameworks with practical design decisions, we offer a replicable model for others developing digital adherence tools.

Our findings contribute to the growing evidence base on how to design rigorous, user-informed digital health interventions. Results from our ongoing feasibility study will determine whether ChatV is acceptable, usable, and ready for full-scale evaluation in a randomised controlled trial. Ultimately, ChatV represents a scalable, evidence-informed approach to improving medication adherence, increasing quit success, and reducing tobacco-related harms at the population level.

Supplemental Material

sj-pdf-1-dhj-10.1177_20552076251380324 - Supplemental material for Using the behaviour change wheel framework to develop a rule-based chatbot to support varenicline adherence for smoking cessation

Supplemental material, sj-pdf-1-dhj-10.1177_20552076251380324 for Using the behaviour change wheel framework to develop a rule-based chatbot to support varenicline adherence for smoking cessation by Nadia Minian, Kamna Mehra, Jonathan Rose, Scott Veldhuizen, Laurie Zawertailo, Matt Ratto, Ryan Ting-A-Kee, Osnat Melamed, Victor Tang and Peter Selby in DIGITAL HEALTH

Footnotes

Acknowledgements

We gratefully acknowledge Sowsan Hafuth and Mackenzie Earle for their valuable contributions to this project, particularly for their support in conducting interviews. Their dedication and assistance were instrumental in advancing this work.

ORCID iDs

Nadia Minian

Ryan Ting-A-Kee

Ethical considerations

The Research Ethics Board of the Centre for Addiction and Mental Health approved this study (REB No. 50/2022).

Author contributions

Nadia Minian: conceptualisation, funding acquisition, methodology, project administration, supervision, visualisation, writing – original draft preparation, and writing – review and editing; Kamna Mehra: formal analysis, investigation, methodology, project administration, visualisation, writing – original draft preparation, and writing – review and editing; Jonathan Rose, Scott Veldhuizen, Laurie Zawertailo L, Matt Ratto and Peter Selby: funding acquisition, methodology, and writing – review and editing; Ryan Ting-A-Kee, Osnat Melamed and Victor Tang: methodology, and writing – review and editing.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research is funded by a Proof of Concept Intervention Grant in Primary Prevention of Cancer (Action Grant) of the Canadian Cancer Society and the Canadian Institutes of Health Research-Institute for Cancer Research (Grant No. 707218) and a Canadian Institutes of Health Research Project Grant: Funding Reference No. PJT 180405.

Declaration of conflicting interests

The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: NM reports receipt of funding from Canadian Institutes of Health Research, Canadian Cancer Society, and from the Discovery Fund of CAMH. PS reports receipt of funding from Canadian Institutes of Health Research, Canadian Cancer Society, Pfizer Inc., Pfizer Canada, and Ontario Lung Association. PS also reports that through an open-tender process, Johnson & Johnson, Novartis, and Pfizer Inc. are vendors of record for having provided smoking cessation pharmacotherapy for research studies at free or discounted rates.

Guarantor

Nadia Minian

Supplemental material

Supplemental material for this article is available online.

References

Dewa

Jacobs

Thanh

, et al. An estimate of the cost of burnout on early retirement and reduction in clinical hours of practicing physicians in Canada. BMC Health Serv Res 2014; 14: 1–9.

Sadek

MacDonald

Streeter

. Stress and burnout among mental health staff during the COVID-19 pandemic. Clin Investig Med 2021; 44: E2–E10.

2022 CaRMS forum presentation deck. Canadian Resident Matching Service; Available: https://www.carms.ca/pdfs/carms-forum-2022.pdf.

Statistics Canada. Canada tops G7 growth despite COVID, Available: https://www150.statcan.gc.ca/n1/daily-quotidien/220209/dq220209a-eng.htm.

Laymouna

Lessard

, et al. Roles, users, benefits, and limitations of chatbots in health care: rapid review. J Med Internet Res 2024; 26: e56930.

Laranjo

Dunn

Tong

, et al. Conversational agents in healthcare: a systematic review. J Am Med Inform Assoc 2018; 25: 1248–1258.

Sallam

. ChatGPT utility in healthcare education, research, and practice: systematic review on the promising perspectives and valid concerns. Healthcare: MDPI 2023; 11: 887.

Nov

Singh

Mann

. Putting ChatGPT’s medical advice to the (Turing) test: survey study. JMIR Med Educ 2023; 9: e46939.

Clusmann

Kolbinger

Muti

, et al. The future landscape of large language models in medicine. Commun Med 2023; 3: 141.

10.

Mittermaier

Raza

Kvedar

. Bias in AI-based models for medical applications: challenges and mitigation strategies. NPJ Digit Med 2023; 6: 113.

11.

Gichoya

Thomas

Celi

, et al. AI pitfalls and what not to do: mitigating bias in AI. Br J Radiol 2023; 96: 20230023.

12.

Gray

Samala

Liu

, et al. Measurement and mitigation of bias in artificial intelligence: a narrative literature review for regulatory science. Clin Pharmacol Ther 2024; 115: 687–697.

13.

Hong

, et al. A bibliometric study and science mapping research of intelligent decision. Cognit Comput 2022; 14: 989–1008.

14.

Skivington

Matthews

Simpson

, et al. A new framework for developing and evaluating complex interventions: update of medical research council guidance. Int J Nurs Stud 2024; 154: 104705.

15.

Thorat

Jadhav

. A review on implementation issues of rule-based chatbot systems. In: Proceedings of the international conference on innovative computing & communications (ICICC), 2020.

16.

Canadian Substance Use Costs and Harms Scientific Working Group. Canadian substance use costs and harms. Canadian Institute for Substance Use Research and the Canadian Centre on Substance Use and Addiction. 2020.

17.

Reid

Pritchard

Walker

, et al. Managing smoking cessation. CMAJ 2016; 188: E484–E492.

18.

LeSage

Shelley

Ross

, et al. Effects of the nicotinic receptor partial agonists varenicline and cytisine on the discriminative stimulus effects of nicotine in rats. Pharmacol Biochem Behav 2009; 91: 461–467.

19.

Cahill

Stevens

Perera

, et al. Pharmacological interventions for smoking cessation: an overview and network meta-analysis. Cochrane Database Syst Rev 2013: CD009329.

20.

VanFrank

BMA

Cornelius

Schecter

, et al. Adult smoking cessation — United States, 2022. Accessed July 2 2025.

21.

Liberman

Lichtenfeld

Galaznik

, et al. Adherence to varenicline and associated smoking cessation in a community-based patient setting. J Manag Care Pharm 2013; 19: 125–131.

22.

Catz

Jack

McClure

, et al. Adherence to varenicline in the COMPASS smoking cessation intervention trial. Nicotine Tob Res 2011; 13: 361–368.

23.

Peng

Swardfager

Benowitz

, et al. Impact of early nausea on varenicline adherence and smoking cessation. Addiction 2020; 115: 134–144.

24.

Peng

Schnoll

Hawk

Jr. , et al. Predicting smoking abstinence with biological and self-report measures of adherence to varenicline: impact on pharmacogenetic trial outcomes. Drug Alcohol Depend 2018; 190: 72–81.

25.

Peng

Morales

Wileyto

, et al. Measures and predictors of varenicline adherence in the treatment of nicotine dependence. Addict Behav 2017; 75: 122–129.

26.

Perski

Crane

Beard

, et al. Does the addition of a supportive chatbot promote user engagement with a smoking cessation app? An experimental study. Digit Health 2019; 5: 2055207619880676.

27.

Whittaker

Dobson

Garner

. Chatbots for smoking cessation: scoping review. J Med Internet Res 2022; 24: e35556.

28.

Michie

van Stralen

West

. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci 2011; 6: 42.

29.

Nelligan

Hinman

Atkins

, et al. A short message service intervention to support adherence to home-based strengthening exercise for people with knee osteoarthritis: intervention design applying the behavior change wheel. JMIR Mhealth Uhealth 2019; 7: e14619.

30.

Gould

Bar-Zeev

Bovill

, et al. Designing an implementation intervention with the behaviour change wheel for health provider smoking cessation care for Australian indigenous pregnant women. Implement Sci 2017; 12: 1–14.

31.

Chen

Zhang

Cui

, et al. Development of a behavior change intervention to improve physical activity adherence in individuals with metabolic syndrome using the behavior change wheel. BMC Public Health 2022; 22: 1740.

32.

Allison

Purkiss

Lorencatto

, et al. Improving compostable plastic disposal: an application of the behaviour change wheel intervention development method. Front Sust 2022; 3: 968152.

33.

Robertson

Baranowski

Thompson

, et al. Using the behaviour change wheel program planning model to design games for health: development study. JMIR Serious Games 2021; 9: e29964.

34.

Michie

Richardson

Johnston

, et al. The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: building an international consensus for the reporting of behavior change interventions. Ann Behav Med 2013; 46: 81–95.

35.

Marques

Wright

Corker

, et al. The behaviour change technique ontology: transforming the behaviour change technique taxonomy v1. Wellcome Open Res 2023; 8: 308.

36.

Cane

O'Connor

Michie

. Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci 2012; 7: 37.

37.

Michie

Atkins

Gainforth

. Changing behaviour to improve clinical practice and policy. Novos desafios, novas competências: contributos atuais da psicologia Braga: Axioma-Publicações da Faculdade de Filosofia. 2016:41–60.

38.

Minian

Mehra

Earle

, et al. AI conversational agent to improve varenicline adherence: protocol for a mixed methods feasibility study. JMIR Res Protoc 2023; 12: e53556.

39.

Minian

Mehra

Rose

, et al. Cocreation of a conversational agent to help patients adhere to their varenicline treatment: a study protocol. Digit Health 2023; 9: 20552076231182807.

40.

Minian

Wong

Hafuth

, et al. Identifying determinants of varenicline adherence using the theoretical domains framework: a rapid review. BMC Public Health 2024; 24: 679.

41.

Hennink

Kaiser

. Sample sizes for saturation in qualitative research: a systematic review of empirical tests. Soc Sci Med 2022; 292: 114523.

42.

. Qualitative research and evaluation methods. Thousand Oaks, CA: 3rd Sage Publications, 2002.

43.

Selby

Voci

Zawertailo

, et al. Public health impact of a novel smoking cessation outreach program in Ontario, Canada. BMC public Health 2018; 18: 1–11.

44.

Herie

Connolly

Voci

, et al. Changing practitioner behavior and building capacity in tobacco cessation treatment: the TEACH project. Patient Educ Couns 2012; 86: 49–56.

45.

Gale

Heath

Cameron

, et al. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med Res Methodol 2013; 13: 117.

46.

Cane

Richardson

Johnston

, et al. From lists of behaviour change techniques (BCTs) to structured hierarchies: comparison of two methods of developing a hierarchy of BCTs. Br J Health Psychol 2015; 20: 130–150.

47.

Norris

Zhang

Wuerstl

, et al. A data extraction template for the behaviour change intervention ontology. Wellcome Open Res 2024; 9: 168.

48.

Martinengo

Jabir

Goh

WWT

, et al. Conversational agents in health care: scoping review of their behavior change techniques and underpinning theory. J Med Internet Res 2022; 24: e39243.

49.

Veinot

Mitchell

Ancker

. Good intentions are not enough: how informatics interventions can worsen inequality. J Am Med Inform Assoc 2018; 25: 1080–1088.

50.

Figueroa

Luo

Aguilera

, et al. The need for feminist intersectionality in digital health. Lancet Digit Health 2021; 3: e526–e533.

51.

Brewer

Fortuna

Jones

, et al. Back to the future: achieving health equity through health informatics and digital health. JMIR Mhealth Uhealth 2020; 8: e14512.

52.

Ryan

Deci

. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am Psychol 2000; 55: 68–78.

53.

Sweller

. Cognitive load theory. The psychology of learning and motivation: cognition in education, Vol 55. San Diego, CA, US: Elsevier Academic Press, 2011, pp. 37–76.

54.

Gardner

Lally

Wardle

. Making health habitual: the psychology of ‘habit-formation’ and general practice. Br J Gen Pract 2012; 62: 664–666.

55.

Aguiar

Trujillo

Chaves

, et al. mHealth apps using behavior change techniques to self-report data: systematic review. JMIR Mhealth Uhealth 2022; 10: e33247.

56.

Lieder

Chen

Prentice

, et al. Gamification of behavior change: mathematical principle and proof-of-concept study. JMIR Serious Games 2024; 12: e43078.

57.

Figueroa

Deliu

Chakraborty

, et al. Daily motivational text messages to promote physical activity in university students: results from a microrandomized trial. Ann Behav Med 2022; 56: 212–218.

58.

Fujii

Nakano

Tanaka

, et al. Effects of self-management interventions with behavior-change support on long-term adherence in patients with chronic respiratory diseases: a systematic review. GHM Open 2022; 2: 12–24.

59.

Latkin

Knowlton

. Social network assessments and interventions for health behavior change: a critical review. Behav Med 2015; 41: 90–97.

60.

Zeng

Chen

Mastey

, et al. Effects of copayment on initiation of smoking cessation pharmacotherapy: an analysis of varenicline reversed claims. Clin Ther 2011; 33: 225–234.

61.

Hartch

Dietrich

Lancaster

, et al. Effects of a medication adherence app among medically underserved adults with chronic illness: a randomized controlled trial. J Behav Med 2024; 47: 389–404.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.59 MB