Sequencing conversational turns in peer interactions: An integrated approach for evidence-based conversational agent for just-in-time nicotine cravings intervention

Abstract

Background

Risky health behaviors place an enormous toll on public health systems. While relapse prevention support is integrated with most behavior modification programs, the results are suboptimal. Recent advances in artificial intelligence (AI) applications provide us with unique opportunities to develop just-in-time adaptive behavior change solutions.

Methods

In this study, we present an innovative framework, grounded in behavioral theory, and enhanced with social media sequencing and communications scenario builder to architect a conversational agent (CA) specialized in the prevention of relapses in the context of tobacco cessation. We modeled peer interaction data (n = 1000) using the taxonomy of behavior change techniques (BCTs) and speech act (SA) theory to uncover the socio-behavioral and linguistic context embedded within the online social discourse. Further, we uncovered the sequential patterns of BCTs and SAs from social conversations (n = 339,067). We utilized grounded theory-based techniques for extracting the scenarios that best describe individuals’ needs and mapped them into the architecture of the virtual CA.

Results

The frequently occurring sequential patterns for BCTs were comparison of behavior and feedback and monitoring; for SAs were directive and assertion. Five cravings-related scenarios describing users’ needs as they deal with nicotine cravings were identified along with the kinds of behavior change constructs that are being elicited within those scenarios.

Conclusions

AI-led virtual CAs focusing on behavior change need to employ data-driven and theory-linked approaches to address issues related to engagement, sustainability, and acceptance. The sequential patterns of theory and intent manifestations need to be considered when developing effective behavior change CAs.

Keywords

Smoking cessation online health communities virtual conversational agents behavior change techniques speech acts health promotion

Introduction

Risky health behaviors such as tobacco use, opioid addiction, and polysubstance abuse are major contributors to premature morbidity and mortality, of which tobacco consumption alone is responsible for killing about eight million people every year across the globe.¹ It is also responsible for rising healthcare expenditures as about US$1.4 trillion is spent annually for treating smoking-related illnesses.¹ According to a recent report, tobacco cessation rates in the USA remain low; even though about 68% of adult smokers expressed their desire to quit tobacco use, 55.4% attempted to quit and only 7.4% reported successful quitting in 2015.² Behavior modification is a core component of the management of these risky health behaviors.³ It is a complex process and research has shown that a range of psychological and social processes influence the engagement of an individual in the sustenance of positive health behaviors.⁴ Behavior change can be a daunting task, and oftentimes individuals relapse as they attempt to embrace and sustain a positive health change. Nearly 75%–80% of the smokers who attempt to quit, relapse before achieving six months of abstinence.⁵ Therefore, supporting individuals to make a successful quit attempt and preventing relapse become a public health priority.¹

The best practice guidelines to support relapse prevention include a combination of pharmacological support such as nicotine replacement therapy (patches, gums, lozenges, etc.) or medications (e.g., Bupropion) and behavioral support delivered via face-to-face counseling sessions with a doctor or trained healthcare professional is found to be most effective for tobacco cessation.⁶ However, only 25% of cigarette smokers reported using nicotine patches or gum during their most recent quit attempt, only 12.2% reported using tobacco cessation medications approved by the FDA, and only 15% sought help from a doctor or other healthcare professional.⁷ One of the barriers to using these evidence-based conventional quit tools is that a majority of smokers quit all at once unassisted due to misperceptions regarding the effectiveness and safety of pharmacotherapy options.^7,8 Other barriers include accessibility issues such as lack of time, transportation issues, and cost, especially among the low socio-economic status populations.⁹

Digital behavior change interventions (DBCIs) have been gaining popularity and have emerged as a mainstay in the behavior change ecosystem. Previous reviews have already established the effectiveness of DBCIs in promoting the adoption of positive health behaviors.^10,11 More recently, given the wide reach and accessibility of mobile and sensing technologies, the use of just-in-time adaptive interventions that aim at providing tailored support at the right time to individuals is becoming increasingly prevalent for supporting health-related behavior change.^12,13 These seminal works highlight the necessity for behavior change interventions to adapt in real-time to an individual's changing context and internal state, which is particularly relevant for relapse prevention with risky behaviors such as tobacco use, where behavioral context can change rapidly and unexpectedly. In the context of tobacco cessation, such interventions function in a variety of areas including, but not limited to, delivering tailored content in real-time via mobile apps, providing peer support via online health communities (OHCs), and conversational agents (CAs) via simulated peer advice, lifestyle tips, and programmed education.^14,15 CAs are computer-based programs that simulate human-like conversations and use natural language to interact with end users.^16,17 They capitalize on the recent advances in voice recognition, artificial intelligence, and natural language processing to extend our ability to provide complex dialog management (DM) methods and improved conversational flexibility.^16,17 Previous research has explored the use of CAs for tobacco cessation interventions. One study implemented a CA within social media instant messenger.¹⁸ It showed that user engagement was higher when CA was involved in the conversation, and also users had a higher rate of abstaining from tobacco use when they were involved in conversations with the agent than without it.¹⁸ Another study added a motivational CA within a tobacco cessation application called the Smoke Free app, and the results showed that the users interacting with the CA had higher chances of success during quitting.¹⁹ Calvaresi et al.²⁰ also implemented CA within Facebook Messenger and obtained similar promising results. One classic example of a commercial conversational text-based intervention designed to help adult smokers begin their quit journey is called Bella.²¹ Bella is an AI (artificial intelligence) powered chatbot for tobacco cessation, developed primarily based on expert guidance and was released in January 2018 in the UK.²¹

Despite taking on the role of “digital companions,” CAs have their limitations; for example, one of the most advanced commercial chatbots targeting tobacco cessation gets stuck when users input responses that are not prompted.²² Other barriers to implementing CA within the context of health care are difficulty in understanding users’ perceptions and cultural attitudes, being repetitive and less engaging, and their inability to form personal connections (humaneness) as users would do with their peers or family in real-life settings.^23,24 Thus, there is a need for CAs that are (a) data-driven to anticipate conversational turns and intent drifts to provide seamless health communications guidance, which may promote higher user engagement and ultimately provide long-term tobacco cessation support, and (b) theory-informed, grounded in behavioral theory and human communication models as research has shown that theoretical fidelity improves the robustness of DBCIs, making them more efficacious than their counterparts.

Given these research gaps, the specific objective of this work is to develop a methodological framework for the emulation of human-CA interactions that build on social media sequencing. Such empirical grounding will enable us to identify organic sequential patterns in theory and intent manifestation as peers interact in digital social settings to facilitate the design of just-in-time adaptive CA-based interventions to support behavior change. This approach would bridge the abovementioned gaps with sustained user engagement at a more fundamental level by examining the syntactic and semantic turns embedded within online peer conversations. Also, to the best of our knowledge, no CAs have explicitly focused on craving management to reduce the chances of relapse among individuals who have quit tobacco use.

Methods

Figure 1 provides an overview of the methodological framework, which consists of (a) a data modeler, (b) a social media sequencing core, and (c) a scenarios builder.

Figure 1.

Overview of the methodological framework for designing a CA.

Their descriptions are provided in the section below. In the scope of this research, we apply this framework to develop a CA for nicotine cravings management to prevent relapse during tobacco cessation. To first understand the critical hallmarks of human conversations in the domain, we leveraged online peer conversations from QuitNet, an OHC for tobacco cessation.²⁵ QuitNet is one of the most popular OHCs that promotes tobacco cessation, with more than 800,000 users since its beginning in 1997.²⁵ The community users are predominantly smokers who are preparing to quit or ex-smokers who aim to stay abstinent. QuitNet provides social support to its users via multiple channels (e.g., private emails, threaded forums, and chat rooms). The users are also encouraged to share their quit dates during their forum conversations as a self-reported abstinence signature. Initial studies have shown a strong association between an individual's participation in QuitNet and abstinence, as compared with individuals who do not participate in such a community.^25–29

Data modeler

The data modeler facilitates the characterization of thematic preferences, theoretical behavior change constructs, and implicit user needs as manifested in online peer interactions. This understanding has enabled us to emulate content boundaries, response tone, and response content in our CA architecture. We utilized the behavior change technique (BCT) taxonomy³⁰ to identify manifestations of theory-linked BCTs embedded in online peer interactions. The BCT taxonomy is a cross-disciplinary taxonomy from domains such as psychology, engineering, and behavioral science that provides a foundation for the design and evaluation of reliable behavior change interventions. This taxonomy has facilitated the study of mechanisms of action related to the change in behavior and the evaluation of such components. Effective BCTs have been identified in interventions aiding in tobacco cessation.³¹

Previous studies have analyzed speech acts (SAs) to understand users’ general attitudes and state of mind as they interact with their online peers.³² Some studies have also identified users’ intent to be able to determine the response of the chatbots.^33–35 To model the communication context underlying online peer interactions (user's implicit intent), we have utilized a modified version of Searle's speech act theory³⁶ which helps to describe how humans express themselves during their communication with others using 10 different categories of SAs. The SA theory³⁶ is a subfield of pragmatics, a branch of linguistics that tries to understand how individuals produce and understand meanings through language.³⁶

Social media sequencing core

The social media sequencing core identifies the sequential patterns describing the usage of various categories of BCTs and SAs embedded within the conversation threads from a tobacco cessation OHC. This understanding has enabled us to capture changes in the user's context that can impact the expression of BCTs within the responses of the CA. Using the Frequent Pattern Growth algorithm,³⁷ we identified the underlying relationships or associations that occur sequentially between various categories of BCTs and SAs embedded within the obstacles theme labeled peer conversations (n = 339,067). Such sequential pattern mining algorithms have been utilized to understand the emotional and contextual features underlying depressive user behaviors from online postings of social media users.³⁸ One study used sequential pattern mining techniques to predict the users’ moods based on digital technology usage patterns.³⁹ An association rule BCT (or SA) category A > BCT (or SA) category B indicates that category A is likely to occur with category B. We used the following measures to identify relevant association rules within our dataset – support, confidence, and lift. Support of a rule refers to the probability of category A and B occurring together in a conversation; confidence represents the probability of also having category B along with category A. Lift refers to the increase in the ratio of presence of category B when category A is also expressed.

Scenarios builder

The scenario builder extracts the relevant nicotine cravings-related scenarios from our tobacco cessation OHC. From a total of obstacles theme labeled peer conversations (n = 339,067), we randomly selected a subset of messages (n = 1000) for extracting the scenarios that best describe users’ needs when dealing with nicotine cravings using grounded theory-based qualitative analysis.⁴⁰ The grounded theory technique is a method of analysis that inductively enables the extraction of concepts, which in this case are the nicotine cravings-related scenarios as they emerge from the user-generated text. We used two main characteristics of this technique to derive meaningful representative scenarios from the peer interactions of an OHC, i.e., open coding and constant comparison.⁴⁰ In open coding, the researchers initially immerse themselves in the text by reading and re-reading it multiple times.⁴⁰ After this, the text is divided into meaningful units that capture a complete thought. Units carrying similar meanings are then grouped into larger-order categories, and these categories are continuously updated and compared throughout the analysis by including or excluding additional meaningful units.⁴⁰ It is referred to as constant comparison, which enables the creation of a hierarchy of categories from which overarching core categories that are consistent with the data at hand are generated.⁴⁰ This process is usually repeated unless saturation is obtained. Saturation is when the addition of new data does not lead to the extraction of any additional concepts or the generation of any additional categories.

Using the subset of obstacles theme labeled peer messages, two researchers performed a line-by-line analysis of the messages using grounded theory techniques (open coding, axial coding, and constant comparison described above) to derive abstract concepts regarding user needs when dealing with nicotine cravings. This process was repeated until no new scenarios were produced from this dataset, i.e., until saturation in extracting of new concepts was achieved. Using this approach, 22 concepts were identified, and similar concepts were then grouped to generate nicotine cravings-related scenarios from the dataset of QuitNet messages. The emergence of new concepts stopped at message number 140; we further coded the remaining messages to ensure the saturation of the concepts. These scenarios were then used to further define the environmental boundaries of the CA. Cohen's kappa measure of 83% was achieved between the two coders and any disagreements were discussed among the researchers until a mutual agreement was reached.

CA architecture design

We generated user intents and responses using the scenarios from the prior step by performing a line-by-line analysis of every message using directed content analysis techniques where each message was chosen as the coding unit for analysis.⁴¹ The textual data of forum messages was first preprocessed using the Natural Language Toolkit (NLTK), and this step included the following tasks: converting text to lowercase, removing punctuation, removing white spaces, removing special characters, and lemmatization.⁴² Using these intents and responses, we trained the RASA framework as described below.

RASA is one of the most popular open-source frameworks for building task-oriented chatbots and virtual assistants. This framework makes use of machine learning techniques to recognize the intent of the user, extract entities, execute and action, and manage the flow of the conversation (described below). The architecture and basic workflow of CA is outlined in Figure 2. Once the user initiates the conversation, the user's message is sent to the Natural Language Understanding (NLU) model of the framework; which preprocesses the text from the user's message and then further parses the data to perform two tasks–intent classification and entity extraction. The output of the NLU model is then sent to the DM core model of the framework, which classifies the following action which is appropriate given what has happened in the conversation. It then chooses a response that is sent back to the user based on the list of responses the agent has been trained to return. Using our conversation scenarios from the prior step, we created 12 user intents (of which five were specifically based on the extracted scenarios, and the remaining seven were to support conversations within the agent) with more than 10 training examples for each intent to train the NLU model. We further created approximately 100 responses for training the DM model. We used the default NLU pipeline for training the model (epochs = 100); preprocessing was implemented using the spaCy language model. We further trained the DM model with epochs = 100, and max_history = 5 (memorization_policy). The fallback_policy was executed if the intent classification pipeline confidence was below the threshold value of 0.3; in this scenario, a default fallback message will be provided to the end users (e.g., “Sorry, did not understand”), which would then revert the conversation to the state before the end user's message that caused the fallback to not influence future actions.

Figure 2.

Architecture and basic workflow of the CA.

Ethical considerations

This study was exempted from human subjects review by the Institutional Review Board at the University of Texas Health Science Center at Houston (HSC-SBMI-15-0697), thus consent was not required. We extracted only the forum messages that were in the public domain, i.e., peer exchanges that were marked public by the users of the forum. To maintain user anonymity, we de-identified the data obtained from the forum by assigning every community user a unique user identifier. In addition, the researchers had no direct contact with the community users.

Results

Data modeler

The data modeler used was semi-automated in nature, given that we used the manually annotated messages (n = 2005) to train deep learning models and used the best-performing model that resulted in 2.23 million labeled peer interactions spanning from 2000 to 2015. The results from this core are thoroughly discussed in Singh et al.⁴³

Social media sequencing core

Table 1 presents the top 10 discovered association rules for BCTs and their corresponding support, confidence, and lift values. In our dataset, comparison of behavior and feedback and monitoring occur together frequently with the support value of 0.38. The confidence value of 0.96 indicates that of all the peer conversations that have a comparison of behavior BCT embedded in them, about 96% contain feedback and monitoring BCT as well. The lift value of 1.02 indicates that expression of the feedback and monitoring BCT is 1.02 times more likely to be with the comparison of behavior BCT than the default likelihood of comparison of behavior BCT alone in obstacles theme labeled peer conversations. Similarly, the support value of the rule comparison of behavior, self-belief → feedback and monitoring is 0.31, and the confidence value of 0.98 indicates that that of all the peer conversations that have the comparison of behavior and self-belief BCTs embedded in it, about 98% contain feedback and monitoring BCT as well, and it is 1.03 times more likely a chance that the expression of comparison of behavior and self-belief BCTs occurs with feedback and monitoring BCT than its expression alone in the peer conversations.

Table 1.

Top 10 sequential patterns for BCTs within the obstacles theme labeled peer conversations.

Sequential patterns	Support	Confidence	Lift
Comparison of behavior → Feedback and monitoring	0.38	0.96	1.02
Comparison of behavior → Self-belief	0.32	0.80	1.03
Comparison of behavior → Natural consequences	0.29	0.73	0.98
Comparison of behavior, Self-belief → Feedback and monitoring	0.31	0.98	1.03
Comparison of behavior, Feedback and monitoring → Self-belief	0.31	0.81	1.03
Comparison of behavior → Self-belief, Feedback and monitoring	0.31	0.78	1.04
Comparison of behavior, Natural consequences → Feedback and Monitoring	0.28	0.96	1.02
Comparison of behavior, Feedback and monitoring → Natural consequences	0.28	0.73	0.98
Comparison of behavior → Feedback and monitoring, Natural consequences	0.28	0.70	0.99
Comparison of behavior, Self-belief → Natural consequences	0.24	0.76	1.01

Table 2 presents the top 10 discovered association rules for SAs and their corresponding support, confidence, and lift values. In our dataset, statement and assertion occur together frequently with the support value of 0.33. This rule has a confidence value of 0.75 which means that of all the peer conversations that have a statement SA embedded in it, about 75% contain an assertion SA as well. The lift value of 1.02 indicates that the presence of the statement SA is 1.02 times more likely to be with assertion SA than the default likelihood of the statement SA alone in obstacles theme labeled peer conversations. Similarly, the support value of the rule directive → assertion is 0.38, and the confidence value of 0.82 indicates that of all the peer conversations that have the directive SA embedded in it, about 82% contain an assertion SA as well; and there is 1.10 times more chance that the expression of the directive SA occurs with the assertion SA than its expression alone in peer conversations.

Table 2.

Top 10 sequential patterns for SAs within the obstacles theme labeled peer conversations.

Sequential patterns	Support	Confidence	Lift
Statement → Assertion	0.33	0.75	1.02
Statement, Directive → Assertion	0.17	0.84	1.13
Statement, Assertion → Directive	0.17	0.52	1.12
Directive → Assertion	0.38	0.82	1.10
Assertion → Directive	0.38	0.51	1.10
Question → Assertion	0.08	0.55	0.75
Question → Statement	0.07	0.51	1.15
Directive, Question → Assertion	0.04	0.69	0.93
Directive, Question → Statement	0.03	0.55	1.26
Statement, Directive, Question → Assertion	0.02	0.75	1.01

Scenarios builder

A total of five nicotine cravings-related scenarios were derived using the grounded theory techniques (open coding and constant comparison).⁴⁰ The derived concepts, scenarios, and example messages that outline the user needs that will be supported by our CA to help individuals overcome nicotine cravings are described in Table 3.

Table 3.

Derived nicotine cravings-related scenarios, their description, and example messages.

Scenario	Description	Example messages
1	The users' express the desire to smoke as they have been experiencing nicotine cravings given their present situation. This scenario was composed by combining multiple concepts such as “have strong urges,” “want to smoke,” and “bad craves day today.”	“It's been a rough day for some reason (strong craves today)” “I am on day 21, and I want to smoke so bad today!!!”
2	The users' specifically ask for help or guidance to deal with nicotine cravings. In this scenario, the users may or may not mention their stage of quit i.e., whether they have been abstinent for a long time or are early in their quit. This scenario was generated by combining concepts such as “help me,” “looking for guidance,” “provide suggestions,” and “how to overcome cravings.”	“Is it normal to still have cravings 35 days into the quit???? I really hope this isn't going to be a permanent thing because honestly, this is ridiculous!!!!” “I am going through a rough patch in my quit, have been experiencing terrible craves since morning. What should I do?”
3	The users' specifically state their intention to quit, their plans of starting the quit journey, and thus, ask their peers for guidance on how to deal with nicotine cravings during the initial quit days. The core concepts of this scenario were “planned quit date is nearing,” “quitting real soon,” and “provide ideas for the initial tough days.”	“My planned quit date is approaching. How to get through the first 3 weeks? How to deal with the nicodemon cravings during that time? I could use a few more ideas to put into my plan” “I have picked a quit date am going to stick to it. Do you have any tips on how to avoid the cravings in the beginning? I have tried to quit before and it is the first thing after waking up that gets to me.”
4	The users' from the community share their experiences of how they overcame the challenges related to nicotine cravings to encourage other users to sustain their quits. The users', within this scenario, provide descriptions of facing such challenges such as providing practical approaches to deal with stressful situations or underlying motivations to ward off the cravings. The key concepts that lead to the emergence of this scenario were – “pledged to not smoke,” “cravings become less intense after a few weeks,” “not giving up,” logged in here instead,” “distractions,” “remove triggers,” and “motivations.”	“I am on my fourth day, using nicotine gum as a replacement and so far so good. I can breathe better already and don't feel sick/tired all the time, so I think about that all the time to stay motivated. Also, I moved my triggers around, like I only drink coffee/alcohol after my gum, so I will not crave smoking. I'm also making plans on the trips I'll be taking with the money I'll save by the end of next year.” “Today is my first day. My morning was really rough (it's my worst time of day), but I got through it and am now at work. I pledge to walk off any craves and if that doesn't work, I come and chat with you. My co-workers are my quit buddies because they are now non-smokers. And with a lot of prayers and support, I know I can do it! Almost 12 h quit! Hurray!.”
5	The users' specifically asked for help to deal with nicotine cravings during the first few days of their quit. Some of the core concepts within this scenario were – “quit last night,” “new quitter,” “first week of quit,” “came here as a precaution,” “provide help for those initial days.”	“It is my first day of quit, I woke up with bad chest pains and I am only 22 so I have decided I have had enough. I haven't had a cigarette in 5 h and the withdrawals are already in full swing. I need strength” “I quit cold-turkey last Wednesday, and started taking deanxit to ward off the nerves. This morning I woke up with terrible cravings and I cannot get my mind of it. Please help me!!”

CA architecture design

In the section below, we outline various user intents and responses specific to nicotine cravings-related scenarios that were extracted via scenario builder core. Within each scenario, we also provide the mapping of user intents to various categories of SAs and the mapping of responses to various categories of BCTs (see Figures 3 and 4). For instance, in scenario 1, where users specifically state their longing to smoke, desire and declarative SAs captured such user needs (see Figure 3). In scenario 2, the users express SAs of question, emotion, desire, and commissive since they express their problems and ask questions about handling nicotine cravings. In scenario 3, SAs of question, declarative, and commissive captured the inquiries of the users as they plan to start their quits. In scenario 4, users describe their beliefs as they deal with obstacles and share their inspirations for staying committed to their quit goals, thereby expressing SAs of assertion, directive, and declarative. Directive and assertion SAs occur together frequently with the support value of 0.38. In scenario 5, the users express SAs of question, declarative, commissive, and desire to describe their desire to smoke given they have just started their quit. The most prevalent SAs within the extracted user intent are declarative, desire, question, and commissive.

Figure 3.

User intents, example messages, and their mapping to SA theory.

Figure 4.

CA responses, example messages, and their mapping to the taxonomy of BCTs.

The BCTs are responsive to the user's intent as captured via QuitNet community peer interactions and are based on the responses provided by successful quitters in the community (see Figure 4). For instance, the responses to the user's intent within scenario 1 are composed of BCTs like feedback and monitoring, antecedents, and repetition and substitution, where users are given evaluative feedback on their behaviors and provided information about how they can incorporate new habits and restructure their environments to distract themselves (e.g., go for a walk) and ward off nicotine cravings. In scenario 2, since the users have expressed their problems and asked questions about how to handle nicotine cravings, the responses are composed of BCTs like feedback and monitoring, antecedents, and natural consequences to highlight the consequences (health, social, emotional) of tobacco use if they continue to do so and again adding distractions or restructuring physical or social environment (e.g., drink more water) to deal with nicotine cravings. In scenario 3, since the users specifically asked for help to deal with nicotine cravings as they plan their quits, we have incorporated BCTs, including shaping knowledge, antecedents, social support, and natural consequences in the responses since these techniques provide advice to users on how they can go about formulating a plan which can include antecedents (e.g., gums, candies, books). In addition, they also highlight the consequences that are associated with their continued tobacco use, such as lousy health or loss of money as it is being spent on buying cigarettes. In scenario 4, the users have described how they dealt with nicotine cravings and did not give into tobacco use again; therefore, the BCTs of feedback and monitoring and social support should be incorporated in responses to provide social support to users for their efforts in dealing with nicotine cravings. In scenario 5, the users described their desire to smoke given they have just started their quit, and thus, the responses are composed of BCTs such as feedback and monitoring, antecedents, and social support which again emphasizes adding distractions to one's environment and providing feedback on one's behavior and support to deal with any obstacles. Overall, the most prevalent BCTs within the response are feedback and monitoring, antecedents, and social support. The presence of sequential patterns identified from SMS core within these user intents and responses demonstrates how such sequential pattern mining can guide the flow of conversation to make the CAs more adaptive and supportive of behavior change.

Discussion

In this study, we present an innovative framework for scoping and designing a preliminary communication architecture of a CA. We apply the framework to design a CA to support individuals struggling with nicotine cravings. Through secondary analysis of the QuitNet community's peer interactions, we developed our training data from real-life peer-to-peer conversations and peer mentoring/advice for craving management. We have sequenced the expression of various categories of BCTs and SAs embedded within the obstacles theme labeled peer conversations. We have extracted relevant nicotine cravings scenarios from our dataset to create appropriate training data for RASA's NLU and DM models to be able to understand users' intent (mapped to SAs) and generate appropriate responses (mapped to BCTs) to help them overcome nicotine cravings and thus, prevent relapse in such users. Through this study, we have gained insights into the frequently occurring categories of BCTs and SAs within peer interactions. Such sequential patterns can be integrated into the design of CA to be able to provide just-in-time adaptive support for individuals trying to resist obstacles during their quits. The need for such an approach has been highlighted in previous works as they allow tailoring the content of DBCIs and enable them to intervene at the right moments (more adaptive interventions).¹² Anastasiadou et al.⁴⁴ used the RASA framework to build a chatbot prototype that helps to improve individuals’ engagement in diabetes self-management. Another study used the RASA framework to develop a stress management application by identifying the stress levels and supporting its management.⁴⁵ Researchers have also utilized this framework for designing a chatbot that can help individuals assess the severity of symptoms from home and predict whether the individual has COVID-19.⁴⁶

Our dataset revealed that individuals mention nicotine cravings in different contexts; for example, an individual who just quit tobacco use might express different intents regarding tobacco use as compared to an individual who craves after maintaining the quit for a long time. It is necessary to understand the context of users’ needs to make digital tools such as CAs more responsive to the latent unspecified needs of users who struggle with obstacles such as nicotine cravings. In the focus area of this study, there has been some prior work on developing chatbots for tobacco cessation. Almusharraf et al.⁴⁷ developed a motivational interviewing-based chatbot via interactive interactions with the smokers themselves to help them quit smoking. Initially, the agent prompted the users to articulate the pros and cons of tobacco use followed by the agent responding in a non-judgmental manner, summarizing their thoughts, and asking them to reflect upon them. They tested the prototype of the agent on a sample of smokers and used the iterations of those interactions to train and improve the chatbot. This study reported that participants found it beneficial to engage with the chatbot and work towards their quit. Similarly, in our study, we used the messages from a real OHC, where peers engage in more natural and non-judgmental conversations with one another and provide them the required social support to motivate one another towards a common goal of not smoking. In our study, we used theoretical frameworks such as the taxonomy of BCTs and SA theory; since theory-grounded behavior change interventions have proven to be effective in facilitating behavior change among its users.⁴⁸ One study identified that the incorporation of BCTs within internet-based tobacco cessation programs can lead to improved chances of tobacco cessation.³¹ In addition to these, relatable real-world experiences are also important to motivate individuals to quit tobacco use as evidenced by “This is Quitting,” a text messaging program designed to help young adults quit smoking.⁴⁹

Some of the limitations of our work include the small sample size to extract nicotine cravings-related scenarios using the grounded theory approach. Such a sample size might have been insufficient to ensure saturation of concepts to derive all possible scenarios related to nicotine cravings. In our future work, we plan to incorporate additional data by scaling the labeling of the online communication themes to a large dataset and using those messages to ensure saturation of concepts for creating relevant nicotine cravings-related scenarios. Secondly, only two researchers were involved in the extraction of concepts and scenarios from the given dataset and thus these scenarios might be limited by subjective bias inherent in the coding process. In future work, we plan to involve multiple researchers in the scenario extraction process to compute inter-rater reliability to achieve a gold-standard corpus of nicotine cravings-related scenarios. We also plan to include interdisciplinary expert reviews to ensure the relevance of the patterns identified using sequential pattern mining algorithms. Another limitation in the current design of the CA is that it might not be able to handle the exception case or problematic scenarios very well; for which we need to generate additional data so that the conversations can be continued with the end-users more productively. In future work, we plan to test and evaluate the quality of responses generated by the CA using the confidence levels (ability to correctly classify the intent from the user's message) reported by the framework as well as a manual review upon feeding previously unseen nicotine cravings-related messages to the CA.

Conclusions

The technological advancements facilitate the use of AI-based CAs in the healthcare domain. This study describes our efforts to define a preliminary architecture of a CA whose goal is to interact with individuals as they report struggles with nicotine cravings. Our work builds on data insights and pattern mining of peer interactions among active and abstinent tobacco users which enables us to model user, behavior, and environment contexts. This deep and naturalistic understanding allows us to emulate guided interactions with a CA with the ultimate goal of reducing their chances of relapse. Specifically, the utilization of pattern mining algorithms to model peer messages grouped at the topic level is essential to simulate CA interactions, thus shifting the unit of analysis from single messages to threads of conversations. Our study provides insights into the implicit needs of individuals through consideration of the expression of content in social media interactions and how those needs can be addressed through appropriate responses grounded in behavior change techniques in a CA interaction. Such theory-informed and data-driven CA interactions may prove to be more effective for the management of nicotine cravings, relapse prevention, and long-term sustenance of positive health behaviors.

Footnotes

Acknowledgements

The authors would like to thank the members of the Center for Digital Health Analytics and UTHealth Houston-Cancer Prevention Research Institute of Texas for support and assistance throughout this study.

Contributorship

TS and SM researched literature and conceived the study. TS and MT were involved in data analysis. TS wrote the first draft of the manuscript. All authors reviewed and edited the manuscript and approved the final version of the manuscript.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical approval

This study was exempted from human subjects review by the Institutional Review Board at the University of Texas Health Science Center at Houston (HSC-SBMI-15-0697).

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Research reported in this publication was supported by the National Library of Medicine and National Cancer Institute of the National Institutes of Health under award numbers 1R01LM012974-01A1. TS is supported by UTHealth Houston-Cancer Prevention Research Institute of Texas Innovation for Cancer Prevention Research Pre-Doctoral Fellowship (Grant #RP210042). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or the Cancer Research Institute of Texas.

Guarantor

SM.

ORCID iD

Tavleen Singh

References

WHO report on the global tobacco epidemic, 2023: addressing new and emerging products: executive summary [Internet], https://www.who.int/publications-detail-redirect/9789240032842 (cited 4 March 2022).

Babb

. Quitting smoking among adults — United States, 2000–2015. MMWR Morb Mortal Wkly Rep. 2017; 65: 1457–1464. https://www.cdc.gov/mmwr/volumes/65/wr/mm6552a1.htm (cited 4 March 2022).

Dietz

Brownson

Douglas

, , et al. Improving physical activity and nutrition and reducing tobacco use and obesity to prevent chronic disease. In: discussion paper, vital directions for health and health care series. Washington, DC: National Academy of Medicine, 2016, pp.2–3.

Glanz

Rimer

Viswanath

. Health behavior and health education theory, research, and practice. 4th ed. San Francisco: Jossey-Bass, 2008, p. 590.

Zhou

Nonnemaker

Sherrill

, et al. Attempts to quit smoking and relapse: factors associated with success or failure from the ATTEMPT cohort study. Addict Behav 2009; 34: 365–373.

Heydari

Masjedi

Ahmady

, et al. A comparative study on tobacco cessation methods: a quantitative systematic review. Int J Prev Med 2014; 5: 673–678.

Caraballo

RS.

Quit methods used by US adult cigarette smokers, 2014–2016. Prev Chronic Dis. 2017; 14, https://www.cdc.gov/pcd/issues/2017/16_0600.htm (cited 4 March 2022).

Smith

Carter

Chapman

, et al. Why do smokers try to quit without medication or counselling? A qualitative study with ex-smokers. BMJ Open 2015; 5: e007301.

Skojac

CTM

Ackerman

. Evaluating patient barriers to tobacco cessation treatment. Fed Pract 2014; 31: 22–26.

10.

Laranjo

Arguel

Neves

, et al. The influence of social networking sites on health behavior change: a systematic review and meta-analysis. J Am Med Inform Assoc 2015; 22: 243–256.

11.

Michie

Yardley

West

, et al. Developing and evaluating digital interventions to promote behavior change in health and health care: recommendations resulting from an international workshop. J Med Internet Res 2017; 19: e7126.

12.

Hardeman

Houghton

Lane

, et al. A systematic review of just-in-time adaptive interventions (JITAIs) to promote physical activity. International Journal of Behavioral Nutrition and Physical Activity 2019; 16: 31.

13.

Nahum-Shani

Smith

Spring

, et al. Just-in-time adaptive interventions (JITAIs) in mobile health: key components and design principles for ongoing health behavior support. Ann Behav Med 2018; 52: 446–462.

14.

Hébert

Alexander

, et al. A mobile just-in-time adaptive intervention for smoking cessation: pilot randomized controlled trial. J Med Internet Res 2020; 22: e16907.

15.

Taylor

GMJ

Dalili

Semwal

, et al. Internet-based interventions for smoking cessation. Cochrane Database Syst Rev 2017; 2017(9). https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD007078.pub5/full

16.

Radziwill

Benton

. Evaluating quality of chatbots and intelligent conversational agents. arXiv:170404579 [cs] [Internet]. 2017, http://arxiv.org/abs/1704.04579 (cited 4 March 2022).

17.

Laranjo

Dunn

Tong

, et al. Conversational agents in healthcare: a systematic review. J Am Med Inform Assoc 2018; 25: 1248–1258.

18.

Wang

Zhang

, et al. Social media–based conversational agents for health management and interventions. Computer (Long Beach Calif) 2018; 51: 26–33.

19.

Perski

Crane

Beard

, et al. Does the addition of a supportive chatbot promote user engagement with a smoking cessation app? An experimental study. Digit Health 2019; 5: 2055207619880676.

20.

Calvaresi

Calbimonte

J-P

Debusson

, et al. Social network chatbots for smoking cessation: agent and multi-agent frameworks. In: IEEE/WIC/ACM international conference on web intelligence [internet]. New York, NY: Association for Computing Machinery, 2019 [cited 2022 Mar 4]. p. 286–292. (WI ‘19). https://doi.org/10.1145/3350546.3352532

21.

The World’s first AI powered stop smoking coach! [Internet]. Solutions 4 Health, https://www.solutions4health.co.uk/news/ai-stop-smoking-coach/ (2018, cited 4 March 2022).

22.

Szalacsi

AI and quitting addictions: how I quit the “Quit with Bella” app and stayed a smoker [Internet]. infoNation, https://medium.com/infonation-monthly/ai-and-quitting-addictions-how-i-quit-the-quit-with-bella-app-and-stayed-a-smoker-b6087d78b3a4 (2019, cited 4 March 2022).

23.

Milne-Ives

de Cock

Lim

, et al. The effectiveness of artificial intelligence conversational agents in health care: systematic review. J Med Internet Res 2020; 22: e20346.

24.

Christoforakos

Feicht

Hinkofer

, et al. Connect with me. exploring influencing factors in a human-technology relationship based on regular chatbot use. Front Digit Health 2021; 3. https://www.frontiersin.org/article/10.3389/fdgth.2021.689999

25.

Cobb

Graham

Bock

, et al. Initial evaluation of a real-world internet smoking cessation system. Nicotine Tob Res 2005; 7: 207–216.

26.

Cobb

Graham

Byron

, et al. Online social networks and smoking cessation: a scientific research agenda. J Med Internet Res 2011; 13: e119.

27.

Graham

Cobb

Raymond

, et al. Effectiveness of an internet-based worksite smoking cessation intervention at 12 months. J Occup Environ Med 2007; 49: 821–828.

28.

Graham

Cobb

Papandonatos

, et al. A randomized trial of Internet and telephone treatment for smoking cessation. Arch Intern Med 2011; 171: 46–53.

29.

Bialous

Sarna

Wells

, et al. Characteristics of nurses who used the Internet-based nurses QuitNet® for smoking cessation. Public Health Nurs 2009; 26: 329–338.

30.

Michie

Richardson

Johnston

, et al. The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: building an international consensus for the reporting of behavior change interventions. Ann Behav Med 2013; 46: 81–95.

31.

McCrabb

Baker

Attia

, et al. Internet-Based programs incorporating behavior change techniques are associated with increased smoking cessation in the general population: a systematic review and meta-analysis. Ann Behav Med 2019; 53: 180–195.

32.

Vosoughi

Roy

(2016, March 31). Tweet acts: a speech act classifier for Twitter. Tenth International AAAI Conference on Web and Social Media. Tenth International AAAI Conference on Web and Social Media. https://www.aaai.org/ocs/index.php/ICWSM/ICWSM16/paper/view/13171

33.

Abedin

Mamun

AIA

Nowrin

, et al. (2021). A deep learning approach to integrate human-level understanding in a chatbot (arXiv:2201.02735). arXiv. https://doi.org/10.48550/arXiv.2201.02735

34.

Hefny

Dafoulas

Ismail

(2020) Intent classification for a management conversational assistant. In: 2020 15th International Conference on Computer Engineering and Systems (ICCES), pp. 1–6. https://doi.org/10.1109/ICCES51560.2020.9334685

35.

Helmi Setyawan

Awangga

Efendi

(2018) Comparison of multinomial naive Bayes algorithm and logistic regression for intent classification in chatbot. In: 2018 International Conference on Applied Engineering (ICAE), pp. 1–5. https://doi.org/10.1109/INCAE.2018.8579372

36.

Searle

. Speech acts: a essay in the philosophy of language [Internet]. Cambridge: Cambridge University Press; 1969, https://www.cambridge.org/core/books/speech-acts/D2D7B03E472C8A390ED60B86E08640E7 (cited 4 March 2022).

37.

Aggarwal

Bhuiyan

Hasan

. Frequent pattern mining algorithms: a survey. In: Aggarwal

Han

(eds) Frequent pattern mining. Springer International Publishing, 2014, pp.19–64. https://doi.org/10.1007/978-3-319-07821-2_2

38.

Giuntini

de Moraes

KLP

Cazzolato

, et al. Tracing the emotional roadmap of depressive users on social Media through sequential pattern mining. IEEE Access 2021; 9: 97621–97635.

39.

Alibasa

Calvo

Yacef

. Sequential pattern mining suggests wellbeing supportive behaviors. IEEE Access 2019; 7: 130133–43.

40.

Strauss

Corbin

. Grounded theory in practice. Sage Publications, Inc, 1997, p.296.

41.

Hsieh

H-F

Shannon

. Three approaches to qualitative content analysis. Qual Health Res 2005; 15: 1277–1288.

42.

Loper

Bird

. NLTK: the Natural Language Toolkit. In:Proceedings of the ACL-02 workshop on effective tools and methodologies for teaching natural language processing and computational linguistics, Philadelphia, Pennsylvania, USA, pp.63–70. Association for Computational Linguistics, 2002.

43.

Singh

Roberts

Cohen

, et al. Discerning conversational context in online health communities for personalized digital behavior change solutions using Pragmatics to Reveal Intent in Social Media (PRISM) framework. J Biomed Inform 2023; 140: 104324.

44.

Anastasiadou

Alexiadis

Polychronidou

, et al. (2020) A prototype educational virtual assistant for diabetes management. In: 2020 IEEE 20th international conference on bioinformatics and bioengineering (BIBE), pp. 999–1004. https://doi.org/10.1109/BIBE50027.2020.00169

45.

Sharma

Parihar

Singh

(2021) Intelligent chatbot for prediction and management of stress. In: 2021 11th International Conference on Cloud Computing, Data Science & Engineering (Confluence), pp. 937–941. https://doi.org/10.1109/Confluence51648.2021.9377091

46.

Anushka

Thelijjagoda

(2022) Healbot: NLP-based health care assistant for global pandemics. In: 2022 International research conference on smart computing and systems engineering (SCSE), 5, pp. 61–67. https://doi.org/10.1109/SCSE56529.2022.9905107

47.

Almusharraf

Rose

Selby

. Engaging unmotivated smokers to move toward quitting: design of motivational interviewing–based chatbot through iterative interactions. J Med Internet Res 2020; 22: e20251.

48.

Davis

Campbell

Hildon

, et al. Theories of behaviour and behaviour change across the social and behavioural sciences: a scoping review. Health Psychol Rev 2015; 9: 323–344.

49.

This is quitting [Internet]. Truth Initiative, https://truthinitiative.org/thisisquitting (cited 9 March 2022).