Public perceptions of a healthy lifestyle change conversational agent in Singapore: A qualitative study

Abstract

Objective

Conversational agents (CAs) are increasingly used for the delivery of healthy lifestyle behaviour interventions. This qualitative study aimed to explore the barriers and facilitators to participants’ usage of a healthy lifestyle change CA and collect their views on areas for its improvement.

Methods

Twenty participants were recruited from a convenience sample of users interacting with a CA promoting healthy lifestyle changes to the general population in Singapore. This CA, Precilla, educated users on healthy living, specifically: diet, exercise, sleep and stress; for four weeks. The volunteers participated in semi-structured interviews where an interview guide was used, with questions on acceptability, satisfaction and critical appraisal of the CA. Interviews were transcribed and analysed in parallel by two researchers using thematic content analysis.

Results

Four main themes were identified: (1) enjoyable and acceptable experiences, (2) suboptimal experience(s), (3) alterations to Precilla for enhanced interaction and (4) suggestions for the future. Enjoyable experiences referenced the CA's friendly personality and important content that motivated a positive change to their lifestyle. Some participants were less satisfied and found the content to be too simple or sometimes, the messages too lengthy.

Conclusions

Participants suggested that in the future, CAs should provide regularly updated content on healthy living, specifically pre-diabetes. Multiple answer options should also be provided for more personalisation along with links to external resources to help improve users’ health literacy. Further recommendations include a necessity for a user-centered approach in CA development, employment of engagement strategies, use of a delivery platform most familiar to the target population and stratified message timings to suit the population and purpose of CA. Translating the health CAs to languages relevant to the target group could also enable wider reach and applicability.

Introduction

The ubiquity of Wi-Fi and high smartphone penetration rates globally allow for increased usage of digital health interventions in healthcare.^1,2 Mobile health (mHealth) has the potential to improve the delivery of healthcare services.^3,4 An example of increasingly popular digital health interventions mostly delivered via mobile phones is conversational agents. Conversational agents (CAs), sometimes referred to as chatbots, can be defined as computer programs, which can interact with human users in a way that mimics human–human interactions. ⁵ The mode of communication can vary from speech to text. Their use has spanned many sectors ranging from finance to fitness. ⁶ CAs have been employed to manage existing health conditions, but their use can also be extrapolated from passive treatment to more active prevention of chronic conditions which are causing a high disease burden.^3,4 Diabetes and pre-diabetes are such conditions of interest that both have high prevalence globally, especially in Singapore. ⁷ Singapore ranks second highest amongst developed countries for its diabetes incidence. ⁸ Given the rise in incidence today, even in younger age groups, an approach of proactive prevention could be highly beneficial. ⁹ Methods to achieve such prevention need to be scalable in order to reach the vast majority, especially in a country of high disease prevalence where the population at large is at risk like Singapore.^10,11

Lifestyle changes that promote healthier eating and increased physical activity have proven to be effective in reversing pre-diabetes and thus reducing diabetes prevalence. ¹² In addition to preventing such chronic conditions, healthy lifestyle changes have the added advantage of boosting the general health and well-being of the participating individuals. ¹³ Research has indicated that healthy lifestyle interventions have played a role in improving mental health, which is another growing problem in Singapore with one in seven having a mental health illness in their lifetime. ¹⁴

This approach was adopted in this study where an individual's risk of diabetes, baseline knowledge of diabetes, pre-diabetes and how to prevent the disease were all assessed. Along with this, the participants’ views and opinions on the healthy lifestyle change CA Precilla were also explored. This CA was designed to educate and inform individuals on diet, exercise, sleep and stress to promote healthier living and diabetes prevention. This was done to also address the issue around the dearth of evidence exploring the use of conversational agents for disease prevention or healthy lifestyle promotion in Singapore.

Aims

This qualitative study aimed to gain more in-depth insight into participants’ experience of interacting with a CA, which promotes healthy living to the general population in Singapore. This would provide valuable insights to hone subsequent healthy lifestyle change CA interventions for populations where the risk of developing diabetes is very high for the general population, such as in Singapore. We were keen to learn more about the informational content in the CA which was well received, the methods of delivery employed which were satisfactory as well as some insights on areas for improvement.

Methods

This qualitative study was conducted in November 2019 and involved semi-structured interviews with 20 participants who were recruited from a convenience sample of 60 individuals actively engaging with a healthy lifestyle change CA, Precilla. ¹⁵ The study was approved by the NTU (Nanyang Technological University) ethics committee (IRB-2018-11-032). All participants read a study information sheet before giving written consent. Specific consent was obtained for the recording of interviews. We report this study in line with the consolidated criteria for reporting qualitative research (COREQ) guidelines ¹⁶ (Multimedia appendix 1).

Healthy lifestyle behaviour change conversational agent and the feasibility study

Precilla was a rule-based, Facebook messenger conversational agent developed using an online software ‘Chatfuel’. The content delivered by Precilla was derived from evidence-based sources and peer-reviewed literature to inform the users on topics pertaining to healthy eating, physical activity, sleep improvement and stress management (Figure 1). In addition to providing advice on how to make healthy lifestyle changes, Precilla also aimed to improve the participants’ knowledge of diabetes and pre-diabetes.

Figure 1.

An example of an interaction between user and CA Precilla during the study.

Message delivery

Precilla initiated conversations with users four times a week for four weeks and sent text messages through Facebook Messenger, often supplemented with visual aids such as images or video links. Messages were sent at 12-noon each time to act as a neutral lunchtime point where users may be available to engage in a short exchange with Precilla. Her tone was informal and friendly, she used a typing function and had a profile picture to appear more human-like and improve the user experience. Being a rule-based conversational agent, Precilla provided users with pre-defined options and delivered a default error message when faced with unfamiliar inputs, to help redirect the user back to the main menu.

Content delivered

A comprehensive breakdown of the topics covered in each of the 16 sessions along with the sources for the information is presented in another paper on the web-based feasibility study. ¹⁷ Here, a brief summary of the content the participants received and provided their feedback on in this qualitative study. The introductory interaction with the user-outlined information on diabetes and pre-diabetes to orient them and establish a constant baseline for all participants.

Healthy diet content was centered around the different food groups and how individuals can plan their meals to incorporate the right quantities of each food type.^18,19 The main food groups discussed were fats, fibre and protein. Further suggestions were made specific to the ‘Hawker’ style of eating ubiquitous in Singapore. Advice on sleep improvements cited common issues associated with sleep disorders and insomnia as well as how to overcome these.^20,21 Specific recommendations were associated with setting a bedtime routine and establishing stimulus control.

Physical activity had two main facets to it. ²² Firstly, the distinction between aerobic and anaerobic exercise was presented and the importance of each emphasised. To make the advice actionable, a variety of suggestions on how to implement the exercise options were presented to suit the preferences of the user. Finally, stress management techniques were selected based on feedback from the literature on remotely delivered stress management techniques which have fared well.^23,24 The advice delivered considered deep breathing, mindfulness and a couple of other relaxation techniques. Content in all four domains was repeated after two weeks to reinforce the information and for better retention by the user.

Participants and recruitment

Participants were recruited using an electronic poster on healthy living focussed pages and groups on Facebook. Of the 60 participants who responded, (and agreed to participate in our corresponding feasibility study) 20 volunteered to participate in the interviews conducted for this qualitative study. Ten interviews were conducted in person and 10 were over the telephone (Figure 2).

Figure 2.

Flow of participants from initial CA feasibility study to participate in telephone and in-person interviews in this qualitative study.

They all had to complete an eligibility questionnaire. We included adults aged 21 and above, fluent in English with a Facebook account. We excluded individuals with a history of major illness, those with physical disabilities or pregnant women as advice provided by the conversational agent was not optimised to consider diet and exercise requirements for these populations. Additionally, as the content in questionnaires, informed consent, interviews and conversational agent were all in English, illiterate and non-writing individuals were excluded.

A purposeful random method of sampling was employed to increase the credibility of the results. ²⁵ The participants were recruited from healthy living–focused Facebook groups and pages where individuals already had an interest in a healthy lifestyle and were technologically savvy enough to navigate Facebook and Messenger. A diverse range of age and ethnic groups was an added consideration to replicate the multiracial society in Singapore. The interest in healthy living along with an ability to navigate social media were important considerations as participants needed to be inclined towards healthy living and be willing and capable to interact with a CA over a four-week period.

Participation in the interviews was entirely voluntary and participants who completed the feasibility study were all compensated with a digital voucher of $25–35 (regardless of involvement in the interviews). $35 vouchers were offered to those who completed the follow-up questionnaire within seven days, whilst the rest were eligible for the $25 voucher.

Data collection

Interviews were conducted by a female PhD student (DAD) in designated private meeting rooms at Lee Kong Chian School of Medicine, NTU, Singapore. An interview guide was employed, and interviews were either in person or over the telephone depending on the volunteer's preference (Multimedia appendix 2). Participants were acquainted with the interviewer at the time of the interview and were aware of their credentials. No relationship was established prior to or after the interviews. DAD was provided with sufficient details, resources and exposure to online courses on qualitative research and conducting telephone interviews prior to study commencement. Field notes were made during the interview. The initial interview guide was informed by the literature on the development of digital and conversational agent-delivered health interventions.^26–29 The interview was adapted further as the study proceeded, to take account of emerging themes and each interview came to an end when we reached saturation of novel topics for each emerging theme.

Basic demographic data were obtained from all participants including gender, age, ethnicity, marital status, monthly household income, number and age of children and occupation. Questions focused on their existing sources of health information, advice and health tracking and whether this was sufficient. Participants were also asked about their experience interacting with the healthy living conversational agent ‘Precilla’. We asked them to comment on their impressions of Precilla (personality, ease of use, comprehensiveness etc.), their thoughts on the methods of message delivery used by the CA (Facebook messenger, timing and frequency of messages), the effects these interactions with Precilla had on their lifestyle (if any) as well as their opinions on how this CA could be improved in the future.

Both short- and long-answer questions were asked. An example of a short answer question is ‘at some points in the conversation you were provided with links to external webpages, did you find this useful or not so useful?’ An example of a question that required a more comprehensive answer is ‘The messages were designed to be conversational, what is your view on this? Would you have preferred it to be one-way communication, or did you like that you had some part to play?’

All in-person interviews were recorded using a portable audio recorder for transcription. Telephone interviews were conducted via speaker phone using the departmental iPhone and also recorded using the audio recorder placed in close proximity to the speaker phone and the interviewer. The average duration of the interviews was 20 min. The audio files were transcribed verbatim. All transcripts were checked with the corresponding sound files for accuracy, and they were subsequently corrected for errors and cleaned.

Data analysis

The data were analysed by two researchers in parallel, via thematic content analysis. ³⁰ Firstly, the researchers familiarised themselves with the transcripts by reading them multiple times. Secondly, initial codes were proposed based on pilot coding of the first five interviews. Thirdly, themes were derived from the codes. Fourthly, the two authors discussed and combined their themes for comparison. Finally, they reached a consensus on the themes to be used and how to define them. This codebook was then used to code the remaining 15 interviews. The finalised codebook is presented in Multimedia Appendix 3. Coding of transcripts was done using the ‘review’ and ‘comments’ functions on a word processor.

Results

Participant demographics

A total of 60% of participants were female; the mean age was 33 (median 31–35; range 22–55; SD 9.23) and 45% were married. 65% were working, 30% were full-time students (undergraduate or postgraduate) and one participant was unemployed. More than half of the participants (75%) were Chinese, 15% were Indian, 5% were Malay and 5% were Caucasian. The monthly household income was S$10,000–12,000 for 30% of participants, less than S$4000 for 25%, S$4000–6000 for 10%, S$6000–8000 for 10%, S$8000–10,000 for 10%, S$2000–3999 for 5% and more than S$14,000 for 5%. When the interviews were conducted (2019), the median monthly household income in Singapore was S$9425. ³¹

Thematic analysis

Four main themes were identified from thematic analysis of the interview transcripts discussing participant interactions with the health conversational agent ‘Precilla’: (1) enjoyable and acceptable experiences, (2) suboptimal experience(s), (3) alterations to Precilla for enhanced interactions and (4) suggestions for the future (Figure 3).

Figure 3.

The main four themes and subthemes identified from interview transcripts.

We present each theme below, along with subthemes and excerpts that highlight different facets of the theme.

Theme 1: Enjoyable and acceptable experiences with Precilla

Precilla's positive traits

More than half of the participants had a generally positive impression of Precilla and the experience of interacting with her. Some participants attributed this to her positive traits, such as the content being comprehensive and yet easy to understand. They also appreciated the fact that the information was always readily available for them to access at any point via the Facebook Messenger app on their smartphones.

‘The content was adequate. It wasn’t too much, wasn’t too dense, it was in a very lay manner, so I think that bit was good.’ – (P01)

‘But what I liked about it is there is the information that she feeds, I may or may not actually read it, but I know that the information is there. If I want to, I can always refer back’ – (P02)

Participants perceived Precilla and the information shared was reliable, accurate and informed by a team of healthcare professionals or academics. They thought Precilla had a likable personality and used an appropriate tone when conversing. Furthermore, they appreciated the visual aids and found the user interface and images employed to be aesthetically pleasing. ‘In terms of diabetes, I think it's a mixture of a team of people. You can have the nutritionists on the food and maybe the sport scientists about the activities, and maybe the doctors like, we need to assess how long we do this. I think it's a very multidisciplinary team, doesn't have to be a physician. I say any professionals can tackle this from different disciplines. It's like public health precisely.’ – (P03)

‘I liked that she was approachable. I liked the fact that the conversation was friendly.’ – (P04)

‘It did not become too much images, which could be a bit jarring because it's only images, and neither was it too text or too wordy, so it presented a good blend.’ – (P06)

Participants shared their exposure to health information from their field of work, advice from the Ministry of Health and health tracking apps. Some preferred using Precilla over these existing sources of health information because of its ease of use and easily digestible content.

‘Yes, I definitely preferred using Priscilla…. It was very informative, and the information is very basic and easy enough to follow without having to go through too many processes to achieve a desired result.’ – (P06)

Positive lifestyle changes and helpful content

Some participants expressed their appreciation towards Precilla based on the positive effects the interactions with Precilla had on their lifestyle, especially in terms of sleep improvement and stress management.

‘Yes, definitely. Me, personally, from all this information, especially for sleep in particular, just like the example, there were some tips and tricks that Priscilla shared. I'm actively trying to use them in my day to day because sleep, personally, is one of my issues.’ – (P06)

‘It did impact on me, especially the stress management. The stress management and sleep management part because I wasn't aware of how it could better managed.’ – (P07)

The content shared was generally well received for the four areas we targeted: healthy eating, physical activity, sleep and stress management. Many participants noted that the content was detailed and informative. Some found this content to be relevant and helpful in learning new content whilst others felt the benefit arose from prior knowledge being reinforced.

‘I did learn some new interesting facts and information that can help with my decisions on how to improve my healthy living style.’ – (P08)

‘I liked the physical activity one. The sleep one was quite comprehensive also. Generally it was good, the information was good.’ – (P04)

Recommendation to friends and family

Finally, some participants expressed their approval for Precilla by indicating that they would recommend Precilla to their friends and family. Their recommendations would be directed to those with sedentary lifestyles and those with a higher risk of diabetes.

‘Yes, I will. For the very sedentary people. It's a good start.’ – (P07)

‘I think it will serve as a good reminder for them because my parents are tech savvy. For Priscilla, maybe those who are in their late 20s to maybe their early 40s, those who are at a higher risk of getting prediabetes and diabetes serves as a good reminder for them especially.’ – (P03)

‘If there is different languages, yes, maybe.’ – (P11)

Theme 2: Suboptimal experience(s) with Precilla

Minimal impact on lifestyle

Other participants reported minimal changes to their lifestyle over the four-week study period. This was sometimes attributed to them having no intentions of changing their lifestyle, hence, having Precilla's input had minimal effects on healthier living. Others purported a lack of longer-term guidance resulting in an inability to make meaningful lifestyle changes in four weeks and maintain those changes over time.

‘It presented some new information, but I wasn’t planning to make any changes to my lifestyle to begin with.’ – (P01)

‘Not really. I don’t think it's sustainable because it's just a tip for the four weeks.’ – (P12)

‘I did try it out, I did try not drinking so much caffeine and going out of the room and then coming back. I didn’t try in a consistent enough manner to actually be able to tell whether it works.’ – (P13)

Unhelpful content

A fair group of individuals were already well versed in healthy living and diabetes prevention or had come across similar advice in the past and found the information shared to be too simple and so, did not learn anything new. Furthermore, the recap sessions which were included to reinforce knowledge for better retention were often perceived as repetitive. Others found some of the content difficult to put into practice and indicated the need for physical instruction for some of the more abstract concepts like mindfulness.

‘Too simple. If differs by people. Because we are working on diabetes so we know more or less.’ – (P05)

‘Yes, somewhat, not a lot because on the dieting and physical activity side of it it's something that I'm already doing’ – (P08)

‘The least useful was probably the exercise part because that information is pretty much found everywhere else, so I think wasn’t useful at all.’ – (P12)

‘I feel like sometimes I didn't really like the part where they recap because I feel like it's redundant, you're doing it again’. – (P03)

‘The least helpful for me is what I mentioned, it's just informative but it doesn't really push you to put the information into real work.’ – (P14)

‘I think mindfulness is possibly something that requires more, I guess, someone to really teach you properly. I think it's a difficult concept to grasp’.

The information shared by Precilla was fairly general and this was not applicable to individuals going through specific life circumstances.

‘Least helpful, I think is about sleep because I have a six-month-old so I can't sleep even if I want to. I think that wasn’t useful to me.’ – (P09)

Some mentioned that with some minor changes such as the translation of Precilla into different languages or with more personalisation, they would be more inclined to recommend Precilla to others.

‘Not at the current stage. But when she's a bit more human I suppose and perhaps a tad more personalised, that will be great.’ – (P10)

‘My family, no, because they don’t understand English.’ – (P05)

Unfavourable messaging style

Suboptimal interactions with Precilla were sometimes attributed to the user experience with some participants finding the typing delay between messages time-consuming, dragging out the whole conversation. This seemed to contradict the concept of mHealth, which is thought to be efficient and convenient.

‘It comes up one line, then wait three seconds, then comes up another line, then. It's a bit annoying. Yes, or you can send multiple paragraphs, but in one chunk. I think the whole point of this space, e-learning thingy, is such that it would be convenient, like, the participants can do it at a time of their convenience. But then it takes quite a bit longer than expected.’ – (P01)

Others felt like there were insufficient answer options to select from and the answers provided did not always indicate the response they wished to give.

‘The button design is one thing, in that there were oftentimes where the function of the button is just to review more information, instead of giving users a choice. The user intents could have been better thought out, maybe for future iterations.’ – (P15)

Some participants preferred using existing platforms over Precilla because of the familiarity and their ability to find what they’re for looking easily. Others had no preference for one platform over another and perceived Precilla as equal to other conversational agents they had used.

‘In terms of finding things online, it's something that we are habituated to doing already, so you can look for specific queries pretty quickly. With the chatbot, also because I wasn't too familiar with the flow, but it wasn't that easy to find that information because it felt more like I had to go through a programme.’ – (P15)

‘It was fine. It's no different from what I have seen in some standalone chatbots like Wysa or Woebot, those mental health chatbots.’ – (P04)

Theme 3: Alterations to Precilla for enhanced interactions

Timing

Timing of the messages was an important part of Precilla's message delivery and largely determined whether or not the messages would be read by participants. The consistent timing of 12 noon was selected as a neutral lunchtime with the notion that individuals would be available to interact with Precilla at this time. However, this timing was not necessarily well received as some individuals had varied lunch timings or were with colleagues and so did not think it was a convenient time to be on their phones. Instead, participants suggested timings either before or after work, when they would be alone and at home where they wouldn’t have to worry about Wi-Fi availability or data usage. Some even thought the idea of stratified timings could be interesting where messages for specific purposes were sent at specific times.

‘No, not for me because I don’t have a set lunch break, so sometimes I didn't get a chance to look at them until nine pm that night.’ – (P17)

‘Maybe morning would be a good time also when you're starting work and you're just about to get started, so you are trying to clear your messages on your phone or something and you can have a quick glance.’ – (P09)

‘And because I don’t want to waste my data, so that's one key issue. If you do it at the timing when you’re at home, maybe it will be better, especially for those with external links.’ – (P03)

‘Yes, if it's about sleep time, maybe eight or nine pm, and then we can just follow the suggestions.’ – (P05)

Bi-directional communication

Although most participants showed a preference for more bi-directional communication when it comes to conversational agent interactions, a few were accepting of a more one-way-driven platform with minimal, if any, input from the user.

‘I'm okay to just skim through rather than actively participate, actually. I barely reply to messages anyway even from friends and stuff. I'm okay with just a one-way conversation.’ ‘… I feel that if I were to ask question, I can just google.’ – (P09)

The remaining participants shared that in the future there would be a need to open up the answer choices and give the users a variety of options to select from. Some even proposed free text as a medium for users to ask questions or clarify certain doubts.

‘I would have liked it if she was a bit more interactive. Because when I type stuff in, she just brings me to the main menu. That's a bit annoying. Because if that is the case, then I don’t see how that's any different from me reading off a brochure or even an email.’ – (P10)

‘Some things didn't really require a reply, because it was just like us taking in information and such, but it didn’t have any outlet for if we had any questions or if we wanted to find more information for example.’ – (P17)

‘Especially when a particular topic is of interest to me is to give me the option to find out more on that particular topic, rather than just a pre-set of the information flow and then we move on to the next topic and so on.’ – (P08)

Some participants didn’t see Precilla as just a health information agent but genuinely wanted to converse in a more human-like manner.

‘Sometimes we want to interact with it, so we don’t want to follow exactly what it gives us.’ – (P05)

Despite these preferences, some individuals were technologically savvy enough to detect that Precilla was indeed a rule-based conversational agent which helped to manage their expectations. However, they did mention a need for future iterations to be smarter in order to keep users engaged and interested.

‘No. I showed the chatbot to my friends and we all had the same feeling because we must follow whatever is given to us. We cannot type anything. For example, when it recommends exercise, I just ask, do you exercise, and then the chatbot said, it seems like you typed something, I don’t know whether I can understand or not, so I will direct you to the main menu. Then I will lose interest I understand it's rule based, chatbot, so it's okay.’ – (P05)

Delivery platform

Participants commented on the suitability of Facebook Messenger as a delivery platform for Precilla. Some shared their acceptability of Facebook Messenger but also mentioned that when the notifications are switched off, they are unaware when messages come in. Five others acknowledged that other messaging platforms are more actively used (especially by younger populations). For example, Instagram and WhatsApp were suggested along with the possibility of Precilla being a stand-alone app (ie. not embedded in any pre-existing platform).

‘Probably Telegram or WhatsApp would have been a better option because, for instance, me, personally, if I did not actually log in or switch on my notifications for Facebook Messenger, I would miss out that that Priscilla has an update for me on that particular day.’ – (P06)

‘Well, for me, I think Facebook is fine, because I’m primarily on Facebook, but the younger people seem to be more Instagramming.’ – (P01)

Theme 4: Suggestions for the future

Blending conversational agents with other platforms

Some participants indicated that the information shared by Precilla was quite dense and may be more suited to an online course. In this way, Precilla could be administered in combination with an online page or course. Resultantly, participants could engage with the conversational agent and receive reminders whilst also learning the necessary content of interest from the course. Others shared a preference for a combination of Precilla with in-person consultations with healthcare professionals for more interactivity.

‘If that was the case, wouldn't it be easier to just have it as a wiki page, or in a more, quote, unquote, traditional online course format rather than someone I have to converse with.’ – (P15)

‘Because I had done another research probably a year ago, one that actually has got a dietician. A one-off, they have a bot also, but separately they also have a dietician so the chatting was more interactive.’ – (P18)

Different message types

Participants made an indication for the preference of different modes of messaging such as voice or video messages for the future for some novelty besides just text messages. They also pointed out a need for a better default or error message should the user type in something which the conversational agent could not recognise. This should be something that clarifies the limits of the conversational agent and points the user towards help.

‘Maybe a voice. We always find that listening is easier than just reading the text.’ – (P05)

‘It's like when you try to find things on how to do certain exercises. When they have either drawings or when it's just descriptive, it's usually a little bit hard. I think what can be improved possibly is videos.’ – (P10)

‘I’m not saying that she should be able to answer that, but it would be nice to at least have an answer that's at least Siri-like that says, I do not have the authority to answer that, or something like that.’ – (P10)

‘On a more basic level, to go back on the main menu, like, because when I tried typing like main menu and stuff, there isn’t a command, or at least to my knowledge, in using the chatbot. So the only way to trigger Precilla to take you back to the main menu was to type something wrong or type anything, so that was slightly annoying because you didn't know.’ – (P15)

Personalisation

The content provided by Precilla was fairly general and could be more targeted towards specific age groups. For example, the advice to younger populations may be more towards cultivating healthier habits whilst the advice for older populations may be more seriously targeted towards specific diabetes prevention techniques. Some individuals were already well versed in healthy eating and physical activity–related advice and found the information provided to be too simple and requested for options to delve into more detail on topics of particular interest.

‘Missing? I’m not sure whether there was age related content for different age groups.’ ‘And also different age groups have different requirements.’ – (P09)

‘I think the app can be more specialised because it's quite a wide topic on general health. So let's say you want to delve into a topic, you can actually choose it and it will be more better for people who want to know more about certain topics, not just everything in general.’ – (P12)

Participants proposed having a search function to recall specific information in their conversation flow and follow-up notifications for unread messages so that they do not completely miss out on interactions if they are unavailable to attend to Precilla immediately.

‘But I’m not sure if there is actually a search function where you can look through the conversation that you had. Because she sends quite a lot of messages, sometimes you want to look back into certain things that she mentioned before. I think it's much of a hassle to really scroll up through the conversation, because it can be quite lengthy. Maybe, I’m not sure, if it's possible to implement a search function in the Facebook chat.’ – (P02)

Finally, participants mentioned that their likelihood of recommending Precilla to their friends and family would be elevated if the content was translated into the other languages relevant to Singapore (i.e., Tamil, Malay, Chinese).

‘But the only thing I suppose that would be of concern is if let's say you are looking to broaden your target audience, then you might have to send it in other languages like Chinese and Malay. Because there are definitely, especially the older generations, they are not going to be able to read English.’ – (P10)

Suggestions for website links

During the course of conversations with Precilla, participants were provided with links to some web pages where they could access further information on a topic area or links for relaxing music to play alongside their mindfulness practice. The usage of these links was not found to be very popular as evidenced by some participants not clicking the links or some unable to recall such an exchange. However, participants did point out that the use of website links was beneficial but provided suggestions on the types of links they’d be most interested in. These included links to helplines, educational talks, courses to attend and links to more in-depth information.

‘For me it's more like links to helplines (…) or maybe there are talks or courses that are around in some medical institution, in the hospitals, poly clinics, anything like that. So, then I know if I want to learn more about stress management somebody is giving a talk somewhere.’ – (P07)

Novel, additional content

The feasibility study lasted for four weeks, and some participants mentioned they’d be open to a longer duration of interactions with Precilla, provided that novel content was introduced. Some suggestions on the type of content they welcomed included more educational information on pre-diabetes (since diabetes is more commonly talked about), signs and symptoms of pre-diabetes and diabetes, context or race-specific food recommendations in Singapore and more clarity on the difference between healthy foods and ‘healthier options’.

‘They promote like, this milk tea is a healthier choice, but it's not healthy. This difference between healthier choice and whether something is healthier or not, it should be the emphasis for the group who are really educated’. – (P03)

‘Yes, especially since I think Singapore, the bigger problem for diabetes in particular is, e.g., for Indians, so, yes, it would be good to… It can bring in race specific food.’ – (P13)

‘I don’t recall, was there advice to look up what the symptoms are if you’re prediabetic?’ ‘The signs or symptoms to look up for.’ – (P18)

Discussion

Key findings

This qualitative study explored the acceptance, satisfaction and critical appraisal of a health CA promoting healthy living to the general population in Singapore. Twenty participants were interviewed about their perceptions of ‘Precilla’, and they shared their experience of interacting with a Facebook Messenger–delivered healthy living and diabetes prevention conversational agent and provided recommendations for the future. Enjoyable experiences were denoted by Precilla's friendly personality, informal tone and helpful content which encouraged some positive changes to their lifestyle. Furthermore, her language and content were easy to understand, it was conveniently accessible from the messenger app on their smartphones and the information was trustworthy, in their opinion, as it appeared to come from reliable healthcare sources. Others were less satisfied with Precilla and found her content to be too simple and delivery suboptimal due to the lengthy messages and insufficient answer options to choose from. When compared with other existing sources of information, Precilla was sometimes not as ‘intelligent’ as users anticipated her to be. However, in other cases, she was thought to be more interactive and engaging than existing apps or sources participants had access to. Timing of messages, sufficient bi-directional interaction and selecting an appropriate delivery platform were important considerations for Precilla's success. Finally, for the future, participants suggested maintaining her kind and friendly personality along with a preference for other messaging formats (video, image, speech, etc.), more personalised content and delivery, links to helplines or more in-depth information on signs and symptoms for diabetes and pre-diabetes and specific healthy food recommendations relevant to and accessible in Singapore. These findings can be extrapolated and applied to the development of further health CAs in Singapore and countries/communities resembling Singapore.

Comparisons with existing literature

A study in Singapore looking at the health education and communication of individuals with pre-diabetes indicated that there was a preference for messages on understanding risks associated with diabetes and how to prevent diabetes, health and family and to avoid using the term ‘prediabetes’ in messages. ³² In our interviews, participants also indicated an interest in understanding the signs and symptoms of pre-diabetes and diabetes and were interested to learn more about pre-diabetes (as diabetes is more commonly talked about) and did not indicate a dislike for the term ‘prediabetes’. Perhaps, unlike individuals in Lim et al. our participants were not pre-diabetic and may have had less of an emotional aversion to the term which might reinstate one's illness. ³² The study also concluded that healthcare professionals at polyclinics could provide preliminary advice before directing patients to community-based programmes. ³² A similar proposition was suggested by some of our participants who were interested in receiving updates and details on where to attend community-based programmes or educational talks on diabetes prevention and healthy living.

Zhang et al. present a timely review on artificial intelligence conversational agents for healthy behaviour change, focussing chiefly on healthy diet and physical activity. ³³ This reiterates the point highlighted by participants in our study that plenty of existing interventions and sources of information focus on diet and physical activity. Hence, the inclusion of additional elements, namely, sleep advice and stress management was well-received for their novelty and distinction from existing health behaviour change programmes.

Some participants in our study faced some challenges in navigating the app in terms of returning to the main menu, asking for help, or finding the default error message insufficient. This could potentially be addressed by providing a link to a user guide as implemented in another study using a smartphone app to promote self-management of diabetes. ³⁴ A user guide may help resolve some of the concerns raised by our participants by outlining the aims and limitations of the conversational agent, additional relevant resources, or a ‘contact us’ option to reach relevant personnel for help.

There are some similarities and differences between the findings in this study and what has been presented in the literature with some overlap and some less discussed concepts. For example, in a study discussing the development of a positive body image CA, focus group discussions were conducted to assess the types of content to be included in the CA. ³⁵ The researchers identified ‘incorporating the feedback of lived experiences’ as a valuable source of information in nurturing a positive body image and averting eating disorders. ³⁵ Although some participants did mention how their personal experiences with diabetes affected their understanding of the condition in an earlier needs assessment study, ³⁶ this was not raised in the current qualitative study and may be valuable when improving CA, Precilla. Another suggestion made by Park et al. in the design of a CA for stress management was to have contextualised feedback and promote self-reflection to encourage healthier behaviours related to stress management. ³⁷

In contrast, some items mentioned by participants in this qualitative study are less prevalent in the literature. For example, the suggestion for specific website links related to diabetes and pre-diabetes advice and upcoming events to grow awareness was a more unique suggestion in this study. Similarly, the discussion around delivery platforms was also specific to this study where the preference for messaging apps such as Telegram and WhatsApp instead of Facebook messenger was raised by participants. Other studies also reported the positive reception of a CA with a consistently friendly personality. ³⁸ They also pointed out the benefits of fostering a CA with traits and intentions similar to Precilla's such as being non-judgemental, emotionally sensitive and motivating. ³⁸

Main recommendations

The main recommendations on how to further improve a conversational agent such as Precilla for healthy living are outlined in Figure 4. These involve selecting appropriate message timings, involving participants in answer option generation and selecting an acceptable delivery platform based on target population usage and finally, engagement strategies such as gamification to keep users entertained and interested in future conversations.

Figure 4.

Main recommendations on how to enhance a healthy living conversational agent as suggested by study participants.

Timing is an important determinant of conversational usage by participants. It would be preferable to set a timing that is in the evenings when individuals are likely to be at home (where Wi-Fi is available and there can be no concerns regarding data usage) and free to interact with the conversational agent. We could attempt to integrate a reminder function or follow-up notification to prompt them if the message has yet to be opened. Message timings could be streamlined further to be personalised based on each individual's needs and preferences. For example, some individuals may wish to receive message prompts before dinner every day, to remind them to have a healthy meal. Others may prefer an interaction at the end of the workday which encourages them to use their free time to get some exercise.

Participants want to be more involved in the conversation. Although free text is a challenge for rule-based agents, providing more options for them to choose from would be helpful. Participants indicated a need to better map out more user intents to make the options more applicable. This is where co-design could be really helpful. Different answer options could be posed to prospective users in a ‘thinking-aloud’ testing setting where they can share their thoughts and opinions in real time of their acceptability of the responses generated by the research group.^39,40 Additionally, translating the conversational content into languages of interest for the target population can be effective. This would enable a greater reach to wider age and ethnic groups who may benefit from the conversational agent but are currently restricted by a language barrier.

Conversational agents could be delivered as part of a hybrid healthcare delivery programme. Having an in-person and mobile health care programme may be an effective means of transitioning patients away from just traditional care and more accepting of hybrid care. Having healthcare professionals linked to one's conversational agent can make them more accessible to patients, can help keep track of behaviours and potentially reduce hospital visits or consultation times.

A conversational agent, not embedded in Facebook Messenger, could be better received, especially by individuals who are not avid users of Messenger. Precilla is a rule-based conversational agent, designed and developed by non-experts to reiterate the accessibility of such digital health interventions for researchers. For such non-developers, plenty of chatbot creation software exists such as Instabot or Collect.chat. ⁴¹ These can be employed to create a conversational agent that can be embedded in a website, for example. Alternatively, a conversational agent could be delivered through a stand-alone app. This could involve hiring a programmer, collaborating with an app development company or using an app builder such as ‘Appy Pie’ or ‘App Sheet’. Most of these builders charge a monthly subscription fee. ⁴²

Participants indicated an interest to be ‘entertained’ in their experience of engaging with a conversational agent. The idea that they wanted to converse with Precilla for fun, referred to her by name, and previously mentioned an interest in rewards/points which is something they have enjoyed on other health apps. Similarly, allowing users to sign up with a username and password, profile image and ability to collect points may be appreciated. A similar approach was adopted by Baptista et al. who incorporated simple gamification elements, including goal setting, progress monitoring, feedback and quizzes within their embodied conversational agent for diabetes management. ³⁴ User authentication via username and password could also promote security and eliminate the need to disclose their phone numbers – which some participants noted as a privacy issue.

Strengths and limitations

The recruited sample included individuals from all the ethnic groups prevalent in Singapore and covered a wide age range from 22 to 55. As the participants were recruited online, their digital literacy was strong enough to use the CA in its entirety over four weeks, hassle-free. This meant the views they shared during the interviews were representative of their interactions with the CA and were not limited by technical difficulties. Although 20 participants was an adequate number for this qualitative analysis, we would aim for larger sample size in future studies to increase the generalisability of the findings.

It is possible that the sample may have been biased as the individuals recruited into the online feasibility trial were from existing healthy living and lifestyle change Facebook groups and so already had a keen interest in healthy living and may have had higher levels of health and technological literacy than the general population in Singapore. The interviews were conducted exclusively in English, and we may have missed out on the opinions of individuals from different generations who are possibly not as fluent in English but may greatly benefit from mobile health usage. Future work should consider translating conversational agent content and conduct interviews in the languages relevant to Singapore (Malay, Tamil and Chinese).

Conclusions

Based on the feedback from our participants, future conversational agents should provide users with regularly updated content on healthy living, multiple, relevant answer options to choose from and links to external resources to improve their health literacy. Further recommendations include (1) a need for a user-centered approach in CA development, (2) employment of engagement strategies such as reminders or gamification, (3) implementing the health CA via a delivery platform most familiar to the target population and (4) use of stratified message timings to suit the population and purpose of CA. Future work should also look into translating health CAs to other relevant languages for the target population, to reach older generations and should employ more messaging options (such as speech or videos) to make the interactions more engaging.

Supplemental Material

sj-docx-1-dhj-10.1177_20552076221131190 - Supplemental material for Public perceptions of a healthy lifestyle change conversational agent in Singapore: A qualitative study

Supplemental material, sj-docx-1-dhj-10.1177_20552076221131190 for Public perceptions of a healthy lifestyle change conversational agent in Singapore: A qualitative study by Dhakshenya Ardhithy Dhinagaran and Lorainne Tudor Car in Digital Health

Supplemental Material

sj-docx-2-dhj-10.1177_20552076221131190 - Supplemental material for Public perceptions of a healthy lifestyle change conversational agent in Singapore: A qualitative study

Supplemental material, sj-docx-2-dhj-10.1177_20552076221131190 for Public perceptions of a healthy lifestyle change conversational agent in Singapore: A qualitative study by Dhakshenya Ardhithy Dhinagaran and Lorainne Tudor Car in Digital Health

Supplemental Material

sj-docx-3-dhj-10.1177_20552076221131190 - Supplemental material for Public perceptions of a healthy lifestyle change conversational agent in Singapore: A qualitative study

Supplemental material, sj-docx-3-dhj-10.1177_20552076221131190 for Public perceptions of a healthy lifestyle change conversational agent in Singapore: A qualitative study by Dhakshenya Ardhithy Dhinagaran and Lorainne Tudor Car in Digital Health