Sage Journals: Discover world-class research

Abstract

Embodied Conversational Agents (ECAs) can increase user engagement and involvement and can strengthen the effect of an intervention on health outcomes that is provided via an ECA. However, evidence regarding the effectiveness of ECAs on health outcomes is still limited. In this article, we report on a study that has the goal to identify the effect of a match between a health topic and the ECAs’ appearance on ratings of personality characteristics, persuasiveness and intention to use. We report on an online experiment with three different ECAs and three different health topics, conducted among 732 older adults. We triangulated the quantitative results with qualitative insights from a focus group. The results reveal that older adults prefer an ECA that has an appearance matching a certain health topic, resulting in higher ratings on persuasiveness and intention to use. Personality characteristics should be measured embedded within a health topic, but are not rated higher because of a match. We furthermore provide guidelines for designing the content of the ECA.

Keywords

embodied conversational agent design guidelines health behavior change persuasiveness

Introduction

eHealth is in a transition phase, whereby text- or video-based communication is gradually being replaced by more engaging means of interaction. Examples of such means are online videos and games, and also Embodied Conversational Agents (ECAs). ECAs have been defined as “more or less autonomous and intelligent software entities with an embodiment used to communicate with the user”.¹ Most often, they take the form of virtual persons or cartoon characters on websites, accompanied by a scripted chat functionality. Compared to traditional text, video or audio communication, ECAs provide a form of simulated human to human interaction, whereby natural dialogues are aimed to be mimicked (including interactive scripts and body language).

Previously, ECAs have been implemented within the health context for a variety of contexts, such as health education,² mental disorders,³ or as a decision aid for shared decision making in healthcare.⁴ Using an ECA to convey information, to offer an intervention, or to support decision making has been found to sort different effects. A review by Provoost and colleagues⁵ found that ECAs can increase user engagement and involvement, and can strengthen the effect of an offline intervention that is provided alongside the ECA. At the same time, they conclude that the evidence regarding the effectiveness of ECAs is still limited, as it is an emerging field and most studies have focused on usability and user acceptance. A similar conclusion was drawn in a review by Kramer et al.,⁶ who found that, on the one hand, using ECAs leads to higher usability and use of a digital intervention, but that the effect of using ECAs on health outcomes is still unclear. Often, motivation to change increased, but health behavior and health literacy did not. A prerequisite for use and effect of an ECA within any context, including health, is a developed and engaging design. Multiple studies have concluded that the design of an ECA should instill emotional bonding with the end-user.^7–9 Most of the studies, aimed at generating design guidelines for ECAs in the health context, have focused on the design of speech or textual output, ECA gaze and facial expressions, and body gestures, while the design of the ECA’s appearance seems to be somewhat neglected.^6,10

The influence of appearance

As supported by research with humans, an ECA’s appearance is essential in influencing users motivation, attitude and future behaviors. For example, people prefer for a health-care provider to wear a white coat or a professional dress,¹¹ for a physiotherapist to wear a tailored dress,¹² and for female therapists to wear casual attire.¹³ These preferences are found to communicate expertise and authority.^12,14 Hence, they have an important influence on patient trust levels, establish confidence, influence the perception of empathy, and increase the likelihood that patients will comply with care instructions.^14,15 This effect goes beyond attire; it is long known that relative to a physically unattractive counsellor, an attractive counselor is perceived more favorably with regard to her competence, professionalism, assertiveness, interest, and relaxation, and her ability to help with problems.¹⁶

Similar to the finding that humans are influenced by the appearance of humans, they are also influenced by the appearance ECAs. Different studies on appearance of ECAs conclude that different appearances lead to different outcomes in terms of motivation and behavior change,¹⁷ regardless of the underlying technical system. Baylor and colleagues showed, for example, that an attractive and ‘cool’ agent leads to higher levels of motivations among youngsters, because it most closely reflected themselves.¹⁸ Furthermore, a recent study by ter Stal and colleagues¹⁹ found that an ECA’s appearance effects the users’ perception of authority, whereby older male agents were seen as more authoritative than young female agents. In another study, ter Stal and colleagues showed that the agent’s role (e.g. a peer or expert) also effects the perception of the agent’s characteristics, such as trust and friendliness, but also the likeliness of following the agent’s advice.²⁰ Thus, adapting the appearance not only influences first impressions, but also future interactions between the user and ECA.

Persuasiveness and personality

In general, the goal for an eHealth intervention is to persuade end-users into using the intervention and complying with the desired behavior (e.g., losing weight or monitoring their health status). By taking into account persuasive strategies or functionalities in the design of an eHealth intervention, one can increase end-user adherence.²¹ An overview of these design principles has been created by Oinas-Kukkonen and Harjumaa²² and includes features such as rewards, third-party endorsement, and instilling authority. However, implementing these features (either with or without an ECA) should not be seen as the optimal solution for creating a successful eHealth intervention. Rather, the design should be tailored towards the target group and the health-related behavior goal.^23,24

Tailoring the design of an ECA can be done by adapting the dialogue script, its body language, or its appearance, all part of an ECA’s personality. Leading to the question which personality traits are ideally incorporated in an ECA. Existing studies on ECAs measure user satisfaction via characteristics as liking, trust and friendliness.^25,26 In a previous study, we set up multiple co-design sessions with community-dwelling older adults in which we discussed which personality traits they would prefer for a health ECA. We found that the most valued traits among this group were friendliness, warmth, trustworthiness, concern and competence.²⁷ These traits, with the exception of ‘warmth’, were also identified as most important among older adults in a card-sorting task study by ter Stal et al.²⁰ These findings indicate that associating the right personality traits with appearance to an ECA for a specific health-related behavioral goal, might increase the persuasiveness, and hence appreciation and effect.

Research objectives

In this article, we report on an experiment with three different ECAs and three different health topics. The aim is to identify the effect of a match between a health topic and the ECAs’ appearance on ratings of personality characteristics, persuasiveness and intention to use. The ultimate goal is hereby to create design guidelines for ECA design that ensures high persuasiveness and intention to use. To this goal, we conducted an online experiment among older adults and triangulated the quantitative results with qualitative insights from a focus group. The main research question that we formulated goes: How does the match between a health topic addressed by an ECA and the ECAs’ appearance effect end-users’ evaluation of the ECAs personality characteristics and persuasiveness?

To guide this study, we formulated three hypotheses, based on the literature discussed above:

1. When an ECA is embedded in a health topic, it is rated higher on positive personality traits (friendliness, warmth, trustworthiness, concern, and competence).

2. Older adults prefer an ECA that has an appearance matching a health topic (cooking, food, and loneliness).

3. An ECA that has an appearance matching a health topic is rated higher on positive personality traits, persuasiveness and intention to use.

Method

Method online experiment

Participants

Participants were recruited via a Dutch research panel of the National Foundation for the Elderly, consisting of approximately 1350, mainly community-dwelling, older adults. Participants received an email asking whether they were willing to participate in the online questionnaire. The only inclusion criterion was that participants should be fluent in the Dutch language. In addition, community-dwelling older adults whom participated in a previous study of the same project were invited per newsletter to complete the questionnaire.²⁷ Since four participants previously indicated a preference to receive documents per post, they received a paper version of the questionnaire.

Stimuli

Based on a previous co-creation study with community-dwelling older adults in the Netherlands,²⁷ we created three different ECAs, with different names and personas (see Figure 1). The first ECA represents a female peer (Ellen), the second ECA a chef (Herman) and the third ECA a fantasy figure (Bo).

Figure 1.

The three different ECAs (Ellen, Herman and Bo).

In addition, and also based on the co-creation study, we created three storyboards (see Appendix 1). Each storyboard addresses a different health topic, and is based on a different behavior change technique (BCT). The first context is ‘Cooking’, and includes a recipe book with the BCT tailoring, with the aim to improve eating behavior. The second context is ‘Food,’ consisting of a food diary via which users self-monitor their eating behavior. The third and last context is ‘Loneliness,’ and consists of a bundle of audio-fragments from other older adults about social activities they performed, based on the BCT social learning, with the aim to decrease loneliness.

Procedure

The online survey tool Qualtrics was used for the questionnaire. After providing written informed consent and completing the questions on the socio-demographics, participants were randomized. A quarter of the participants were randomly assigned to questionnaire A, three quarters of all participants were randomly assigned to questionnaire B (see Figure 2).

Figure 2.

Flow of the questionnaire. Note: E = Ellen, H = Herman, B = Bo. F = Food, C = Cooking, L = Loneliness.

In questionnaire A, participants were asked to rate the appearance of the three different ECAs on five personality characteristics. They only viewed the image of the ECA (similar to Figure 1). Next, participants were asked to indicate the importance of the five characteristics. Last, participants viewed three different storyboards without an ECA (see Appendix 1), and were asked which of the three ECAs would be able to help them best.

In questionnaire B, participants were further randomized over 1 of 3 groups. Each group of participants viewed three storyboards, each with a different ECA addressing a different health topic (thus, there were 9 combinations). Participants were then asked to rate the ECA on five personality characteristics, and to assess its persuasiveness and intention to use.

Measurements

All participants were asked questions about their socio-demographics, including: age, gender, retirement (y/n), highest finished education, living situation (with partner, without partner, with someone else), whether they had home-cooked dinners (in order to control for community-dwelling, y/n) and chronic diseases. In addition, we also measured health literacy, using the Three Brief Health Literacy Screeners.²⁸

For questionnaire A, participants reviewed the three ECAs without a health topic, or only the health topics. The following data were collected:

• Ratings of the personality characteristics per ECA: friendliness, warmth, trustworthiness, concern and competence. These five characteristics were based on the previous co-creation study. The questions were measured using participant agreement with a 5-point Likert scale ranging from “1 = Strongly disagree” to “5 = Strongly agree”

• Ratings of importance of the five personality characteristics. The following statement was provided: “In general, I think it is important for a coach to show the following characteristics”. The question was answered on the same 5-point Likert scale.

• Preferred ECA per health topic, after seeing only the health topic. The following question was asked: “Which virtual coach would be able to help you best?” The answer option consisted of an image of each of the three ECAs.

For questionnaire B, participants reviewed three different ECAs, addressing three different health topics. The following data were collected:

• Ratings of the personality characteristics per ECA. The questions were measured using the same 5-point Likert scale.

• Ratings on persuasiveness and intention to comply per ECA. This was measured by a validated perceived persuasiveness scale, adapted from Drozd et al.²⁹ The scale consists of four questions: i) “The system would influence me.”; ii) “The system would be convincing.”; iii) “The system would be personally relevant for me.”; iv) “The system would make me reconsider my (eating) behavior. The questions were measured using the same 5-point Likert scale.

• Intention to use, measured via the question “I would like to use this program”, and answered on the same 5-point Likert scale.

Data analysis

Data were analyzed using SPSS/WIN 25.0 software (IBM Corp., Armonk, NY, USA). Data are presented as mean ± standard deviation (SD) for normally distributed continuous variables, and frequencies and percentages for categorical variables. For H1, we first controlled whether all personality characteristics were deemed important ( $>$ 3.0). Next, we generated the average score of the five personality characteristics per ECA, split into with and without health topic, and used the independent t test to test statistical difference. For H2 we counted how often an ECA was preferred for a certain health topic, using 40% as a cut-off score to identify a match. In addition, we used ANOVA to test for significant influence of gender or age. For H3, we first created categories for matching and other combinations, and generated a single score for persuasiveness (α was between 0.915 and 0.963). We then compared scores on average personality, persuasiveness and intention to use between both categories, using the independent t test. Normality was assumed based on the sample size.³⁰ A two-sided p value of $<$ 0.05 was used as the cut off for statistical significance. All calculations were discussed with a statistician.

Method focus group

Participants

We invited all 13 participants from our previous study²⁷ to participate in this focus group. Invitations were send per mail and post, based on the preference of the participant. In total, six participants signed up and joined the focus group.

Study design

In the focus group we discussed the findings and improved the appearance of the ECA. The duration of the session was 4 hours, including a lunch and one-hour break. The aim of the analyses was to provide deeper insights on the match between the ECA and the health topic, the appreciation of ECAs in terms of positive personality traits, and the persuasiveness of ECAs. More specific:

• How should the appearance of the ECAs be improved?

• How should the personality of the ECAs be designed?

Procedure

As a result of the online questionnaire, two ECAs were selected for the three different health topics. Participants were picked-up at home and we (LK and StS) met at the headquarters of the National Foundation for the Elderly. After a short introduction, participants were asked to provide written informed consent. In total, the focus group consisted of three assignments.

During the first assignment, the designer (StS) showed an image of the first ECA, and LK provided a recap of the cooking topic. Participants were asked why they thought this particular ECA was preferred for this health topic. Next, we asked how participants thought the design of the ECA could be improved. When a participant mentioned, for example, the haircut, this was adjusted at the spot and improved until satisfaction was reached. We created a set of design characteristics upfront to ask participants’ opinions about, if it had not been mentioned yet (including age, skin color, BMI, outfit, hair, eye color, accessory, other). For the second assignment we asked participants to write a background story of the ECA and share it afterwards. Next, we repeated the first and second assignment for the other ECA.

During the last assignment, we invited participants in groups of three to write a short dialogue between an ECA and a user. We provided them with post-it notes, and discussed a short general example. They were instructed to write an opening, explain something about the health topics, and write an ending. This was repeated for the other ECA. Afterwards, we discussed the dialogues plenary, and created a list of conditions for the dialogues.

Data analysis

Work-sheets were scanned. Audio recordings of the sessions were transcribed verbatim by an independent agency and reviewed by the research team for accuracy by comparing the audio recordings with the written transcripts. Pseudonyms were developed for each participant to maintain confidentiality. All data was uploaded in ATLAS.ti 8. qualitative data analysis software. Analysis was guided by a thematic analysis approach, and combined a deductive and inductive approach.³¹ One researcher (LK) created a first list of codes based on the script for the session. Both LK and StS then coded the transcripts independently, and added extra codes if needed. Differences in codes and coded fragments were discussed, leading to a final and agreed upon codebook and coded transcript.

Results

Questionnaire

Characteristics of participants

The questionnaire was filled in by 732 study participants. Five participants only filled in the demographics and were therefore excluded. In total, 724 participants completed the questionnaire online, and three on paper. The mean age of the 727 study participants was 72.7 ± 8.11 years, with 83.8% being retired. Women accounted for 62.6% of participants. Half of the participants lived alone (50.5%), other participants lived with their partner (47.7%), or with someone else (1.8%). Almost all participants had home-cooked dinners (91.6%). In total, 52.5% had completed high school or some associate degree, 40.5% of all participants had completed college or university. The mean health literacy score was 6.51 ± 1.68 out of 12.

Personality characteristics of the embodied conversational agents

The descriptive statistics of the importance of the separate personality characteristics show that all characteristics were rated important (score between 3.84 and 4.36, see Table 1 for all details). After controlling for violation of the assumption of sphericity, a repeated measure ANOVA showed that the ECAs were rated significantly different on all five characteristics: friendliness (F (1,324) = 85.322, p < .001), warmth (F (1, 324) = 61.459, p <.001), trustworthiness (F (1, 324) = 174.401, p <.001), concern (F (1, 324) = 70.455, p <.001) and competence (F (1, 324) = 146.948, p <.001). Pairwise comparisons, with Bonferroni correction, reveal that Ellen (the peer) scored significant higher than Herman (the chef) on all characteristics (p = <.001 and p = 0.036), with the exception of warmth (p = 0.094). Furthermore, Bo (the fantasy figure) differs significantly from both Ellen and Herman on all five characteristics (p = <.001).

Table 1.

Rating of the importance and personality characteristics per ECA.

Characteristics	Importance	Appreciation per ECA
Characteristics	General	Ellen	Herman	Bo
Friendliness	4.06 ± 0.83	3.21 ± 0.98^A	3.05 ± 0.96^B	2.55 ± 1.14^C
Warmth	3.84 ± 0.90	2.94 ± 0.89^A	2.82 ± 0.82^A	2.40 ± 1.02^B
Trustworthiness	4.25 ± 0.88	3.12 ± 0.78^A	2.81 ± 0.75^B	2.31 ± 0.95^C
Concern	4.16 ± 0.89	2.99 ± 0.78^A	2.86 ± 0.77^B	2.34 ± 0.96^C
Competence	4.36 ± 0.90	3.01 ± 0.77^A	2.89 ± 0.86^B	2.27 ± 0.94^C

Note: Statistically significant differences among these means do not share a superscript.

As can be seen in Table 2, Ellen received the highest score for overall personality, both with and without health topic. Results of the independent t test showed a significant difference on rating of personality characteristics without and with addressing a health topic for Ellen (t (563) = −5.004, p <.001), Herman (t (567) = −7.034, p <.001) and Bo (t (568) = −12.656, p <.001).

Table 2.

Average rating of personality characteristics without and with health topic.

	Ellen	Herman	Bo
Without health topic	3.05 ± 0.69	2.89 ± 0.70	2.37 ± 0.92
With health topic	3.35 ± 0.72	3.29 ± 0.70	3.26 ± 0.69
Difference (p)	<.001	<.001	<.001

Embodied conversational agents and health topic

For Cooking and Food, Herman is the preferred ECA (72.3% and 45.3% respectively, see Table 3). For Loneliness, Ellen is preferred (60.5%). Thus, Bo is not the ECA of choice for any of the three topics. There was no significant effect of gender (p was between 0.153 and 0.650) or age (p was between 0.243 and 0.892).

Table 3.

Preferred ECA per health topic.

	Ellen	Herman	Bo
Cooking	64 (22.7%)	204 (72.3%)¹	14 (5.0%)
Food	107 (41.1%)¹	115 (45.3%)¹	32 (18.0%)
Loneliness	141 (60.5%)¹	50 (21.5%)	42 (18.0%)

¹A superscript indicates a match.

Effect of a match

As can be seen in Table 3, the match category consisted of Herman for Food, Herman for Cooking, Ellen for Food, and Ellen for Loneliness. The other category consisted of Herman for Loneliness, Ellen for Cooking, Bo for Food, Bo for Cooking, and Bo for Loneliness. Results of the independent t test showed no significant difference for personality (t (417) = 1.833, p = 0.068), but a significant difference for persuasiveness (t (417) = 3.264, p = 0.001), as well as intention to use (t (417) = 3.046, p = 0.002) (see Table 4).

Table 4.

Difference between match and other on personality, persuasiveness and intention to use.

	Match	Other	Difference (p)
Personality	3.35 ± 0.64	3.24 ± 0.66	0.068
Persuasiveness	2.78 ± 0.95	2.49 ± 0.92	0.001
Intention to use	2.77 ± 1.02	2.46 ± 1.05	0.002

Focus group

Link to previous study and characteristics of participants

Based on the online experiment, Ellen was chosen as the ECA for the health contexts Food and Loneliness. Herman was chosen as the ECA for the health context Cooking. The mean age of the six study participants was 83.5 ± 7.71 years. Figure 3 shows the setting of the focus group. According to participants in the focus group, Ellen was chosen for these modules because of her competency. Participants also assumed that she knew some things of food, and that she was easy to approach. Herman was chosen for cooking because he ‘had studied for it’. Participants also assumed he was familiar with food for older adults.

Figure 3.

Setting of the focus group.

Improving the design

In general, participants were quite positive about Ellen. They agreed she had a ‘nice, open and friendly face’. A first suggestion was to broader her neck, and make her eyebrows bigger. Also, her shadow should be less visible. There was an extensive discussion about her glasses. Some participants thought it was too ‘fussy’ or ‘too serious’. While other participants thought it looked nice. In the end we agreed to change the color of the frame, so it was less remarkable. Figure 4 displays Ellen before and after the focus group.

Figure 4.

Ellen before and after the focus group.

Participants thought Herman had the right age. This was extensively mentioned by one of the participants as following:

“I choose a young cook who just left the cooking school, or where does he come from. Know the last things, be aware of the possibilities that are available (…). An old cook does not know all that exactly, and a young man does.”

Participants further agreed the moustache should be smaller, but was a nice addition. His shirt should indeed be white, without any stains, and he should wear a traditional chef’s heat. Figure 5 displays Herman before and after the focus group.

Figure 5.

Herman before and after the focus group.

Personality

Although the background stories written for Ellen were different from each other, the general line was rather similar. Ellen is a loving person with some life experience, has a family and multiple children. She is easy to feel at home with. An example of a story is the following:

“Ellen, a familiar appearance, loves people, a people person, has great colleagues, does this work from her heart, enjoys the work but not too many hours. Because otherwise she gets upset and you get another Ellen. Has a good home base, time for her family, a sweet husband. She is a good listener with empathy. Does she have a nice hobby, nice family, dear husband, I already said that-”

Herman is a nice and fun person, who loves to eat nice food, and is a bit overweight. He is always busy with work. Multiple participants also wrote that he played soccer in his free time, and again, is competent.

“My cook should be a pleasant person. He likes good food, but together. Because he often stands alone in the kitchen and then has to taste everything. It makes him fat. So that actually belongs to a cook. The advantage is also that he is married so that his projects can be criticized. How would the cook be at home? Does he cook there?”

During the writing process of the dialogues, various requirements were discussed. There were personal differences, especially regarding the formality of the tone. Table 5 lists the requirements that stood out.

Table 5.

Requirements and example quote.

Requirements	Example quote
Use short sentences	“Good day sir, may I introduce myself? I am Ellen the food coach, sent by your doctor, Dr. P. I’m going to give a short explanation about the food project.”
Include small talk	“How are you?”
Allow user to tell about their own life	[user] “I try to eat healthy, but I do not always succeed.” But also, off-topic: “Car got rid of, you shouldn’t change 60 years of damage-free driving”
Ask user which name to use	“Can I say Annie?”
Use a formal tone	“Good day sir or madam?”
Adjust greeting on time of day	“Good morning.”
Provide the user with choice	“We have an app for cooking, food and loneliness. Would that be something for you?”
Provide suggestions for activities and details	You may enjoy meeting other people in the community center from time to time… [user] “And then I ask how it works and how much it costs I always ask. And then she tells you how it works and how much it costs.”

Discussion

This study investigated whether a match between the health topic and an ECA was associated with a more positive evaluation on five personality traits, persuasiveness and intention to use. The results reveal that older adults prefer an ECA that has an appearance matching a certain health topic, resulting in higher ratings on persuasiveness and intention to use. Positive personality traits are not appreciated better as a result of a match. However, we found that it is important to measure these traits embedded in a health topic in order to gain a realistic rating for when the ECA is used in practice. In a focus group we explored how both design and personality evaluations should be further improved. Multiple specific design changes were made, and we developed two background stories and a list of requirements for the tone of voice in the dialogue script.

Match with health topics

Our results show that a match between the appearance of an ECA and its health topic is preferred by end-users over a non-match. These findings are in line with a large body of literature that uncovered the effects of appropriate appearances by healthcare professionals. For example, Hatfield and colleagues³² concluded that a standardized uniform increases perceptions of professionalism and recognition among patients. Their conclusion may also explain our findings. However, we also found that a match between ECA and a health topic did not positively effect the appreciation of the ECA personality. It was expected that, by matching appearance with a health topic, the positive aspects of stereotypes can be used, thereby boosting the end-user’s judgment of personality traits that are typically favored for a specific stereotype. For the case of an informative website about cancer screening, a match between health information design and stereotypes was shown to lead to increased message credibility and informativeness and positively effects the attitude towards cancer screening.³³ The authors explained these effects via the social judgment theory.³⁴ The theory posits that when people are confronted with new (sources of) information, they relate this to their current knowledge or attitudes (in this case, stereotypes). If their previous conceptions are confirmed, their appreciation of this new source is higher, and vice versa. So, when the appearance of an ECA confirms with the end-users’ initial, stereotypical conceptions, given the health topic, the persuasiveness of the ECA will increase, but not necessarily the appreciation of its personality. Tests such as the implicit association test might provide further understanding of these mechanisms.^35,36

Embodied conversational agents personality

In line with previous studies,^25–27 our results confirm that the traits friendliness, warmth, trustworthiness, concern, and competence, are all deemed important for an ECA. However, when only provided with an image of an ECA, people rate them differently. Thus, without considering a health topic, we found that people rated the personality traits of a peer higher than those of a cook, and the personality traits of a cook higher than those of a fantasy figure. Earlier review on ECAs in the health context already showed that fantasy figures are not often used,⁶ and showed that human agents are generally preferred over cartoon-like agents.^6,10 When a health context is provided, and thus dialogue is added in the form of a mock-up, it becomes evident that the personality traits of all three ECAs are rather higher, and that the order of preference remains consistent. This finding that people appreciate personality traits higher when provided with dialogue can be explained by the fact that personality is reflected in dialogue. Kampman et al.³⁷ proposed a neural network based fusion method, and showed that personality traits of ECAs are best predicted when audio, language and appearance are combined. This indicated that it is indeed important to consider the tone of voice, when writing dialogue for a specific ECA. In our results we provided various requirements, and argued to include disclosure. Revealing information makes people likable to others, we disclose more to those we like, and we like others we have disclosed to.³⁸ This general rule for humans, is also proven to be true for ECAs.^6,20 Following the golden standard when designing ECAs for health, a background story and tone of voice is best created together with the potential end-user in order to increase the changes of higher use and greater effect.⁶

Embodied conversational agents persuasiveness

The ECAs were perceived as more persuasive with than without a health topic. Thus, they were perceived as more influencing, convincing, relevant, and made them reconsider their behavior more. Furthermore, the intention to actually use the ECA also increased when the health topic was shown. However, one should aim to increase the persuasiveness of an ECA, rather than simply measure it. We asked end-users for their preferences regarding the appearance, and thereby aimed to tailor the design of the ECA further to the needs and wishes of community-dwelling older adults. The overview of design principles created by Oinas-Kukkonen and Harjumaa²² was not intended to use as a checklist, but rather to select the principles most important for the system at hand. For ECAs in the health context, these seem to include tailoring and similarity, in the form of offering relevant suggestions and a background story similar to that of a potential user. With regard to the personality of the ECA, both trustworthiness and expertise are important design features. Last, we showed that the social role should be matched to the health context. Incorporating these design features, increases the changes for adherence, and maximizes the changes of actual health behavior change.

Limitations of the study

Our results show the preference of Dutch community-dwelling older adults towards the appearance and personality of various ECAs in the health context. Among other things, this includes the preference of a humanoid ECA over a fantasy figure. However, earlier studies show that certain preferences are clearly context dependent, and differ from person to person. Hence, our findings are not simply generalizable to curative interventions where the focus might be on other personality traits, or design features. Furthermore, it is known that older adults have other preferences regarding ECAs compared to youngsters.²⁰ Thus, research outside the context of health ECAs for older adults, should always consider tailoring their ECA to their specific target group. We hope to have offered a method to do so. Although it should be noted that our aim was to uncover design guidelines. While these guidelines are needed to develop interventions that best match a certain target group, the possibility and added value of personalizing an intervention could be considered in later stages of development.

Another limitation which should be taken into account is the setting of the study. During the questionnaire, we asked participants to rate the ECAs after an initial and single interaction. These results might be different if participants were exposed to the ECA for a longer period of time. Hence, one should ideally retake these questionnaires in the evaluation process of the ECA to make sure the suggested design guidelines still remain valid. In addition, researchers should consider the influence of possible sensory issues during the development and evaluation.

Concluding remarks

In this article, we have uncovered design guidelines for developing ECAs within the health context, with a particular focus on a match between the ECA and the health topic, ECA personality, and ECA persuasiveness. Since more engaging means of communication are rapidly taking over the text-based information which was favored in healthcare for so long, it is an important task for the human-computer interaction community to develop guidelines that can aid the visual and dialogue design of this modality. Our efforts have completed a part of the puzzle. It is now up to future studies to develop further guidelines and complete it.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This was supported by ZonMw (40-44300-98-110).

ORCID iDs

Lean L Kramer

Lex van Velsen

Bob C Mulder

Silke ter Stal

Emely de Vet

Appendix

References

Ruttkay

Pelachaud

. From brows to trust: evaluating embodied conversational agents, Vol. 7. Springer Science & Business Media, 2004. DOI: 10.1007/1-4020-2730-3

Sebastian

Richards

. Changing stigmatizing attitudes to mental health via education and contact with embodied conversational agents. Computers in Human Behavior 2017; 73: 479–488. DOI: 10.1016/j.chb.2017.03.071

Philip

Dupuy

Auriacombe

, et al. Trust and acceptance of a virtual psychiatric interview between embodied conversational agents and outpatients. NPJ Digital Medicine 2020; 3(1): 1–7. DOI: 10.1038/s41746-019-0213-y

Robertson

Solomon

Riedl

, et al. The visual design and implementation of an embodied conversational agent in a shared decision-making context (ecoach). In: International Conference on Learning and Collaboration Technologies. Springer 2015, pp. 427–437. DOI: 10.1007/978-3-319-20609-7_40

Provoost

Lau

Ruwaard

, et al. Embodied conversational agents in clinical psychology: a scoping review. Journal of Medical Internet Research 2017; 19(5): e151. DOI: 10.2196/jmir.6553

Kramer

Ter Stal

Mulder

, et al. Developing embodied conversational agents for coaching people in a healthy lifestyle: Scoping review. Journal of Medical Internet Research 2020; 22(2): e14058. DOI: 10.2196/14058

Montenegro

JLZ

da Costa

da Rosa Righi

. Survey of conversational agents in health. Expert Systems with Applications 2019; 129: 56–67. DOI: 10.1016/j.eswa.2019.03.054

Scholten

Kelders

Van Gemert-Pijnen

. Self-guided web-based interventions: scoping review on user needs and the potential of embodied conversational agents to address them. Journal of Medical Internet Research 2017; 19(11): e383. DOI: 10.2196/jmir.7351

Yalcın

. Empathy framework for embodied conversational agents. Cognitive Systems Research 2020; 59: 123–132. DOI: 10.1016/j.cogsys.2019.09.016

10.

ter Stal

Kramer

Tabak

, et al. Design features of embodied conversational agents in ehealth: a literature review. International Journal of Human-Computer Studies 2020; 138: 102409. DOI: 10.1016/j.ijhcs.2020.102409

11.

Petrilli

Mack

Petrilli

, et al. Understanding the role of physician attire on patient perceptions: a systematic review of the literature—targeting attire to improve likelihood of rapport (tailor) investigators. BMJ Open 2015; 5(1). DOI: 10.1136/bmjopen-2014-006578

12.

Mercer

MacKay-Lyons

Conway

, et al. Perceptions of outpatients regarding the attire of physiotherapists. Physiotherapy Canada 2008; 60(4): 349–357. DOI: 10.3138/physio.60.4.349

13.

Richards

. College students’ preferences for therapist attire, gender, and office décor. Marywood University, 2010.

14.

Rehman

Nietert

Cope

, et al. What to wear today? effect of doctor’s attire on the trust and confidence of patients. The American Journal of Medicine 2005; 118(11): 1279–1286. DOI: 10.1016/J.AMJMED.2005.04.026

15.

Chung

Lee

Chang

, et al. Doctor’s attire influences perceived empathy in the patient–doctor relationship. Patient Education and Counseling 2012; 89(3): 387–391. DOI: 10.1016/j.pec.2012.02.017

16.

Lewis

Walsh

. Physical attractiveness: its impact on the perception of a female counselor. Journal of Counseling Psychology 1978; 25(3): 210. DOI: 10.1037/0022-0167.25.3.210

17.

Baylor

. Promoting motivation with virtual agents and avatars: role of visual presence and appearance. Philosophical Transactions of the Royal Society B: Biological Sciences 2009; 364(1535): 3559–3565. DOI: 10.1098/rstb.2009.0148

18.

Rosenberg-Kima

Baylor

Plant

, et al. Interface agents as social models for female students: the effects of agent visual presence and appearance on female students’ attitudes and beliefs. Computers in Human Behavior 2008; 24(6): 2741–2756. DOI: 10.1016/j.chb.2008.03.017

19.

Ter Stal

Broekhuis

van Velsen

, et al. Embodied conversational agent appearance for health assessment of older adults: Explorative study. JMIR Human Factors 2020; 7(3): e19987. DOI: 10.2196/19987

20.

ter Stal

Tabak

op den Akker

, et al. Who do you prefer? the effect of age, gender and role on users’ first impressions of embodied conversational agents in ehealth. International Journal of Human–Computer Interaction 2020; 36(9): 881–892. DOI: 10.1080/10447318.2019.1699744

21.

Kelders

Kok

Ossebaard

, et al. Persuasive system design does matter: a systematic review of adherence to web-based interventions. Journal of Medical Internet Research 2012; 14(6): e152. DOI: 10.2196/jmir.2104

22.

Oinas-Kukkonen

Harjumaa

. Persuasive systems design: key issues, process model, and system features. Communications of the Association for Information Systems 2009; 24(1): 28. DOI: 10.17705/1CAIS.02428

23.

Ludden

Van Rompay

Kelders

, et al. How to increase reach and adherence of web-based interventions: a design research viewpoint. Journal of Medical Internet Research 2015; 17(7): e172. DOI: 10.2196/jmir.4201

24.

van Velsen

Broekhuis

Jansen-Kosterink

, et al. Tailoring persuasive electronic health strategies for older adults on the basis of personal motivation: web-based survey study. Journal of Medical Internet Research 2019; 21(9): e11759. DOI: 10.2196/11759

25.

Bickmore

Caruso

Clough-Gorr

, et al. It’s just like you talk to a friend’relational agents for older adults. Interacting with Computers 2005; 17(6): 711–735. DOI: 10.1016/j.intcom.2005.09.002

26.

Friederichs

Bolman

Oenema

, et al. Motivational interviewing in a web-based physical activity intervention with an avatar: randomized controlled trial. Journal of Medical Internet Research 2014; 16(2): e48. DOI: 10.2196/jmir.2974

27.

Kramer

Blok

Van Velsen

, et al. Supporting eating behaviour of community-dwelling older adults: co-design of an embodied conversational agent. Design for Health 2021; 5(1): 120–139. DOI: 10.1080/24735132.2021.1885592

28.

Chew

Bradley

Boyko

. Brief questions to identify patients with inadequate health literacy. Health 2004; 11: 12.

29.

Drozd

Lehto

Oinas-Kukkonen

. Exploring perceived persuasiveness of a behavior change support system: a structural model. In: International Conference on Persuasive Technology. Springer 2012, pp. 157–168. DOI: 10.1007/978-3-642-31037-9_14

30.

Rosenblatt

. A central limit theorem and a strong mixing condition. Proceedings of the National Academy of Sciences of the United States of America 1956; 42(1): 43.

31.

Braun

Clarke

. Using thematic analysis in psychology. Qualitative Research in Psychology 2006; 3(2): 77–101. DOI: 10.1191/1478088706qp063oa

32.

Hatfield

Pearce

Del Guidice

, et al. The professional appearance of registered nurses: an integrative review of peer-refereed studies. The Journal of Nursing Administration— JONA 2013; 43(2): 108–112. DOI: 10.1097/NNA.0b013e31827f2260

33.

Yang

. The color of gender stereotyping: The congruity effect of topic, color, and gender on health messages’ persuasiveness in cyberspace. Computers in Human Behavior 2016; 64: 299–307. DOI: 10.1016/j.chb.2016.07.001

34.

Sherif

Hovland

. Social judgment: assimilation and contrast effects in communication and attitude change. American Journal of Sociology 1961.

35.

Nosek

Greenwald

Banaji

. Understanding and using the implicit association test: II. Method variables and construct validity. Personality and Social Psychology Bulletin 2005; 31(2): 166–180.

36.

Vianello

Bar-Anan

. Can the implicit association test measure automatic judgment? the validation continues. Perspectives on Psychological Science 2021; 16(2): 415–421.

37.

Kampman

Barezi

Bertero

, et al. Investigating audio, visual, and text fusion methods for end-to-end automatic personality prediction, 2018. arXiv preprint arXiv:180500705. DOI: 10.18653/v1/P18-2096

38.

Collins

Miller

Spielman

et al. How good is good enough? an ethical analysis of software construction and use. Communications of the ACM 1994; 37(1): 81–91. DOI:10.1145/175222.175229

Optimizing appreciation and persuasion of embodied conversational agents for health behavior change: A design experiment and focus group study

Abstract

Keywords

Introduction

The influence of appearance

Persuasiveness and personality

Research objectives

Method

Method online experiment

Participants

Stimuli

Procedure

Measurements

Data analysis

Method focus group

Participants

Study design

Procedure

Data analysis

Results

Questionnaire

Characteristics of participants

Personality characteristics of the embodied conversational agents

Embodied conversational agents and health topic

Effect of a match

Focus group

Link to previous study and characteristics of participants

Improving the design

Personality

Discussion

Match with health topics

Embodied conversational agents personality

Embodied conversational agents persuasiveness

Limitations of the study

Concluding remarks

Footnotes

Declaration of conflicting interests

Funding

ORCID iDs

Appendix

References