Effect of nonverbal behaviors and speech characteristics of online medical ECAs on patients’ satisfaction: The mediating roles of empathy and trust

Abstract

Objective

Embodied conversational agents (ECAs) play a crucial role in digital healthcare. While existing studies have examined the effect of ECAs’ appearances and nonverbal behaviors on acceptance, the mechanism by which nonverbal behaviors and speech characteristics jointly influence patients’ satisfaction remains unclear. This study explores the impact of medical ECAs’ nonverbal behaviors and speech characteristics on patients’ perceived empathy, trust, and satisfaction.

Methods

This study adopted a mixed factorial experimental design. Independent variables included nonverbal behaviors (facial expressions: smiling/painful; body postures: open/closed) and verbal behaviors (speech rate and pitch: low/medium/high). Dependent variables included perceived empathy, trust, and satisfaction. Speech rate served as a between-subjects variable and pitch as a within-subjects variable. 75 participants were recruited to watch and interact with the 12 pre-recorded ECA videos presented in a random order. Perceived empathy, trust and satisfaction were measured.

Results

Body postures significantly affected satisfaction (p< .001), while facial expressions showed no significant impact (p=.302). Speech rate significantly influenced satisfaction (p=.034), with low rate yielding higher satisfaction than high rate (p= .044); pitch showed no significant main effect (p=.295). Trust fully mediated the relationship between nonverbal behaviors and satisfaction, while empathy and trust serially mediated this effect. Trust partially mediated speech characteristics’ effect on satisfaction. The combination of painful expression, open posture, low speech rate, and high pitch yielded significantly higher satisfaction than other conditions (p< .001).

Conclusions

Open body postures and low speech rate directly enhanced patients’ satisfaction. Trust mediated the effects of both nonverbal behaviors and speech characteristics on satisfaction. The optimal combination of open postures, painful expressions, high pitch, and low speech rate most effectively improved patient satisfaction. These findings provides empirical evidence for multimodal medical ECA design and enhancing digital health service experience.

Keywords

embodied conversational agents nonverbal behavior speech characteristics trust empathy satisfaction

Introduction

Embodied Conversational Agents (ECAs) are artificial intelligence systems integrating language and non-verbal interaction capabilities,¹ particularly by combining large language models and techniques such as retrieval-enhanced generation to enhance the accuracy, transparency, and adaptability of interactions.² As an emerging human-computer interaction technology, its application in medical support has become increasingly widespread.³ ECAs can communicate with patients in real time and in a multimodal manner through human-like avatars, bringing new possibilities to traditional healthcare service models, particularly demonstrating significant potential in disease management, mental health support, and patient education.^4,5 They offer functions such as emergency medical consultations, symptom checks, appointment scheduling, and post-treatment follow-ups, all without time or location constraints.⁶

These capabilities position ECAs as a central component of the Digital Front Door strategy—an approach in which technology serves as the primary gateway between patients and healthcare providers. The COVID-19 pandemic accelerated this structural shift from face-to-face care toward distributed, remote-first delivery models, with evidence suggesting that patients’ experiences at this initial digital touchpoint meaningfully shape their subsequent engagement, adherence, and healthcare utilization.^7,8 Yet this transition also introduces a critical design challenge: without the physical presence of a clinician, the communicative warmth and nonverbal cues that naturally foster empathy and trust must be deliberately embedded within the virtual interface.⁹ In this context, the behavioral and communicative design of ECAs as the first point of digital contact may determine not only immediate patient experience, but also longer-term engagement with the broader care system.

However, the unique nature of medical contexts presents distinct challenges for ECAs in practical applications. The successful implementation of digital healthcare systems or technologies depends not only on the technological sophistication itself, but also on users’ understanding and willingness to adopt the technology, as well as meeting their performance expectations, effort expectations, and social impact.¹⁰ Furthermore, Unlike customer service or e-commerce, medical consultations involve health-related anxiety, privacy concerns, and emotional vulnerability,^11,12 making patient trust and perceived empathy key factors in the success of medical ECAs.^13,14 Traditional online medical consultations primarily relied on text-based dialogue, lacking the emotional expression and non-verbal cues in face-to-face interactions. It could lead to inadequate information transmission and insufficient emotional resonance.¹⁵ In contrast, ECAs can provide a richer and more intuitive communication experience through multimodal interaction, such as visual imagery, facial expressions, gestures, and speech characteristics.¹⁶ The appearance and communication style of ECAs significantly influence trust, and the sensitivity of medical contexts makes trust-building particularly important.^13,14 Optimizing ECAs’ visual appearance, speech characteristics, and interactive behaviors could effectively enhance patients’ trust and acceptance of the healthcare system.¹⁷

Trust and empathy are widely recognized in doctor-patient interactions as core factors influencing patients’ satisfaction and treatment adherence. Trust reflects patients’ beliefs in healthcare professionals’ competence, kindness, and integrity,¹⁸ while empathy reflects a doctor’s ability to understand, feel, and share patients’ emotional experiences.¹⁹ Researches indicated that a doctor’s expression of empathy could significantly enhance patient trust, thereby improving overall satisfaction.^20,21 For example, direct eye contact and a forward-leaning or upright posture,²² mimicking others’ facial expressions, using facial expressions that convey concern, smiling, nodding, using a warm tone of voice, a lower pitch, and touching others’ arms or hands could influence patients’ perceived empathy.^23–25 Therefore, the behavioral characteristics of ECAs influence patients’ perceptions of trust and empathy.

Existing researches suggest that the design features of ECAs show a significant impact on patients’ psychological perceptions. These design features primarily include two dimensions: nonverbal behaviors, including facial expressions, body posture, and eye contact; and speech characteristics, such as speech rate, pitch and voice quality.²⁶ Regarding nonverbal behaviors, facial expressions could rapidly influence patients’ trust judgments toward ECAs, with expressions of pain more effective than smiles in eliciting perceived empathy in medical contexts.²² Body posture also plays a significant role, with open body postures such as uncrossed arms and leaning forward typically interpreted as more approachable and trustworthy.²⁷ In particular, empathetic ECAs could improve patients’ experience during the consultation.²⁸ According to studies on speech characteristics, speech rate and pitch together could greatly affect users’ emotional reactions and levels of trust; better trust was generally linked to lower pitch.^29,30 Gao et al.³¹ also explored the impact of virtual agents’ stylization and speech characteristics on trust building, providing new insights for optimizing the multimodal design of ECAs.

Most existing studies have focused on the influence mechanisms of ECAs in a single modality, such as studying facial expressions or speech characteristics alone, lacking systematic studies on the comprehensive effects of nonverbal behaviors and speech characteristics. Second, research on the mechanisms through which these design features influence patient satisfaction via the mediating effects of trust and perceived empathy remains limited. Third, the optimal configuration of different multimodal behavioral cues in medical contexts has not been sufficiently validated.

This study aims to explore how nonverbal behaviors and speech characteristics of medical ECAs influence patient satisfaction through the mediating effects of trust and perceived empathy. Specifically, this study will: (1) examine the independent effects of ECAs’ facial expressions, body posture, pitch, and speech rate on patients’ satisfaction; (2) reveal the mediating role of trust and perceived empathy in the relationship between design features and satisfaction; (3) validate the interactive effects of nonverbal behavior and speech characteristics, and explore the optimal multimodal design combination. Furthermore, the results are expected to be applied in empathy training for healthcare professionals by analyzing effective empathy expression patterns, improving medical professionals’ communication skills, and ultimately improving patients’ overall healthcare experience and satisfaction. The findings can also guide the optimized design of medical ECAs, assisting developers in creating more trustworthy and patient-friendly virtual medical assistants.

Research model and hypothesis development

Nonverbal behaviors

Nonverbal behaviors refer to conveying information through nonverbal means such as facial expressions, body posture, and gestures.³² In healthcare settings, the nonverbal behaviors are crucial for shaping patient-clinician interactions and influence patients’ perceptions of care quality.³³ Related studies have identified specific nonverbal behaviors that influence how patients perceive clinicians.^34,35 These behaviors include eye contact, maintaining eye level with the patient, leaning forward, adopting an open posture, mirroring others’ expressions, displaying caring facial expressions, smiling, nodding, using a warm tone of voice, and appropriate physical contact.²⁴ These nonverbal behaviors have a significant impact on offline medical consultations, affecting patients’ satisfaction and healthcare service utilization.³⁶ Rosenthal-von Der Pütten et al.³⁷ found that the nonverbal behaviors of ECAs significantly affected patient acceptance, with approximately 65% of patients’ acceptance stemming from nonverbal behaviors. However, the absence of vocal inflection and natural body movements may also affect patient acceptance of ECAs. Therefore, ensuring consistency between verbal and nonverbal cues is crucial for patient experience.

Facial expressions represent one critical aspect of nonverbal behavior. Facial expression interaction can improve subjective happiness with the patient experience by increasing their emotional engagement.³⁸ Previous investigations have revealed that smiling can indirectly increase happiness by generating satisfaction and elevating patients’ sense of respect.³⁹ Smiling in the medical settings can improve overall satisfaction by increasing patients’ comfort, trust, and positive perception of doctors.⁴⁰ Doctors who display smiling expressions are perceived as more empathetic, enthusiastic, and capable of improving patient satisfaction than those without nonverbal behaviors.²⁷ Patients often perceive doctors who lack smiles as less patient-centered and less willing to seek for medical care.⁴¹ It was also found that negative expressions reduce patients’ satisfaction.⁴²

Consistent with facial expressions, body posture is another significant nonverbal cue that affects the initial impression of doctor-patient interactions.⁴³ Open and relaxed body postures are generally perceived as conveying greater warmth, calmness, informality, and approachability than closed or tense body postures.⁴⁴ In particular, nodding, leaning forward, facing the patient directly, and keeping arms and legs uncrossed could improve patients’ satisfaction.⁴⁵ The research on power posing of body posture by Carney et al.⁴⁶ was applied explicitly to doctor-patient interactions by Forkin et al.⁴⁷ They demonstrated that doctors who adopted high-power positions, or open postures, were more likely to be seen as self-assured, wise, and focused on leadership than those who adopted low-power postures, or closed postures. As a result, they were more likely to gain the trust and satisfaction of their patients. In light of these results, the following hypotheses are proposed:

H1a

ECAs displaying smiling expressions will significantly increase patient satisfaction compared to painful expressions.

H1b

ECAs exhibiting an open body posture will significantly increase patient satisfaction compared to a closed posture.

Chain mediation mechanism through trust and empathy

The ECAs’ nonverbal behaviors directly influence patients’ perceptions, and the effects can also be mediated through psychological variables. Trust, as a core concept in interpersonal relationships, has been widely applied in the field of interaction research between humans and virtual agents.^48,49 Based on the computer-as-social-actor paradigm, researchers have confirmed the applicability of interpersonal trust dimensions.⁵⁰ Existing studies indicated that patients remained more skeptical in interactions with virtual agents compared with interpersonal interactions,⁵¹ resulting in varying satisfaction levels. Patient satisfaction with virtual agents largely depends on their trust.⁵² The transparency of interactive intent demonstrated by digital agents during interactions can effectively reduce patients’ perceived vulnerability, thereby fostering trust and promoting the adoption of new technologies.^53,54

Facial expressions are an essential part of nonverbal behaviors when building trust. People could establish trust judgments about others in a very short time, about 38 milliseconds.⁵⁵ People make unconscious decisions about whether or not to trust others based on their observations of facial expressions.⁵⁵ According to studies based on the theory of emotional generalization, positive expressions were regarded as the most trustworthy, while negative emotional expressions diminished trust and positive expressions significantly increased trust.⁵⁶ Nevertheless, the function of facial expressions differed in varied situations.^44,57 Patients in discomfort were more likely to believe that a doctor is sincere and emotionally engaged, particularly when combined with open body postures.⁴⁷ In contrast, smiling could make people appear more friendly, but its positive effects might be reduced when addressing serious health issues, as it might be perceived as lacking the necessary seriousness. According to Lennart Seitz et al.,⁵⁸ lacking an emotional foundation for trust restrict patients’ confidence in ECAs in medical contexts. While open postures conveyed a sense of acceptance,⁴⁷ virtual agents consistently displaying pain alongside patients demonstrated empathy.⁵⁹ Therefore, we propose a hypothesis that painful facial expression with open posture could be the most effective in building trust and improving satisfaction.

Empathy is another significant mediating factor that is associated with trust. Eklund and Meranius⁶⁰ defined empathy as the capacity to feel, comprehend, and share others’ experiences. In other words, it is the process of recognizing and comprehending patients’ cognitive and emotional requirements and reacting appropriately.^51,61,62 Previous studies have primarily focused on two categories of empathetic reactions. The first is a cognitive response, sometimes referred to as perspective-taking, which is the capacity to comprehend the intentions, feelings, and ideas of others.⁶³ The second is emotional response, which is demonstrated by trying to experience other people’s emotions and reacting to them in a way that aligns with how they perceive them.⁶⁴ This study primarily focuses on the emotional responses elicited by ECAs. Although ECAs may lack empathy,⁶⁵ they possessed the ability to express empathy, primarily through nonverbal behaviors that reflected human emotions.⁶⁶ Facial expressions, eye contact, and open body postures that demonstrated concern or alignment with the patient were considered expressions of empathy.²²

Trust and empathy are intimately linked. When ECAs exhibited both a forward-leaning posture and a painful facial expression at the same time, participants felt the most empathy.²² Kraft-Todd found that physicians who adopted open, patient-facing positions were seen as more capable and sympathetic than those who wore more introverted with closed positions.²⁷ Doctors’ empathy could facilitate positive doctor-patient interaction development, and previous studies and theoretical frameworks suggested that empathy was essential to patient trust.⁶⁷ According to patients, doctors’ empathic actions, such as listening to their thoughts, comprehending their viewpoints, providing comfort, and providing individualized care were the cornerstone of trust.²⁰ Additionally, empathic actions significantly increased patient happiness, and patients’ opinions of doctors’ empathy directly influenced their trust.^68,69 In light of this analysis, this study proposes the following hypotheses:

The effect of ECAs’ nonverbal behaviors on satisfaction is mediated by enhanced patients’ trust. Specifically, the combination of painful expressions and open posture can increase patient trust, thereby improving satisfaction.

ECAs’ nonverbal behavior combining painful expressions and open posture enhances patients’ satisfaction through increased perceived empathy, which subsequently strengthens trust.

The role of speech characteristics and their mediating effect

In virtual medical interactions, auditory speech characteristics are as important as visible nonverbal behaviors. Voice, the most basic form of human communication, is vital to the sense of service quality. The speech characteristics of service providers had a major impact on consumers’ perceptions of service quality, product assessments, level of satisfaction, and willingness to use.^70,71 In their study on voice-based consultation satisfaction, Gray et al. found that virtual and in-person consultations had very high patients’ satisfaction levels and performed comparably, providing crucial support for the application of speech characteristics in medical settings.⁷²

Among speech characteristics, the influence mechanism of pitch requires further investigation. As an essential perceptual feature of speech, pitch enables listeners to identify pitch attributes such as high and low.⁷³ Given the uncertainty of disease information, patients often feel anxious and seek positive cues from doctors.⁷⁴ Researches indicated that high pitch could convey physicians’ positive emotions, reduce psychological stress through emotional contagion, and enhance patient confidence.^75–77 Therefore, pitch positively influenced patients’ satisfaction, notably higher pitch.⁷⁸ However, some psychological studies also indicated that when voice pitch was higher, it might be perceived as emotionally immature⁷⁹ or lacking calmness,⁸⁰ thereby reducing patients’ satisfaction. However, in medical consultation settings, patients’ expectations of improved health often led to overlook those negative factors. In contrast, low pitch was typically associated with sadness or lack of joy,⁸¹ aggression, and threat. Thus, low pitch might evoke negative emotions, exacerbate patients’ psychological burden, and lead to low satisfaction.

Another essential speech characteristic that correlates with pitch is speech rate, which is the quantity of speech units generated in a given amount of time and is strongly associated with information support.⁸² People who speak more quickly are regarded as more compelling than those who speak more slowly, and a faster speech rate usually means that the doctor can give more information, demonstrating the degree of concern for the patient.⁸³ However, rapid speech shortens the time that patients use to acquire and retain information. It may decrease their capacity to process and interpret information.⁸⁴ A decrease in speech rate was found to be a way of expressing sadness.⁸⁵ Doctors can help patients relax in medical situations by speaking more slowly.⁸⁶

Additionally, ECAs’ speech characteristics may influence trust perception of patients.⁸⁷ Researches on the relationship between voice and trust has produced mixed results. Some studies found that people are more likely to trust individuals with lower-pitched voices,^88,89 while other studies have found that individuals are more likely to trust people with higher-pitched voices.⁹⁰ During doctor-patient interactions, doctors communicate with patients through voice, and the interactive sounds contain emotional and attitudinal information, which is conveyed nonverbally through emotional vocal cues.⁹¹ Doctors who slow down their speech rate were perceived as more caring and empathetic when conveying information. Slowing down speech rate can help patients relax and alleviate anxiety, and it significantly influences patients’ perceptions when delivering bad news.^86,92 Based on these findings, this study proposes the following hypotheses:

H4a

ECAs with higher pitch will significantly increase patients’ satisfaction compared to medium or lower pitch.

H4b

ECAs with a slower speech rate will be associated with higher patients’ satisfaction compared to faster speech rates.

The effect of speech characteristics on patient satisfaction is mediated by enhanced patients’ trust. Specifically, the combination of low speech rate and high pitch increases patients’ trust, thereby improving patients’ satisfaction.

Combining a slower speech rate and higher pitch in ECAs further enhances patients’ trust by increasing their perceived empathy, thereby improving satisfaction.

Integrative effects of multimodal behavioral cues

The preceding analysis focused on the independent effects of nonverbal behaviors and speech characteristics. However, the integrative effects of ECAs’ behavioral cues during the interaction needed to be investigated in the online medical settings. The theory of emotional consistency proposes that when facial expressions, body posture, and speech characteristics align emotionally, observers are more likely to perceive authenticity and reliability, thereby enhancing emotional perception.^93–95 Additionally, de Gelder and Vroomen⁹³ found in their multimodal emotional integration study that visual and auditory channels exhibited interactive effects in emotional perception, and inconsistent facial expressions and speech reduced the accurate identification of emotional information. Wang and Gratch⁹⁴ further found that when the vocal emotions of ECAs align with their facial expressions, patients were more likely to perceive them as empathetic individuals with a sense of social presence. Behavioral consistency helped evoke patients’ psychological safety, enhancing trust and perceived empathy.⁹⁶ Specifically analyzing the multimodal interaction combinations in this study, in a medical setting, virtual agents and patients displaying consistent expressions of pain can express concern and empathy for the patient’s predicament,⁵⁹ conveying understanding and attention to patient suffering. Open postures, as a form of body language, convey an attitude of acceptance, support, and willingness to listen.⁴⁷ Together, these two nonverbal cues create a pattern of caring emotional expression. According to speech characteristics, a high pitch could be used to express warmth and positive emotions, giving patients emotional comfort and enhancing their trust.^75,76,78 On the other hand, a slower speech rate can help patients relax⁸⁶ and improve comprehension of information.⁸⁴ Additionally, this combination of speech characteristics creates a pattern of care-oriented emotional expression.

Current research predominantly focused on single-modal interaction behaviors of ECAs, with limited attention paid to identifying which specific combinations of speech characteristics and nonverbal behaviors most effectively convey empathy and trust. Studies on multimodal emotion integration suggest that behaviors from different sensory channels undergo mandatory integration, which occurs automatically and unconsciously.⁹³ Individuals simultaneously integrate visual and auditory information to form impression-based judgments of others.⁹⁷ On the visual behavioral cues, combinations of nonverbal behaviors are crucial for communicating empathy. Riess and Kraft-Todd³⁵ proposed the E.M.P.A.T.H.Y. tool, highlighting posture and facial expressions as key nonverbal elements in medical communication. Specifically, painful facial expressions effectively convey empathy and emotional resonance with patients’ distress,⁵⁹ while open postures signal receptiveness and willingness to listen.⁴⁷ Recent studies indicate that slower speech rates and higher pitch are reliably recognized by listeners as conveying greater empathy⁹⁸ and trust,⁸⁷ thereby enhancing satisfaction. Thus, when nonverbal behaviors and speech characteristics are presented simultaneously, they may produce a synergistic enhancement effect, more comprehensively addressing patients’ dual needs for empathy and trust during medical consultations.

Rahsepar Meadi et al.⁹⁹ also examined the difficulties associated with virtual consultations, stressing the value of multimodal behavioral cues in delivering individualized and reliable healthcare services. Their findings revealed that patients’ perceptions of physician empathy and dependability significantly influenced satisfaction with virtual medical consultations. Drawing on these insights, the present study investigates how specific combinations of behavioral cues influence patient perceptions of ECAs in virtual healthcare contexts. This study proposes the following hypotheses:

The combination of ECAs’ nonverbal behaviors featuring painful expressions and open posture with speech characteristics featuring lower speech rate and higher pitch will significantly enhance patient satisfaction.

Therefore, a research model was developed based on hypotheses H1-H7, as shown in Figure 1.

Figure 1.

Research model.

Methods

Study design

ECAs’ speech characteristics (speech rate: low, medium, high; pitch: low, medium, high) and nonverbal behaviors (facial expressions: smiling and painful; body posture: open and closed) were used as independent variables in a mixed factorial experimental design to test the research hypotheses. To reduce the possibility of fatigue effects, speech rate was manipulated as a between-subjects factor, while nonverbal behaviors and speech pitch were manipulated as within-subjects factors. For each between-subjects experiment, 25 participants were assigned to watch 12 video clips and complete a self-report scale following each clip. This study followed the APA Journal Article Reporting Standards (JARS) for experimental research in reporting the methods and results,¹⁰⁰ incorporating the AGReMA guidelines as a reference for reporting the process and outcomes of the mediation analysis.¹⁰¹

All data were collected in an independent laboratory with a quiet and private environment to minimize external interference. Participants sat alone in front of a computer screen and randomly watched and interacted with 12 pre-recorded ECAs videos. Each video depicted an ECA conducting a virtual consultation with a patient in a telemedicine setting. Three common mild symptoms were presented to simulate a real medical consultation, coughing, rash, and abdominal discomfort. The ECAs provided two standardized statements, the patient asking about symptoms and the ECA offering medical treatment recommendations, as shown in Table 1. After the inquiry, participants had a 7-second waiting time to describe their symptoms, enhancing immersion, shown as Figure 2.

Table 1.

Standardized verbal descriptions provided by ECAs.

Inquiry	Treatment suggestions
Hello, I am the physician responsible for your treatment. Please describe your symptoms, such as pain, fever, or discomfort.	Rash — Don’t worry. Based on the symptoms you described, you may have urticaria. This is a common skin condition characterized by itchy red welts or rashes. It is recommended to avoid known allergens as much as possible to relieve the symptoms and keep your skin cool and dry.
	Cough — Don’t worry. Based on the symptoms you described, you may have a cold or pharyngitis with fever. This is a common respiratory infection. You can take medications for a dry cough, such as amoxicillin, to alleviate the symptoms, and drink sufficient water.
	Abdominal discomfort — Don’t worry. You may have indigestion, bloating, or other gastrointestinal issues based on the symptoms you described. You should avoid greasy and spicy foods, and may consider taking antacids or prokinetic agents, such as omeprazole. After meals, avoid lying down immediately or engaging in strenuous exercise.

Figure 2.

Schematic diagram of ECA diagnosis and treatment process in each video clip.

Materials

The experiment employed a video-based paradigm in which participants watched pre-recorded stimuli. Following the experimental design, 36 video clips were created using DAZ Studio 4.23 and Blender 3.4. The ECAs were created using DAZ Studio 4.23 (64-bit) human models and clothing, with facial modifications, animations, and scene setup performed in Blender 3.4. The video clips were created using the keyframe method in Blender 3.4. Various keyframe interpolation methods were used to adjust transitions between different postures and expressions, with referenced open and closed postures by Forkin et al.⁴⁷ Finally, the professional video editing software was used for voice synthesis and editing.

The ECA featured a male ECA wearing medical attire, set in a hospital environment. The choice of a male ECA was based on researches indicating that while female-voiced assistants might be perceived as more trustworthy, there was no significant difference in the degree to which medical advice was accepted between the male and female agent.¹⁰² All ECAs displayed 36 combinations of two dynamic nonverbal behaviors (ie, facial expressions, body posture) and three variations in pitch and speed rate (low, medium, high). Facial expressions either aligned with the patient’s described symptoms (moderate painful expression) or not aligned (smiling). Body posture followed open and closed posture standards.⁴⁷

The voice was created using the Alibaba Cloud platform, with pitch and speech rate adjusted by modifying basic parameters. The default medium pitch and speech rate were set to 0, high pitch adjusted to +40, low pitch to -40, high speech rate to 80, and low speech rate to -120. This study employed AI-based lip-sync technology (Lip-sync by Sieve platform) to enhance the naturalness and immersion of the ECAs’ speech in the videos and ensure synchronization between speech and lip movements, enabling the ECA to exhibit lip movements consistent with the speech contents. In total, 36 video vignettes were pre-recorded and an example was presented in supplementary file. Each video was approximately 30 seconds long, showcasing a front-view perspective of the ECA from the waist up, shown as Figure 3. A representative video clip has been provided as supplementary material.

Figure 3.

Schematic diagram of nonverbal behaviors of medical ECAs. (a) Smiling facial expression with an open body posture; (b) smiling facial expression with a closed body posture; (c) painful facial expression with an open body posture; and (d) painful facial expression with a closed body posture.

Experiment procedure

Before the formal experiment, a detailed description and consent of the study were given to the participants. Participants’ demographic data (name, age, and gender) and their experience with mobile medical consultation services (ie, whether they had previously used online medical services, how satisfied they were with those services, whether the responses from those services were useful, and whether they would use those services again) were collected at the beginning of the experiment. Each participant was randomly assigned to one of three speech rate conditions and E-Prime 3.0 software was used to present 12 videos in a random order. Participants were asked to fill out a questionnaire assessing their perceptions of empathy, trust, and satisfaction following each video. After every four video parts, participants were asked to identify the ECAs’ facial expressions to make sure that they were focused. Those who gave a wrong response were deemed inattentive and were not included in the analysis. After completing the about 30-minute-long experiments, participants were given a gift to compensate for their time.

Participants

This study used the GPower 3.1 to estimate the sample size, with a moderate effect size of 0.25, a significance level of 0.05, and a statistical power of 80%.¹⁰³ It was estimated that at least 60 valid participants were needed. A total of 80 participants were recruited from the university community. Five participants were excluded due to failed attention checks, resulting in a final sample of 75 participants (47 females, 28 males; M_age=23.0 years, SD=2.5). Detailed demographic characteristics were presented in Table 2. Written informed consent forms were obtained from all the participants before the experiment. This study was approved by Ningbo University Ethics Committee (approval number: NBU-2025-483) and conducted in accordance with the Declaration of Helsinki.

Table 2.

Demographic characteristics and online healthcare service experience of participants (N = 75).

Variable	Category	Frequency (n)	Percentage(%)
Gender	Female	47	62.7
	Male	28	37.3
	Total	75	100.0
Ever used online medical services	Yes	47	62.7
Ever used online medical services	No	28	37.3
Satisfaction (users only, n = 47)	Very dissatisfied	2	4.3
	Dissatisfied	4	8.5
	Neutral	27	57.4
	Satisfied	12	25.5
	Very satisfied	2	4.3
Perceived helpfulness of the responses (n = 47)	Helpful	34	72.3
Perceived helpfulness of the responses (n = 47)	Not Helpful	13	27.7
Willingness to reuse (n = 47)	No	4	8.5
	Depends	24	51.1
	Yes	19	40.4

Measures

To assess the participants’ trust, empathy, and satisfaction with ECA, a 10-item scale was used for trust assessment, adapted from the studies by McKnight et al.¹⁰⁴ and Guo et al.,¹⁰⁵ including three dimensions: competence, benevolence, and integrity. Perceived empathy was measured using a 9-item scale adapted from Paiva et al.¹⁰⁶ Satisfaction was measured using a 7-point single-item scale ranging from very dissatisfied to very satisfied. All measurements utilized a 7-point Likert scale. The detailed contents of the scales were appended as Table 3.

Table 3.

Trust and perceived empathy measurement scales.

Scale/Measurement	References	Items
Trust – Competence	McKnight et al., 2002.¹⁰⁴	The ECAs perform their work very well.
		The ECAs are very knowledgeable and experienced in the medical field.
		The ECAs fulfill the physician’s role very well.
	Guo et al., 2016.¹⁰⁵	The ECAs are competent and able to provide helpful advice.
Trust – Benevolence	McKnight et al., 2002.¹⁰⁴	The ECAs genuinely care about my health and try to help me.
		If I have questions, the ECAs will do their best to answer them.
		I believe the ECAs act in my best interest.
Trust – Integrity	McKnight et al., 2002.¹⁰⁴	I believe the ECAs are very sincere.
		I believe the ECAs’ responses are reliable.
		I believe all the ECAs’ responses are honest.
Perceived Empathy	Paiva et al., 2017.¹⁰⁶	The ECAs treat me with warmth and care.
		The ECAs see things from my perspective.
		The ECA feels sorry for my illness.
		The ECA considers various aspects when responding.
		If the ECA saw me injured, they would want to protect me.
		The ECA seems to understand my concerns.
		The ECA answers my questions thoroughly and from multiple perspectives.
		I think the ECA is a compassionate person.
		The ECA adopts a positive approach and attitude, putting themselves in my position.

Data analysis

First, the collected data was organized and screened to ensure all questionnaires were fully completed and data quality was maintained. Scale data was entered based on the scoring of each item. During data entry, reverse-scored items were processed accordingly, and metrics such as the mean and median values for each variable were calculated to serve as indicators for subsequent analysis. Then, normality and reliability tests were performed prior to data analysis. Except for satisfaction, which was measured using a single 7-point Likert scale without assessing reliability, all other items demonstrated excellent internal reliability. Normality was assessed using the Kolmogorov-Smirnov and Shapiro-Wilk tests, and all variables significantly deviated from a normal distribution (ps < .05). Therefore, nonparametric tests were employed: Friedman tests were used to examine within-subjects effects of facial expression and body posture (H1a, H1b) and pitch (H4a), while a Kruskal-Wallis test with Dunn-Bonferroni post hoc comparisons was used for the between-subjects effect of speech rate (H4b). No demographic covariates were included in these analyses; the potential influence of age, gender, and prior online medical service experience is acknowledged as a limitation.

All three dependent variables (perceived empathy, trust, and satisfaction) were measured simultaneously at a single time point immediately following each video clip. Although this cross-sectional measurement limits causal inference, the hypothesized serial mediation pathway was established based on established theoretical frameworks in doctor-patient interaction research.^20,21 Mediation analyses were conducted using the PROCESS macro (Model 4 for simple mediation; Model 6 for serial mediation) with 5,000 bootstrap samples, and significance was determined by 95% bias-corrected confidence intervals excluding zero. All data processing was performed in IBM SPSS Statistics 27, and graphs were drawn using OriginLab 2024.

Results

Descriptive statistics and reliability analysis

Reliability referred to the consistency of a set of measurements, meaning that when similar methods are used to measure the same psychological construct, the two results show a high degree of correlation.¹⁰⁷ Descriptive statistical analysis and reliability measurements were performed for all variables. The results showed that satisfaction had a median of 5.00 (IQR = 2.00, range = 1-8), trust had a median of 5.15 (IQR= 1.50, range = 1.10-7.40, α = 0.945), and empathy had a median of 4.50 (IQR = 1.88, range = 1.00-7.63, α = 0.938). Satisfaction was measured using a single item on a 7-point Likert scale. Since it only included one item, internal consistency was not assessed. All other variables demonstrated acceptable internal reliability (αs>.76). Exploratory factor analyses confirmed the construct validity of both multi-item scales, with single-factor structures explaining 67.32% and 70.25% of the variance for trust and empathy, respectively (KMOs >.92, all ps<.001). Normality was assessed using the Kolmogorov-Smirnov and Shapiro-Wilk tests. Since all variables deviated significantly from normal distribution, nonparametric tests were used to ensure the robustness of the results.

Effects of nonverbal behaviors and speech characteristics on satisfaction

To investigate the effects of nonverbal behaviors and speech characteristics on satisfaction, nonparametric tests were conducted to examine the main effects of nonverbal behaviors (facial expressions, posture) and speech characteristics (pitch, speech rate) on satisfaction. The results were presented in Table 4. The Friedman test results indicated that the main effect of facial expressions (smiling vs. painful) on satisfaction was not significant (χ²(1) = 1.07, p=.302). Although the mean rank of the smiling expression (M=1.52) was slightly higher than that of the painful expression (M = 1.48), the difference was insufficient to support statistical significance, suggesting that facial emotional cues had a limited impact on patient satisfaction in medical settings, and thus did not support hypothesis H1a. In contrast, under the body posture condition, satisfaction with open postures (M=1.66) was significantly higher than that with closed postures (M=1.34), with the difference being statistically significant (χ²(1)=69.66, p < .001), supporting hypothesis H1b. Regarding speech characteristics, there was no significant difference in satisfaction levels across pitch levels (low, medium, high) (χ²(2)=2.44, p=.295), thus not supporting hypothesis H4a. The Kruskal-Wallis test revealed a significant main effect of speech rate on satisfaction (H(2)=6.77, p =.034). Further Dunn-Bonferroni post hoc analysis indicated that satisfaction with low speech rate (mean rank=481.08) was significantly higher than that with high speech rate (mean rank=430.68, p=.044), supporting hypothesis H4b.

Table 4.

Statistical analysis results of the effects of different nonverbal behaviors and speech characteristics on patients’ satisfaction.

Variable	Conditions	Mean ranks	Test statistic	p
Facial Expression	Smiling, Painful	1.52 (smiling), 1.48 (painful)	χ²(1) = 1.07	.302
Body Posture	Open, Closed	1.66 (open), 1.34 (closed)	χ²(1) = 69.66	< .001
Vocal Pitch	Low, Medium, High	1.94, 2.03, 2.03	χ²(2) = 2.44	.295
Speech Rate	Low, Medium, High	481.08, 439.74, 430.68	H(2) = 6.77	.034*

*Note. Only the comparison between low and high speech rate was significant after Bonferroni adjustment (p=.044).

The mediating role of trust

To examine whether the nonverbal behaviors of ECAs (the combination of painful expressions and open postures) indirectly influenced patient’ satisfaction by enhancing their trust, a mediation analysis was conducted using the PROCESS Model 4,¹⁰⁸ with 5,000 bootstrap resampling iterations. The independent variables were dichotomized, with the combination of painful expressions and open postures coded as 1 and all other combinations as 0. The results were shown in Table 5.

Table 5.

Mediation and serial mediation analysis results.

Path	B	SE	t	p	95% CI
Mediation Results: Nonverbal Behavior → Satisfaction
Nonverbal Behavior → Trust	0.291	0.084	3.44	0.001	[.125, .456]
Trust →Satisfaction	1.012	0.025	40.17	<.001	[.962, 1.061]
Total Effect	0.388	0.106	3.64	0.001	[.179, .597]
Direct Effect	0.094	0.064	1.47	0.143	[–.032, .220]
Indirect Effect	0.294	0.077	-	-	[.147, .448]
Mediation Results: Speech Characteristics → Satisfaction
Speech Characteristics → Trust	0.285	0.117	2.44	0.015	[0.056, 0.514]
Trust → Satisfaction	1.02	0.025	40.65	< .001	[0.971, 1.069]
Total Effect	0.129	0.148	0.87	0.384	[-0.161, 0.419]
Direct Effect	-0.162	0.088	-1.84	0.066	[-0.334, 0.011]
Indirect Effect	0.291	0.1	-	-	[0.095, 0.483]
Serial Mediation: Nonverbal Behavior → Empathy → Trust → Satisfaction
Nonverbal Behavior→ Empathy	0.519	0.103	5.06	< .001	[0.318, 0.720]
Empathy → Trust	0.646	0.017	37.9	< .001	[0.612, 0.679]
Nonverbal Behavior→ Trust	0.177	0.072	2.45	0.014	[0.035, 0.318]
Trust → Satisfaction	0.623	0.037	16.78	< .001	[0.550, 0.696]
Direct Effect	-0.005	0.059	-0.08	0.937	[-0.121, 0.111]
Indirect Effect	0.209	0.043	-	-	[0.126, 0.296]
Serial Mediation: Speech Characteristics → Empathy → Trust → Satisfaction
Speech Characteristics → Empathy	0.169	0.143	1.18	0.24	[-0.112, 0.450]
Empathy → Trust	0.642	0.017	38.29	< .001	[0.609, 0.675]
Speech Characteristics → Trust	0.177	0.072	2.45	0.014	[0.035, 0.318]
Trust → Satisfaction	0.627	0.037	16.87	< .001	[0.554, 0.700]
Direct Effect	-0.118	0.08	-1.47	0.141	[-0.276, 0.039]
Indirect Effect	0.247	0.106	-	-	[0.042, 0.462]

Note. Nonverbal behavior combinations (painful expression and open posture combination), Speech characteristics combination (low speech rate and high pitch), Bootstrap sample size = 5,000.

The mediation analysis revealed that the nonverbal combination of painful expressions and open postures significantly and positively predicted patients’ trust in ECAs (B=0.291, p < .001), and trust further positively predicted satisfaction (B=1.012, p < .001). Although the direct effect of nonverbal behaviors on satisfaction was not significant (B=0.094, p=.143), the indirect effect was significant (B = 0.294, 95% CI [0.147, 0.448]), supporting hypothesis H2, indicating that trust fully mediated the relationship between nonverbal behaviors and satisfaction.

To test whether the combination of speech rate and pitch influenced patients’ satisfaction by mediation role of trust, a mediation analysis was conducted using the PROCESS Model 4. The results were shown in Table 5. The combination of low speech rate and high tone of voice significantly enhanced patients’ trust (B=0.285, p = .015). Trust also had a significant positive predictive effect on satisfaction (B=1.020, p < .001). Although the direct effect of the combination on satisfaction was marginally significant (B=–0.162, p = .066), its indirect path through trust was significant (B = 0.291, 95% CI [0.095, 0.483]), supporting hypothesis H5 and validating the partial mediating effect of verbal behavior combinations by a mediation role of trust.

Serial mediation effects of perceived empathy and trust

A serial mediation analysis was conducted using PROCESS Model 6 to examine whether nonverbal behaviors (painful expressions and open postures) influenced patient satisfaction through perceived empathy and trust. The results were shown in Table 5. Nonverbal behaviors significantly enhanced perceived empathy (B=0.519, p < .001), empathy strengthened trust (B=0.646, p < .001), and trust further enhanced satisfaction (B=0.623, p <.001). The serial path was significant (B=0.209, 95% CI [0.126, 0.296]), while the direct path was not significant (B=-0.005, p=.937). Empathy and trust played a complete mediating role in this path, supporting hypothesis H3.

Further examining whether empathy influenced satisfaction through a sequential path leading to trust. The PROCESS Model 6 results, as shown in Table 5, indicated that the combination of low speech rate and high pitch did not significantly affect perceived empathy (β = 0.169, p = .24). Still, it had a significant positive effect on trust (β = 0.177, p = .014). Both empathy and trust significantly predicted satisfaction. Although the total impact of voice on satisfaction was not significant (β = -0.118, p = .141), the separate mediating path through trust was significant (Boot 95% CI = [0.034, 0.195]). The total effect was significant through the indirect impact of empathy and trust but not through the direct effect, thus partially supporting H6, indicating that only trust played a significant mediating role, while empathy did not, suggesting that speech characteristics primarily influenced satisfaction through trust.

Interaction effects of nonverbal behaviors and speech characteristics

Since the data did not follow a normal distribution, the Mann-Whitney U test was used to compare patient satisfaction differences between the group with speech characteristics and nonverbal behavior (coded as 1) and the other groups (coded as 0). The analysis results were shown in Table 6, indicating that the combination group’s satisfaction scores were significantly higher (p < .001). Specifically, participants in the group exhibiting a combination of painful facial expressions, open posture, slow speech rate, and high pitch had significantly higher satisfaction scores than those in the non-combination group (U=17598.000, Z=-3.442, p < .001). This result supported hypothesis H7.

Table 6.

Mann-Whitney U test results for the effect of behavioral combinations on patients’ satisfaction.

Group	Sample size (N)	Mean rank	Sum of ranks
Non-combination Group (coded = 0)	875	458.11	400848.00
Combination Group (coded = 1)	55	583.04	32067.00

Test statistic	Value
Mann-Whitney U	17598.000
Wilcoxon W	400848.000
Z value	-3.442
Asymptotic significance (2-tailed)	<.001

Based on all analyses, the results of hypothesis tests regarding the effects of nonverbal behaviors (facial expressions, body posture) and speech characteristics (speech rate, pitch) on patients’ satisfaction were summarized, and the mediating role of empathy and trust in this process was demonstrated, as shown in Table 7.

Table 7.

Hypothesis test results.

Hypothesis	Findings	Test/model used	Significance (p < .05)	Supported
H1a (Facial Expression → Satisfaction)	Facial expression had no significant effect on satisfaction.	Friedman test	Not significant (p = .302)	Not supported
H1b (Body Posture → Satisfaction)	An open posture significantly increased satisfaction.	Friedman test	Significant (p< .001)	Supported
H4a (Vocal Pitch → Satisfaction)	Vocal pitch had no significant effect on satisfaction.	Friedman test	Not significant (p = .295)	Not supported
H4b (Speech Rate → Satisfaction)	A low speech rate significantly increased satisfaction.	Kruskal–Wallis test & Dunn–Bonferroni	Significant (p = .034, only low vs high significant)	Supported
H2 (Nonverbal Behaviors^a→ Trust→ Satisfaction)	Trust played a full mediating role in the effect of nonverbal behaviors on satisfaction.	PROCESS Model 4	Significant (indirect effect: p < .001)	Supported
H3(Nonverbal Behaviors^a →Empathy → Trust → Satisfaction)	Empathy and trust played a full mediating role in the effect of nonverbal behaviors on satisfaction.	PROCESS Model 6	Significant (serial mediation: p < .001)	Supported
H5 (Speech Characteristics^b → Trust → Satisfaction)	Trust played a partial mediating role in the effect of speech characteristics on satisfaction.	PROCESS Model 4	Significant (indirect effect: p = .015)	Supported
H6 (Speech Characteristics^b → Empathy → Trust → Satisfaction)	Only trust was a significant mediator; empathy was not significant.	PROCESS Model 6	Partially significant (Trust: p = .014, Empathy: p = .240)	Only trust mediation supported
H7 (Combined Behaviors^c → Satisfaction)	Combined behaviors significantly increased satisfaction.	Mann-Whitney U test	Significant (p < .001)	Supported

^aNonverbal behaviors: Painful expression and open posture.

^bSpeech characteristics: Low speech rate and high pitch.

^cCombined behaviors: Painful expression, open posture, low speech rate and high pitch combination.

Discussion

This study systematically examined how nonverbal behaviors and speech characteristics of medical ECAs influenced patients’ satisfaction, focusing on perceived empathy and trust as mediators. This study found that the body posture of ECAs significantly affected patient satisfaction, whereas the facial expression’s main effect was insignificant. This result contrasted to some extent with established studies. Empirical studies such as Kraft-Todd et al.²⁷ and Forkin et al.⁴⁷ showed that open body posture significantly enhanced patients’ first impressions, trust and satisfaction with their physicians. The present study similarly found that patients reported higher levels of trust and satisfaction when the ECA adopted an open posture. In contrast, facial expression’s effect on patients’ satisfaction did not reach significant levels, which was not entirely consistent with the finding that smiling enhanced satisfaction in some studies.^39,40,109 A possible explanation is that body posture affects participants’ whole range perception.¹¹⁰ Future studies could utilize eye-tracking technology to more accurately capture the distribution of patients’ attention during interactions with ECAs.

Furthermore, when paired with an open posture, painful expressions considerably improved patients’ perceptions of ECAs empathy without lowering satisfaction. This finding was in line with the study by Marcoux et al.²² and Choi et al.,¹¹¹ which implied that the nonverbal combination of open postures and painful expressions was interpreted as a sign of greater empathy. Specifically, individuals with high empathy traits were more likely than people with low empathy to unconsciously display facial responses that matched the facial expressions of the observed subject.^112–116 This automatic facial consistency reflected emotional empathy with the observed subject.^117–119 For example, observers could feel another person’s pain and display matching painful expressions. Mediation analyses further revealed that the effect of nonverbal behaviors on satisfaction was fully mediated by trust, an essential antecedent of perceived empathy. This finding is further consistent with UTAUT-based models applied in digital health contexts, in which trust has been identified as a pivotal variable mediating the relationship between system-level features and adoption or satisfaction outcomes.^53,54 Notably, the role of social influence documented in AI system adoption studies² aligns with our finding that the social and emotional cues conveyed by ECAs such as open postures and empathic vocal characteristics function as social presence signals that strengthen trust and ultimately drive satisfaction.

The present study furtherly validated the interaction effects among different nonverbal behaviors to enhance their perceived empathy and trust of ECAs. In contrast, a single smiling facial expression that lacked support from other behaviors might instead be interpreted as unserious or out of place,¹²⁰ weakening its positive effect on satisfaction. Multidimensional combinations of nonverbal behaviors increased patients’ satisfaction more than single dimensions, and their path of action was primarily through influencing perceived empathy and trust.

Unlike earlier studies, pitch had no direct impact on patients’ satisfaction when it came to speech characteristics. Our study found that slower speech rate resulted in higher levels of satisfaction, inconsistent with the findings of Liu et al.’s⁷⁷ study, which found that the speed at which a doctor spoke in their questioning tone was positively correlated with patients’ satisfaction and that doctors who spoke at a high speed tended to have high patients’ satisfaction, while faster speech rate decreased patients’ satisfaction. For patients did not have enough time to receive and analyze the information, fast speech might diminish cognitive trust, making it more difficult for patients to understand and perhaps leading to reduced satisfaction.¹²¹

Although past studies have explored the effects of a single dimension of speech rate or pitch on patients’ trust and satisfaction,^76,77 the interaction effect between pitch and speech rate has not been systematically examined in healthcare contexts. In this study, we found that the combination featuring low speech rate and high pitch significantly enhanced patients’ trust in ECAs compared to other combinations, thereby increasing overall satisfaction. The serial mediation analysis also indicated that although this combination failed to enhance perceived empathy directly, it indirectly affected satisfaction through enhanced trust. Previous studies showed that low speech rate made utterances more explicit, reducing patients’ cognitive load and information anxiety.^85,86 Meanwhile, high pitch stimulated positive emotional responses, conveying rapport and warmth while making it easier for patients to establish social connections with physicians.⁷⁸ This combination helped to weaken the sense of distance caused by virtual identity and enhance the judgment of ECAs’ trust in virtual medical scenarios. Prior researches have shown that a lower pitch was often perceived as more authoritative and enhanced listeners’ trust in the speaker.^88,122 However, our findings suggested that patients’ trust when interacting with medical ECAs relied more on the emotional cues conveyed by the voice than on the power cues. Specifically, the combination of low speech rate and high pitch was more in line with the need to build social connection and emotional support during medical consultations, and had a greater impact on enhancing ECAs’ trust and satisfaction of ECAs. The interaction between pitch and speech rate was not only a tool for information transfer, but also constituted an essential tool for physicians’ emotional expression and social presence construction.

The findings of this study also revealed the importance of multimodal emotional integration. It has been shown that mandatory integration of emotional information from different sensory channels occurs and that this integration process was automatic and unconscious.^93,123 The theory of emotional congruence states that when facial expressions, body postures, and speech characteristics are emotionally congruent, it is easier for observers to perceive authenticity and trust.^93,94 This study also validated this theory, which found that combining nonverbal behaviors (painful expressions and open postures) with speech characteristics (low speech rate and high pitch) of ECAs significantly enhanced patient satisfaction. This study empirically confirmed that this combination of nonverbal behaviors and speech characteristics significantly impacted patient satisfaction in digital healthcare consultation, and future ECA design should fully consider combining the two to enhance patients’ experience.

Limitations and future research

While this study revealed how nonverbal behaviors and speech characteristics of ECAs in digital health settings affected patient satisfaction through empathy and trust, its scope of study also demonstrated several limitations. In terms of experimental design, this study used a relatively simplified medical consultation scenario in which participants watched video clips and assessed them. It was a single-exposure experimental design that may not fully reflect the complexity and dynamics of fundamental medical interactions. Moreover, nonverbal communication is an ongoing and dynamic process.¹²⁴ As Lal and Neduncheliyan⁵ pointed out, patients’ perceptions of virtual healthcare agents change as the duration of the interaction increases. Thus, future researches should adopt a longitudinal design to explore the persistence of the effects of virtual agents in long-term use situations. Additionally, all dependent variables were measured simultaneously at a single time point, which limits causal inference in the observed mediation pathways.

Another limitation is the use of questionnaires for measurement only. Although questionnaires provide essential results, they may not capture subconscious or transient impressions triggered by nonverbal behaviors. Future studies may introduce physiological measures such as heart rate variability, galvanic responses, or eye tracking that can provide a more nuanced understanding of patients’ emotional responses to physician cues.¹²⁵ The sample was drawn primarily from the university community, and the participants were mostly young students, which may have limited the external validity of the results. Since age, medical experience, and digital literacy can influence patients’ responses to medical ECAs,^126,127 and these variables were not included as covariates in the current analyses, future studies should both expand the sample and account for these demographic factors in statistical models.

Beyond methodological limitations, this study raises ethical concerns that warrant attention. While optimized combinations of nonverbal cues painful expressions, open postures, low speech rate, and high pitch enhance perceived empathy and trust, their persuasive effectiveness also carries risks. Overly convincing ECAs may foster uncritical patient reliance on AI agents, potentially impairing critical evaluation of medical information. The WHO’s ethics guidelines on AI for health caution that emotionally engaging systems can induce automation bias, wherein patients inappropriately defer medical judgment to automated agents.¹²⁸ Furthermore, ECAs that exhibit near-human behavioral characteristics may precipitate the Uncanny Valley Effect, which can paradoxically induce discomfort or evasive responses in certain users.¹²⁹ Bach and Männikkö furtherly emphasize that justified patient trust in AI healthcare must rest on transparency and human oversight rather than emotional cues alone.¹³⁰ Future work should therefore balance behavioral optimization with clear ethical boundaries—including safeguards against over-reliance, accurate information delivery, disclosure of artificial identity, and alignment with national and international AI ethics frameworks.^128,131

Conclusions

This study focused on the nonverbal behaviors and speech characteristics of medical ECAs in digital healthcare contexts, exploring how they influenced patients’ perceptions of empathy and trust, thus affecting patients’ satisfaction. Through a mixed experimental design and empirical analysis, the study revealed the independent effects of various behavioral cues and clarifies their interaction mechanisms and mediating pathways. Results showed that open body postures and low speech rate significantly improved patient satisfaction, whereas facial expressions and vocal pitch alone did not produce significant effects. Trust served as a key mediator in the relationship between ECAs’ behaviors and patient satisfaction, with empathy and trust jointly forming a serial mediation pathway. The findings indicated that combining ECAs’ nonverbal behaviors, specifically painful facial expressions and open body posture, with speech characteristics, such as low speech rate and high pitch, significantly enhanced patient satisfaction. This combination could lead to the highest level of patient satisfaction compared to other combinations, ultimately improving the patients’ consultation experience. As AI healthcare systems become more widespread, the emotional expression capabilities of virtual agents will become a key factor in influencing patients’ perceptions of empathy, trust, and satisfaction. The behavioral design framework and validated pathways proposed in this study provide theoretical guidance and design standards for developing future digital healthcare systems.

Supplemental material

Supplemental material - Effect of nonverbal behaviors and speech characteristics of online medical ECAs on patients’ satisfaction: The mediating roles of empathy and trust

Supplemental material for Effect of nonverbal behaviors and speech characteristics of online medical ECAs on patients’ satisfaction: The mediating roles of empathy and trust by Fei Fang, Xuanning Chen, Faren Huo, Xuanhui Liu in Digital Health.

Footnotes

Acknowledgements

We would like to acknowledge all of the participants and all of the researchers in this study.

ORCID iD

Fei Fang

Ethical considerations

Ethical approval was received from Ningbo University Ethics Committee (approval number: NBU-2025-483).

Consent to participate

Written informed consent was obtained from all participants prior to participation.

Author contributions

FF: Conceptualization, methodology, writing. XC: Methodology, writing—original draft preparation. FH: Writing—review & editing. XL: Analysis, writing—review & editing.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The study was funded by Zhejiang Provincial Philosophy and Social Sciences Planning Project (24NDQN027YB); Ningbo Natural Science Foundation (202003N4145); Major Humanities and Social Sciences Research Projects in Zhejiang higher education institutions (2024QN034); Fundamental Research Funds for the Provincial Universities of Zhejiang; Ningbo Philosophy and Social Sciences Planning Project (G2024-1-38).

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data Availability Statement

De-identified data will be made available upon reasonable requests to the corresponding author and with data use agreements in place.*

Guarantor

Fei Fang is the guarantor for this article and accepts full responsibility for the integrity and accuracy of the work.

Supplemental material

Supplemental material for this article is available online.

References

Cassell

(ed). Embodied conversational agents. MIT Press, 2000.

Al Atraqchi

OMA

Abdulmuhsin

Yaseen

, et al. Large language models and talent management: Developing RAG model in healthcare. In: Yaseen

(ed). Applied Artificial Intelligence in Business. Springer Nature Switzerland, 2024, pp. 197–218.

Provoost

Lau

Ruwaard

, et al. Embodied conversational agents in clinical psychology: A scoping review. J Med Internet Res 2017; 19: e151. https://doi.org/10.2196/jmir.6553

Jiang

Huang

Wang

, et al. Embodied conversational agents for chronic diseases: Scoping review. J Med Internet Res 2024; 26: e47134. https://doi.org/10.2196/47134

Lal

Neduncheliyan

. The evolution and potential of conversational agents in healthcare. In: Khurana

Thakur

Kantha

, et al. (eds). Machine Learning Algorithms. Springer Nature Switzerland, 2025, pp. 209–220.

Davenport

Kalakota

. The potential for artificial intelligence in healthcare. Future Healthcare J 2019; 6: 94–98. https://doi.org/10.7861/futurehosp.6-2-94

Keesara

Jonas

Schulman

. Covid-19 and health care’s digital revolution. N Engl J Med 2020; 382. https://doi.org/10.1056/NEJMp2005835. Epub ahead of print 4 June 2020.

Brands

Gouw

Beestrum

, et al. Patient-centered digital health records and their effects on health outcomes: Systematic review. J Med Internet Res 2022; 24: e43086. https://doi.org/10.2196/43086

Ramachandran

Brinton

Wiljer

, et al. The impact of eHealth on relationships and trust in primary care: A review of reviews. BMC Prim Care 2023; 24: 228. https://doi.org/10.1186/s12875-023-02176-5

10.

Alkhwaldi

. Understanding the acceptance of business intelligence from healthcare professionals’ perspective: An empirical study of healthcare organizations. IJOA 2024; 32: 2135–2163. https://doi.org/10.1108/ijoa-10-2023-4063

11.

Sundar

. Humanizing chatbots: The effects of visual, identity and conversational cues on humanness perceptions. Comput Hum Behav 2019; 97: 304–316. https://doi.org/10.1016/j.chb.2019.01.020

12.

Araujo

. Living up to the chatbot hype: The influence of anthropomorphic design cues and communicative agency framing on conversational agent and company perceptions. Comput Hum Behav 2018; 85: 183–189. https://doi.org/10.1016/j.chb.2018.03.051

13.

Laranjo

Dunn

Tong

, et al. Conversational agents in healthcare: A systematic review. J Am Med Inf Assoc 2018; 25: 1248–1258. https://doi.org/10.1093/jamia/ocy072

14.

Nundy

Montgomery

Wachter

. Promoting trust between patients and physicians in the era of artificial intelligence. JAMA 2019; 322: 497. https://doi.org/10.1001/jama.2018.20563

15.

Kaptein

De Ruyter

Markopoulos

, et al. Adaptive persuasive systems: A study of tailored persuasive text messages to reduce snacking. ACM Trans Interact Intell Syst 2012; 2: 1–25. https://doi.org/10.1145/2209310.2209313

16.

Ismail

NAS

Mageswaran

Bujang

, et al. Beyond words: Analyzing non-verbal communication techniques in a medical communication skills course via synchronous online platform. Front Med 2024; 11: 1375982. https://doi.org/10.3389/fmed.2024.1375982

17.

Lupetti

Hagens

Van Der Maden

, et al. Trustworthy embodied conversational agents for healthcare: A design exploration of embodied conversational agents for the periconception period at erasmus MC. In: Proceedings of the 5th International Conference on Conversational User Interfaces, Eindhoven Netherlands, 2023, pp. 1–14. ACM.

18.

Harrison McKnight

Chervany

. Trust and distrust definitions: One bite at a time. In: Falcone

Singh

Tan

Y-H

(eds). Trust in Cyber-societies. Springer Berlin Heidelberg, 2021, pp. 27–54.

19.

Davis

. Measuring individual differences in empathy: Evidence for a multidimensional approach. J Pers Soc Psychol 1983; 44: 113–126. https://doi.org/10.1037/0022-3514.44.1.113

20.

Mercer

Maxwell

Heaney

, et al. The consultation and relational empathy (CARE) measure: Development and preliminary validation and reliability of an empathy-based consultation process measure. Fam Pract 2004; 21: 699–705. https://doi.org/10.1093/fampra/cmh621

21.

Jin

Wang

. The relationship between the physician-patient relationship, physician empathy, and patient trust. J Gen Intern Med 2022; 37: 1388–1393. https://doi.org/10.1007/s11606-021-07008-9

22.

Marcoux

Tessier

M-H

Jackson

. Nonverbal behaviors perceived as most empathic in a simulated medical context. Comput Hum Behav 2024; 157: 108268. https://doi.org/10.1016/j.chb.2024.108268

23.

Halim

Foster

Ayala

, et al. The physiological nature of caring: Understanding nonverbal behavior. In: Foster

Yaseen

(eds). Teaching Empathy in Healthcare. Springer International Publishing, 2019, pp. 41–61.

24.

Hall

Harrigan

Rosenthal

. Nonverbal behavior in clinician—patient interaction. Appl Prev Psychol 1995; 4: 21–37. https://doi.org/10.1016/s0962-1849(05)80049-6

25.

Lorié

Reinero

Phillips

, et al. Culture and nonverbal expressions of empathy in clinical settings: A systematic review. Patient Educ Couns 2017; 100: 411–424. https://doi.org/10.1016/j.pec.2016.09.018

26.

Straßmann

Krämer

. A categorization of virtual agent appearances and a qualitative study on age-related user preferences. In: Beskow

Peters

Castellano

, et al. (eds). Intelligent Virtual Agents. Springer International Publishing, 2018, pp. 413–422.

27.

Kraft-Todd

Reinero

Kelley

, et al. Empathic nonverbal behavior increases ratings of both warmth and competence in a medical context. PLoS ONE 2017; 12: e0177758. https://doi.org/10.1371/journal.pone.0177758

28.

Mari

Mandelli

Algesheimer

. Empathic voice assistants: Enhancing consumer responses in voice commerce. J Bus Res 2024; 175: 114566. https://doi.org/10.1016/j.jbusres.2024.114566

29.

Knowles

Little

. Vocal fundamental and formant frequencies affect perceptions of speaker cooperativeness. Q J Exp Psychol 2016; 69: 1657–1675. https://doi.org/10.1080/17470218.2015.1091484

30.

Stirrat

Perrett

. Valid facial cues to cooperation and trust: Male facial width and trustworthiness. Psychol Sci 2010; 21: 349–354. https://doi.org/10.1177/0956797610362647

31.

Gao

Dai

Zhang

, et al. Trust in virtual agents: Exploring the role of stylization and voice. IEEE Trans Vis Comput Graphics 2025; 31: 3623–3633. https://doi.org/10.1109/TVCG.2025.3549566

32.

Hall

Horgan

Murphy

. Nonverbal communication. Annu Rev Psychol 2019; 70: 271–294. https://doi.org/10.1146/annurev-psych-010418-103145

33.

Harrigan

Oxman

Rosenthal

. Rapport expressed through nonverbal behavior. J Nonverbal Behav 1985; 9: 95–110. https://doi.org/10.1007/bf00987141

34.

Mast

. On the importance of nonverbal communication in the physician–patient interaction. Patient Educ Couns 2007; 67: 315–318. https://doi.org/10.1016/j.pec.2007.03.005

35.

Riess

Kraft-Todd

. E.M.P.a.T.H.Y: A tool to enhance nonverbal communication between clinicians and their patients. Acad Med 2014; 89: 1108–1112. https://doi.org/10.1097/ACM.0000000000000287

36.

Roter

Frankel

Hall

, et al. The expression of emotion through nonverbal behavior in medical visits. Mechanisms and outcomes. J Gen Intern Med 2006; 21: S28–S34. https://doi.org/10.1111/j.1525-1497.2006.00306.x

37.

Rosenthal-von Der Pütten

Straßmann

Yaghoubzadeh

, et al. Dominant and submissive nonverbal behavior of virtual agents and its effects on evaluation and negotiation outcome in different age groups. Comput Hum Behav 2019; 90: 397–409.

38.

Vugt

HCV

Bailenson

Hoorn

, et al. Effects of facial similarity on user responses to embodied agents. ACM Trans Comput-hum Interact 2010; 17: 1–27. https://doi.org/10.1145/1746259.1746261

39.

Otterbring

. Smile for a while: The effect of employee-displayed smiling on customer affect and satisfaction. JOSM 2017; 28: 284–304. https://doi.org/10.1108/josm-11-2015-0372

40.

Schneider

Zerbini

Reicherts

, et al. Smiling doctor, satisfied patient—the impact of facial expressions on doctor-patient interactions. Front Med 2025; 12: 1518517. https://doi.org/10.3389/fmed.2025.1518517

41.

Hall

Ruben

Swatantra . First impressions of physicians according to their physical and social group characteristics. J Nonverbal Behav 2020; 44: 279–299. https://doi.org/10.1007/s10919-019-00329-8

42.

Small

Verrochi

. The face of need: Facial emotion expression on charity advertisements. J Mark Res 2009; 46: 777–787. https://doi.org/10.1509/jmkr.46.6.777

43.

Decar

Conover

Misra

. Effective communication in the pre-operative environment. Intern J Otolaryngol Head Neck Surg 2020; 09: 1–7. https://doi.org/10.4236/ijohns.2020.91001

44.

Burgoon

. Relational message interpretations of touch, conversational distance, and posture. J Nonverbal Behav 1991; 15: 233–259. https://doi.org/10.1007/bf00986924

45.

Beck

Daughtridge

Sloane

. Physician-patient communication in the primary care office: A systematic review. J Am Board Fam Pract 2002; 15: 25–38.

46.

Carney

Cuddy

AJC

Yap

. Power posing: Brief nonverbal displays affect neuroendocrine levels and risk tolerance. Psychol Sci 2010; 21: 1363–1368. https://doi.org/10.1177/0956797610383437

47.

Forkin

Dunn

Kaperak

, et al. Influence of sex and body language on patient perceptions of anesthesiologists. Anesthesiology 2019; 130: 314–321. https://doi.org/10.1097/ALN.0000000000002527

48.

Benbasat

Wang

. Trust in and adoption of online recommendation agents. JAIS 2005; 6: 72–101. https://doi.org/10.17705/1jais.00065

49.

Wang

Benbasat

. Empirical assessment of alternative designs for enhancing different types of trusting beliefs in online recommendation agents. J Manag Inf Syst 2016; 33: 744–775. https://doi.org/10.1080/07421222.2016.1243949

50.

Reeves

Nass

. The media equation: How people treat computers, television, and new media like real people and places. 1. paperback ed. CSLI Publications, 1998.

51.

Adam

Wessel

Benlian

. AI-based chatbots in customer service and their effects on user compliance. Electron Markets 2021; 31: 427–445. https://doi.org/10.1007/s12525-020-00414-7

52.

Huang

Markovitch

Stough

. Can chatbot customer service match human service agents on customer satisfaction? An investigation in the role of trust. J Retail Consum Serv 2024; 76: 103600. https://doi.org/10.1016/j.jretconser.2023.103600

53.

Dbesan

Abdulmuhsin

Alkhwaldi

. Adopting knowledge-sharing-driven blockchain technology in healthcare: A developing country’s perspective. VJIKMS 2025; 55: 683–709. https://doi.org/10.1108/vjikms-01-2023-0021

54.

Abdulmuhsin

Dbesan

Rehman

, et al. Redefining knowledge-generation-driven blockchain for healthcare use: Insights from medical institutions. Health Serv Manage Res 2026; 39: 37–54. https://doi.org/10.1177/09514848251358332

55.

Bar

Neta

Linz

. Very first impressions. Emotion 2006; 6: 269–278. https://doi.org/10.1037/1528-3542.6.2.269

56.

Van Der Biest

Cracco

Riva

, et al. Should I trust you? Investigating trustworthiness judgements of painful facial expressions. Acta Psychol (amst) 2023; 235: 103893. https://doi.org/10.1016/j.actpsy.2023.103893

57.

Henry

Fuhrel-Forbis

Rogers

MAM

, et al. Association between nonverbal communication during clinical interactions and outcomes: A systematic review and meta-analysis. Patient Educ Couns 2012; 86: 297–315. https://doi.org/10.1016/j.pec.2011.07.006

58.

Seitz

Bekmeier-Feuerhahn

Gohil

. Can we trust a chatbot like a physician? A qualitative study on understanding the emergence of trust toward diagnostic chatbots. International Journal of Human-Computer Studies 2022; 165: 102848. https://doi.org/10.1016/j.ijhcs.2022.102848

59.

Ersöz

Nissen

Schütte

. Risk, trust, and emotion in online pharmacy medication purchases: Multimethod approach incorporating customer self-reports, facial expressions, and neural activation. JMIR Formative Research 2023; 7: e48850. https://doi.org/10.2196/48850

60.

Håkansson Eklund

Summer Meranius

. Toward a consensus on the nature of empathy: A review of reviews. Patient Educ Couns 2021; 104: 300–307. https://doi.org/10.1016/j.pec.2020.08.022

61.

Asada

. Towards artificial empathy: How can artificial empathy follow the developmental pathway of natural empathy? Int J of Soc Robotics 2015; 7: 19–33. https://doi.org/10.1007/s12369-014-0253-z

62.

Murphy

. Introduction to AI robotics. 2nd ed. MIT Press, 2019.

63.

Devoldre

Davis

Verhofstadt

, et al. Empathy and social support provision in couples: Social support and the need to study the underlying processes. J Psychol 2010; 144: 259–284. https://doi.org/10.1080/00223981003648294

64.

Preston

De Waal

FBM

. Empathy: Its ultimate and proximate bases. Behav Brain Sci 2002; 25: 1–20. https://doi.org/10.1017/s0140525x02000018

65.

Davenport

Guha

Grewal

, et al. How artificial intelligence will change the future of marketing. J of the Acad Mark Sci 2020; 48: 24–42. https://doi.org/10.1007/s11747-019-00696-0

66.

Bänziger

Patel

Scherer

. The role of perceived voice and speech characteristics in vocal emotion communication. J Nonverbal Behav 2014; 38: 31–52. https://doi.org/10.1007/s10919-013-0165-x

67.

Bendapudi

Berry

Frey

, et al. Patients’ perspectives on ideal physician behaviors. Mayo Clin Proc 2006; 81: 338–344. https://doi.org/10.4065/81.3.338

68.

Kerasidou

. Artificial intelligence and the ongoing need for empathy, compassion and trust in healthcare. Bull World Health Organ 2020; 98: 245–250. https://doi.org/10.2471/BLT.19.237198

69.

Walsh

O’Neill

Hannigan

, et al. Patient-rated physician empathy and patient satisfaction during pain clinic consultations. Ir J Med Sci 2019; 188: 1379–1384. https://doi.org/10.1007/s11845-019-01999-5

70.

Bartsch

. “what sounds beautiful is good?” how employee vocal attractiveness affects customer’s evaluation of the voice-to-voice service encounter. In: Stauss

(ed). Aktuelle Forschungsfragen im Dienstleistungsmarketing. Gabler, 2007, pp. 45–68.

71.

Lowe

Haws

. Sounds big: The effects of acoustic pitch on product perceptions. J Mark Res 2017; 54: 331–346. https://doi.org/10.1509/jmr.14.0300

72.

Gray

Michael

Hoffmeister

, et al. Patient satisfaction with virtual vs in-person voice therapy. J Voice 2024; 38: 1088–1094. https://doi.org/10.1016/j.jvoice.2022.03.011

73.

Wang

T-Y

Kawaguchi

Kuzuoka

, et al. Effect of manipulated amplitude and frequency of human voice on dominance and persuasiveness in audio conferences. Proc ACM Hum Comput Interact 2018; 2: 1–18. https://doi.org/10.1145/3274446

74.

Ong

LML

De Haes

JCJM

Hoos

, et al. Doctor-patient communication: A review of the literature. Soc Sci Med 1995; 40: 903–918. https://doi.org/10.1016/0277-9536(94)00155-m

75.

Chen

Guo

, et al. Exploring the online doctor-patient interaction on patient satisfaction based on text mining and empirical analysis. Inf Process Manage 2020; 57: 102253. https://doi.org/10.1016/j.ipm.2020.102253

76.

Liu

Zhang

Gao

, et al. Physician voice characteristics and patient satisfaction in online health consultation. Inf Manage 2020; 57: 103233. https://doi.org/10.1016/j.im.2019.103233

77.

Liu

Gao

. Which voice are you satisfied with? Understanding the physician–patient voice interactions on online health platforms. Decis Support Syst 2022; 157: 113754. https://doi.org/10.1016/j.dss.2022.113754

78.

Liu

Jiang

. Exploring the effects of online physician voice pitch range and filled pauses on patient satisfaction in mobile health communication. Health Communication 2024; 39: 1–14. https://doi.org/10.1080/10410236.2024.2313791

79.

Montepare

Zebrowitz-McArthur

. Perceptions of adults with childlike voices in two cultures. J Exp Soc Psychol 1987; 23: 331–349. https://doi.org/10.1016/0022-1031(87)90045-x

80.

Pogacar

Shrum

Lowrey

. The effects of linguistic devices on consumer information processing and persuasion: A language complexity × processing mode framework. J Consum Psychol 2018; 28: 689–711. https://doi.org/10.1002/jcpy.1052

81.

Ramakrishnan

El Emary

IMM

. Speech emotion recognition approaches in human computer interaction. Telecommun Syst 2013; 52: 1467–1478. https://doi.org/10.1007/s11235-011-9624-z

82.

Goldman-Eisler

. The determinants of the rate of speech output and their mutual relations. J Psychosom Res 1956; 1: 137–143. https://doi.org/10.1016/0022-3999(56)90015-0

83.

Levelt

WJM

. Speaking: From intention to articulation. The MIT Press, 1989.

84.

Cheng

, et al. Scoring metrics of assessing voiceprint distinctiveness based on speech content and rate. IEEE Trans Depend Sec Comput 2024; 21: 1–18. https://doi.org/10.1109/tdsc.2024.3380603

85.

Scherer

. Vocal communication of emotion: A review of research paradigms. Speech Commun 2003; 40: 227–256. https://doi.org/10.1016/s0167-6393(02)00084-5

86.

Knowlton

Larkin

. The influence of voice volume, pitch, and speech rate on progressive relaxation training: Application of methods from speech pathology and audiology. Appl Psychophysiol Biofeedback 2006; 31: 173–185. https://doi.org/10.1007/s10484-006-9014-6

87.

Maltezou-Papastylianou

Scherer

Paulmann

. How do voice acoustics affect the perceived trustworthiness of a speaker? A systematic review. Front Psychol 2025; 16: 1495456. https://doi.org/10.3389/fpsyg.2025.1495456

88.

Klofstad

Anderson

Peters

. Sounds like a winner: Voice pitch influences perception of leadership capacity in both men and women. Proc R Soc B 2012; 279: 2698–2704. https://doi.org/10.1098/rspb.2012.0311

89.

Tigue

Borak

O’Connor

JJM

, et al. Voice pitch influences voting behavior. Evol Hum Behav 2012; 33: 210–216. https://doi.org/10.1016/j.evolhumbehav.2011.09.004

90.

Montano

Tigue

Isenstein

SGE

, et al. Men’s voice pitch influences women’s trusting behavior. Evol Hum Behav 2017; 38: 293–297. https://doi.org/10.1016/j.evolhumbehav.2016.10.010

91.

Luangrath

Peck

Barger

. Textual paralanguage and its implications for marketing communications. J Consum Psychol 2017; 27: 98–107. https://doi.org/10.1016/j.jcps.2016.05.002

92.

McHenry

Parker

Baile

, et al. Voice analysis during bad news discussion in oncology: Reduced pitch, decreased speaking rate, and nonverbal communication of empathy. Support Care Cancer 2012; 20: 1073–1078. https://doi.org/10.1007/s00520-011-1187-8

93.

De Gelder

Vroomen

. The perception of emotions by ear and by eye. Cogn Emot 2000; 14: 289–311. https://doi.org/10.1080/026999300378824

94.

Wang

Gratch

. Don’t just stare at me! In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Atlanta Georgia USA, 2010, pp. 1241–1250. ACM.

95.

Föcker

Gondan

Röder

. Preattentive processing of audio-visual emotional signals. Acta Psychol (amst) 2011; 137: 36–47. https://doi.org/10.1016/j.actpsy.2011.02.004

96.

Mondloch

Nelson

Horner

. Asymmetries of influence: Differential effects of body postures on perceptions of emotional facial expressions. PLoS ONE 2013; 8: e73605. https://doi.org/10.1371/journal.pone.0073605

97.

Oleszkiewicz

Pisanski

Lachowicz-Tabaczek

, et al. Voice-based assessments of trustworthiness, competence, and warmth in blind and sighted adults. Psychon Bull Rev 2017; 24: 856–862. https://doi.org/10.3758/s13423-016-1146-y

98.

Spooner

Taylor-Jones

Canessa-Pollard

. Vocal behaviour affects the expression of empathy in trained and untrained speakers. British Journal of Guidance & Counselling 2025: 1–12. https://doi.org/10.1080/03069885.2025.2470429

99.

Rahsepar Meadi

Sillekens

Metselaar

, et al. Exploring the ethical challenges of conversational AI in mental health care: Scoping review. JMIR Ment Health 2025; 12: e60432. https://doi.org/10.2196/60432

100.

Appelbaum

Cooper

Kline

, et al. Journal article reporting standards for quantitative research in psychology: The APA publications and communications board task force report. Am Psychol 2018; 73: 3–25. https://doi.org/10.1037/amp0000191

101.

Lee

Cashin

Lamb

, et al. A guideline for reporting mediation analyses of randomized trials and observational studies: The AGReMA statement. JAMA 2021; 326: 1045–1056. https://doi.org/10.1001/jama.2021.14075

102.

Goodman

Mayhorn

. It’s not what you say but how you say it: Examining the influence of perceived voice assistant gender and pitch on trust and reliance. Appl Ergon 2023; 106: 103864. https://doi.org/10.1016/j.apergo.2022.103864

103.

Faul

Erdfelder

Buchner

, et al. Statistical power analyses using G*power 3.1: Tests for correlation and regression analyses. Behav Res Methods 2009; 41: 1149–1160. https://doi.org/10.3758/BRM.41.4.1149

104.

McKnight

Choudhury

Kacmar

. Developing and validating trust measures for e-commerce: An integrative typology. Inf Syst Res 2002; 13: 334–359. https://doi.org/10.1287/isre.13.3.334.81

105.

Guo

Zhang

Sun

. The privacy–personalization paradox in mHealth services acceptance of different age groups. Electron Commer Res Appl 2016; 16: 55–65. https://doi.org/10.1016/j.elerap.2015.11.001

106.

Paiva

Leite

Boukricha

, et al. Empathy in virtual agents and robots: A survey. ACM Trans Interact Intell Syst 2017; 7: 1–40. https://doi.org/10.1145/2912150

107.

Campbell

Fiske

. Convergent and discriminant validation by the multitrait-multimethod matrix. Psychol Bull 1959; 56: 81–105.

108.

Hayes

. Introduction to mediation, moderation, and conditional process analysis: A regression-based approach. The Guilford Press, 2013.

109.

Greene

Adelman

Friedmann

, et al. Older patient satisfaction with communication during an initial medical encounter. Soc Sci Med 1994; 38: 1279–1288. https://doi.org/10.1016/0277-9536(94)90191-0

110.

Grün

Heibges

Westfal

, et al. “you never get a second chance”: First impressions of physicians depend on their body posture and gender. Front Psychol 2022; 13: 836157. https://doi.org/10.3389/fpsyg.2022.836157

111.

Choi

Park

Loeser

, et al. Improving counseling effectiveness with virtual counselors through nonverbal compassion involving eye contact, facial mimicry, and head-nodding. Sci Rep 2024; 14: 506. https://doi.org/10.1038/s41598-023-51115-y

112.

Dimberg

Andréasson

Thunberg

. Emotional empathy and facial reactions to facial expressions. J Psychophysiol 2011; 25: 26–31. https://doi.org/10.1027/0269-8803/a000029

113.

Holland

O’Connell

Dziobek

. Facial mimicry, empathy, and emotion recognition: A meta-analysis of correlations. Cogn Emot 2021; 35: 150–168. https://doi.org/10.1080/02699931.2020.1815655

114.

Riess

. Biomarkers in the psychotherapeutic relationship: The role of physiology, neurobiology, and biological correlates of E.M.P.A.T.H.Y. Harv Rev Psychiatry 2011; 19: 162–174. https://doi.org/10.3109/08941939.2011.581915

115.

Rymarczyk

Żurawski

Jankowiak-Siuda

, et al. Empathy in facial mimicry of fear and disgust: Simultaneous EMG-fMRI recordings during observation of static and dynamic facial expressions. Front Psychol 2019; 10: 701. https://doi.org/10.3389/fpsyg.2019.00701

116.

Sonnby-Borgström

Jönsson

Svensson

. Emotional empathy as related to mimicry reactions at different levels of information processing. J Nonverbal Behav 2003; 27: 3–23. https://doi.org/10.1023/a:1023608506243

117.

Prochazkova

Kret

. Connecting minds and sharing emotions through mimicry: A neurocognitive model of emotional contagion. Neurosci Biobehav Rev 2017; 80: 99–114. https://doi.org/10.1016/j.neubiorev.2017.05.013

118.

De Waal

FBM

Preston

. Mammalian empathy: Behavioural manifestations and neural basis. Nat Rev Neurosci 2017; 18: 498–509. https://doi.org/10.1038/nrn.2017.72

119.

Olszanowski

Wróbel

Hess

. Mimicking and sharing emotions: A re-examination of the link between facial mimicry and emotional contagion. Cogn Emot 2020; 34: 367–376. https://doi.org/10.1080/02699931.2019.1611543

120.

Hildenbrand

Perrault

Switzer

. Smiling versus resting B**ch face: Patients’ evaluations of male and female healthcare providers’ facial expressions. Commun Q 2023; 71: 390–413. https://doi.org/10.1080/01463373.2023.2205025

121.

Yang

Zhou

. Modulating your speech rate: The effect of speech rate on crowdfunding performance. Electron Commer Res Appl 2022; 56: 101211. https://doi.org/10.1016/j.elerap.2022.101211

122.

Tsantani

Belin

Paterson

, et al. Low vocal pitch preference drives first impressions irrespective of context in male voices but not in female voices. Perception 2016; 45: 946–963. https://doi.org/10.1177/0301006616643675

123.

Gao

Wedell

Shinkareva

. Evaluating non-affective cross-modal congruence effects on emotion perception. Cogn Emot 2021; 35: 1634–1651. https://doi.org/10.1080/02699931.2021.1973966

124.

Burgoon

Manusov

Guerrero

. Nonverbal communication. 2nd ed. Routledge, 2021.

125.

Melamed

Jackson

. Psychometric instruments: Potential benefits and practical use. Ind Commer Train 1995; 27: 11–16. https://doi.org/10.1108/00197859510087569

126.

Pak

McLaughlin

Bass

. A multi-level analysis of the effects of age and gender stereotypes on trust in anthropomorphic technology by younger and older adults. Ergonomics 2014; 57: 1277–1289. https://doi.org/10.1080/00140139.2014.928750

127.

Seo

. When female (male) robot is talking to me: Effect of service robots’ gender and anthropomorphism on customer satisfaction. Int J Hosp Manag 2022; 102: 103166. https://doi.org/10.1016/j.ijhm.2022.103166

128.

Ethics and governance of artificial intelligence for health: WHO guidance. 1st ed. World Health Organization, 2021.

129.

Cihodaru-Ștefanache Ștefania Podina

. The uncanny valley effect in embodied conversational agents: A critical systematic review of attractiveness, anthropomorphism, and uncanniness. Front Psychol 2025; 16: 1625984. https://doi.org/10.3389/fpsyg.2025.1625984

130.

Bach

Männikkö

. The importance of justified patient trust in unlocking AI’s potential in mental healthcare. Front Hum Dyn 2025; 7: 1519872. https://doi.org/10.3389/fhumd.2025.1519872

131.

Mennella

Maniscalco

De Pietro

, et al. Ethical and regulatory challenges of AI technologies in healthcare: A narrative review. Heliyon 2024; 10: e26297. https://doi.org/10.1016/j.heliyon.2024.e26297

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

27.88 MB