Sage Journals: Discover world-class research

Abstract

Background and objectives

Communication is a complex skill and key component for professionals in healthcare. Complex skills are best learned through experiential learning like role-plays or simulated patient encounters. The aim of the present study is to explore how students and lecturers assess the conditions under which the use of an artificial intelligence (AI)-based feedback system can promote the learning process.

Methods

An interview study with health profession students and lecturers was conducted using a qualitative descriptive design. Recorded audio data was transcribed and evaluated by structuring qualitative content analysis. The research process was conducted and continually reflected by an interprofessional research team. Ethical approval was obtained.

Results

Using qualitative content analysis, 4 major themes were identified. These are “conditions,” “communication scenarios,” “AI-based learning platform” and “debriefing.” Lecturers and students welcome the idea of AI providing output on verbal and para-verbal aspects. To implement AI-based output into a teaching programme AI functionality should be adaptable to the specific situation. Lecturers and students highlighted that AI could be particularly valuable for speech qualities which are often difficult for humans to assess. AI could give freedom to focus on additional aspects of the conversation by documenting desirable speech qualities. Lecturers and students prefer for the AI-based output to be given at the end of rather than within the role-play. Furthermore, they wish for communication scenarios to be analysed repeatedly in order to track progress.

Conclusion

Practicing in a safe environment and receiving competent credible feedback, with lecturers trained in facilitation, is a pre-requisite for the entire learning progress. The integration of an AI-based feedback system should be characterised by both flexibility of the AI application and standardisation of the communication scenario.

Keywords

AI acceptance communication debriefing experiential learning healthcare interviews learning platform speech recognition

Introduction

Communication skills are essential for healthcare professionals’ ability to deliver patient-centred care. Empathetic conversations foster positive relationships between healthcare professionals and patients¹ and contribute to the success of therapy.² The best way to acquire communication skills is to train in a simulated environment as complex skills like communication are best learned through experiential learning.³ Evidence suggests that communication skills training improves students’ communication abilities^4,5 and equips them with enhanced communication skills for clinical practice.⁶

As a result, simulation-based medical education has gained prominence in healthcare profession programmes, including those at the University of Lübeck. The University of Lübeck offers 9 health degree programmes, which provides an ideal environment for patient-oriented and interprofessional learning, as well as the acquisition of communication skills. Across all healthcare programmes, various teaching modules include active training in communication skills. These modules emphasise self-awareness, involving role-plays and exercises based on case studies, with actors simulating patients or interprofessional counterparts. The role-play scenarios or simulated patient encounters vary in scope and content across different healthcare programmes, encompassing emergency situations, explaining pain mechanisms to patients, and communicating with patients who have dementia. Over the past decades, a considerable number of students and lecturers have participated in communication training and gained relevant experience. Both groups – particularly experienced lecturers – possess a substantial body of practical knowledge, which includes the facilitation of debriefings. There is a consensus in literature that these ‘learning conversations’⁷ following the communication scenarios are a key element in enabling trainees to reach their full learning potential.^7-9 This is the research context of the current project LABORATORIUM in which an interdisciplinary team is presently developing artificial intelligence (AI)-supported individual learning assistance hereafter called CoSy (Communication System). AI-supported systems are increasingly making inroads into education, as evidenced by the growing body of research in this domain.¹⁰ AI is defined as ‘the theory and development of computer systems able to perform tasks normally requiring human intelligence, such as visual perception, speech recognition, decision making and translation between languages’.¹¹ A valuable application of AI lies in the field of communication skills training^12,13 – as applied in CoSy. CoSy addresses the fact that debriefing is a highly demanding task and aims to contribute to deepening this important step in communication training. CoSy will measure specific communication parameters from the audio data concerning linguistic characteristics and make them available for feedback discussions and debriefing. The idea is that CoSy will provide information from the conversations between students. This in turn allows the actors involved, lecturers and students, to concentrate on other aspects of communication. CoSy is, at this point, an imaginary system. In contrast to communication platforms like SimConverse^©, which uses interactive AI characters to play the role of any patient or colleague, or ChatGPT, users won't be able to engage in conversations with CoSy. CoSy will be a speech recogniser, able to give output to verbal (what did you say) and para-verbal (how did you say it) foundations of communication. To the best of our knowledge, communication skills training in higher education incorporating a speech-recogniser during debriefing has not been reported. A preliminary version of this article appeared as a preprint.¹⁴

The project LABORATORIUM comprises several phases. The analysis presented in this article preceded the ELSI workshops (Ethical, Legal and Socio-cultural Implications) and serves as the foundation for the development of CoSy. It is widely acknowledged that technology providers should align their developments with user intentions in order to enhance acceptance.¹⁵ Accordingly, the primary objective was to define criteria for a system that is both highly usable and capable of delivering a satisfactory user experience, thereby facilitating the effective integration of CoSy into educational settings.

To support this objective, the present study explores the experiences and perceptions of both students and lecturers regarding the potential implementation of an AI-based feedback system. In line with the goals of a comprehensive requirement analysis, this study aims to gain insights into users’ perspectives on possible applications of the system, as well as their expectations for its use –particularly within emotionally challenging learning environments.

From the students’ and lecturers’ perspectives

What are the required pre-requisites for integrating an AI-based learning platform into communication training in healthcare programmes?

How should the communication scenarios be designed?

Which speech qualities are considered useful when integrating an AI-based learning platform?

In what way should the output of the AI-based learning platform be didactically embedded in the communication situation?

Methods

Design: This interview study used a qualitative, descriptive design to explore students’ and lecturers’ experiences and perceptions of communication training. The research was conducted and continuously reflected by an interprofessional research team. In addition, the procedure and the results of the data analysis were presented, discussed and reflected within the project advisory board consisting of experts from research and practice. In order to ensure the rigour and trustworthiness of the study the recommended standards for reporting qualitative research (SRQR) were followed.¹⁶

Sample and recruitment: Participants were recruited by specifically addressing lecturers and students (purposive sampling). An a priori determination of the sample structure aimed to reflect the diversity of the field of research and to maximise the variations of perspectives. Lecturers and students from every single healthcare programme, which holds communication as a central part of the teaching, were included. Students must have previously participated in practical communication exercises during their studies and attend the corresponding healthcare programmes. Lecturers of the healthcare programmes must have experience in communication education. The sample must also include lecturers and students from mono-professional as well as interprofessional modules. Information about the study, its aims and data protection regulations concerning the interviews were provided to potential participants via email.

Data collection: On the premise that all interviewees were aware of the study procedure and provided their written consent, data collection was conducted on focus groups of health profession students and individual interviews with health profession lecturers. These diverse methodological approaches were chosen to properly address the different groups. Therefore, individual interviews were conducted with experts (lecturers) while interactive discussion among students was made possible through focus groups. The interviews were based on 2 semi-structured interview guides, 1 for lecturers (individual interviews) and 1 for students (focus groups). Both interview guides were developed based on the research questions and discussions with the interprofessional research team. In an initial expert workshop, in the form of a brainstorming session, a list of speech qualities, like ‘terminology’, ‘length of pauses’ or ‘adjusting one's volume depending on the patient’ were compiled. Both interview guides included questions on the same 4 central themes: (1) current communication scenarios in teaching and characteristics of communication, (2) current debriefing for the development of communication skills in the health professions, (3) the use of an AI-based learning platform in communication situations and in relation to speech qualities and (4) conditions for integrating the AI-based learning platform. The guidelines included open questions or narrative invitations. The wordings for lectures and students were for example: ‘What are your wishes for feedback by an AI-based learning platform after a conversation with a patient?’ Or: ‘What additional benefits could an assistance system bring to teaching?’ In order to explore participants’ views on the speech qualities, they were asked towards the end of the interviews to rate which of these speech qualities an AI should provide feedback on. For this purpose, the participants were asked to assign previously prepared cards with speech qualities to the categories ‘very important’, ‘important’ and merely ‘nice-to-have’. The interview guide also included an explanation to give interviewees a realistic description of CoSy and what it will actually be capable of. The wording was as follows: ‘In many everyday appliances, like voice-controlled assistance systems in cars, and entertainment programmes artificial intelligence is already integrated to give feedback. The game Sing-Star^© is an example of what feedback on voice pitch can look like. (Sing Star^© is a karaoke game that analyses a singers timing and pitch which is then compared to the original track.) Imagine that there is an assistance system, that systematically evaluates conversations and speech behaviour. The following questions focus on what is spoken in a situation and about speech qualities on which the assistance system can provide feedback’.

Prior to the start of data collection, the interview guide was pretested in 2 interviews, and minor adjustments were made, especially regarding the order of the questions and the openness of the wording. Data from these 2 pilot interviews were used solely to refine the interview guide and were not included in the subsequent data analysis. Socio-demographic data of the participants was collected using a structured data collection sheet. The interviews took place between May and August 2022. To create a trusting atmosphere and avoid socially desirable answers, focus groups were conducted by members of the interprofessional research team from differing programmes (including the authors HB, AD and MMK). All interviews were audio-recorded. Following the data collection, the interviewers prepared a protocol (interview postscript).

Data analysis: The interviews and focus groups were transcribed verbatim according to the rules of Dresing and Pehl.¹⁷ In order to maintain confidentiality, it was ensured that even context-related narratives of the interviewees did not allow any identification. The interviewer was consulted in case of uncertainties within the transcription process. During the transcription, the data was anonymised. The analysis was based on structural content analysis¹⁸ using the software MAXQDA 22 (VERBI Software, 2021). A structuring qualitative content analysis allows the development of central themes within a data-based deductive-inductive approach. The data analysis for this article specifically focused on AI-related findings from the interviews. The interviews were analysed separately in the first step. In order to develop an initial understanding of the data, interviews were read carefully and intensively, taking into account the research questions (initiating text work). Using a deductive approach, a preliminary category system was developed for the main categories based on the central themes of the interview guidelines. Data was coded with these main categories. Subcategories were then formed inductively from the data collected during the initial interviews. The procedure consisted of reading each interview line by line. Codes were formed directly from the text. These codes were systematised and grouped into subcategories. Definitions and anchor examples were listed for each category. These definitions and examples provided guidance in the event of discrepancies in the evaluation process. Subsequently, the entire dataset was analysed using this category system. During the data analysis the category system was adapted and refined. Coding was initially performed in teams of 2 researchers (HB, AD, MMK, KR) to ensure a consistent approach. Disagreements between 2 researchers were resolved by discussion in the research team until consensus was reached. The remaining data was then coded individually and cross-checked by a third researcher. Afterwards, the results of the analysis were contrastively related to each other (perspective triangulation) and relationships and connections between the topics were identified using the constant comparative method.¹⁹ These interrelations between the themes were visualised. The results of the analysis were continuously discussed and reflected upon by the entire research team.

Results

Twenty-seven health professionals participated in this study: 7 lecturers and 20 students (Table 1). A total of 8 health teaching modules (mono- and interprofessional) and 7 health professions, respectively: medicine with 3 teaching modules: general practice (M1), emergency medicine (M2) and medical psychology (M3), and nursing sciences, midwifery sciences, occupational therapy, physiotherapy, and speech and language therapy as well as clinical psychology (each represented in at least one teaching module). In addition to the medical modules, there were also the following mono-professional modules: the physiotherapy module on pain therapy (M4) and the midwifery module on communication and counselling (M5). The paediatrics (M6) and dementia module (M7) were interprofessional. Clinical psychology (M8) is a future module in a new study programme in which no teaching took place at the time of data collection. It was therefore not possible to conduct a focus group or expert interview. CoSy is to be used in this module in the future. There were no focus groups for modules M1 and M3 as there were no students willing to participate. For every other module a focus group was conducted in which students, who had already completed the module, took part and an expert interview was held with a lecturer, who taught the module.

Table 1.

Characteristics of Study Population.

	LECTURERS N = 7(7 INDIVIDUAL) INTERVIEWS)	STUDENTS N = 20(5 FOCUS GROUPS)
Gender, women (N)	5	17
Age M (min-max)	44 (33-52)	24.5 (21-41)
Health profession
Medicine (general practice)	1	-
Medicine (emergency medical care) M”	1	5
Medicine (medical psychology)	1	-
Midwifery	1	3
Nursing	1	5
Occupational therapy	1	3
Physiotherapy	1	4
Clinical psychotherapy	-	-
Speech and language therapy	-	-
Teaching experience, years M (min-max)	13.0 (6-23)	-
Number of semesters, M (min-max)	-	6 (4-14) missing = 2
Clinical experience, years M (min-max)	12 (0-20)	2.25 (.42-16) missing = 1

The 7 individual interviews with the lecturers lasted an average of 71 min (ranging from 59 to 101 min), while the 5 focus group interviews lasted an average of 112 min (ranging from 95 to 130 min). Overall, a total of 17 h and 35 min of audio has been recorded.

Results in this article specifically focus on AI-related findings and revealed 4 major themes which were addressed in the interviews and focus groups. The 4 major themes are ‘conditions’, ‘communication scenarios’, ‘AI-based learning platform’ and ‘debriefing’ (Figure 1). The analysis of the interviews shows that the participants have confidence that the AI output will have the potential to deepen the learning process and to complement the debriefing. This shows that CoSy is not perceived as an alien piece of the puzzle. The optimal framework conditions and pre-requisites for the communication training became apparent. They are summarised under the topic ‘conditions’. Analysis shows that the following presented 3 individual parts ‘communication scenarios’, ‘AI-based learning platform’, and ‘debriefing’ influence each other and must be well harmonised.

Figure 1.

Identified Major Themes.

Conditions

The participants assume that the use of CoSy has an impact on the achievement of communication skills. They have the understanding that CoSy can record and analyse what is being said. According to the interviewees, CoSy needs to have information about a conversation in order to be able to analyse what is being said in a dialogue. Examples of important contextual information was mentioned, for example, the age of the patient, the room in which the conversation takes place and the time frame.

KL5-Lecturer: “Does it [AI] need information about the speakers? (…) so it should know who the patient is and who the therapist is (..) I would think (..) it probably needs information whether it is a man or a woman (..) because something like pitch has a large range of variance but probably differs. […].” #00:45:31-9#

KL6-Student2: “Maybe also an overview of the setting, so if we imagine a practice, in which room is it carried out, what are the room conditions and what is in front of it, behind it? If there are perhaps colleagues in a motor activity room and it is super loud, then of course I have to speak up. […].” #0:58:51.0#

Assumed added value: In addition, many interviewees assume that CoSy can give more comprehensive feedback on communication in comparison to the lecturers and peers, as it can record and process more data than humans. Students and lecturers highlighted that CoSy could be particularly valuable for speech qualities which are often difficult for humans to assess. Participants think beyond face-to-face teaching and imagine usability in many other settings such as self-study in the SkillsLab or at home on a computer. The students also imagine the application in practical everyday clinical settings – carrying CoSy in their pocket.

KL2-Student3: “Well, what I was just thinking about, which would also be cool, is if you simply had the chance to go back to the ward after university, if there are patients who voluntarily take part in it. […] you go alone or in pairs to the patient's room, have conversations and record them, […] that you have it for yourself as feedback. […].” #01:34:31-4#

Nature of the data: As in many technology or AI-based cases, the interviewees assume that CoSy will provide objective and neutral feedback. It is expected that the AI output states facts about communication that are less subjective than feedback from lecturers or fellow students. This presumed objective view of a conversation seems to be an important perspective for students to add.

KL2-Lecturer: “(…) I see a neutral evaluation of situations. Um. Highly standardised. With the possibility of being able to focus on other things in the situations simultaneously. And to create a certain evaluability.” 00:58:22-4#

KL6-Lecturer: “That has something objective about it, doesn't it? Because otherwise you have the feeling that I have used a lot of technical terms or the students report back and then it was still only three times, but somehow it felt very dominant. (…) We know this ourselves, when we conduct an interview, you think you have left huge pauses in the interview situation, but when you transcribe it, it was a maximum of one minute and it felt very, very long in the interview. […].” #00:51:55-8#

Time and teaching organisation: Lecturers are aware that the use of CoSy will require workload outside the classroom. They assume a higher time requirement because they expect more intensive preparation and follow up for skills training. Lecturers see a need for preparatory training with the practical application, like handling the user interface and the didactic implementation, which includes the decisions when and how to discuss the AI output. Both lecturers and eventually students need to train in the application of the AI system. Lecturers must be instructed in the use of the system outside of teaching. For the students, sensitising instruction should take place within the framework of the respective teaching module.

KL5-Lecturer: “Yes, I think (.) at least the lecturers should be trained (.), what they can do and what they can't do, so that you know where you can use it [the AI] well, so where you see your own possibilities. I think it would be nice if you could shift part of the teaching to this SkillsLab. […] and that in the self-learning time that belongs to each module you can then also motivate (..) to practise this situation. So I believe that in teaching, when it is here on campus (…) the lecturers must first be activated. They are certainly obliged to (…) really use it (…) and not to find it funny or (…) to be afraid of the technology.” #01:10:00-0#

KL4-Lecturer: “But what you can already gather is that there is still a moment when a human being has to be there to pick up the AI again. Someone who also says, what does this mean now […]. At least for the first time. […] Um, and then you can let the student train independently. […].” #00:50:21-4#

Students and lecturers favour a reasonable degree of data sovereignty and control over the data. Students like the data to be stored in order to use CoSy output to show long-term trends in their learning processes.

KL6-Student2: “Maybe also (…) that once you have received feedback, that there is a follow-up situation or several, that you can also compare things, […] or has something improved or not and if not, why not. So that you can use that again. That this AI is simply used several times in situations.” #1:32:18.7#

Rooms and equipment: For various steps, such as the first interaction of lecturers and students with the AI system, but also for the later skills training with CoSy itself, certain premises are needed. These include technical equipment as well as suitable materials and furniture for different scenarios.

KL8-Lecturer: “So, you can also (…) / you could also play it more abstractly in a seminar room, of course. But then it certainly loses some of its authenticity and perhaps makes it more difficult for the students to put themselves in this situation and imagine how they would react. And also for the actors, that's how we perceive it, it's also helpful to have a few CORE utensils with them. So it doesn't have to be the full programme but the bag, the bed, the pyjamas um yes.” #0:10:57.1#

Communication Scenarios

This theme outlines the essential pre-requisites and preferences of interviewees regarding the design of communication scenarios in general. Additionally, the limitations perceived by the interviewees concerning the use of CoSy are described.

Range of content and themes: Participants identified several communication scenarios that could potentially be enhanced with AI support, including anamnesis, goal-setting and counselling interviews, emergency management and interprofessional communication situations like handovers and case discussions. What they all have in common is that they are largely standardised and follow a structured conversational format.

KL1-Lecturer: “Yes, the scenario must be clearly defined in some way, like / it should be, for example / it should be a feedback conversation, and in a feedback conversation, we define / need to include these specific aspects. And these aspects can be verbally represented by the following words or something like that. […].” #00:48:11-2#

In contrast to standardised situations, there are more complex situations, such as scenarios primarily involving manual techniques and procedural workflows, or face-to-face patient-care scenarios. In these scenarios, there is an expectation for more variability in the content of the conversation and therefore it is more difficult to specify which terms or aspects need to be mentioned. Often, in the more complex situations, the atmosphere of the conversation and feelings matter more than the specific content. The flow of these conversations is less predictable and exhibits greater variability in each individual scenario. In these cases, meaningful AI output seems to be only limitedly conceivable.

KL8-Lecturer: “Only very, very limited scenarios where it might be about structuring a conversation or A LOT OF INFORMATION / uh, for example, using the SBAR [Situation, Background, Assessment and Recommendation] -framework. […] I could imagine it for highly standardised situations, which actually exclude almost every classic patient-care situation.” #00:49:04#

Composition of actors: The communication training sessions used in Lübeck are structured as one-on-one scenarios, with 2 possible conversation configurations. In some cases, one student acted in their own profession and the other as a patient or a patient's relative. In most instances, a professional (paid) actor portrayed the patient's role to create a more realistic atmosphere. The background information on their role is known to the actors in both options, which is particularly important to bring the scenario close to reality.

KL6-Student2: “I'm thinking of this situation, the patient is in bed as an old woman in need of care and I'm supposed to motivate her to eat breakfast or something. […] And she didn't make it easy for me. I really had to think a bit about “how do I do this now?” […] And of course, it's completely different if someone you know was lying there. Then you might lose the actual role. (…) That's what I would have liked to see more of throughout my studies, […] And this professional actress also did it very well (laughs). Yes.” #0:45:45.3#

In the interests of realism, lecturers and students prefer actors (standardised patients). However, this is only the case for adult patients who are played by actors. Lecturers and students criticise the connection to reality in children as patients; these cannot be staged by adults, regardless of whether they are actors or fellow students. At this point, it can be mentioned that the participants are again thinking beyond the planned capabilities of CoSy and imagined AI-controlled avatars as communication partners alongside actors.

KL7-Lecturer: “Maybe that would be a form where something could be done. That means that they have different forms of avatars that have certain information that is important or that is not important. […].” #01:05:41#

Overall, students had good experiences with both options, actors and fellow students, in the role of patients. Beyond communication skills, it can help students to step into the patient role and experience that side of the interaction. There were no mentions made of the potential for real patients to volunteer for training purposes.

Frequency and location of training: As aforementioned, especially students describe the learning of communication skills as a process with a need for repetition. Interviewees consider CoSy suitable to be used throughout the learning process and thus in every session.

KL6-Student2: “And that it's also used regularly, that […] that it's somehow a continuous part of it. […]” #1:53:44.7#

KL5-Lecturer: “I'd prefer shorter situations, practicing them more frequently.” #01:04:55-4#

Students shared and reflected their experiences due to the COVID-19 pandemic-related restrictions. There was a clear consensus that communication training in attendance is seen as more beneficial. Benefits of conducting in-person sessions include a better group atmosphere and a more realistic scenario setup, as nonverbal aspects of communication cannot be adequately expressed in an online setting.

KL8-Student3: “I found the online communication quite challenging because in communication, there's a lot of body language involved. […] I found it quite difficult to sit at home and try to do something, […] I think body language is an important tool in communication, which was completely absent. […].” #00:15:53-2#

AI-based Learning Platform – CoSy

The ideas on what special functions and characteristics CoSy must have, what CoSy should look like, and how the system should be accessed in detail vary. There is an agreement that it must be accessible to both students and lecturers.

Features: The categorisation of speech qualities, on which participants would like to have AI output, into ‘very important’, ‘important’ or merely ‘nice-to-have’, was part of the interviews and focus groups. The 7 lecturers were given some time to categorise each of the proposed features during their interviews (Figure 2). In the focus groups, the categorisation process was designed with greater flexibility. Initially, students categorised the speech qualities individually before discussing their choices in the group. Some qualities were perceived as largely irrelevant and were therefore not categorised by the students at all. Additionally, parts of the categorisation were conducted in pairs, which led to variation in the number of categorised qualities compared to the lecturers (Figure 3). In both figures, the qualities are presented in descending order of priority.

Figure 2.

On Which Speech Qualities Would You Like to Have Artificial Intelligence (AI) Feedback – Lecturer.

Figure 3.

On Which Speech Qualities Would You Like to Have Artificial Intelligence (AI) Feedback – Students.

The need to repeat communication training creates requirements for the function of CoSy. Students and lecturers suggest that CoSy should document the speech qualities and the development of learning over time. CoSy output should be stored to enable students to compare their CoSy output parameters of current communication with CoSy output parameters of previous communication situations. This additionally implies the existence of a user profile, so that the students gain access to their data. This proposal applies to both the comparison of the own individual performance and the comparison with a reference group. Interviewees suggest that the reference group should consist of students from the same profession or students who have a similar level of learning (from beginners to experts).

KL8- Lecturer: “It would certainly be good if the student could somehow save it, especially when he wants to practice it again, so that you can see a progression. I would also like that.” #00:50:37#

Design: For the CoSy output there are different forms imaginable, which seem almost contradictory. On the one hand, students and lecturers envision neutral and nonjudgemental feedback. This form of feedback should be as quantitative as possible including facts and figures. On the other hand, evaluative feedback is envisioned classifying into desirable and undesirable or using a traffic light system, for example, which would relieve lecturers and students of the task of interpretation. Furthermore, they envision that CoSy output uncovers gaps in the conversation and reports back if something in the communication is debatable. In this context CoSy output could point out suggestions for a better performance. A physical output could be a visual or textual print of the feedback.

KL4-Lecturer: “Yes, it is so exciting with transcripts, how people talk. And sometimes you're not even aware of it, because you pay attention to many other things […]. That's very conspicuous when it's there in black and white.” #00:38:44-9#

Overall, the results show that lecturers and students would like CoSy to be clearly structured and designed, allowing an intuitive navigation. Although it is imaginable in principle that the function of CoSy is to provide output on a wide range of speech qualities, participants prefer the selection of specific speech qualities. A suggested function is to be able to select the desired speech qualities individually directly before the communication situation, as not all speech qualities are considered important for feedback in every single situation. This would narrow down the feedback in terms of content.

KL6-Student2: “Maybe you select certain topics for feedback, which are then only a topic and which are then only reported back. So that it's not too much.” #1:36:12.9#

Timing: The analysis revealed that both students and lecturers are accustomed to post-event debriefing. Providing within-event feedback appears to be uncommon. Students address speech qualities that can be provided as feedback during the scenario, such as volume or speech rate, if defined in advance. In addition, within-event feedback also requires a reference value, such as ‘too loud’ or ‘too quiet’, to directly adapt one's own speech behaviour and thus creating added value. Even though feedback during the communication situation is conceivable, post-event debriefing is clearly favoured overall.

KL2-Student5: “So, I think I would be distracted during that. I mean, that's a good point about the volume, then I can react to it in the moment. But I believe, for me, it would be better to receive it afterward.”

KL2-Student3: “I would also say, in the end is actually better because otherwise, during the conversation, you would be constantly preoccupied with what this system is telling me right now. It shifts the focus away from the conversation.” #01:20:38-3#

Debriefing

Role-play scenarios are commonly framed by briefing and debriefing phases as stated by Eppich and Cheng²⁰ or Zigmont.²¹ In this current article the ‘learning conversations’⁷ following the role-play scenarios are called ‘feedback’ or ‘debriefing’ interchangeably as the interviewees did not differentiate between these terms. Debriefing is considered important by the interviewees, also quite independently of CoSy, and takes up a large part of the narratives. Students in particular share many of their experiences.

Facilitation: According to the participants of this study, debriefing should be structured using a manual ‘feedback guide’ (for students) or an ‘observation guide’ (for lecturers), which includes aspects to be observed and questions to promote learning. In preparation for this conversational phase, participants also describe the need to review and establish rules or conversation principles. They pointed out that conversation rules should be considered both in the communication scenario and in the debriefing.

KL4-Lecturer: “Feedback rules, the normal standard, that you always learn. To always convey I-messages, how do things affect me, what impression does it make, how can I imagine that it could perhaps be brought into a more optimised direction. And [I] give concrete examples […] also give concrete ideas on how it could have been packaged differently if necessary. […]” #00:11:54-9#

KL1-Student: “I could imagine that with the help of a tool like this, you wouldn't forget anything and would also emphasise such things more, […] so yes, exactly, maybe that wouldn't be such a bad idea, to get a little guide for reflection of communication.” #00:48:12-4#

Even though self-reflection of both simulation participants was addressed in some examples, the interviews mainly dealt with the lecturers’ responsibility and their (desired) behaviour. All in all, interviews show that the guiding through the debriefing phase is mostly the responsibility of the lecturers. Students expect lecturers to navigate them through the debriefing and to focus on learning objectives. A self-guided facilitation did not emerge as a central theme.

Learning strategy: All in all, participants have an idea of the requirements of debriefing and its process elements. It often comes through in the interviews how important and formative this phase is, especially for the students. Interdisciplinary feedback is rated as extremely helpful by the students; one lecturer also emphasises that peer feedback is more effective than feedback from the lecturers. The analysis also confirms that debriefing is a critical component in the process of learning. Even though students stated a positive learning attitude in this context, it comes with little surprise that they also described negative experiences. Too little time for debriefing was discussed. It was mentioned that the debriefing took place afterwards ‘in the corridor’ with the consequence that conversation did not go as deep as it was expected. Additionally, there was mention of a performance paramount. Students compared their conversational performance with each other, thinking about who was better and who was worse. Students said that they felt more evaluated and judged by lecturers than constructively criticised. Signs that the learning environment was no longer perceived as safe are expressions such as ‘lucky to come out of it’ or feeling ‘paraded’.

KL8-Student2: “Well, I still remember that we always had very little time for the feedback and I was often somehow still in the situation before and then the next situation was already going on. So that somehow things were brought up and then not finished […].” #00:30:48-4#

KL2-Student3: “[…] I remember, for example, that some of the professional drama patients gave feedback; saying ‘Yes, there was something I didn't like, but I can't really say what’. […] then you get ‘I didn't like something’ […], and you can't handle that, because you don't have anything to start with, to improve on.” #00:28:07-1#

Attitude: The requirements for rules of conversation and the negative experiences already indicate that participants, lecturers as well as students, expect an attitude of appreciation. In this context, students indirectly point out once again the processual nature of gaining communication skills. They suggest that lecturers should state that students’ communication does not have to be perfect. This is complemented well by the attitude of the lecturers who indicate that they should encourage students and stimulate processes of reflection.

KL8-Student: “That perhaps it is also communicated – by the lecturers – that not everything has to be perfect at the end of the training, because, a lot still, yes, comes afterwards. So there will be many situations that I have not yet experienced in my training and that I cannot respond perfectly to every situation […].” #00:55:07-7#

Implementation of AI output: Lecturers and students welcome the idea of CoSy providing feedback on verbal and para-verbal foundations of communication. For the didactic implementation, the interviews show that there are speech qualities that are considered as highly relevant. But students do not consider themselves as passive recipients of feedback. They state that they want to be involved in the feedback process actively and they phrase that they do not want to be left alone with feedback. Students point out, that CoSy output should not be used to evaluate performance. It must be interpreted and discussed, which requires time. In terms of content, the lecturers and students would like to talk about concrete ideas for improving performance and to identify gaps in professional knowledge, such as forgotten anamnesis modules or treatment errors.

KL1-Student1: “But for example in the choice of words there can be alternative suggestions, so to say: ‘look, you have used these words, there are possibly better alternatives’ um (…), so I imagine that is quite feasible.” #01:01:33-9#

KL8-L: “[…] “The terms that occur in this case were missing”, for example, somehow pain, visual analogue scale, delirium, assessment, that would be helpful, I think. […].” #00:49:04#

In addition to this imaginable added value of the CoSy output, it also becomes clear that not all speech qualities are considered highly relevant. Students are less interested in feedback on the pitch of their voice. They do however wish for feedback on nonverbal communication such as gestures, facial expressions, and body language. Overall, CoSy output alone is not sufficient, but is a piece of the puzzle.

Discussion

The interviews have given insight into the experiences and perceptions of students and lecturers in communication training in higher education. The 4 research questions were answered in varying degrees of depth. There are clear indications that the acceptance of CoSy depends on specific framework conditions that affect the entire communication skills training. The acceptance particularly depends on how the CoSy output will be handled. (It is a tool that must be applied in accordance with its intended use.) Overall, the participants are very open to the use of CoSy as a supplementary tool in communication skills training. Without stating it directly and from the perspective of quantitative criteria, the ideas and perceptions of the interviewees show that participants assume that the CoSy output will meet the quality criteria of objectivity and reliability. The issues raised by the interviewees are therefore not related to the quality of the CoSy output, but mainly to the nature of the output and the framework in which the output is embedded.

The first question, ‘What are the required prerequisites for integrating an AI-based learning platform into communication training in healthcare programmes?’ led to 3 main requirements. These requirements can be simply interpreted to mean that CoSy is an addition and not a replacement. Firstly, the social interaction with a demand for an appreciative and respectful attitude. As the most mentioned and experienced communication scenarios were lecturer-guided, the attitude of the lecturers rather than that of the students was the subject of discussion. These finding is in accordance with previous research literature since the most used and most studied method for simulation debriefing is the facilitator-guided method and not the self-guided method.²² Students expressed a desire for emotional support from their lecturers, including encouragement when their communication skills are still developing. This aligns with empirical findings, as numerous studies have consistently demonstrated that learning processes are positively affected when students perceive their lecturers as supportive, empathetic and committed to their academic success.²³

Secondly, the analysis shows that participants wish to be guided through the debriefing process. This was already mentioned while talking about the communication scenarios but was demanded more clearly for the debriefing phase. Our results are consistent with literature which stated that it is important that debriefers or facilitators be aware of how they communicate and which questions they ask to promote optimal learning.^24,25 The interviewees themselves have mentioned the idea of using a guideline during the debriefing. A debriefing tool, a scripted guide, seems to be an optimal tool to help to transform experience into learning. It would support both, the high level of facilitation by following a defined structure and a high student-centred reflection by using (preformulated) phrases. Students in our interviews provide valuable insights and clearly demand that they want to be involved in the process and participate actively. This agrees with theory which describes that reflection is an essential learning component of debriefing²⁶ and that students should be iteratively engaged in the reflective cognitive work.²⁷

Thirdly, even though an evaluation of the reported speech qualities by CoSy would be possible, and was mentioned as an idea in the interviews, the interpretation of the data, the cognitive work, should be carried out by the participants. It is much more likely that communication training becomes even more demanding when the implementation of CoSy output is added since this introduces objective data. It is not a question of whether one perceived a speech rate to be fast or slow, because the speech rate can now be quantified. This could lead to taking the CoSy output as a diagnosis, as a fact which does not need to be discussed. Against the background of the interviews, however, CoSy output should be interpreted more as an instrument that gives the participants clues. Just as an X-ray gives clues about a fracture, the CoSy output gives clues about our speech behaviour. However, in both examples, the way one deals with the information received must be interpreted individually: What do I do with the data supplied, how do I interpret it? Is my fast speech rate appropriate or do I want to take care and speak more slowly in similar situations in the future? It seems to be all about the right questions. By exchanging and engaging with other people's knowledge and thinking, ones’ views and the perception of the communication situation can be broadened, with CoSy output being a piece of knowledge. Lecturers and students emphasise that CoSy is an additional and complementary tool and cannot replace existing communication training due to the non-verbal and emotional aspects in communication. These are highly complex and individual, so that, according to the interviewees, corresponding AI-based derivations alone are viewed sceptically. We assume that CoSy output can be used to deepen and to stimulate the discussion and a stimulating discussion in turn leads to the lecturer being able to retreat. Fanning and Gaba²⁸ already formulated this paradox our interviewees described: A high level of facilitation implies a low level of involvement by the facilitator.

To answer the second question, many elements of the design of the communication scenarios could be identified. The participants had positive experiences with both actors and peers, but in the interest of realism, lecturers and students prefer the involvement of actors. For the communication situations themselves, the integration of the CoSy is more easily imaginable in standardised situations which are more predictable in their structure. Closely related to this, participants state the need to repeat communication training sessions to make skills development visible. Furthermore, standardised situations would have the advantage of higher comparability, when learning trajectories are to be presented idiosyncratically.

This in turn presupposes a capability on the part of CoSy, namely the data storage, which leads to many ethical questions like to whom does the data belong, how long and where should it be stored? In this context, the question of who has access to the CoSy output must be answered not only for data storage in the long run, but also be answered directly for the didactic embedding during the debriefing phase. Do all involved, peers, lecturers and actors, have access to the output, does the individual output belong to the actors or is the lecturer the one who introduces the output in a suitable place and puts it up for discussion during debriefing? These are questions that need to be discussed in the future ELSI workshops. The interviews showed that the experiences to date were more lecturer-guided than peer-guided. In this current study mainly lecturers were the ones keeping an eye on the target. This suggests that the CoSy output in the debriefing process should be provided by the lecturers. On the other hand, the students emphasised that they want to be actively involved in the process, from which it could be concluded that the actors themselves decide which speech qualities should be discussed in the debriefing and when. Although experiences were mentioned and shared by the lecturers, information on applied methods was vague. Very little mention was made on how the CoSy output should be didactically embedded in the communication training (question 4). This question could not be answered sufficiently and needs to be sensitively tried out in practice and evaluated in future training sessions.

The amount of time needed for the whole training depends on different factors. This is an important aspect, as elements that were repeatedly rated negatively by students were time constraints. An experienced lecturer may be highly efficient in the appropriate timing and use of questions in contrast to a novice. Lecturers might place different emphasis on self-guided discussions in contrast to lecturer focused discussions, with learner-centred discussions requiring more time. Overall, communication training needs time. As aforementioned CoSy is seen as an additional piece of the puzzle. For the framework of future skills training with CoSy, interpretation of interviews leads to the assumption that more time will be needed at various points for teaching and for courses themselves. For this, it cannot be assumed that the use of CoSy will save time, quite the contrary.

The thought of conversation optimisation by CoSy reflects the high level of openness towards CoSy implementation. This, in connection with the desire and demand for a safe learning environment, means that care must be taken, especially on the part of lecturers, to handle this output responsibly. The implementation of CoSy requires high quality and reliability of AI analysis. To prevent the use of false data and produce a best didactical result CoSy must quantify and explain its output reliably. It is recommended to integrate cognitive forcing functions that provoke analytical thinking.²⁹ The CoSy output should be used to track the learning objective and to report back relevant speech qualities. The lecturers should see themselves as co-learners. CoSy output should not be used, and this was clearly stated by students, to measure performance.

Even though the interviewees were asked about their concerns they expressed few reservations about the use of CoSy. We did not expect this, as a global study in the trust in AI has documented that many people feel ambivalent about the use of AI, reporting optimism and excitement as well as fear and worry.³⁰ Therefore, the future ELSI workshops should be mentioned again at this point. They will be conducted to identify and discuss the specific challenges and risks in relation to the opportunities and the potential of CoSy. Speech contains a lot of information that provides insight into the speaker's inner state, mood, goals and motives.³¹ CoSy will only be able to reproduce a limited amount of this information. This is partly because CoSy will hold more information on textual aspects than on emotional aspects. The combination of both research steps, the currently described requirement analysis and the ELSI workshops, will provide a more complete picture on how students and lecturers perceive the risks of CoSy.

Factors, that influence user acceptance and ultimately determine the actual use of CoSy, were addressed to varying degrees. Based on the interviews, it can be assumed that students as well as lecturers believe that training with CoSy will improve students’ performance. The Technology Acceptance Model (TAM) and the Theory of Acceptance and Use of Technology (UTAUT) are the most prevailing models in explaining what affects the acceptance of various healthcare technologies.¹⁵ The UTAUT describes the factor Performance Expectancy,³² the TAM Perceived Usefulness.³³ The TAM is based on the assumption that 2 main factors influence the acceptance of a technology: The aforementioned Perceived Usefulness, that is, if users believe that a technology will help them to perform their tasks better, they are more willing to accept it. Secondly, the Perceived Ease of Use, that is if users believe that a technology is easy to learn and use, they are more likely to accept it.³³ According to the UTAUT user-friendliness leads to greater acceptance, summarised under the term Effort Expectancy. Additionally, the factors Social Influence and Facilitating Conditions influence user acceptance and the use of technologies. Resources and support that students and lecturers need to be able to use CoSy effectively, such as technical support, were mentioned in the interviews (Facilitating Conditions). Social Influence, for example, whether other colleagues or fellow students will use CoSy, remains to be seen. These factors must be considered and evaluated not only in the current development phase of CoSy, but also in its practical use.

The interviews provided valuable information to understand which expectations the interviewees have for the use and implementation of CoSy. The future use of CoSy will show which of the reported framework conditions and subcategories will be particularly important in communication training. It is possible that a hierarchical structure will emerge that did not yet become clear in the interviews. Some subcategories could fade into the background, while others become more important.

The study has several limitations. The results reflect the perspectives of lecturers and students at the University of Lübeck and are applicable to this context. They cannot be transferred to other universities without limitations. Purposive sampling and sampling maximal variation¹⁹ allowed for the participation of faculty and students from a variety of disciplines. However, due to the different levels of development of the healthcare programmes, not all perspectives could be considered. At the time of data collection, the clinical psychology and psychotherapy programme was still in the process of being established, so no experience of the students could be drawn on. Likewise, the perspective of the speech and language therapy students is missing. Overall, the recruitment and scheduling of appointments for focus groups with the students proved to be challenging. The reason for this was probably the high workload in the healthcare programmes and the number of examinations during the period of data collection. As a result, only 3 to 5 students participated in the focus groups, which possibly limited the interactions between them. In addition, only students with positive attitudes towards communication training may have participated, so negative attitudes are underrepresented in the sample. To support critical voices in the discussions the interviewers were from another faculty. The interviewed lecturers had a broad background of experience in teaching. Again, no lecturer from the clinical psychology and psychotherapy and speech and language therapy programmes could be included.

At the time of data collection, students and lecturers had no experience with the integration of an AI-based platform into teaching. The identified perspectives and attitudes of the interviewees are based on their imaginations. Especially the students thought beyond the prelims. They imagined interaction with CoSy, assuming that it would have the characteristics of a voice chatbot or conversational agent, by expressing the wish that it could make suggestions for conversational optimisation or by providing an avatar. This goes beyond the planned capabilities of CoSy as a learning platform and speech recogniser without the capability of response in a natural way. It is possible that more detailed and concrete settings for the integration of the AI-based learning platform would be described if practical experience had already been gained.

Overall, the sample showed diversity in teaching and study experience, age and clinical expertise. The structuring qualitative content analysis¹⁸ enabled a systematic approach, the identification of central themes and the interrelationships of the categories. Strengths of the analysis are the systematic linking of perspectives through triangulation and the analysis by an interprofessional team. Individual pre-assumptions and category development were continuously reflected and discussed in the interprofessional team. In addition, the results were presented and reflected several times in the entire team and in the scientific advisory board of the project LABORATORIUM. Students are also members of the advisory board; unfortunately, a member checking¹⁹ with the students participating in the data collection was not possible due to organisational reasons. Data saturation was neither achieved nor intended as the objective of the study. Instead, the focus was gathering the perspectives of students and lectures from the modules where CoSy will be implemented in the next phase of the project. Further qualitative surveys are planned in the project to collect the experiences of lecturers and students in the use of CoSy in communication training.

Conclusions

By implementing CoSy output in an appropriate way it seems to be able to support the process of learning and to enrich the communication skill training. Lecturers are key to students learning in simulation training, but facilitating through debriefing is not a linear, one-way distribution of learning. As the skill of the debriefer is of great importance in ensuring the best learning experience, lecturers should receive facilitation training. A tool for structured debriefings should be used to facilitate the debriefing phase and sufficient time must be available for debriefing sessions. Although the participants, especially the lecturers, see great potential for contributions to reflective learning, concrete ideas for methodological embedding need to be developed in the future. More homogeneity in the curricula and evaluation of communication training should be sought in the future. This will allow for a better understanding of how communication skills training works.

Supplemental Material

sj-docx-1-mde-10.1177_23821205251358089 - Supplemental material for Student and Lecturer Perceptions of the Use of an AI to Improve Communication Skills in Healthcare: An Interview Study

Supplemental material, sj-docx-1-mde-10.1177_23821205251358089 for Student and Lecturer Perceptions of the Use of an AI to Improve Communication Skills in Healthcare: An Interview Study by Hanna Brodowski, Anna Dammermann, Muriel Marieke Kinyara, Martina Andrea Obst, Fabian Samek, Corinna Peifer and Katharina Roese in Journal of Medical Education and Curricular Development

Footnotes

ORCID iDs

Hanna Brodowski

Katharina Roese

Ethical Considerations and Consent to Participate

This study was conducted in accordance with the Declaration of Helsinki and approved by the ethics committee of the University of Lübeck (number 2022-408). Informed consent to participate was obtained from all the participants for study. Participants had the opportunity to withdraw their written consent for the interview at any time. The participating students were compensated with 20 €.

Author Contributions

HB was the major contributor in writing the manuscript, recruited participants, conducted interviews, analysed, and interpreted the data. AD contributed to the writing of the results, recruited participants, conducted interviews, analysed, and interpreted the data. MMK contributed to the writing of the results, recruited participants, conducted interviews, analysed, and interpreted the data. MAO contributed to the writing of the introduction, critically revised the manuscript, and interpreted the data. FS and CP critically revised the manuscript and interpreted the data. KR conceived the original idea and supervised this step of the project, contributed to the writing of the methods, analysed and interpreted the data, and critically revised the manuscript. The manuscript has been read and approved by all the authors. All the authors have contributed substantially to the work presented in this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work is funded by the German Federal Ministry of Education and Research (BMBF) and the regional government of the state of Schleswig-Holstein, Germany under project number 16DHBKI075.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data Availability Statement

The anonymised dataset used and/or analysed during the current study is available from the corresponding author on reasonable request.

Supplemental Material

Supplemental material for this article is available online.

References

Howick

Moscrop

Mebius

, et al. Effects of empathic and positive communication in healthcare consultations: a systematic review and meta-analysis. J R Soc Med. 2018;111(7):240-252.

Horvath

Del Re

Flückiger

Symonds

. Alliance in individual psychotherapy. Psychotherapy. 2011;48(1):9-16.

Maureen

. A practical guide for teachers of communication skills: a summary of current approaches to teaching and assessing communication skills. Educ Prim Care. 2007;18(1):1-10.

Liu

Rohrer

Luo

Fang

Xie

. Doctor–patient communication skills training in mainland China: a systematic review of the literature. Patient Educ Couns. 2015;98(1):3-14.

Smith

Hanson

Tewksbury

, et al. Teaching patient communication skills to medical students: a review of randomized controlled trials. Eval Health Prof. 2007;30(1):3-21.

Moral

Andrade-Rosa

Molina

Barba

De Torres

Martín

. Effectiveness of a communication skills training program for medical students to identify patients communicative clues. Patient Educ Couns. 2020;103(11):2384-2387.

Tavares

Eppich

Cheng

, et al. Learning conversations: an analysis of the theoretical roots and their manifestations of feedback and debriefing in medical education. Acad Med. 2020;95(7):1020-1025.

Karnieli-Miller

. Reflective practice in the teaching of communication skills. Patient Educ Couns. 2020;103(10):2166-2172.

Taha

. Feedback in an epidemic? Cureus J Med Sci. 2022;14(2):e22008.

10.

Ouyang

Zheng

Jiao

. Artificial intelligence in online higher education: a systematic review of empirical research from 2011 to 2020. Educ Inf Technol. 2022;27(6):7893-7925.

11.

Knowles

. The Oxford dictionary of phrase and fable. 2nd ed. Oxford University Press; 2005.

12.

Almansour

Soliman

Aldekhyyel

Binkheder

Temsah

Malki

. An academic viewpoint (2025) on the integration of generative artificial intelligence in medical education: transforming learning and practices. Cureus. 2025;17(3):e81145.

13.

Asif

Gouqing

. Innovative application of artificial intelligence in a multi-dimensional communication research analysis: a critical review. Discov Artif Intell. 2024;4(1):37. Available from: https://link.springer.com/10.1007/s44163-024-00134-3 .

14.

Brodowski

Dammermann

Kinyara

, et al. Student and lecturer perceptions of the use of an AI to improve communication skills in healthcare: an interview study. Preprint. 2024; Available from: https://www.researchsquare.com/article/rs-4225671/v1.

15.

AlQudah

Al-Emran

Shaalan

. Technology acceptance in healthcare: a systematic review. Appl Sci. 2021;11(22):10537. Available from: https://www.mdpi.com/2076-3417/11/22/10537.

16.

O’Brien

Harris

Beckman

Reed

Cook

. Standards for reporting qualitative research: a synthesis of recommendations. Acad Med. 2014;89(9):1245-1251.

17.

Dresing

Pehl

. 2015. Manual (on) transcription: transcription conventions, software guides and practical hints for qualitative researchers.

18.

Kuckartz

Rädiker

. Qualitative content analysis: methods, practice and software. Sage; 2023.

19.

Flick

. An introduction to qualitative research. 7th ed. Sage; 2023.

20.

Eppich

Cheng

. Promoting excellence and reflective learning in simulation (PEARLS): development and rationale for a blended approach to health care simulation debriefing. Simul Healthc J Soc Simul Healthc. 2015;10(2):106-115.

21.

Zigmont

Kappus

Sudikoff

. The 3D model of debriefing: defusing, discovering, and deepening. Semin Perinatol. 2011;35(2):52-58.

22.

Sawyer

Eppich

Brett-Fleegler

Grant

Cheng

. More than one way to debrief: a critical review of healthcare simulation debriefing methods. Simul Healthc J Soc Simul Healthc. 2016;11(3):209-217.

23.

Zhang

. Exploring the relationships among teacher–student dynamics, learning enjoyment, and burnout in EFL students: the role of emotional intelligence. Front Psychol. 2024;14:1329400.

24.

Husebø

Dieckmann

Rystedt

Søreide

Friberg

. The relationship between facilitators’ questions and the level of reflection in postsimulation debriefing. Simul Healthc J Soc Simul Healthc. 2013;8(3):135-142.

25.

Kihlgren

Spanager

Dieckmann

. Investigating novice doctors’ reflections in debriefings after simulation scenarios. Med Teach. 2015;37(5):437-443.

26.

Nagle

Foli

. Student-centered reflection in debriefing: a concept analysis. Clin Simul Nurs. 2020;39:33-40.

27.

Kolb

. Experiential learning: experience as the source of learning and development. 2nd ed. Pearson Education LTD; 2015.

28.

Fanning

Gaba

. The role of debriefing in simulation-based learning. Simul Healthc J Soc Simul Healthc. 2007;2(2):115-125.

29.

Buçinca

Malaya

Gajos

. To trust or to think: cognitive forcing functions can reduce overreliance on AI in AI-assisted decision-making. Proc ACM Hum-Comput Interact. 2021;188:1-21.

30.

Gillespie

Lockey

Curtis

Pool

Akbari

. Trust in artificial intelligence: a global study. The University of Queensland; KPMG Australia; 2023 [cited 2024 Aug 4], Available from: https://espace.library.uq.edu.au/view/UQ:00d3c94.

31.

Alnuaim

Zakariah

Shukla

, et al. Human-computer interaction for recognizing speech emotions using multilayer perceptron classifier. Bhagyaveni MA, editor. J Healthc Eng. 2022;2022: 6005446.

32.

Venkatesh V, Morris MG, Davis GB, Davis FD. User acceptance of information technology: toward a unified view. MIS Q. 2003;27(3):425. Available from: https://www.jstor.org/stable/10.2307/30036540 .

33.

Venkatesh

Davis

. A theoretical extension of the technology acceptance model: four longitudinal field studies. Manag Sci. 2000 [cited 2025 Mar 9];46(2):186-204. Available from: https://pubsonline.informs.org/doi/10.1287/mnsc.46.2.186.11926 .

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB