Sage Journals: Discover world-class research

Abstract

Objectives: Pilot 5 utilizes AI and robotics to develop a robotic nurse assisting hospital staff in response to workforce shortages and rising care demands due to an aging population. This project aims to optimize resources, reduce errors, and improve patient satisfaction through personalized care. Methods: The Living Lab approach was implemented to split the study into sprints. The first split involves working with project partners and stakeholders to define the problem, brainstorm functionalities, and identify limitations (24 participants). The second split focuses on further requirement gathering, exploring real-world use cases, and considering ethical and privacy concerns (51 participants). Results: The project used iterative development cycles (5-8 months) to continuously improve the solution. Surveys revealed high satisfaction rates, with average scores of 4.0 and 3.6 for Sprints 1 and 2, respectively. Similarly, a team morale survey indicated a positive trend, with average scores of 7.6 and 8.18 for Sprints 1 and 2, respectively. Conclusion: Pilot 5 offers a promising solution to the evolving needs of modern hospitals. This study explores the integration of a social robotic system into nursing care to enhance quality and emphasizes stakeholder engagement, participatory design, and user-centered approaches in AI healthcare solutions.

Keywords

co-creation stakeholder engagement socially assistive humanoid robots living lab hospital care

Introduction

HosmartAI project is developing a common, open-source integration platform to streamline the incorporation and evaluation of digital technologies (including robotics and AI) within the healthcare system. This platform will be piloted across eight large-scale initiatives focused on enhancing medical diagnosis, surgical interventions, disease prevention, and treatment, as well as rehabilitation and long-term care support within diverse hospital and care settings. Notably, Pilot 5 specifically targets advancements in cardiovascular disease and thoracic disorder management in Slovenia.

In this study, we describe the co-creation steps towards to Pilot 5 study which were achieved before the actual testing in real world environment. Pilot 5 (Assistive Care in Hospital: Robotic Nurse) focuses on cardiovascular diseases and Thoracic Disorders and was carried out by the UKCM Hospital, Slovenia. Its goal is to evaluate a social assistive humanoid robotic system (SAHR) that supports nursing care and improves the quality of care. SAHR assumed communicative functions such as assistance (providing medical information, reminding patients to take medication, drinking behavior, etc.) and monitoring the patient’s objective (measuring blood pressure, temperature, hydration) and subjective state of health (emotions, pain, PROMs). The robot interconnects with the clinical data repository to support clinicians during a grand round.

In Pilot 5, the emphasis is on values-sensitive design and co-creation, highlighting the significance of incorporating ethical considerations and stakeholder input into the development process.^1,2 This approach underscores the importance of ensuring that the design and implementation of solutions align with ethical principles, societal values, and user preferences. By engaging in co-creation with stakeholders, including end-users, researchers, and experts, Pilot 5 aims to foster a collaborative environment where diverse perspectives are considered, leading to more inclusive and user-centered solutions.

Values-sensitive design (VSD) is a crucial approach for developing socially assistive robots in hospitals. VSD emphasizes integrating human values throughout the design process. This ensures robots align with societal values and minimize harm while promoting positive outcomes.³ In healthcare settings, VSD can address ethical challenges and prioritize patient care.⁴ By incorporating VSD principles, developers can create robots that respect patient privacy, promote well-being, and ultimately improve the quality of care provided by hospital staff.⁵

Pilot 5 heavily relied on Participatory Design (PD), a crucial approach for effective technology development. PD involves active stakeholder engagement, transforming them from passive recipients to co-designers.⁶ This approach is linked to increased trust, user acceptance, and overall effectiveness of socially assistive robots.⁷ By actively involving users, stakeholders, and caregivers, PD allows for addressing concerns, incorporating feedback, and tailoring the robot design to specific needs and preferences. The literature highlights various factors influencing user acceptance, including robot characteristics,⁸ interaction dynamics,⁹ user perceptions based on observed interactions,¹⁰ self-efficacy,¹¹ and visual design.¹²

The core position of co-creation (integrating co-design and co-production) as a strategy for social and health innovation is unquestionable.^13,14 Public engagement in shaping research and innovation fosters a sense of ownership and empowerment, ensuring projects are truly “with” and “by” the people, not simply delivered on their behalf. Still, it is highly challenging and due to different limitations, barely implemented in the research, development, and innovation cycles.¹⁵ Therefore, systematizing the introduction of a user-centered design culture is a challenge that requires a continuous and evolving effort to ensure that value is created ensuring users’ long-term engagement which was followed through in this study. In the healthcare field, the concept of the method of co-creation is based first on acknowledging a silo culture and then on the difficulties to work across silos.¹⁶ From that point, co-creation has a built-in provision to sort immanent conflicts of interests and opinions upfront. It does so by having the participants in the process learn each other’s perspectives during the development of innovation.

Living Labs methodology is a user-centered approach to innovation that takes place in real-world settings.¹⁷ It emphasizes co-creation, where stakeholders from various backgrounds work together throughout the entire innovation process. By bringing together stakeholders and testing ideas in a real-world setting, our living lab approach helped to increase the success rate of HosmartAI project pilots.

The levels of acceptance among stakeholders regarding the implementation of socially assistive humanoid robots (SAHRs) in nursing and care routines and, additionally, the factors that influence higher or lower acceptance within these groups, focusing on demographic variables, technology expectations, ethical considerations, and negative perceptions were recently examined by.¹⁸ That study revealed that most patients and healthcare providers view SAHRs positively. Specifically, many participants were willing to be treated or work in a hospital where an SAHR collaborates with humans. The structural models of overall acceptance highlighted specific personal traits that lead to greater acceptance of SAHRs. In particular, patient acceptance was significantly influenced by realistic expectations of SAHR functionality, rather than unrealistic depictions portrayed in the entertainment industry.

This study employed a co-creation approach in Pilot 5, divided into two distinct phases: Sprint 1 and Sprint 2. During Sprint 1, ethical approval for patient interaction was pending. Consequently, data collection relied on a cross-sectional online survey (completed by 282 healthcare employees and 346 patients) and a face-to-face workshop (involving 24 participants from diverse stakeholder groups: researchers, nurses, and project coordinators). These activities aimed to gather initial insights and inform subsequent phases. In Sprint 2, the focus shifted towards co-creation through a series of face-to-face workshops (four in total, with 51 participants from diverse stakeholder groups). These workshops addressed key aspects of the robot’s development, including finalizing its real-world architecture and placement, defining the conceptual design for human-robot interaction, designing the care plan, identifying and scheduling digital interventions, and developing and deploying the first iteration of vision models on the robot. As a last step, a cross-sectional study was designed with 110 patients to evaluate the SAHR’s readiness for the clinical trials.

In the following sections, the co-creation activities of Pilot 5 which were completed before starting the actual studies within the project are explained within the living lab methodology. In the next sections, stakeholders’ engagement, participatory design, and deployment scenarios are explained according to the results of co-creation sessions with stakeholders.

Methodology

This section details the methodological approach employed in Pilot 5. This study has a mixed-method approach. Surveys, workshops, and interviews were used as data collection methods. Collected data was analyzed by basic statistical methods. We will delve into the implementation of the Living Lab approach, specifically focusing on the design of extended sprints tailored for Pilot 5. Additionally, we will explore the strategies undertaken for stakeholder community engagement. Furthermore, we will discuss the Pilot’s adherence and elaborate on the deployment scenarios envisioned for the Pilot’s outcomes.

Agile schema

The agile methodology was applied during the whole task and several sprints were conducted to optimize the technologies within this study. Following the guidelines provided by the hybrid methodology, an adaptation of the CAPTAIN 2020 project methodology was carried out to adjust the sprint approach into our study’s framework.^19,20

Inclusion/exclusion criteria

This study recruited participants from various stakeholder groups (Table 3). The clinician and nurse sample included clinicians who regularly participate in grand rounds within the vascular surgery or thoracic surgery ward. Engineers/researchers and project team members were selected from those actively involved in the project.

Implementation of living lab approach: Design of sprints for pilot 5

After gathering the considerations within the initial questionnaire, 4 sprints were decided as the implementation of the sub-phases for the whole project. In this study, we adopt a broader definition of “sprint” signifying an extended time-boxed period (5-8 months) dedicated to accomplishing a specific set of user activities, deviating from the conventional Scrum framework’s one-week to one-month sprints. Table 1 lists the planned periods for each sprint and the determined time frames of each sub-phase for the whole project whereas the co-creation phase of Pilot 5 was implemented during only the first two sprints.

Table 1.

Planned HosmartAI sprints.

Sprint	Start	End	Duration	Focus related to the project stages
1	Jul 21	Nov 21	5 months	Co-design with project partners and project stakeholders. Action to better understand the problem; what seems a good value proposal (e.g. expected functionalities); understand limitations (e.g. environment; working flow dynamics) and how to address them; orientations on “to Dos” and “not to Dos”)
2	Dec 21	Apr 22	5 months	Co-design to continue eliciting requirements, exploring real implementation scenarios and pilot specificities, exploring ethics and privacy by design and feedback for the design of the interventions (e.g. what kind of information to display). Focus widely on value creation on the AI platform, the technical implementation, and business case feasibility
3	May 22	Dec 22	8 months	Testing available minimum viable product (MVP) and continuing co-creation
3	May 22	Dec 22	8 months	• 1st implementation of AI platform
4	Jan 23	Jun 23	6 months	Testing available MVP and continuing co-creation
Final MVP				Jul 24 to be explored

In Sprint 1, common co-creation activities were discussed in groups with the participants regarding the concept to be explored. Conclusions were extracted from this discussion as to where and when possible, the robotic nurse would be utilized in co-creation. Interaction with patients could be facilitated only after ethics approval has been obtained. Thus 1st co-creation is being carried out as a cross-sectional survey and can also include health professionals participating in a workshop.

In Sprint 2, each sub-phases were defined for a time frame (Table 2). Within the Sprint 2 period, 4 face-to-face workshops or focus group discussions with different co-creation methodologies were organized with different stakeholders to define and model the robot’s engagement in clinical workflow, evaluate the new prototypes by clinicians under laboratory conditions, and analyze and match between requirements of the system against conditions and capacity of the environment.

Table 2.

Sprint 2 sub-phase breakdown with duration.

Sprint 2	Time frame
1. Sprint planning	4 weeks
2. Design of technology	2 weeks
3. Development of technology	2 weeks
4. Design the co-creation/testing procedures	2 weeks
5. Laboratory technical assessment	1 week
6. Preparation of the living lab sessions	3 weeks
7. Technical field testing	1 week
8. Running Co-creation/testing sessions	2 weeks
9. Feedback synthesis	2 weeks
10. Sprint review	1 week

Stakeholders’ community engagement: Strategy

The initial custom-made feedback questions and focus group studies were used as a strategy to establish the local stakeholders’ groups.

Those questions consisted of a phase where the objective was to have a first understanding and gather a general idea of stakeholders’ preferences for technical, pilot, and business partners, their backgrounds on the methodology adopted by HosmartAI project, the barriers foreseen, the mitigation strategies, etc.

The second consultation consisted of a workshop that was carried out during the consortium general meeting in June 2021. This workshop served as a focus group to discuss the major subjects within the stakeholders’ community engagement, and from it, preferences about important stakeholders and main barriers to involving those stakeholders were explored and concreted (Table 3).

Table 3.

The details of stakeholders who participated in Pilot 5 Studies during Sprint 1 and Sprint 2.

Role	Gender	Count
Nurse	F	30
Nurse	M	5
Medical doctor	F	6
Medical doctor	M	13
Engineer/researcher	M	6
Engineer/researcher	F	2
Physiotherapist	F	4
Project team member	F	5
Project team member	M	4

Participatory design: Sprints of pilot 5

As stated in the methodology, this co-creation study involves only the first two sprints. The details of the workshops during those sprints are given in the following text.

Sprint 1

Pilot 5 began with a stakeholder workshop introducing the project goals, potential robot applications, and limitations. Medical professionals discussed potential uses and identified suitable tasks for the robot. (Table 4).

Table 4.

Details of the workshop of Sprint 1.

Workshop	1
Number of participants	24
Groups of stakeholders involved	3 (researcher, nurse, project team member)

Sprint 2

During workshop 1, researchers (healthcare professionals) from the Department of Thoracic Surgery expressed interest in the presented scenarios. One area for future exploration is the potential impact of surgical robots on workload, as identified by the participants. The workshop resulted in the following key outcomes: Established goals for upcoming workshops and developed a clear action plan for the forthcoming period.

Sprint 2’s initial workshops addressed two crucial aspects: exploring integration possibilities and identifying implementation challenges. Researchers presented five potential scenarios for robot-assisted workflows in thoracic surgery, fostering discussion and exploration.

In workshop 3, the project aimed to achieve three main objectives: introducing the robot “Frida” to participants, evaluating its core functionalities, and collaboratively defining its role within the clinical setting. Participants first encountered Frida at the University of Maribor facilities. To determine the appropriate complexity for the robot’s future training content, researchers assessed its basic functions such as speech recognition, gestural communication, movement capabilities, and object manipulation.

Workshop 4 focused on codeveloping a video guide for post-surgical rehabilitation delivered by robot. Held at the UKCM Hospital in Slovenia, the session defined an exercise protocol for Frida to broadcast to patients, promoting faster recovery through breathing and lower body exercises (Table 5).

Table 5.

Details of the workshops of Sprint 2.

Workshop	1
Number of participants	13
Groups of stakeholders involved	3 (researcher, nurse, project coordinator)
Workshop	2
Number of participants	25
Groups of stakeholders involved	3 (researcher, nurse, project coordinator)
Workshop	3
Number of participants	9
Groups of stakeholders involved	3 (researcher, physiotherapist, project team member)
Workshop	4
Number of participants	4
Groups of stakeholders involved	2 (researcher, project team member)

Deployment scenarios

In Sprint 1 and Sprint 2, the Pilot 5 study defined the following deployment scenarios (test cases) to be delivered. Wizard of Oz (WoZ) experiments are a valuable tool for researchers to gain early insights into user needs and preferences for new technologies. They help bridge the gap between theoretical concepts and real-world user experiences. Thus, for the experiments with the robot, we designed from scratch and applied WoZ experiments within the project after the face-to-face workshop with stakeholders.

Scenario 1: Robot in data collection mode

Within this test case, the socially assistive robotic unit acted as a tool to collect patient data and store the data in the FHIR (Fast Healthcare Interoperability Resources) based patient record. The robotic unit navigates to the patient’s room, scans the QR code to identify the patient, reports its location and activity to the blockchain, and executes the activity (or multiple activities in a sequence). The activities were scheduled using FHIR CareWorkflow (version 3). The following UML outlines the test case.

Figure 1 depicts the SAHR undergoing a data collection test. The right side of the figure illustrates data flow from the human user. It shows a patient interacting with the system, possibly through a physical interface labeled «Human». The left side of the figure depicts how the system responds to the user. The Multimodal Sensing module gathers health data like temperature or blood pressure. The Converse module enables communication between the user and the robot, through a Frida interface.

Figure 1.

SAHR in data collection mode test case.

Scenario 2: Robot in the group activity

Within this test case, the socially assistive robotic unit acted as a tool to motivate patients to carry out physical activities and assess the correctness of exercise execution. In this test, case the physiotherapist explained the exercises to the patient and trained them in the first session (with the presence of the robot). Using the FHIR CareWorkflow (version 3) resource, the exercises were scheduled and the robot navigated to the room and tried to engage with patients to carry out the activities. The robot helped patients with breathing exercises in a WoZ experiment. The following UML outlines the test case.

As seen in Figure 2, the initial deployment (v1.0) of scenario 2 aimed to assess the robot’s effectiveness as a physical therapy coach, specifically for breathing exercises. SLAM ensured navigation and blockchain securely stored robot location and activity data. While a multi-modal sensing component observed patients’ exercise execution, a one-way communication module only demonstrated and guided exercises without unrelated conversation unless an emergency arose.

Figure 2.

SAHR in group activity test case.

Scenario 3: Robots during grand round routine

Within this test case, the socially assistive robotic unit acted as a tool to enable the clinicians’ intelligible and real-time access to patient sources, such as diagnostic reports, laboratory results, radiology images, etc. The robotic units follow the clinician, scan the QR codes, and load patient resources. Speech recognition and natural language understanding was used to formulate queries and a tablet was used to display the data. In the first deployment, a series of WoZ Experiments were planned, where the operator navigated the robot to simulate follow functionality and also re-interpreted queries in natural language to compensate for possible recognition failures due to the real-life environment. The following UML outlines the test case:

Test Case 3 builds upon the foundation of test Case 1 by significantly enhancing the FHIR interface (Figure 3). This upgraded interface boasts two key features: a clinician’s interface specifically designed for data visualization, and a RESTful interface that seamlessly connects with the hospital’s existing digital system and database.

Figure 3.

SAHR in grand round routine.

Satisfaction survey

A satisfaction custom-made questionnaire was prepared and filled by the facilitators’ team. The aim is to understand possible problems in the procedure, the communication, and the assigned work and improve it in the next sprint. It includes 3 questions with Likert 5 responses (Tables 7 and 9) and two questions with free-text responses (Tables 8 and 10). Custom-made questionnaires are often used to study the job satisfaction of hospital staff.²¹ In general, custom-made inventories to examine the job satisfaction of nurses seem to be constructed with practical purposes in mind²² instead of focussing on the theoretical and methodological adequacy of the instruments.

Results

This section presents the findings of Pilot 5 during Sprint 1 and Sprint 2. We will begin by detailing the process of establishing stakeholder groups, a crucial step in the Living Lab approach. Subsequently, we will delve into the participatory design process employed during Pilot 5’s extended sprints. This will be followed by an overview of the specific actions carried out within the Pilot’s co-creation phases. Finally, we will present a retrospective analysis of the Pilot’s sprints, alongside a cross-sectional study conducted with participating patients. This combined approach will provide a comprehensive understanding of the Pilot’s outcomes.

Building the stakeholders’ groups

Pilot 5 study constituted a stakeholders’ group made up of individuals representing each of the categories of stakeholders identified according to counting on the personal feedback, experiences, and expectations of each of the potential participants. Within the initial questionnaire, we identified the relevant stakeholders. After consultation within the team during the consortium meeting face-to-face workshop in June 2021, additional analysis was made, and stakeholders were prioritized with identifying the barriers to the co-creation process for the pilot 5 study (Table 6).

Table 6.

Perceived barriers to Co-creation processes with stakeholders.

No	Perceived barriers
1	Inherent conflict of interest
2	Accessibility for participants
3	Privacy issues
4	Technical interoperability
5	Quality of results
6	Integration and acceptance
7	Standardization

Outputs and satisfaction results of participatory design

In sprint 1, the study mainly relied on technical partners for detailing and discussing technical components and conducted co-creation sessions with healthcare professionals and patients. The team defined six personas from the results of a cross-sectional survey, optimizing them based on new information, and prioritizing unmet needs.

In sprint 2, work was carried out to finalize the architecture and placement of the robot in the real environment, conceptualize robot interaction, and design care plans, and storylines/exercises. Moreover, the team worked on identifying the blockchain deployment scenario, validating the user interface, creating the first version of the vision models, and deploying them on the robot.

Sprint 1

During the discussion, medical doctors shared their perspectives on how the robot could be integrated into surgical patient care. Nurses identified potential tasks the robot could handle. All participants collaboratively discussed potential challenges and limitations the robot might face in a hospital setting. Based on these discussions, we developed a preliminary plan for the initial interaction between staff and the robot within the hospital environment. Overall, the participants were engaged and enthusiastic about the next steps in piloting the robot technology.

This collaborative session yielded a preliminary plan for initial robot-staff interaction and hospital integration, with positive feedback foreshadowing successful project execution (Tables 7 and 8).

Table 7.

Team’s satisfaction after sprint 1 (likert 5).

Question	Mean value
How do you feel about the amount of work assigned to you in this sprint?	3
How do you feel about the amount of work you did in this sprint?	5
How do you feel about the quality/value of work you did in this sprint?	4

Table 8.

Free text responses from Sprint 1.

Question 1: ‘What would you change?’

We planned to organize the co-creation meeting sooner, but it was impossible due to COVID-19. The first time we prepared a basic, ground presentation, we planned to prepare a specific session for user stories separated in each clinical department

Question 2: ‘What did you like more??’

We anticipate robotic nurses to be included in the work process

Sprint 2

Open dialogue with nurses and doctors revealed interest in these scenarios, alongside concerns about potential workload increases. This emphasizes the need for proactive mitigation strategies to ensure a seamless integration process. In workshop 3, Open discussions were then held with nurses and medical doctors to identify potential challenges associated with Frida’s integration and collaboratively determine suitable tasks for it to perform. Finally, all participants were encouraged to interact with Frida, fostering a sense of familiarity and gauging its potential appeal to patients. This positive reception towards robot interaction served as a promising indicator for future exploration. Discussions in workshop 4 highlighted the need for improved video quality (technical support) and future protocol refinement.

A facilitator satisfaction survey (Table 9) identified areas for improvement. Feedback pointed to scheduling challenges, likely due to staffing shortages and COVID-19 protocols, emphasizing the need for increased time allocation. Despite this, participants remained enthusiastic about Frida’s potential (Table 10).

Table 9.

Team’s satisfaction after sprint 2 (likert 5).

Question	Mean value
How do you feel about the amount of work assigned to you in this sprint?	3
How do you feel about the amount of work you did in this sprint?	4
How do you feel about the quality/value of work you did in this sprint?	4

Table 10.

Free text responses from Sprint 1.

Question 1: ‘What would you change?’
We would need more time since it is hard to coordinate the meetings with departments due to a lack of staff and COVID-19
Question 2: ‘What did you like more??’
We are excited to work with the robot and to explore the possibilities

Retrospective analysis of Sprint1 and Sprint 2

The Retrospective Analysis marks the culmination of each Sprint loop, offering invaluable insights into the team’s dynamics and developmental progress. A thorough analysis of feedback from all partners reveals a positive trend of improvement across all sprints. As the project progressed, the team honed their collaborative skills and problem-solving capabilities, thus demonstrating growing adeptness within their roles.

One of the keys to success is balancing the technical and clinical aspects of the project, making sure that all stakeholders are involved, and considering the varied schedules and commitments of the partners. The continuous improvements seen during Sprint 1 and Sprint 2 demonstrate the team’s commitment to learning and adapting, pointing to a successful project overall.

To assess the overall sentiment and satisfaction of a work team, a team morale survey was applied to all stakeholders for both sprints (Tables 11 and 12). Through this survey, we aimed to measure satisfaction and engagement, identify areas for improvement, gauge team dynamics, and identify strengths and weaknesses.

Table 11.

Team Morale Survey Results (PM: Project manager, D: Developer, PF: Pilot facilitator, R: Researcher, PC: Pilot coordinator/leader: PC, WP: WP Leader).

1st sprint
N° answers	16
Role/Position in the project	PM	D	PF	R	PC	WP
Role/Position in the project	50%	19%	6%	38%	25%	25%
Years of experience in EU projects	0	1-3	4-6	7-9	>10
Years of experience in EU projects	6.25%	37.50%	6.25	31.25	18.75
2nd sprint
N° answers	21
Role/Position in the project	PM	D	PF	R	PC	WP
Role/Position in the project	29%	24%	14%	52%	19%	14%
Years of experience in EU projects	0	1-3	4-6	7-9	>10
Years of experience in EU projects	9.52%	42.86%	19.05%	14.29%	14.29%

Table 12.

Team morale survey questions.

#	Questions	1st	2nd
Q1	I am proud of the quality/value of work I produced at this sprint	7.6	7.9
Q2	I am proud of the work delivered to stakeholders	7.9	8.0
Q3	As a project team, we get stuff done quickly and efficiently	6.8	8.0
Q4	I had a clear and inspiring mission for this sprint	6.7	8.3
Q5	I enjoyed the work done in this sprint	7.3	8.2
Q6	I have learned new things from my engagement with sprint	7.4	8.3
Q7	I get the support I need from the team	8.2	8.7
Q8	I contribute to what will be developed and how	8.4	8.0
Q9	As a project team, we have good communication and collaboration	8.1	8.3

Comparison of results from the team morale web survey

The study team morale web survey has demonstrated an overall positive trend across the sprints. There was a gradual increase in pride regarding the quality of work produced (Q1) and delivered to stakeholders (Q2). The team’s efficiency (Q3) and clarity of mission (Q4) significantly improved from the first to the second sprint. Satisfaction from work (Q5) and learning opportunities (Q6) consistently increased over time. Team support (Q7) maintained high scores throughout, reflecting a strong sense of camaraderie. Contribution to development (Q8) remained relatively high. Finally, team communication and collaboration (Q9) showed a steady improvement over time, indicative of a healthy team dynamic. In summary, the data points to a progressive enhancement in team morale and unity over the sprints, with pride in work, enjoyment, and mutual support emerging as key strengths.

Discussion

This study employed a Living Lab methodology with iterative sprints (5-8 months) to facilitate continuous improvement through co-creation with all stakeholders. Retrospectives conducted after each sprint proved instrumental in this process. By analyzing user feedback, the project team was able to dynamically adapt and refine strategies throughout the project lifecycle.

Key takeaways from this co-creation experience highlight the importance of comprehensive stakeholder involvement, encompassing clinicians, technical teams, and end-users. This inclusive approach fostered nuanced discussions and ensured that developed solutions catered to everyone’s needs. Stakeholder perceptions significantly influenced project success; a positive outlook towards challenges and potential solutions from all parties expedited the development process.

The agile development model required flexibility to accommodate the varying schedules of clinicians and project teams, ensuring optimal participation. Proper timing for stakeholder engagement increased their participation and commitment, while also facilitating the smooth execution of tasks and sprints. Clear, concise, and regular communication across all stakeholders emerged as essential for project success. This included clarifying expectations, sharing progress updates, and fostering open discussions about potential challenges.

Close collaboration between technical and non-technical partners fostered a unified problem-solving approach and ensured the effective execution of tasks. Active participation from end-users throughout all project phases proved vital for its success. A well-organized approach to sprints and tasks, coupled with readily available resources, streamlined project operations.

Finally, the findings revealed positive user acceptance of SAHRs during patient admission. This suggests SAHRs have the potential to alleviate nurses’ workload and enhance patient satisfaction within this specific healthcare context.

Limitations

The study encountered several limitations. Firstly, participation was contingent upon a favorable attitude towards technology and robotics. This inherently limited the age range of participants, typically skewing younger. Additionally, co-creation can be a time-consuming process for both patients and healthcare staff, especially in busy clinical settings. This can restrict the number of participants willing or able to dedicate the required time. Finally, demonstrating the concrete benefits of co-creation initiatives proved challenging. While we measured factors like satisfaction and team morale, different aspects of co-creation likely require different evaluation metrics. In this study, we used custom-made questionnaires to get the initial feedback from employees and measure their satisfaction according to the requirements of the project. However, this approach raises concerns about whether it accurately measures the intended constructs. Those custom questionnaires were specifically designed for this project study within the hospital. This limits the generalizability of the findings to other settings or patient populations.

Future work

This study served as the culmination of a value-sensitive design process of Pilot 5, evaluating the readiness of a proposed socially assistive humanoid robot (SAHR) for inclusion in clinical trials assessing the effects of interactive digital patient assistance (HosmartAI Pilot 5 Trials). As a next step in the project, in Sprint 3 and Sprint 4, a single-session case study involving 110 patients (average age: 59.66 years, 56.4% male) explored patient perceptions of the SAHR. Following informed consent, participants engaged in a face-to-face interaction with the robot, including a question-and-answer session mimicking a patient admission process. Post-interaction, participants provided feedback using the e-acceptability scale and a behavioral intention questionnaire. Results indicated generally positive attitudes towards the SAHR in a patient admission scenario, suggesting high user acceptance. Patients reported favorable evaluations of the robot’s social capabilities, ease of use, and overall experience. Detailed findings will be presented in a forthcoming publication as a future work.

Conclusion

Pilot 5 of the [details omitted] project successfully demonstrated the value of a co-creation approach in developing a Socially Assistive Humanoid Robot (SAHR) for supporting nursing care within a hospital setting. This study employed a Living Lab methodology with iterative sprints, fostering continuous improvement through active participation from all stakeholders, including clinicians, technical teams, and patients.

The co-creation process, divided into two distinct phases (Sprints 1 and 2), leveraged various methods for data collection and stakeholder engagement. Initial insights were gathered through online surveys and face-to-face workshops, informing subsequent co-creation activities in Sprint 2. These activities, consisting of a series of workshops, addressed key aspects of the robot’s development, including design, functionality, and implementation within the real-world hospital environment.

The project benefitted significantly from the diverse perspectives brought forth by stakeholders. A positive outlook towards challenges and solutions from all parties expedited the development process, while clear and consistent communication ensured alignment throughout. Close collaboration between technical and non-technical partners fostered a unified problem-solving approach, further contributing to the project’s success.

This study contributes to the growing body of knowledge around co-creation and its effectiveness in developing user-centered healthcare technologies. Future research can explore the long-term impact of SAHRs on nursing staff and patient outcomes, while further refining co-creation methodologies for optimal stakeholder engagement in healthcare innovation projects.

Footnotes

Acknowledgements

We extend our deepest gratitude to the dedicated nurses, clinicians, and patients who are considering or participating in this study. Their invaluable contributions were instrumental in advancing knowledge in understanding the role of socially assistive robotic nurses in hospitals.

Author contributions

U.A., B.B, I.M, N.P., I.R.R., V.S. devised the project, the main conceptual ideas and proof outline. B.B, I.M, N.P., S.U., I.R.R., R.S., V.S. conceived and planned the experiments. B.B, I.M, S.U., I.R.R., V.S. carried out the experiments. U.A., B.B, I.M, N.P., S.U., I.R.R., R.S., V.S. contributed to the interpretation of the results. U.A. wrote the manuscript with support from I.M, V.S. All authors provided critical feedback and helped shape the research, analysis and manuscript.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The study was carried out under the Horzion H2020 funded project HosmartAI (Grant no. 101016834) and the ARIS research and infrastructure programme (Grant no. P2-0069) The funding source had no impact on the decision to carry out the study or in its design. The funding source had no impact on the decision to publish this research. The content of this paper does not reflect the official opinion of the European Union or any other institution. Responsibility for the information and views expressed therein lies entirely with the authors.

Ethical statement

ORCID iD

Umut Arioz

References

Strikwerda

Van Steenbergen

Van Gorp

, et al. The value sensitive design of a preventive health check app. Ethics Inf Technol 2022; 24(3): 38.

Davis

Nathan

. Value sensitive design: applications, adaptations, and critiques. In: Handbook of ethics, values, and technological design: sources, theory, values and application domains. Berlin, Germany: Springer, 2015, pp. 11–40.

Gerdes

Frandsen

. A systematic review of almost three decades of value sensitive design (VSD): what happened to the technical investigations. Ethics Inf Technol 2023; 25(2): 26.

Chita-Tegmark

Scheutz

. Assistive robots for the social management of health: a framework for robot design and human–robot interaction research. Int J Soc Robot 2021; 13(2): 197–217.

González-González

Violant-Holz

Gil-Iranzo

. Social robots in hospitals: a systematic review. Appl Sci 2021; 11(13): 5976.

Vandekerckhove

De Mul

Bramer

, et al. Generative participatory design methodology to develop electronic health interventions: systematic literature review. J Med Internet Res 2020; 22(4): e13780.

Zhong

Fraile

Castellano

, et al. A case study in designing trustworthy interactions: implications for socially assistive robotics. Front Comput Sci 2023; 5: 1152532.

Klüber

Onnasch

. Appearance is not everything—preferred feature combinations for care robots. Comput Human Behav 2022; 128: 107128. DOI: 10.1016/j.chb.2021.107128.

Bishop

Maris

Dogramadzi

, et al. Social robots: the influence of human and robot characteristics on acceptance. Paladyn 2019; 10(1): 346–358. DOI: 10.1515/pjbr-2019-0028.

10.

Rettinger

Fürst

Kupka-Klepsch

, et al. Observing the interaction between a socially-assistive robot and residents in a nursing home. Int J Soc Robot 2023; 16: 403–413. DOI: 10.1007/s12369-023-01088-9.

11.

Latikka

Turja

Oksanen

. Self-efficacy and acceptance of robots. Comput Human Behav 2019; 93: 157–163. DOI: 10.1016/j.chb.2018.12.017.

12.

Liberman-Pincu

Parmet

Oron-Gilad

. Judging a socially assistive robot by its cover: the effect of body structure, outline, and color on users’ perception. ACM Trans Hum Robot Interact 2023; 12(2): 23–26. DOI: 10.1145/3571717.

13.

Dekker

Franco Contreras

Meijer

. The living lab as a methodology for public administration research: a systematic literature review of its applications in the social sciences. Int J Publ Adm 2020; 43(14): 1207–1217.

14.

Greve

Leminen

De Vita

, et al. Unveiling the diversity of scholarly debate on living labs: a bibliometric approach. Int J Innov Mgt 2020; 24(08): 2040003.

15.

Paskaleva

Cooper

. Are living labs effective? Exploring the evidence. Technovation 2021; 106(C): 102311. DOI: 10.1016/j.technovation.2021.102311.

16.

Kim

Jang

, et al. Living labs for health: an integrative literature review. Eur J Publ Health 2020; 30(1): 55–63.

17.

Alavi

Lalanne

Rogers

. The five strands of living lab: a literature study of the evolution of living lab concepts in HCI. ACM T Comput-Hum Int 2020; 27(2): 1–26.

18.

Mlakar

Smrke

Flis

, et al. Using structural equation modeling to explore patients’ and healthcare professionals’ expectations and attitudes towards socially assistive humanoid robots in nursing and care routine. Int J of Soc Robotics 2024; 16: 105–124. DOI: 10.1007/s12369-023-01039-4.

19.

Petsani

Konstantinidis

Carroll

, et al. Creating a feedback loop between persona development and user research towards better technology acceptance. In: HCI international 2020-late breaking papers: user experience design and case studies: 22nd HCI international conference, HCII 2020, Copenhagen, Denmark, july 19–24, 2020, proceedings 22 2020. Berlin, Germany: Springer International Publishing, pp. 282–298.

20.

Tessarolo

Nollo

Conotter

, et al. User-centered co-design and AGILE methodology for developing ambient assisting technologies: study plan and methodological framework of the CAPTAIN project. In: In2019 IEEE 23rd international symposium on consumer technologies (ISCT). Piscataway, NJ: IEEE, 2019, pp. 283–286.

21.

Pietersen

. Job satisfaction of hospital nursing staff. SA J Hum Resour Manag 2005; 3(2): 19–25.

22.

Traynor

Wade

. The development of a measure of job satisfaction for use in monitoring the morale of community nurses in four trusts. J Adv Nurs 1993; 18: 127–136.

Unlocking the power of socially assistive robotic nurses in hospitals through innovative living lab methodology

Abstract

Keywords

Introduction

Methodology

Agile schema

Inclusion/exclusion criteria

Implementation of living lab approach: Design of sprints for pilot 5

Stakeholders’ community engagement: Strategy

Participatory design: Sprints of pilot 5

Sprint 1

Sprint 2

Deployment scenarios

Scenario 1: Robot in data collection mode

Scenario 2: Robot in the group activity

Scenario 3: Robots during grand round routine

Satisfaction survey

Results

Building the stakeholders’ groups

Outputs and satisfaction results of participatory design

Sprint 1

Sprint 2

Retrospective analysis of Sprint1 and Sprint 2

Comparison of results from the team morale web survey

Discussion

Limitations

Future work

Conclusion

Footnotes

Acknowledgements

Author contributions

Declaration of conflicting interests

Funding

Ethical statement

ORCID iD

References