Abstract
The widespread adoption of telehealth robots remains in the future because home healthcare robot designs do not yet meet healthcare providers’ needs. We explored the potential for a hybrid rigid-soft robot for telehealth. An interdisciplinary team (engineering, human factors, health care) designed a prototype robot with potential to perform telehealth tasks within a home environment. We conducted an exploratory evaluation with healthcare providers to assess their perceptions, ideas for use cases, and design requirements. This research provides design guidance for telehealth robotics and illustrates the potential for a hybrid rigid-soft robot to support healthcare providers to conduct in-home patient evaluations.
Telehealth provides the opportunity to administer health care in a person’s home through technology (Institute of Medicine, 2001). The COVID-19 pandemic exacerbated the need to explore remote healthcare solutions. One approach is telehealth robots located at the user’s home and controlled remotely by a healthcare provider to garner on-demand high quality videos and images for diagnosis and treatment. Telehealth robots have potential to improve health care, reduce healthcare access boundaries, and provide affordances that go beyond other telehealth devices, by affording mobility and manipulation capabilities along with non-invasive tools for data collection (Dagioglou et al., 2014). However, such devices are not yet widely deployed.
Limitations and Barriers with Existing Telehealth Robotic Systems
Common barriers for the deployment and adoption of telehealth and assistive robots in the home include cost (Scott et al., 2018) and size (Aguiar et al., 2021). Further, many assistive telehealth robots are developed using hard, rigid materials that have both advantages and disadvantages relative to emerging soft dexterous robots (Uppalapati et al., 2020). Rigid robot form factors are precise and have high load bearing for manipulation and transfer support, but these systems tend to be costly, bulky in size, and made from non-compliant materials that could, in some cases, be hazardous for human contact.
Soft dexterous robots made from Fiber Reinforced Elastomeric Enclosures (FREEs; Krishnan et al., 2015) offer solutions for the size, cost, and safety barriers that hinder the adoption of existing telehealth robots often made from hard-rigid materials and links. Though rigid manipulator robots that are lightweight, use series elastic actuators, and have safety embedded in their controls may be safe for human interaction, low-cost soft robots with no conventional rigid-structures and actuators could be even safer for humans to touch and engage with (Abidi & Cianchetti, 2017; Bicchi et al., 2022.)
Soft continuum robotic configurations (Hughes et al., 2016; Rus & Tolley, 2015) made using FREEs (Uppalapati et al., 2018) might be an ideal candidate for telehealth robotic monitoring of patients. Soft Continuum Arms (SCAs) are devoid of rigid, metallic structures, actuators, and sensors. The inherent compliance, damping, and lack of sharp corners allow for safe contact with humans as they absorb rather than transmit impact forces (Uppalapati et al., 2020).
Design Process and Context of Study
To explore the potential of a hybrid rigid-soft robotic arm for telehealth applications we assembled an interdisciplinary team that included experts in engineering (n = 4), human factors (n = 3), and health care (n = 2). The team first identified tasks that a soft robot might be useful for in a telehealth context, such as medication checks, wound care, and inspection of different parts of the body. These tasks were based on the telehealth literature (e.g., Charness et al., 2011) and a review of existing telehealth systems. We were also informed by the healthcare subject matter experts (SMEs) on our team, who identified telehealth needs they face in their current practice and use cases that could improve in-home care and reduce the need for in-person appointments.
Using these tasks as a starting point, we developed a prototype of a soft robot performing each task and video-recorded it in the realistic home setting of the McKechnie Family LIFE Home (https://lifehome.ahs.illinois.edu/). We then used an interview process to engage a diverse range of healthcare providers (HCPs) to conduct a needs assessment and explore additional design requirements, possible use cases, overall attitudes, and potential concerns for soft telehealth robots. Individual interviews were conducted by the human factors researchers to enable HCPs to be frank about their opinions and not be concerned about critiquing the prototype in front of the engineering team.
Robot Prototype Materials and Engineering
For the initial design we focused on several aspects of the robot: rigid link arm, hybrid rigid-soft final link, end effector camera, and software architecture (see Figure 1 and Uppalapati & Krishnan, 2018, 2020, 2021). The 3-link rigid arm is made of three high torque capacity servos (Hitec D950 TW). This arm can be used to place the final link in any desired position for the telehealth application. The three joint locations are labeled as J1, J2, and J3 in Figure 2. Hybrid rigid-soft robot components. Note. (a) Retracted variable length soft distal link that can extrude the soft continuum arm (Uppalapati et al., 2020), (b) three examples of spatial configurations of the soft continuum arm; (c) 3D printed end cap that can connect to a tip camera or a 2-finger gripper. J1-J4 illustrate arm joints (degrees of freedom). The dotted box includes a system component could be mounted on any other platform. Video snippets of prototype robot performing tasks with tip camera. Note. World view on the left and the tip camera view from the robot on the right enclosed in the blue box.
As the final link, we incorporated a Variable Length Nested Soft (VaLeNS) arm originally proposed by (Uppalapati et al., 2020). The VaLeNS section is attached to the 3-link rigid arm using a servo (J4 in Figure 2) that can axially rotate it.
The SCA can bend and rotate clockwise and counterclockwise thus providing additional degrees of freedom and dexterity. The retracted configuration is shown in Figure 2 (a). The SCA can undergo complex spatial deformations as shown in Figure 2 (b).
A wireless camera (600TVL 1/3 CMOS Micro AIO FPV Camera) or a 2-finger gripper could be attached at the tip of the SCA, shown in Figure 3 (c). The camera transmits the video feed from a video transmitter (5.8 GHz, 40 Channel) and the receiver at the operator end is used to see and save the video. The robot is controlled via ROS Melodic on Python 2.7 on a Raspberry Pi 4B. The robot was programmed a priori to perform different tasks shown in the video presented to the participants. Video snippets of prototype robot performing wound care tasks with gripper. Note. World view on the left and a close-up version on the right.
Healthcare Provider Interview Study
Participants and Data Collection
We recruited seven HCPs (4 men, 3 women) to obtain insights from different health specialties: cardiologist; cardiopulmonary rehabilitation; emergency medicine; family nurse practitioner; geriatrics; internal medicine; and neurology. Our sample was recruited using snowball sampling (Naderifar et al., 2017) from contacts at a large healthcare system. This phase of the user research was intended to elicit potential health care use cases for soft robotic telehealth systems and understand the required system capabilities for those use cases. The goal was to obtain a diversity of opinions from HCPs about the potential for the robot prototype to be used for telehealth. Participant ages ranged from 39 to 55 (M = 47, SD = 5.7). Study participants were offered a $25 Amazon eCode gift card as compensation. All study procedures were approved by an Institutional Review Board.
Study Procedures and Materials
Healthcare Provider Characteristics (n = 7).
aResponse options for Robot Familiarity ranged from 1 (not sure what this is) to 3 = used or operated this robot only occasionally to 5 (have used or operated this robot frequently).
Participants were shown a 4-minute narrated video to introduce the soft robot prototype carrying out simulated telehealth tasks (see Figures 2 and 3 for snippets from the video). They were informed that the robot was a purely functional prototype and not intended to display the final robot appearance or mobile drive base form factor. The perspectives shown in the video represented what HCPs would see from a remote location. The robot system performed the following tasks: (1) a body scan where the robot arm scanned the length of a person’s body on a bed; (2) zoom on a limb region, where it illustrated the ability to use the camera for a close-up view; (3) obstacle avoidance to show the ability of the soft arm to wind around the leg of a chair to view a wound location; (4) upper body scan to illustrate the range of the robot arm; (5) medication monitoring where the camera focused on medication labels that could be read by the viewer; and (6) wound care to simulate how the soft arm and gripper might be used to change a bandage.
Structured Interview Questions.
Data Analysis
Qualitative interview data were analyzed through top-down and bottom-up coding (Charmaz & Belgrave, 2012). Our research and engineering team met bi-weekly to discuss interview findings that could inform design iterations. We provide illustrative quotes and general themes to illustrate how the HCP insights informed our design.
Results
Attitudes and First Impressions
Participants reported overall positive sentiments and first impressions toward the soft robot prototype. They focused on the affordances of the flexibility and safety of the soft robotic arm and how this type of robot could be used to improve access to health care more generally. “It looks very flexible, and it can really do multiple tasks that a rigid arm may not be able to perform. I really liked how it was able to navigate through the different medicine bottles, which were even further from the edge of the table, and how it was able to look around the leg. And then being flexible, it creates that safety environment for the patients…” “In places that are remote or inaccessible this could be a benefit. For example, we have to drive an hour and a half due to weather limitations, or the patients themselves have the transportation issue… that could be a really good market for that type of technology.”
New Use Case Suggestions
Participants proposed a diverse range of use cases and applications for soft telehealth robotic arms. They were prompted to provide specific feedback on the prototype as well as to broadly ideate on possible use cases for soft telehealth robots in general for home environments. They discussed potential use cases pertaining to their specific areas of healthcare practice as well as generally ideated domains outside their core healthcare specializations. Comments included: “If it has sensors... it can detect the fall of [an] elderly patient at home. That’s one of the major causes of morbidity and mortality for our elderly folks.” “It could be something that can improve the medication compliance for patients by reminding them and bringing them the medicine for them to take.” “The physician can really tailor their exam to what the concern of the patient is. Say somebody’s having respiratory symptoms... They can position the robot to really listen to their lungs from the back and they can listen to the chest from the front, because the robot can kind of manipulate and have the soft touch. It can enhance the evaluation aspect for clinicians [which] can lead to better treatment options.”
System Modification Recommendations
Participants offered suggestions for system modifications and additional tools that could be used by the soft robotic arm. Most design modifications pertained to adding additional sensors to capture various types of sound-based data: “As I do telehealth, the one thing that I can’t assess is sounds and what things feel like... And it is so critical. You want to know what a person’s heart sounds like…what their legs look like and maybe what pitting edema looks like. The patient could do those things for me [but]not the heart sounds, so it’s really the capability to hear through that and to really provide a really good assessment for a patient who is at home, especially one that is possibly immobile and one that does not have the literacy… [it could be] amazing.” “Tools [with] sensors would be things to enhance that data arm capabilities… If you have a way of touching the skin and having a small probe to come in, it can measure their hydration levels [and] can measure the cardiac rhythm, it can measure the blood pressure and respiration rate. So those are the important biometrics that can really help.” “To improve this system would be again optimizing your design product adding more various sensors.” “I think that the sounds and the depth perception those kinds of things, the sounds/hearing, something that is really important. I feel that the robotic arm in what it displayed, and the touch is there, but what it would be missing is the actual sounds.”
Potential Barriers to Adoption
Though safety was not a key apprehension or barrier mentioned by the HCPs, other potential perceived barriers did emerge, such as cost and privacy: “If it is too expensive that clinician is not going to use it…” “The second [barrier] is how do we assure privacy for the patient, so that can be a concern for the patient to have a robot which has a camera that can see them 24–7 and how is their privacy assured who’s watching and how’s that in control.” “With telehealth visits, when you talk about domestic violence, depression, and anxiety you’re not really sure who is present in the room, and [if] the privacy is actually obtained…”
System Trust
Participants elaborated on several key factors that might influence their trust in the soft robotic arm system for telehealth applications, focusing on aspects of robot reliability, appearance, and safety: “I would trust it. In fact, it takes away from some of the human fallibility. I think part of that is there might be actual improvements in some of the ways that we deliver care.” “A rigid arm can be just one directional up down, it has limited range of motion. And then, again, the safety as it can be another one that comes up. And just the feel of it looks softer.” “One can still appear to be not human, but at least it can give a more human kind of an experience to some extent, than a rigid arm.”
Robot Control Preferences
All the HCPs expressed the desire for the soft robotic arm to have both fully autonomous and teleoperation features. Preference for fully autonomous robot capabilities would be useful at different stages of treatment/care, or data collection needs at different times of the day: “I think there’s benefit to being autonomous in that I don’t have to get out and look at it right when it’s happening. Maybe as a specialist at the end of the day I go in and look at all my videos that I make decisions about what information is collected, while I was doing other clinical things.” “But if it’s more of a scenario where things are in question, if we’re on the maintenance phase and [we are] still kind of figuring out with our patients, in those cases I might want more information on something a little more closely or do more of an assessment that maybe isn’t programmed in.” “…If at nighttime the patient is sleeping, the robot can be there, monitoring the patient, monitoring the vitals, monitoring that his heart rate is good, his pulse rate…that data feed can notice any change in the pattern and or if there is a patient that got up in the middle of the night and had a fall. Those things can be alerted to the health care system – that’s the autonomous aspect of it, which is monitoring that patient without any input.” “The other aspect where you can need to control it, is where the exam needs to happen, and you really want to address those specific areas that you want to highlight. And to evaluate… specific area[s] of the body that you want to have further detailed sight of.”
Themes
In general, the HCPs had positive first impressions toward the hybrid rigid-soft robot prototype that we presented. They could see the potential for the soft robotic arm capabilities for in-home telehealth and some explicitly perceived the soft extruding link as safe for human contact. The HCPs explicitly discussed how the dexterity of the SCAs (soft link) allows for safe reaching of intricate and occluded areas of the patient’s body, which may otherwise be difficult to reach with rigid robots.
They identified a range of use cases that would support their clinical goals for telehealth. For some use cases, HCP may require the system to be portable, and compact to move around constrained spaces in the home. The system should be able to perform both video-monitoring and manipulation tasks. They desired soft robotic arms to be capable of procuring sound-based data for a variety of telehealth use cases.
They provided suggestions for increased functionality and design enhancements by adding more sensing, accurate positioning, and manipulation capabilities. Such expanded capabilities may require improving the load bearing ability, and stiffness of the soft manipulator, which presents itself with a tradeoff between softness and rigidity.
HCPs consistently indicated interest in a soft robot that has both fully autonomous and teleoperation control features to perform tasks. The inherent material compliance of soft robots can open the door for high levels of autonomy to be accepted for telehealth robots. For example, instead of the HCPs controlling and maneuvering the robot all the way to the patient’s wound, the robot could automate this process by visually servoing itself to the area of interest.
We learned that the manipulator may need to counter the movement of the patient by an image-guided readjusting of its end position, so that a desired area is always highlighted for the HCP. Robust autonomous navigation and manipulation using images from either the tip camera or the base camera require integrating recent advances in vision-based perception (object recognition, distance, and pose estimation) combined with trajectory optimization and control of soft flexible appendages.
Design Iterations
Research team members used their interview debriefing notes to identify emergent themes and communicate user-driven system requirements and human factors design recommendations. These findings were iteratively communicated to the mechanical and software engineering team to inform design modifications through a series of bi-weekly team meetings.
The resultant second-phase prototype is illustrated in Figure 4. We revised the robot design to an upright configuration to further increase its compactness. The robot parts are made modular to enable easy assembly and disassembly. These modifications resulted in reduction of the robot weight to less than 5 kgs, thus enabling a single adult to lift and place the robot at different locations. Iterated robot design and its capabilities. Note. (a) improved robot design with an integrated wireless camera (with antenna), gripper, and a height-adjustable base. Activities illustrate how the robot can (b) grip a stethoscope and place it on the chest of a person, (c) grip a piece of cotton to touch a simulated wound on the patient’s arm, and (d) pickup up a dropped pen from the floor and hand it to person. Robot tip camera views are in the blue boxes.
The end effector used in the video stimuli consisted of a camera alone, which had to be replaced with a gripper for the wound care task. In the refined design, the camera and a two-fingered gripper were integrated into a single design. The gripper is 3D printed and actuated pneumatically, without any electrical signals at the tip to further enhance safe intractability with older adults. The integrated end effector allows for image-guided manipulation by the HCPs, and to place items on appropriate areas of the patient’s body (e.g., stethoscopes, oximeters).
Figure 4 shows the integrated end effector gripping the end of the stethoscope and placing it safely on the patient’s chest for acoustic feedback. In addition, we demonstrated the feasibility of the manipulator for dressing a wound by performing real-world test verification in the home simulation space. In practice, this could be a difficult task without close-up images of the wound visible to the HCPs. The new integrated end effector provides access to the wound location more closely and can be monitored/cleaned more precisely. Image-guided manipulation afforded by the integrated end effector can enable both tele-operated and autonomous reaching of objects, which we uncovered as desired control preferences from the HCPs. The design iteration process is by no means final. The goal herein was to demonstrate the openness of HCPs to the potential of this type of robot configuration for telehealth and to illustrate how their insights could guide design.
Conclusion
We conducted an exploratory evaluation of HCPs’ perceptions of the potential for soft robotic arms for telehealth and engaged them about the specific prototype shown in video stimuli involving the robot shown. Participants were asked about their overall impressions, potential use cases, required system design modifications, additional possible tools the arm could use, usability and adoption barriers, and aspects of human–robot trust. The HCPs reported overall positive sentiments and first impressions toward the soft robotic arm, especially the affordances of the flexibility and safety of the soft robotic arm.
In addition, we gained a better sense of the use cases, preferred levels of autonomy, and design requirements that meet the needs of HCPs. These insights allowed us to configure the robot system using universal design principles (e.g., Sanford, 2012), to ensure usability and feasibility of deploying the robot for these applications in home environments. Despite existing challenges for hardware design (i.e., soft robot control, perception), our work highlights the feasibility of deploying soft robot systems for telehealth applications in home environments as well as HCPs’ willingness to use the system.
We need to extend this research to a broader range of HCPs in terms of their clinical specialty, telehealth experience, and openness to technology. Future work should also involve other potential users of the system (e.g., patients) to explore variable stiffness capabilities of the SCA to expand the level of functionality of the robot and overall acceptability of soft telehealth robotic systems. Incorporating this feature within existing designs in a compact format is still a challenge (Manti et al., 2016). Informed by this study, future iterations of the SCA will integrate several additional lightweight sensors such as microphones (for sound data collection), tactile sensors, and others for enhanced perception. In addition, the design of the mobile base needs to be explored, with respect to maneuverability as well as potential trip hazards. Despite existing challenges for human–robot interaction, hardware design, along with soft robot control and perception, our work highlights the feasibility of developing soft robots for telehealth applications in home environments as well as HCPs willingness to use the system.
Footnotes
This work was supported by the Jump Applied Research through Community Health through Engineering and Simulation (ARCHES) Grant from OSF HealthCare.
We appreciate the contributions of Dr. Ryan Reich, Sister M. Pieta, Megan Bayles, Tim Hale, and Husna Hussaini.