Abstract
This study aims to characterize contact forces between humans and tools during activities of daily living (ADL) to provide information to a personal care robot. The study was conducted on non-impaired subjects to capture various static and dynamic force levels when making contact with three different robotic tools, each designed to perform an ADL task: hair brushing, face wiping, and face shaving. The static trial of the study involved 21 participants. Forces were collected at fixed locations for each task and used to develop models for each participant. Extraction of the maximum force levels was performed for both the maximum and desired levels of force. The dynamic trial involved 24 individuals. Participants were asked to maintain a comfortable level of force for the duration of their contact with the tool as the robot moved along its path to perform the ADL task. For the static and dynamic trials, higher forces were observed during hair brushing compared to the other two tasks. It was observed that the hair brushing task force at a specific contact point has an overall maximum of 55.66 N, while the maximum forces detected in the face wiping and face shaving tasks were 36.40 and 11.11 N, respectively. The forces collected were analyzed, and no trends were found relating the contact forces to the gender, height, or weight of the subjects. Based on the analysis of the results, recommendations have been made to enhance the force safety limits for the personal care robot working environment.
Keywords
Introduction
The need for assistance during personal care activities is growing as larger proportions of society begin to age and experience disabilities. This increases the demands on the healthcare industry and its workers, contributing to the rising costs of hiring more personnel to help care for these individuals. Activities of daily living (ADL) refers to “a set of common, everyday tasks, the performance of which is required for personal self-care.” 1 Activities that are categorized as ADLs include feeding, toileting, transferring, and hygiene-related tasks. 1 It should be noted that physical contact with a care receiver is required for all of these tasks, whether it is directly from a caregiver or through an everyday tool. An individual’s ability to perform activities of daily living (ADL) is a useful metric in this field as it is predictive of their ability to live independently.
Assistive technology is one approach to helping improve patient independence and relieving demands on the healthcare industry. Assistive technology refers to “any product, device or equipment that is used to maintain, increase, or improve the functional capabilities of individuals with disabilities.” 2 While this definition includes a very wide range of devices, this study will focus on robotics for personal care tasks, and, more specifically, activities of daily living (ADLs) associated with hygiene and personal appearance that are required for an individual’s morning routine. They are hair brushing, face wiping, and shaving.
Using robots as assistive devices to support activities of daily living (ADL) includes feedings and personal care, such as washing their face, brushing their teeth, combing their hair, and shaving. A detailed survey of the current robot-assisted feeding systems was presented by Park et al. 3 Several robot devices that assisted with feed were reported in the literature, including the iCraft, 4 KNRC, 5 Vision Augmented Robotic Feeding, 6 and Assistive Dexterous Arm. 7 Bilyea et al., 8 provided a review of the robotic applications in personal care.
Currently, it is the responsibility of a caregiver to be aware of safe and appropriate forces to apply during each task. They may also have feedback from the user relating to their preferences in terms of these forces. Therefore, in designing an assistive robot to take over the performance of these tasks, the system must also be aware of these force levels and have adaptive capabilities.
Modeling contact forces during human-robot interactions is an extensive topic. Recently, Sharkawy and Koustoumpardis 9 presented a detailed human-robot interactions (HRI) review. This study also investigated and compared the various methods for detecting human-robot collisions through model-based and data-based methods. Wang et al. 10 studied the contact forces for models in simulations for robot-assisted dressing, bathing, and dishwashing tasks. A survey of various real applications for HRI in different environments, including industrial, medical and rehabilitation, agricultural, and educational, is presented in Sharkawy. 11
This section discusses the existing approaches used for modeling contact forces during human-robot interactions for performing three ADLs: hair brushing, face wiping, and face shaving. The literature contains only a few studies that have measured and characterized the contact forces required for an assistive robot to perform face wiping and face shaving.
A study presented by Wiener et al. 1 investigated the practical demands of potential users of assistive robots from a survey performed on patients with functional upper limb impairments. The study highlighted the need to design assistive robot devices for ADLs, such as handling foods, face washing, hairdressing, and tooth brushing.
In 2013, a case study with the title, “Robots for Humanity” was performed at the Georgia Institute of Technology. 12 This project involved adapting a general-purpose PR2 robot to work with an individual who has severe quadriplegia due to a brainstem stroke. The first tasks developed for this project included scratching an itch and shaving facial hair. During testing of the shaving task, it was found that while the individual was able to do the task, he was incurring small nicks and abrasions on his face. 13
Following this observation, the forces were recorded of the individual performing the shaving task with the adapted robot. These forces were then compared to the forces used by his caregiver. It was found that he was applying forces of up to 25 N, which is over eight times greater than the average force of 3 N measured when the caregiver performed the same task. This demonstrated the need for further investigation into the levels of contact forces and for limiting the amount of force that a robot applies in order to prevent damage to the user’s skin.
To develop an answer to this research question, Hawkins et al.14,15 performed a study to collect the forces and torques applied when nine able-bodied subjects were performing face wiping and shaving tasks. Other studies that have examined forces/torques during everyday tasks include a study performed by Redmond et al., 16 who collected data from subjects while they were writing, opening/closing a jar, brushing their teeth, and using a phone. Data was collected using instrumented tools for each of these tasks. Van der Weijden et al. 17 captured the forces used during tooth brushing. The study also used an instrumented tool to observe the relationship between the applied forces and plaque removal. Ajani and Assal 18 proposed an adaptive compliance control scheme-based autonomous robotic system using a Universal UR-10 (6-DOF) robot arm for assistance in beard shaving.
It can be concluded from the literature that while some previous force data has been collected regarding face wiping and shaving, no study has examined hair brushing. Furthermore, while some of the studies were done with the intention of eventually providing a robot with a contact force model, others were performed using a robotic manipulator. Therefore, this study contributes to improving the existing knowledge base in the field by adding an extra task (hair brushing). Also, recording of the force data for the three tasks was done while the robot held and manipulated each tool. The study performed a detailed analysis of safety force limits and developed an algorithm for customizing the safety levels for a target force. Finally, the forces at each point of contact were examined statistically to determine if there were relationships between the gender, weights, and heights of the participants. These were the motivations behind performing the investigational trials that are outlined in this paper.
Methods and materials
A model was developed by observing the forces that the subjects applied to each robotic tool on their head. The trials were done in two parts. The first part involved the robot in fixed positions, and will be referred to henceforth as “static trials.” Then, another round of testing was done to further investigate the characteristics of these contact forces. This involved the robot moving along a tool path that is unique to each task and being in contact with the participant as it was moving. This set of trials will be referred to as the “dynamic trials.”
Equipment
The system used in this study consisted of a robotic manipulator with six degrees of freedom (DOF). A FT 150 force/torque sensor 19 from Robotiq with 6-DOF was used and attached to the robot wrist. The force sensor was attached to the robot manipulator end effector. The force sensor has a measuring range of ±150 N for forces in all directions and ±15 Nm for moments. Three off-the-shelf tools were adapted for this application, one for each task: hair brushing, face wiping, and face shaving. For each of these tools, a custom-made adapter was designed and 3D printed to allow the tools to be rigidly mounted on the force sensor to give accurate force readings. This modular design ensured that all the tools were able to attach to the same plate to provide consistency in the collected data.
For the static trials, a robot program was designed for each of the tasks to step through the same locations for each participant. While the dynamic trials employed a Microsoft Kinect 2.0 sensor with its Face Tracking Software Developers Kit (SDK), to track the face of the participant, plan the tool path, and record its position. Forces were acquired by the Robotiq force sensor and sampled at a frequency of 50 Hz to ensure that the signals collected were a thorough representation of the data. All the hardware components were connected to Intel Core i7-8700K 6-Core 3.7 GHz personal computer (PC).
The controlling and acquisition software program was written in Visual Studio and stored on the PC. The program enables the system to perform each task and allows the researcher to control the robot based on the facial tracking algorithm from a Microsoft Kinect. This enabled data to be provided to define the position of the participant and the robot while recording the relative contact forces.
A flow diagram, which depicts the communication between the system’s hardware components and incorporates the different control methods, has been generated to conceptualize the interactions within the system (Figure 1).
A diagram indicating the interaction of the hardware for the everyday task assistance robotic system.
Data capture
Data from both parts of the trial was recorded on the same computer (PC). The identification of the participants was done through a confidential subject ID and kept separate from any identifying information.
During the static trials, the position information was recorded from the robot using a local coordinate system and transformed to account for the geometry of the tool in order to reproduce each point of contact. Once the robot was stationary at each point, the participant was responsible for moving their head to make contact with the robot for the designated periods of time indicated by the researcher.
The dynamic trials involved tracking the data regarding the position of the participant using the Kinect to read the X, Y, and Z coordinates of 1347 tracked face points at a speed of 30 frames per second. These coordinates were written into a file and used to generate a model indicating the location of the participant’s head. The position data for the robot was recorded once at the beginning of each movement and separately at a frequency of 10 Hz using the FANUC PC Developer’s Kit for Visual Studio. The robot recorded the position of the tool using the known translation from the end of the manipulator to the part of the tool that makes contact with the participant. Finally, the force sensor data was recorded using the included data logger software.
Methodology
The experimental procedures took place in the Robotics Institute at the University of Guelph. Permission was obtained from the University’s Research Ethics Board prior to the study (REB 16-12-583). All interested participants were presented with information about the experiment and then signed a consent form if they agreed to participate. The researcher remained in the room, maintained control of the robot, and ensured the comfort and safety of the participants. Each participant was introduced to the equipment and experiments and given the opportunity to ask questions.
For the static trials, the appropriate end-effector was mounted to the force sensor, and the robot was moved into the first position for initial contact with the subject. The participant was then seated on a chair within the robot’s workspace and was asked to adjust the height and position of the chair so that they could comfortably contact the tool in the desired place. The target and safety limit force measurements were taken at each position. At least 15 s of data were collected both prior to and following contact with the user. The effects of gravity on the force sensor data were removed by normalizing with respect to a baseline value.
In the dynamic trials, the appropriate end effector was mounted to the force sensor. The participant was then seated either facing the robot for the face wiping and shaving tasks or away from the robot for the hair brushing task. If the user was seated facing the robot, Kinect tracked their face to pick a starting point for the tool path, which then caused the robot to start the task. This point was recorded by both Kinect and the robot for later validation of the path planning algorithm. For the hair brushing task, the robot moved to a fixed starting point. Each task was run twice; the first time allowed the participants to learn the tool path and become comfortable working with the robot, while the second time recorded the data that was used in later analyses.
Experimental design
The experiments were conducted on non-impaired subjects to capture various force levels when making contact with three different robotic tools, each designed to perform ADL tasks: hair brushing, face wiping, and face shaving. For the static trials, 21 participants (11 males, 10 females, ages 19–60) were recruited. For the dynamic trials, there were 24 participants (12 males, 12 females, ages 18–60). Fifteen of the subjects participated in both rounds of the trials. For both parts, the order of the tasks was randomized between participants to eliminate any biasing effect that the order may have had on the observed force levels. Participants were required to be at least 18 years of age, and able to communicate with the researcher. All subjects completed a survey before participating, which provided the age, gender, height, and weight data for each subject for further analysis.
Static trials
A robot program was generated for each task to reach the desired points of contact between the tool and the participants’ faces or heads. The robot would not move once it reached the point of contact. The selected points of contact can be seen for each task in Figures 2 to 4. These points were selected based on bony locations on the head and were chosen to provide a clear picture of the force being applied for the task. Medical practitioners were also consulted about the ideal locations.
Locations on the head for points of contact for the hair brushing task.
Locations on the face for points of contact for the face wiping task.
Locations on the face for points of contact for the face shaving task.
Two sets of data were recorded at each of these points. First, a target force level was determined by asking participants to apply the level of force that they felt would be necessary for the robot to complete the task. Next, a maximum force level was recorded beyond which they would begin to experience discomfort. The researcher asked the participant to make contact with the robot by saying “begin” and then prompted the participant to end contact with the robot with the word “stop.” Target force levels were held for 30 s, while maximum forces were recorded for 15 s.
(1) Hair Brushing Task: In the hair brushing task, a detangling brush (approximate contact area: 42.14 cm2) was held at four contact locations by the robot. These locations included above each ear, the middle of the top of the head, and the back of the head (Figure 2). Due to the area of the contact force, most of the scalp was covered between these four points. This provided a starting point to determine a standardized force level for the different locations where the robot would be functioning.
(2) Face Wiping Task: The face wiping tool (approximate contact area: 16.25 cm2) consisted of a 3-D printed plastic base covered by four layers of microfibre cloth. Contact locations for this task were chosen to provide the clearest overall picture of the face. Each cheekbone was selected as a point and there were three points down the center of the face as well (Figure 3). These locations were selected due to their unique positioning and to provide as much force information about different areas of the face as possible. This was ensured by selecting bony locations where contact force readings would be less affected by the elasticity of the skin and underlying tissue.
(3) Face Shaving Task: The shaving tool consisted of an adapted rotary razor (approximate contact area: 21.20 cm2) that was held by the robot in the same locations selected for the face wiping task, with the exception of the forehead contact location (Figure 4). The shaving task was only simulated in this situation to get a baseline of the anticipated forces. Therefore, the razor was switched off and covered with a layer of plastic wrap that was changed between participants.
Dynamic trials
The experimental design for the dynamic trials was very similar to the static trials. A program was written in Visual Studio to track the starting point of each task and inform the robot where to move. Beyond this point the tool paths were all the same relative to that initial point. Participants were asked to maintain a comfortable level of force for the duration of the contact as the robot moved along its path. The paths for each task are described below.
(1) Hair Brushing Task: The hair brushing task consisted of five different strokes moving from the front of the hairline, to near the user’s face, to the hairline at the back of their head. The first stroke occurred around the right ear of the individual and with each stroke, the tool was moved up and around the user’s head until it finished with a stroke around the left ear.
(2) Face Wiping Task: The face wiping task began on the left cheek of the individual, just in front of the ear at the top of the cheekbone. The robot wiped up and down the cheek of the user as it moved forward toward the middle of their face. It then went across the user’s chin and proceeded to move back along their right cheek in the same manner until it reached a point in front of their right ear. The tool then moved away from the user’s face and positioned itself in front of the top right corner of the forehead. It moved across the top of the forehead in a horizontal stroke, then down and back across to the right, covering the full area of the forehead. Once again, the robot moved away from the user and positioned itself in front of them at the top of their nose. It then moved down the nose and across the area between the nose and mouth from left to right.
(3) Face Shaving Task: Contact with the razor for the face shaving task began in the same place on the user’s face as the wiping task. It followed the same path as the face wiping task: along the left cheek, the chin, then the right cheek. It then moved away from the user, positioned itself in front of the participant, and moved across their mustache area.
Experimental results
Static trials
Data analysis began with processing the data to produce graphs of each of the forces observed by the point of contact and subject. An image of the graphs produced for one participant can be seen in Figures 5 to 7 for each task. The colors of the lines for each point correspond to the contact color locations on the heads seen in Figures 2 to 4.
Forces by the point of contact for the hair brushing task.
Forces by the point of contact for the face wiping task.
Forces by the point of contact for the face shaving task.
An extraction of the maximum force levels was performed for both the maximum and desired level of forces that the user had been asked to exert. Upon further visual inspection of the graphs for the target force, it could be seen that the initial contact with the tool often caused a spike. The force would then plateau at a lower level, which was a better representation of the desired force for that point of contact. To account for this, another maximum was taken from beyond a certain time threshold, based on how long the forces took to stabilize following the initial contact. This force measurement from each point of contact for each subject was used in performing statistical analyses on the desired force levels.
Due to the fact that the tool for the face shaving task was held at an angle relative to the force sensor, the forces were decomposed and appeared in two different directions: X and Z. It can be seen from Figure 7 that while the X-axis component is larger, there are significant force readings in the Z direction as well. The components of these forces were recombined mathematically in order to observe the true magnitude of the contact forces for this task.
(1) Extracting Forces: Figures 8 and 9 show the maximum values extracted from the raw data graphs for the hair brushing task, as well as the region that was used to extract the target force, which is marked with black dotted lines. It can be seen in Figure 9 that the peaks are not necessarily indicative of the target force. Therefore, the target force measurement for each subject was taken separately from the flatter region of the force curves. This will be important in setting the regions of desired force levels when designing the algorithm for the personal care robotic system.
Maximum value extraction from the maximum force application trial for the hair brushing task.
Maximum value extraction and region of target force application from the average force application trial for the hair brushing task.
Table 1 shows a summary of the force data extracted from each of the recordings for all of the participants in the static trials. The second and third columns in the table represent the maximum forces observed for the maximum force trials and the average force trials, respectively. The “Target” column of the table accounts for removing the spike often seen upon initial contact, to examine the more consistent target force that was maintained for the rest of the trial.
Summary of the forces extracted from the static trial tasks.
The maximum values from the maximum force application trials will be used to set safety limits that the robot may never exceed during the performance of its tasks. Finally, the target force levels represent a set point in order that the robot can adjust its tool path by small amounts to maintain consistent and comfortable force levels for the areas of the face each time the task is performed.
Dynamic trials
The dynamic trial raw data was normalized and graphed to examine the forces and moments that were present over the entire duration of each task. Examples can be seen in Figures 10 to 12 for hair brushing, face wiping, and shaving, respectively. Positive and negative forces denote the direction of the force.
Forces and moments for the hair brushing task.
Forces and moments for the face wiping task.
Forces and moments for the shaving task.
The black dotted lines on each of the graphs represent the different contact areas that each of the tasks was divided into for further analysis. For the hair brushing task, each of the five sections represents one stroke of the brush, beginning at the hairline around the participant’s face and ending at the nape of the neck. For the face wiping task, contact areas 1–3 represent the left cheek, chin, and right cheek for the face wiping task. Contact area 4 is the forehead, 5 is the nose, and 6 is the upper lip area. Finally, the shaving task has seven contact areas: 1–3 are the same as those for the face wiping task, then areas 4–6 are the right side of the jaw, the middle area of the jaw under the chin, and the left side of the jaw respectively. Contact area 7 for the shaving task represents the mustache.
This normalized data was then used to extract the maximum and average contact forces for the different areas of the head and face of each participant, much like the static trials.
(1) Force Characterization: The dynamic trial force ranges can be seen in Table 2. For the Z direction, the negative forces are those that are pushing into the surface of the sensor, so the majority of the forces applied in the Z direction were negative. The columns entitled “Neg” and “Pos” denote these different directions along each axis.
Summary of the extracted forces from the dynamic trial tasks.
The forces were much larger in magnitude in the hair brushing task, with a minimum force of −31.53 N, but similar for the wiping and shaving tasks with ranges of −9.84 to 6.28 and −10.08 to 9.76 N respectively.
Safety procedure
As reported in Table 1, the maximum force values seen in each static task were used to determine the initial safety limits for the algorithm regarding the dynamic trials. They were 56 N for hair brushing, 37 N for wiping, and 12 N for shaving. For the dynamic trial, the robot algorithm was programmed to move the robot manipulator away from the user and pause the task if the force recorded exceeded the task’s safety limit magnitude. The participant then can select either to continue the task or return the manipulator to its home position. This measurement was taken to ensure the safety of the participants.
Discussion
Statistical analysis
(1) Static Forces: To begin the statistical examination of these contact forces, the minimum, quartiles, and maximum from each average force level trial were determined and reported by the contact location. This can be seen in the box plot in Figure 13. In general, it can be noted that while the quartiles are fairly evenly spaced, the maximum tends to be much larger than the upper quartile number for all of the tasks. This means that the participants who applied these maximum force levels were much further away from the median than the majority of the study participants. This will cause the distribution to be shifted to the lower end of the box. It can be seen for each task that the forces observed have relatively normal distributions, with 2–4 outliers above the rest of the sample.
Box plot for the distribution of the average forces for the different tasks.
Further tests were done by comparing the average force levels across all subjects for different tasks. A linear model was fitted, and ANOVA was performed. It was found that there was a statistically significant (p = 0.0036) difference in at least one of the mean force levels by the task.
In order to determine which task was different, a Tukey’s HSD test was also performed. It was found that there were significant differences between the means for the shaving and hair brushing tasks (p = 0.0272), and between the means for the face wiping and hair brushing tasks (p = 0.0055).
Next, male and female subjects were compared for the hair brushing and face wiping tasks. A two-sample t-test was used and indicated that there was no evidence of a statistically significant difference between the male and female subjects from these trials for either hair brushing (p = 0.6448) or face wiping (p = 0.1762). There were also no significant relationships related to the static data regarding the height or weight of the participants.
Finally, the forces at each point of contact were examined to look for similarities across different participants and tasks. A linear model and ANOVA test were performed on the data and sorted by the point of contact. It was found that there were no statistically significant differences between any of the points for hair brushing (p = 0.2850), face wiping (p = 0.0721), or shaving (p = 0.7040). Therefore, recommendations will be given based on the desired force range and calibration instead of conditioned upon location.
(2) Dynamic Forces: Ranges seen in each of the tasks were defined by the previously extracted forces, which could be used to set safety limits in the force feedback control algorithm. In order to examine these forces and their distributions, a box plot was generated for each task to compare the contact locations. These box plots for the primary directions of force can be seen in Figures 14 to 16.
Distribution of average contact forces for each brush stroke.
Distribution of average contact forces for each wiping contact area.
Distribution of average contact forces for each shaving contact area.
In order to better characterize the dynamic trials forces, statistical analyses were run to compare contact locations and examine trends related to demographic information. The contact location comparison began with running an ANOVA on all of the different contact areas for each task. Statistically significant differences were found to be present in all of the tasks, including hair brushing in the Y (p = 2.81e-10) and Z (p = 4.69e-9) directions, face wiping in the X (p = 1.47e-7), Y(p<2e-16), and Z (p = 1.19e-15) directions, and shaving in the X (p = 7.99e-14), Y (p = 2.85e-8), and Z (p=6.04e-14) directions. A Tukey’s HSD was done for all of these force directions in order to examine where the significant differences were in terms of the contact locations. There were no distinguishing trends related to the forces in terms of symmetry or other contact location relationships that could be used in the force feedback control algorithm.
Furthermore, the demographic data collected from the participants was also used to look for significant trends. This was done first with each demographic characteristic separately and then with all of the characteristics accounted for using a multiple regression model. No significant trends were found that related any of this data to the forces observed in the dynamic trials.
Conclusions
Force sensing to create contact force models has been explored for the purpose of providing assistive robots with allowable force limits. The static and dynamic force data was recorded for the three tasks while the robot held and manipulated the tools for each task. To the best of the authors’ knowledge, this is the first study to present a contact force analysis for an assistive robot during hair brushing.
The results presented in this study have allowed for recommendations to be developed regarding target and maximum forces during each task that the personal care robot has performed thus far. Based on these observations, upper safety contact force limits and an algorithm for the customization of target force levels have been developed. The ranges in the force from the dynamic trials largely agreed with similar tasks presented in the literature, with the exception of significantly higher forces in the hair brushing task. The models developed based on the forces collected were analyzed and no trends were found relating contact forces to the gender, height, or weight of the subjects.
The static forces were initially examined as having the potential for calibrating the force levels in the dynamic task. However, due to the lack of significant relationships between the static and dynamic data, this was not determined to be a valid method for customizing the force levels. While the top of the observed force levels from the trials were used to create general safety recommendations, a further investigation, perhaps with a larger sample size, will be required to test these limits and determine an appropriate method of customizing the dynamic force levels. The further testing could lead to more detailed characterization and, therefore, improvements in creating an intelligent algorithm system for ADL tasks performed by assistive robots.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The authors would like to sincerely thank all those who participated in the study. We also gratefully acknowledge the financial support from the Natural Sciences and Engineering Research Council of Canada through Discovery Grant No. 400697.
