Sage Journals: Discover world-class research

Abstract

Background

Controlling a myoelectric arm prosthesis after amputation is a skill that is difficult to master and requires extensive training since muscles need to be activated in a specific manner. One novel type of training places the prosthesis user in a virtual reality environment (VRE) to train; however, it is not yet clear if skill transfer occurs from the training to actual prosthesis use.

Objective

The aim of the current study is to examine skill transfer of VRE training.

Methods

A total of 20 able-bodied participants were divided into two training groups. One group trained with a newly developed VRE system and the other trained using a prosthesis simulator. Both groups performed five training sessions in a pre-test/post-test design where functional control, motivation and perceived workload were measured.

Results

The results showed that both groups improved from pre-test to post-test on functional control and motivation but not on the workload. Interestingly, almost no differences in improvements between groups were found indicating that there was no difference in effectiveness between training methods.

Conclusion

Based on these results, we can conclude that we found skill transfer of VRE training. Therefore, VRE training is an effective training method for arm prosthesis control and has valuable potential for implementation in a rehabilitation setting.

Keywords

virtual reality upper limb prosthesis prosthesis training myoelectric prosthesis control skill transfer

Introduction

After an upper limb amputation, functionality can be restored using a myoelectric upper limb prosthesis. Such myoelectric prostheses are controlled by muscle activity from the remaining muscles in the residual limb. The activation of these remaining muscles can be measured using surface electromyography (EMG) and can be used to operate the motors in the prosthesis. The current state-of-the-art control systems use pattern recognition (PR) to generate prosthesis motor commands, as is the case in the current study. In PR control, a classifier is trained to recognize the natural muscle activation patterns that the user produces when intending to use a certain grip. The grip detected by the PR classifier is sent as a control command to the prosthesis in real time. However, literature showed that controlling a PR-driven prosthesis is not easy to learn and therefore training is essential, especially for novice prosthesis users.^1–5

Starting prosthesis control training at an early stage after amputation, that is, before an individual receives his/her personal device, would allow to exploit neuroplasticity.^6,7 However, only limited options to train prosthesis control in this pre-prosthetic phase are available, such as training with a tabletop hand or using software such as PAULA by Ottobock.^8,9 Another way of enhancing training effectivity is by providing users with individualized feedback during training.¹ Such feedback could address the aberration someone might have within their prosthesis control abilities. Furthermore, enjoyment and intrinsic motivation can increase training effectivity,^10,11 in addition to repetition and reinforcement learning.¹²

One form of training that can address most of these aspects to maximize training effectiveness is serious game training. In the field of rehabilitation medicine, a serious game is often a type of computer game which is not only entertaining but also trains the player in acquiring a new skill. One of the most important aspects of a serious game is that the skills acquired during training transfer to the task at hand, because only then the training can be considered effective. Even though only a few serious games have been developed for upper limb prosthesis training, the results show that serious game training can be both enjoyable and effective in inducing transfer.^13–18 Serious games can also be implemented within a virtual reality environment (VRE), which places an in a virtual environment using a head-mounted display. In this environment players can move around and interact with virtual objects just like they would with actual objects in real life. A serious game training designed in a VRE to improve upper limb prosthesis control skills is not yet fully explored. Such a training could be more effective than prosthesis training in real life due to individualized real-time feedback in combination with the immersive and realistic aspects of a VRE.

The use of a VRE to train prosthesis control could come with a multitude of benefits. First of all, it could address all aspects to increase training effectivity that were mentioned before. VRE training could allow individuals with a recent amputation to start training in the pre-prosthetic phase, and it can include individualized feedback, while additional serious gaming elements can be exploited to increase intrinsic motivation. Furthermore, skill transfer may occur with less effort because training in a realistic VRE could be more task-specific (see 19). In addition, VRE training could provide individuals with a recent amputation with a realistic experience of having a myoelectric prosthesis. Such experiences could help manage expectations of what a prosthesis can offer to the user, which in turn could decrease the relatively high abandonment rates.²⁰ Lastly, due to the fact that a virtual prosthesis is being controlled in the VRE, the user can train without the added weight of an actual prosthesis. This might seem like a disadvantage at first, since it could lead to a less realistic experience, but removing the weight could result in a lower workload, which in turn could lessen fatigue during training. This could mean that VRE training allows users to train the actual control of the prosthesis more frequently and for longer time periods.

Despite the expected benefits of VRE training, it is currently unknown whether skill transfer from VRE training to real prosthesis control occurs. We therefore aimed to examine skill transfer from the VRE training to actual prosthesis control which was achieved by measuring prosthesis functional control before and after training. Furthermore, we investigated differences in motivation and workload between training with a VRE and with a prosthesis simulator.

Methods

Ethical approval

The current study was approved by the Local Ethical Committee of the department of Human Movement Sciences of the University Medical Center Groningen, Groningen, the Netherlands (Research Registry number 11470). Before the experiment started, participants received an information letter and were asked to sign an informed consent form.

Participants

Participants for the current study were able-bodied individuals recruited from the student population of the Human Movement Science department at the University of Groningen, The Netherlands. Inclusion criteria included the participants to be 18 years or older, right-handed (tested using the Edinburgh Handedness Inventory²¹), to have normal or corrected vision, no injuries to their upper limbs and no experience with myoelectric prosthesis control.

Participants were assigned to one of two groups, the virtual reality training group (VR-TG) or the prosthesis simulator training group (PS-TG) in a pseudo-random manner. As we aimed to prioritize safe usage of the VRE, participants filled in the Motion Sickness Susceptibility Questionnaire (MSSQ) which determined the likelihood of a person experiencing motion sickness.²² We regarded an MSSQ score of 11.3 or higher as an indication that a participant was prone to having cybersickness.²² Participants with a high MSSQ score were only assigned to the PS-TG while participants with a low MSSQ score could be assigned to the PS-TG or the VR-TG.

Materials

Prosthesis simulator

The prosthesis simulator used during the pre-test, post-test and training sessions of the PS-TG (see Figure 1) consisted of a MyoPlus cuff using pattern recognition software from Ottobock (Ottobock Healthcare products, Austria), a Bebionic hand (Ottobock Healthcare products, Austria) which was attached to a modified brace with velcro straps (Rebound Post-Op Elbow, Össur), with a total weight of 1.62 kg. The simulator was attached to the participants’ arm using velcro straps while allowing them to move their elbow joint with a hinge mechanism. The length of the forearm section and upper arm section of the brace could be adjusted to ensure a perfect fit for each participant. The cuff was placed high on the forearm, approximately 10 cm distal to the elbow, ensuring that the active electrodes were located over the muscle bellies of the relevant muscles. Participants also wore a wrist brace (Wrist Lacer, Medical Specialties, USA) when operating the prosthesis simulator to facilitate them in producing isometric contractions. With this simulator the prosthesis hand was located below and slightly forward with respect to the participants’ actual hand, as can be seen in Figure 1.

Figure 1.

The prosthesis simulator with the rebound post-op brace from Össur, the MyoPlus cuff from Ottobock and the Bebionic prosthesis from Ottobock.

Before the prosthesis simulator could be operated, a calibration sequence had to be completed. This sequence started by connecting the MyoPlus system to a Samsung tablet (Galaxy Tab A8) via Bluetooth. When the connection was established, the participant had to calibrate the prosthesis while standing upright in three different postures: (1) the upper arm parallel to the torso with the elbow fixed at 90°, (2) the entire arm parallel to the torso (the elbow at 180°) and (3) the entire arm extended in front of the participant at shoulder height (elbow at 180° and shoulder at 90° flexion). Participants calibrated the prosthesis simulator for two grips, ‘Hand Open’ and ‘Hand Close’, which were activated by extending and flexing the wrist against the splint, respectively.

To assess functional prosthesis control, the following tests were used: the Cylinder test,^23,24 two tasks from the activities of daily living (ADL) section of the Southampton Hand Assessment Procedure (SHAP)²⁵ and a modified Box and Blocks test.²⁶

Cylinder test

For the Cylinder test, participants had to grasp three wooden cylinders of different diameters (2 cm, 4 cm and 6 cm, all had a height of 10 cm) with the prosthesis simulator. This was done in five trials per cylinder size (order was block randomized), while the opening angle of the prosthesis hand was measured using the Optitrack system (Flex 13 system with Motive software, NaturalPoint, Inc., USA) with markers placed on the hand’s dorsum, the distal thumb phalanx, the distal index finger phalanx and on top of each of the three cylinders. Prior to the start of the test, participants were instructed to stand in front of the table and open the prosthesis hand maximally and then close the hand to calibrate the distance between the markers to the maximal and minimal aperture. Participants were then instructed to start with a closed prosthesis hand and grasp one of the three cylinders which was placed in front of them at ±20 cm distance on the table, lift the cylinder about 5 cm and place it back down. The instructions participants received were to complete the task as accurately as possible.

SHAP

Two tasks of the ADL section of the SHAP were chosen: the zipper and jar lid tasks. These tasks were chosen because they showed the largest performance gradient between pre-test and post-test during pilot experiments. The instructions given to the participants were to open the zipper completely using the prosthesis simulator and to take off the lid of the jar with the prosthesis simulator while holding the jar in their intact (left) hand, respectively. Participants were instructed to record their own completion time by hitting a timer before they started and after they finished each task attempt. An attempt was stopped when a participant exceeded the cut-off time of 4 minutes, which was chosen based on participants’ performances during pilot measurements, since there are no set boundaries for the tasks of the SHAP specific for prosthesis users.^25,27,28 Our cut-off time was chosen to give participants enough time to complete the tasks, without fatigue affecting their performance on the subsequent tests and training session.

Box and Blocks

In the Box and Blocks test, participants had to move as many blocks as possible within 1 minute from the right to the left set-up compartment using the prosthesis simulator. Sixteen blocks were laid out in a grid pattern and could only be picked up one at a time. If all blocks were moved to the sides of the box, the participant was allowed to use their left hand to move the blocks to the center for better access with the prosthesis simulator.

Motivation

Four sections of the Intrinsic Motivation Inventory (IMI)²⁹ were used to measure motivation: the Enjoyment/Interest, Perceived Competence, Importance/Effort and Usefulness/Value subscales. Participants scored each item on a seven-point Likert scale at the end of the second training session (T2) and after finishing the functional tests of the post-test. The IMI was not applied during the pre-test, as participants could not rate their motivation regarding an unknown intervention.

Workload

The Prosthesis Task Load Index (PROS-TLX)³⁰ asked participants to first score workload on eight subsections on a 20-point scale and subsequently to assign a weight to each of the subsections, resulting in one score as outcome measure. For the same reasons as the IMI, this questionnaire was also filled in by participants at T2 and after finishing the functional tests of the post-test.

Training

The VR-TG - calibration

Before participants in the VR-TG could start, several calibrations needed to be performed. First, participants put on the Myoplus cuff and the wrist brace. In addition, participants were instructed to hold a tennis ball in their palm which was taped to their hand since previous research revealed that restricting movement provides more distinct EMG-signals.³¹ Then participants had to go through the calibration process of the Myoplus cuff which consisted of calibrating Hand Open and Hand Close for each of the three postures that were used to calibrate the prosthesis simulator.

The Meta Quest 2 virtual reality headset (Meta Platforms Technologies, CA, USA) was used during the training sessions. After the Myoplus cuff calibration was completed, the Meta Quest two right-handed controllers were attached to the participants’ right forearm, directly distal to the EMG cuff, using a velcro band (see Figure 2). After this was completed, the participant could put on the head-mounted display of the Meta Quest 2 and hold the left controller in their left hand.

Figure 2.

The experimental set-up of a participant in the virtual reality training group (VR-TG). The participant is wearing the VR headset, the right controller is strapped to the arm using a velcro band and the wrist motion is limited by a splint and tennis ball. The electrode (EMG) cuff is around the right arm at the location of the muscle bellies.

Before each participant could start the training, the following steps needed to be taken: First, the participant was guided to indicate the boundaries of the playable area, within which the participant could move freely. The playable area, which was a 3 m by 3 m square on the floor indicated by tape, was cleared before the start of the experiment to ensure safety among all participants. Furthermore, a researcher was present at all times during training sessions to fully ensure participants’ safety.

The virtual environment was calibrated to the participants’ eye height by touching the left controller to the ground. Finally, participants were instructed to cast what they saw on the screen of the head-mounted display to a laptop, so that the researchers could see what the participants were seeing, to monitor the training.

After the VRE was started, participants needed to place the virtual prosthesis hand in the correct position, that is, the position of their actual right hand. Therefore, participants fully extended both arms in front of them (shoulders at 90° flexion, extended elbows and wrists neutral). When this calibration was completed, the Myoplus cuff was connected to the Meta Quest 2 through Bluetooth.

The VR-TG – the VRE

The VRE serious game placed the participants in the role of a barista at a seaside mediterranean café. In the VRE, the participants’ avatars had an anatomical left hand which was controlled using buttons on the left-handed controller, and a virtual prosthetic right hand which was controlled by muscle activation of the forearm muscles and the orientation of the right controller. The goal within the VRE serious game was to complete orders of several drinks, indicated by a specific icon for each drink, using a number of consecutive actions: (1) grasp the correct cup (with the corresponding icon from the order) using the virtual prosthesis hand, (2) place the cup under the coffee machine and the hand should release the cup (3) press the button with the corresponding icon using the prosthesis hand, (4) grasp the full cup with the prosthesis hand, (5) place the full cup on a tray without spilling the liquid and (6) after completing the full order and as soon as all cups were put on the tray, a bell had to be rung (see Figure 3). When the bell was rung, the waitress would come over to check the order. If the order was correct, the participant would receive a tip in the form of coins which was based on their performance (i.e. if cups were dropped or liquid was spilled the tip would be lower). If the order was incorrect, the waitress would express her anger and indicate which drinks were incorrect and which needed to be completed again. Participants received feedback on their grasping force through a horizontal bar with a slider whenever a cup was grasped (see Figure 3(b)).

Figure 3.

A depiction of the participants’ view when training with the VRE serious game. (a) The participant had to grasp the correct cup to complete the order given by the waitress. The icons per cup, presented in the black square behind the cups, correspond to the icons on the order (which can be seen in C). This is the layout presented in level three where the cups were placed at different heights, for level 1 and 2 the cups were all stationed on the counter. (b) The cup was placed beneath the coffee machine and the participant had to press the corresponding button, which was indicated by the same icons. Shown here is the horizontal bar with slider which provides the participant with feedback on their grasping force. The colors from left to right indicate: white = the object has not been grasped yet, green = the object is grasped with sufficient amount of force and red = the object is grasped with too much force and will deform or break soon. (c) The order was completed and the full cups were placed on the tray. At this moment, the participant had to press the bell on the left side of the tray to alert the waitress. (d) The waitress came over to inspect whether the order was correct. In this example, the order was correct which prompted the participant to receive a tip presented in coins. The number of coins was determined based on how well the order was prepared (i.e. this number would be lower if the participant had dropped cups or spilled liquid).

The VRE serious game consisted of three levels. The first level consisted of two orders. The first order was used as a tutorial of how the goal of the VRE should be completed, which was done by showing participants blue indicators of what their next action needed to be and consisted of one drink. Subsequently, the second order, consisting of three drinks, was brought by the waitress. The drinks in the orders were randomized but in the first level these concerned only robust cups which were easy to grasp and were presented at waist height.

In the second level participants had to complete three orders of maximally five drinks per order. In this level, three new drinks were added: the first drink concerned a smaller cup which was harder to grasp, the second drink consisted of a flexible cup which could deform if grasped with too much force and the third drink was a glass cup which could break if too much force was applied during grasping; both were indicated with red on the force feedback bar.

The third level challenged the participants further by varying the height from where specific cups needed to be grasped from. The glass cup and smaller cup needed to be grasped from a shelf at shoulder height and the flexible cup needed to be grasped from a cupboard below the counter at knee height.

The PS-TG

Participants in the PS-TG performed their five training sessions in the real world. An experimental set-up was created to mimic the tasks of the VRE as best as possible. Three tables were placed in a ‘U’ shape with coffee cups presented on the left table, a coffee machine on the middle table and a tray on the right table, closely resembling the configuration of the counter in the VRE. Participants also needed to complete orders of specific drinks as in the VRE, in three separate levels. The differences between the levels were the same as in the VRE. However, a few adaptations to the complete set-up were made for practical reasons.

First, no liquid was involved in the experiment. A cardboard cut-out was made for each individual cup that fitted its circumference. To simulate the spilling of liquid, a ping-pong ball was placed on top of the cardboard after the button on the coffee machine was pushed. The ping-pong ball was not allowed to fall off; if it did, the participant had to start over. Second, instead of glass cups that could shatter, we substituted them with paper cups that could be crushed. Researchers overviewing the training sessions judged whether a paper cup would ‘break’ and if so, whether the participant should start over.

Design

The experiment had a pre-test/post-test design, with five training sessions in between tests, which took place over the course of 1 week on consecutive days starting on Monday and ending on Friday. In the first session the pre-test and the first training session were performed, while in the fifth session the fifth training session and the post-test were performed. Each training lasted approximately 25 minutes (i.e. close to the time it took to complete all three levels).

Outcome measures

Cylinder test

The data analysis of the Cylinder test was done using Matlab (2020a, The Mathworks Inc., USA). The raw data of each participant contained the 3D coordinates of all of the markers and the sample frequency of the Optitrack system was 120 Hz. The 3D coordinates of the hand, thumb and index finger were used to define the aperture angle of the prosthesis. To mitigate the effect of discontinuities in the data, the time series of the aperture angle was filtered using a moving average filter with a window length of 150 samples. The maximum aperture during grasping for each trial was calculated and then normalized based on the minimum and maximum opening angle of the prosthesis hand. Then, a regression line was plotted through the normalized maximum opening angle data over the five trials for each individual, cylinder size and test moment individually. The slopes of these regression lines and the normalized maximum aperture were used as outcome measures (see 24).

SHAP

The outcome measure was the completion time which was recorded by the participant pressing a timer.

Box and blocks

The outcome measure used was the number of blocks successfully transferred.

IMI and PROS-TLX

The outcome measure used was the calculated score, conform the instructions of each questionnaire. For the IMI this was done by averaging all subsections resulting in a score ranging from 0 to 7. For the PROS-TLX this was done by multiplying the score of each section by the weight, summing these multiplied scores and divide by 8, resulting in a score ranging from 0 to 70.

Statistical analysis

All analyses were performed in SPSS (SPSS (IBM Corp. Released 2020. IBM SPSS Statistics for Windows, Version 27.0. Armonk, NY: IBM Corp) apart from the effect size calculations of the ANOVA tests, which were done in R (R version 4.3.1 (2023-06-16 ucrt)). The analyses were done separately for the domains of functional control, motivation and workload.

Functional control

The Kolmogorov-Smirnoff test showed a normal distribution of the Box and Blocks data. A repeated measures ANOVA with Group (VR-TG vs PS-TG) as between subjects factor and Test moment (pre-test vs post-test) as within subjects factor was used. For the two tasks of the SHAP only the post-test scores were used because only few participants were able to complete the task in the pre-test (13 participants for the Jar Lid task and five for the Zipper task). In the post-test the number of included participants, after removing missing values and outliers, was 17 (8 in VR-TG, 9 in PS-TG) for the Jar Lid task and 13 (6 in VR-TG, 7 in PS-TG) for the Zipper task, hence a Mann-Whitney U test was used. For the Cylinder test two analyses were performed, one on the slopes of the regression lines and one on the maximum aperture. The Kolmogorov-Smirnoff test showed that the data was not normally distributed. For both analyses the main effect of Group was tested using a Mann-Whitney U test, the main effect of Cylinder Size was tested using a Friedman test and the main effect of Test Moment was tested using a Wilcoxon Signed Rank test. The interaction between Group and Test Moment was examined by calculating difference scores between pre-test and post-test and testing these between groups using a Mann-Whitney U test.

Motivation

We analyzed IMI data parametrically since the Kolmogorov-Smirnoff test showed they were normally distributed. A repeated measures ANOVA with Group (VR-TG vs PS-TG) as between subjects factor and Test moment (T2 vs post-test) and Subsection as within subjects factor was used.

Workload

The Kolmogorov-Smirnoff test showed that the data was normally distributed, therefore PROS-TLX data was tested parametrically. A repeated measures ANOVA with Group (VR-TG vs PS-TG) as between subjects factor and Test moment (T2 vs post-test) as within subjects factor was used.

Due to the large number of statistical tests, the decision was made to adjust the alpha to .01. Sphericity was tested using a Mauchlys test and if the result was significant, the degrees of freedom were corrected using the Greenhouse-Geisser. The effect sizes used were for the ANOVA tests the generalized eta-squared ( $η_{g}^{2}$ ), for the Mann-Whitney U and Wilcoxon tests the Wilcoxon r (r = z/√n) and for the Friedman test the Kendall’s W (W = χ²/n (k – 1)).

Results

A total of 20 participants were included, eight males and 12 females, with a mean age of 21.2 ± 1.9 years and a mean handedness of 90.5% ± 8.7 right-handed. Ten participants were assigned to the VR-TG, four males and six females, mean age 20.6 ± 1.4 years, mean handedness 88% ± 9.8 right-handed, and 10 participants were assigned to the PS-TG, four males and six females, mean age 21.8 ± 2.2 years and mean handedness 93% ± 7.1 right-handed. Almost all participants underwent the pre-test/post-test design with the five training sessions in between. However, the following missing values were identified: in the pre-test 13 participants (6 VR-TG, 7 PS-TG) of the Jar Lid task and seven participants (4 VR-TG, 3 PS-TG) of the Zipper task were able to complete the tasks. One participant in the VR-TG did not complete the post-test due to illness, therefore the VR-TG consisted of nine participants in the post-test analyses. Another participant in the VR-TG did not complete the PROS-TLX questionnaire at the T2 measurement since a section was forgotten to be filled in, hence eight participants in the VR-TG were included in that analysis. In addition, two participants in the VR-TG and two participants in the PS-TG were unable to perform the Zipper task in the post-test, so they were excluded from the analysis of that task resulting in 15 participants in that analysis (7 VR-TG, 8 PS-TG). For the Cylinder test, the data of one participant in the pre-test and one participant in the post-test was lost due to equipment failure, both participants were excluded from the analysis so there were 18 total participants included in that analysis (8 VR-TG, 10 PS-TG).

Unfortunately, there were software bugs present in the current version of the VRE which could have influenced our results. These bugs could be divided into two types: (1) not recognizing a correct order while it was in fact made correctly and (2) glitching when a cup was grasped which caused the cup to fall. These bugs happened 79 times in total occurring across all participants. In some cases, we were unable to fix the bugs instantly, which meant the training sessions needed to be stopped, this happened 8 times divided over the five sessions of four participants.

Functional control

Box and blocks test

The results of the Box and Blocks test showed a significant main effect of Test moment, F (1, 17) = 10.66, p = .005, $η_{g}^{2}$ = .13, indicating that more blocks were moved in the post-test than in the pre-test. No significant differences were found for the group or interaction effect, see Figure 4.

Figure 4.

Findings of the Box and Blocks test. The comparison between the pre-test and post-test is presented in boxplots with a significance of ** = p < 0.01.

SHAP

No significant differences were found for the Jar Lid task (mean VR-TG: 49.3 sec (21.2); mean PS-TG 26.7 sec (9.6)) and for the Zipper task of the SHAP (mean VR-TG: 67.2 (52.1); mean PS-TG: 80.6 (34.8)).

Cylinder test

No significant differences were found between the slopes of the regression lines. The analysis on the prosthetic hand aperture revealed a significant effect of Cylinder size (χ² (2) = 71.1, p < .001, effect size W = .44) and of Test moment (Z = −5.17, p < .001, effect size r = .32), see Figure 5(a) and (b) respectively. Pairwise comparisons with Bonferroni correction revealed a significant difference between the small and large object sizes (p < .001) and between the medium and large object sizes (p < .001), where the aperture was smaller for the small and medium cylinder sizes than the large cylinder size, respectively. The main effect of Test moment showed that participants used a smaller aperture in the post-test compared to the pre-test.

Figure 5.

(a) Boxplots of the normalized aperture for the different cylinder sizes used in the Cylinder test. Outliers are shown in small circles. (b) Boxplots of the normalized aperture of the prosthetic hand for the pre-test and post-test of the Cylinder test. Outliers are shown in small circles. Significant differences in both A and B are indicated as: *** = p < .001.

Motivation

The analysis of the IMI showed a significant main effect for Test moment, F (1, 18) = 16.79, p < .001, $η_{g}^{2}$ = .04, indicating that an overall increase in motivation was found between T2, median 5.0 (IQR 1.41), and the post-test, median 5.4 (IQR 1.17). Also, a significant main effect of Subsection was found, F (3, 54) = 25.97, p < .001, $η_{g}^{2}$ = .36, indicating that the overall score was different between the subsections. After post-test comparisons with Bonferroni correction, differences showed that the Effort/Importance and Perceived Competence subsections scored lower than the Enjoyment/Interest and the Usefulness/Value subsections, see Figure 6 for the pairwise comparisons and corresponding p-values.

Figure 6.

Boxplots of the different sections of the IMI with separate plots for the 2nd training and the post-test. It can be seen that for only for the Effort/Importance and Perceived Competence section a significant change occurred during training, indicating an interaction effect. Significant differences are indicated using ** = p < .01, *** = p < .001.

In addition, a significant interaction effect between Test moment and Subsection was found, F (3, 54) = 7.00, p = .002, $η_{g}^{2}$ = .05. Further testing using a paired t-test showed that a significant increase from T2 to post-test was only found for the Perceived Competence section (t (19) = −3.55, p = .002) and the Effort/Importance section (t (19) = −3.56, p = .002), see Figure 6. No significant differences between groups were found.

Workload

The workload did not differ between the groups. Also, no differences were found between pre-test and post-test. The mean score at T2 for the VR-TG was 40.0 (8.1) and for the PS-TG 46.1 (5.4). The mean score at the post-test for the VR-TG was 37.1 (7.7) and for the PS-TG 41.5 (10.3).

Discussion

The present study set out to examine skill transfer of prosthesis control after training with a VRE serious game, which was done by analyzing differences in functional control, motivation and workload between VRE training and prosthesis simulator training. This was achieved using a newly developed VRE with game-like elements specifically designed for upper limb prosthesis training. What we found was that participants improved from pre-test to post-test on functional tests. These improvements did not differ between training groups indicating that the training methods used in each group provided similar improvement with regard to functional prosthesis control. This can be interpreted as the occurrence of skill transfer from VR training to actual prosthesis control.

We also found that motivation improved from T2 to the post-test, most notably in the perceived competence section and the importance section of the IMI. Similar to the functional control results, this improvement did not differ between groups, which indicated that in the motivation aspects both training methods also provided comparable results. This is an unexpected finding because participants were expected to enjoy the VR training more than the simulator training due to the novelty of the VRE and the gamification aspects that were included. However, when looking at the absolute scores of the IMI, we found medium to high scores across both groups. This ceiling effect might explain why no change was found. Moreover, in the current version of the VRE, the serious gaming element was not exploited to its full potential, leaving room for the VRE training to increase motivation (see 32). It has to be noted that motivation scores in healthy participants might not be generalizable to prosthesis users, which is why future research regarding training motivation in prosthesis users is necessary.

In the workload domain, we also found no differences between the groups. The data showed that both groups had a score of approximately 40 on a scale of 0–70, which translates to a medium-high workload. This score was not surprising for the PS-TG because they were operating a relatively heavy prosthesis simulator during their training sessions, but it was surprising to see a similar score for the VR-TG. The high workload in the VR-TG could be a result of the technical issues we encountered with this early version of the VRE. Participants reported frustration and increased fatigue during the occurrence of bugs, which forced participants to repeat the same motion multiple times. This could explain the high workload in the VR-TG.

Virtual reality in rehabilitation

In the recent literature, many articles show the benefits that different forms of VR can have for rehabilitation training. Research shows that different types of VR training could be an effective form of training for patients with stroke,³³ Parkinson patients,³⁴ neurorehabilitation,³⁵ soldiers³⁶ and upper limb prosthetic users.^37–44 These studies show that training with a form of VR could result in a decrease in stress, a decrease in workload, an increase in motivation and provide skill transfer to actual functional skills (also see Levac et al., 2019⁴⁵). The results of the current study are in line with these findings in showing that training with a VRE could have multiple benefits in rehabilitation. However, the current study also adds to the existing literature by (1) explicitly assessing skill transfer in virtual reality instead of an augmented reality set-up and by (2) comparing the VRE training effects to conventional prosthesis training using a prosthesis simulator.

For upper limb prostheses specifically, multiple studies suggest that loading the affected limb to simulate the weight of a prosthesis should be done to make VRE training more effective.^38,41,43 However, our results suggest that VRE training without limb loading seems to be an effective method, at least in able-bodied individuals. It should be noted that while we used a VRE, other studies used a form of augmented reality which could be a reason why the results of these studies would not be comparable to ours. Furthermore, the current VRE allows participants to roam around the playable area, which is not possible in the other systems, which could influence immersion and possibly skill transfer. Therefore, future research seems to be necessary to determine what type of VR system is most effective for training prosthesis control and whether limb loading is relevant, especially in prosthesis users, to create a training with the highest effectiveness.

Strengths and limitations

The strengths of the current study are mostly reflected in the design of the VRE. We have already discussed positive design aspects of the system such as the inclusion of real-time feedback of the grasping force, the objects that were presented at different heights, forcing participants to use a wide range of postures in the arm and the inclusion of breakable objects, which stimulated participants to learn dexterous control. However, one of the major benefits of the system is that skill transfer was observed, which is the most clinically important finding. Based on the current findings, training with the current VRE improved actual prosthesis simulator control, suggesting that VRE training could be valuable for future rehabilitation training programs and benefit individuals with a recent amputation. However, validation of these training effects in prosthesis users, needs to be determined in future research.

However, the current VRE design also had drawbacks such as the number of technical issues we faced. As was mentioned in the results section, since we used an early version of the VRE, bugs were encountered during the execution of this study. Furthermore, by performing the current study with able-bodied individuals, it is questionable whether the results are generalizable across the population of arm prosthesis users. In addition, only participants who scored low on the MSSQ were assigned to the VR-TG, which may have introduced a selection bias. In hindsight, with regard to the use of the IMI to determine motivation, the addition of an interview might have been useful.

Future research

Based on the results of the current study compared with the existing literature, future research should expand the possibilities of the VRE further. One way this can be done is adding more tasks to the VRE system such as bimanual tasks or tasks that resemble certain hobbies an individual might have. Providing more variation in the VRE training, and eliminating the technical issues, may reduce the workload. A lower workload would allow for longer training sessions, which might lead to a more effective training program. Moreover, as a next step to continue the current study, the VRE training could be compared to usual care of prosthesis users to examine differences. Furthermore, future research should focus on exploring whether limb loading during VRE training would consolidate or even enhance skill transfer while maintaining a low workload. Such a system would most likely be a highly effective training tool from which all prosthesis users could benefit.

Conclusions

In the current study we set out to examine skill transfer from VR training to real-world prosthesis use and we compared these training effects with conventional prosthesis training using a prosthesis simulator. Skill transfer was established, which seems promising for the clinical applicability of the VRE. In addition, we found no differences in motivation and workload between VRE training and prosthesis simulator training. However, the current state of the VRE could have contributed to a higher workload compared with a finalized version of the VRE, this needs to be examined in future research. With these findings in mind, the claim can be made that VR training could be an effective method to learn prosthesis control.

Statements and declarations

Footnotes

Acknowledgments

We would like to thank The Simulation Crew (Nijmegen, The Netherlands) for the collaboration in this project and the development of the virtual reality environment.

Author contributions

Conception: CvdS and RM.

Performance of work: BM, BvD and JT.

Interpretation or analysis of data: BM and JT.

Preperation of the manuscript: BM.

Revision for important intellectual content: AM, CvdS and RM.

Supervision: AM, CvdS and RM.

Conflicting interests

The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: The authors report there are no competing interests to declare. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 965731.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: HORIZON EUROPE Framework Programme (965731).

Ethical considerations

ORCID iDs

Bart Maas

Jack Tchimino

Alessio Murgia

Corry K van der Sluis

Raoul M Bongers

References

Kristoffersen

Franzke

Van Der Sluis

, et al. The effect of feedback during training sessions on learning pattern-recognition-based prosthesis control. IEEE Trans Neural Syst Rehabil Eng 2019; 27: 2087–2096. DOI: 10.1109/TNSRE.2019.2929917.

Powell

Thakor

. A training strategy for learning pattern recognition control for myoelectric prostheses. J Prosthet Orthot 2013; 25: 30–41. DOI: 10.1097/JPO.0b013e31827af7c1.

Kristoffersen

Franzke

van der Sluis

, et al. Serious gaming to generate separated and consistent EMG patterns in pattern-recognition prosthesis control. Biomed Signal Process Control 2020; 62: 102140. DOI: 10.1016/j.bspc.2020.102140.

Powell

Kaliki

Thakor

. User training for pattern recognition-based myoelectric prostheses: improving phantom limb movement consistency and distinguishability. IEEE Trans Neural Syst Rehabil Eng 2014; 22: 522–532. DOI: 10.1109/TNSRE.2013.2279737.

Kuiken

Miller

Turner

, et al. A comparison of pattern recognition control and direct control of a multiple degree-of-freedom transradial prosthesis. IEEE J Transl Eng Health Med 2016; 4: 2100508. DOI: 10.1109/JTEHM.2016.2616123.

Di Pino

Guglielmelli

Rossini

. Neuroplasticity in amputees: main implications on bidirectional interfacing of cybernetic hand prostheses. Prog Neurobiol 2009; 88: 114–126. DOI: 10.1016/j.pneurobio.2009.03.001.

Makin

Flor

. Brain (re)organisation following amputation: implications for phantom limb pain. Neuroimage 2020; 218: 116943. DOI: 10.1016/j.neuroimage.2020.116943.

Vasluian

Bongers

Reinders-Messelink

, et al. Learning effects of repetitive administration of the southampton hand assessment procedure in novice prosthetic users. J Rehabil Med 2014; 46: 788–797. DOI: 10.2340/16501977-1827.

Terlaak

Bouwsema

Van Der Sluis

, et al. Virtual training of the myosignal. PLoS One 2015; 10: e0137161. DOI: 10.1371/journal.pone.0137161.

10.

Verrienti

Raccagni

Lombardozzi

, et al. Motivation as a measurable outcome in stroke rehabilitation: a systematic review of the literature. Int J Environ Res Publ Health 2023; 20: 4187. DOI: 10.3390/ijerph20054187.

11.

McNaughton

. Intrinsic motivation. Practical Neurol 2023; 23: 489–492. DOI: 10.1136/pn-2023-003763.

12.

Roche

Vujaklija

Amsüss

, et al. A structured rehabilitation protocol for improved multifunctional prosthetic control: a case study. J Vis Exp 2015; 2015: e52968. DOI: 10.3791/52968.

13.

Boschmann

Neuhaus

Vogt

, et al. Immersive augmented reality system for the training of pattern classification control with a myoelectric prosthesis. J NeuroEng Rehabil 2021; 18: 25. DOI: 10.1186/s12984-021-00822-6.

14.

Tabor

Bateman

Scheme

. Evaluation of myoelectric control learning using multi-session game-based training. IEEE Trans Neural Syst Rehabil Eng 2018; 26: 1680–1689. DOI: 10.1109/TNSRE.2018.2855561.

15.

Van Dijk

Van Der Sluis

Van Dijk

, et al. Learning an EMG controlled game: task-specific adaptations and transfer. PLoS One 2016; 11: e0160817. DOI: 10.1371/journal.pone.0160817.

16.

Prahm

Vujaklija

Kayali

, et al. Game-based rehabilitation for myoelectric prosthesis control. JMIR Serious Games 2017; 5: e3. DOI: 10.2196/games.6026.

17.

Winslow

Ruble

Huber

. Mobile, game-based training for myoelectric prosthesis control. Front Bioeng Biotechnol 2018; 6: 94. DOI: 10.3389/fbioe.2018.00094.

18.

Razak

Hashim

Gholizadeh

. Video games based rehabilitation approach for upper limb amputee (preprint). JMIR Serious Games 2021; 9: 1–10. DOI: 10.2196/17017.

19.

Savelsbergh

Van Der Kamp

. Information in learning to co-ordinate and control movements: is there a need for specificity of practice? Int J Sport Psychol 2000; 31: 467–484.

20.

Biddiss

Chau

. Upper limb prosthesis use and abandonment: a survey of the last 25 years. Prosthet Orthot Int 2007; 31: 236–257. DOI: 10.1080/03093640600994581.

21.

Oldfield

. The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia 1971; 9: 97–113.

22.

Golding

. Motion sickness susceptibility questionnaire revised and its relationship to other forms of sickness. 1999.

23.

Bouwsema

Van Der Sluis

Bongers

. Clinical Biomechanics Movement characteristics of upper extremity prostheses during basic goal-directed tasks. JCLB 2010; 25: 523–529. DOI: 10.1016/j.clinbiomech.2010.02.011.

24.

Van Dijk

Van Der Sluis

Van Dijk

, et al. Task-oriented gaming for transfer to prosthesis use. IEEE Trans Neural Syst Rehabil Eng 2016; 24: 1384–1394. DOI: 10.1109/TNSRE.2015.2502424.

25.

Light

Chappell

Kyberd

. Establishing a standardized clinical assessment tool of pathologic and prosthetic hand function: normative data, reliability, and validity. Arch Phys Med Rehabil 2002; 83: 776–783. DOI: 10.1053/apmr.2002.32737.

26.

Mathiowetz

Volland

Kashman

, et al. Adult norms for the box and block test of manual dexterity. Am J Occup Ther 1985; 39(6): 386.

27.

Resnik

Borgia

Cancio

, et al. Psychometric evaluation of the Southampton hand assessment procedure (SHAP) in a sample of upper limb prosthesis users. J Hand Ther 2023; 36: 110–120. DOI: 10.1016/j.jht.2021.07.003.

28.

Burgerhof

JGM

Vasluian

Dijkstra

, et al. The Southampton Hand Assessment Procedure revisited: a transparent linear scoring system, applied to data of experienced prosthetic users. J Hand Ther 2017; 30: 49–57. DOI: 10.1016/j.jht.2016.05.001.

29.

Ryan

Mims

Koestner

. Relation of reward contingency and interpersonal context to intrinsic motivation: a review and test using cognitive evaluation theory. J Pers Soc Psychol 1983; 45: 736–750.

30.

Parr

JVV

Galpin

Uiga

, et al. A tool for measuring mental workload during prosthesis use: the Prosthesis Task Load Index (PROS-TLX). PLoS One 2023; 18: e0285382. DOI: 10.1371/journal.pone.0285382.

31.

Kristoffersen

Franzke

Van Der Sluis

, et al.

Should hands be restricted when measuring able-bodied participants to evaluate machine learning controlled prosthetic hands?

IEEE Trans Neural Syst Rehabil Eng 2020; 28: 1977–1983. DOI: 10.1109/TNSRE.2020.3007803.

32.

Garske

Dyson

Dupan

, et al. Serious games are not serious enough for myoelectric prosthetics. JMIR Serious Games 2021; 9: 1–12. DOI: 10.2196/28079.

33.

Wang

Chen

Wong

AMK

, et al. Game-based virtual reality system for upper limb rehabilitation after stroke in a clinical environment: systematic review and meta-analysis. Game Health J 2022; 11: 277–297. DOI: 10.1089/g4h.2022.0086.

34.

Pazzaglia

Imbimbo

Tranchita

, et al. Comparison of virtual reality rehabilitation and conventional rehabilitation in Parkinson’s disease: a randomised controlled trial. Physiotherapy 2020; 106: 36–42. DOI: 10.1016/j.physio.2019.12.007.

35.

Georgiev

Georgieva

Gong

, et al. Virtual reality for neurorehabilitation and cognitive enhancement. Brain Sci 2021; 11: 1–20. DOI: 10.3390/brainsci11020221.

36.

Lackey

Salcedo

Szalma

, et al. The stress and workload of virtual reality training: the effects of presence, immersion and flow. Ergonomics 2016; 59: 1060–1072. DOI: 10.1080/00140139.2015.1122234.

37.

Woodward

Hargrove

. Adapting myoelectric control in real-time using a virtual environment. J NeuroEng Rehabil 2019; 16: 1–12. DOI: 10.1186/s12984-019-0480-5.

38.

Hunt

Sun

Wang

, et al. Limb loading enhances skill transfer between augmented and physical reality tasks during limb loss rehabilitation. J NeuroEng Rehabil 2023; 20: 16. DOI: 10.1186/s12984-023-01136-5.

39.

Levin

Weiss

Keshner

, et al. Emergence of virtual reality as a tool for upper limb rehabilitation: incorporation of motor control and motor learning principles. Phys Ther 2015; 95(3): 415.

40.

Boschmann

Neuhaus

Vogt

, et al. Immersive augmented reality system for the training of pattern classification control with a myoelectric prosthesis. J NeuroEng Rehabil 2021; 6: 1–15. DOI: 10.1186/s12984-021-00822-6.

41.

Ramirez-Fernandez

Moran

Garcia-Canseco

. Haptic feedback in motor hand virtual therapy increases precision and generates less mental workload. In: Proceedings of the 2015 9th international conference on pervasive computing technologies for healthcare, PervasiveHealth 2015, Istanbul, Turkey, 20–23 May 2015, pp. 280–286. DOI: 10.4108/icst.pervasivehealth.2015.260242.

42.

Shi

, et al. Current status and clinical perspectives of extended reality for myoelectric prostheses: review. Front Bioeng Biotechnol 2023; 11: 1334771. DOI: 10.3389/fbioe.2023.1334771.

43.

Chappell

Son

Clark

, et al. Virtual reality pre-prosthetic hand training with physics simulation and robotic force interaction. IEEE Rob Autom Lett 2022; 7: 4550–4557. DOI: 10.1109/LRA.2022.3151569.

44.

Gaballa

Cavalcante

Lamounier

, et al. Extended reality “X-reality” for prosthesis training of upper-limb amputees: a review on current and future clinical potential. IEEE Trans Neural Syst Rehabil Eng 2022; 30: 1652–1663. DOI: 10.1109/TNSRE.2022.3179327.

45.

Levac

Huber

Sternad

. Learning and transfer of complex motor skills in virtual reality: a perspective review. J NeuroEng Rehabil 2019; 16: 121. DOI: 10.1186/s12984-019-0587-8.

Skill transfer of upper limb prosthesis control after training in a virtual reality environment

Abstract

Background

Objective

Methods

Results

Conclusion

Keywords

Introduction

Methods

Ethical approval

Participants

Materials

Prosthesis simulator

Cylinder test

SHAP

Box and Blocks

Motivation

Workload

Training

The VR-TG - calibration

The VR-TG – the VRE

The PS-TG

Design

Outcome measures

Cylinder test

SHAP

Box and blocks

IMI and PROS-TLX

Statistical analysis

Functional control

Motivation

Workload

Results

Functional control

Box and blocks test

SHAP

Cylinder test

Motivation

Workload

Discussion

Virtual reality in rehabilitation

Strengths and limitations

Future research

Conclusions

Statements and declarations

Footnotes

Acknowledgments

Author contributions

Conflicting interests

Funding

Ethical considerations

ORCID iDs

References