Abstract
Benefits of immersive virtual reality rehabilitation (VRR) include increased motivation and improved transfer of skills to real-world tasks. The introduction of Oculus hand-tracking technology allowed for the development of VRR games that do not need virtual reality (VR) hand controllers. This is beneficial as participants with upper limb impairments/injuries may have difficulties with/be limited in using/manipulating VR hand controllers. In this project, a VRR game was developed and evaluated. The aim of this study was to determine patient experience when using VRR as an adjunct to upper-limb rehabilitation. N = 20 participants receiving upper limb rehabilitation completed a series of VRR tasks by playing the “smoothie bar” VRR game. After the completion of the VRR tasks, the participant experience was evaluated via a study-specific questionnaire. Key findings include 95% agreement that VRR tasks were fun and engaging and 75% agreed that VR tasks will be helpful to include in their rehabilitation. Hands-tracking VRR has a high potential to be used as an adjunct intervention in upper limb rehabilitation.
Introduction
Impairments to upper limb function can result from conditions, injuries and illnesses such as neurological, orthopedic and trauma. 1 The level of impairment and the impact on the individual can range from mild to severe. 1 This change in upper limb function impacts how individuals participate in daily activities such as using cutlery or driving. 1 The severity of the impairment is indicative of the rehabilitation interventions required, with intervention times ranging from days to months. 1 In some cases, medical or surgical interventions are provided in conjunction with therapy interventions to restore as much function to the upper limb as possible. The clinical rehabilitation pathways and time frames are determined by the condition, illness or injuries sustained. The timing and intensity of the rehabilitation programme are determined by the severity of the impairment. Interventions can include various activities and exercises to improve movement and engagement in daily activities. Therefore, it is timely to look at rehabilitation interventions to ensure practice is contemporary.
Background
Occupational therapy interventions for upper limb rehabilitation are person-centered and are designed to facilitate the return to everyday tasks. 2 Restorative rehabilitation involves repetition, practice and rehearsal of movements and activities until improvements across several domains (including but not limited to range of motion, pinch and grip strength and dexterity) have been made. 3 The modalities which occupational therapists (OTs) use to deliver their interventions are wide-ranging, and are often limited by physical space, resources and time. 4 A core principle of occupational therapy practice is engagement in activities therefore the interventions provided need to be meaningful to the individuals receiving it. 5 This in turn has resulted in OTs looking for contemporary and alternative interventions for the individuals with whom they work. Virtual reality has been increasingly used in rehabilitation as a contemporary method of delivery, this is known as virtual reality rehabilitation (VRR). 6 Examples include cognitive training for memory and attention and retraining of motor function of the upper limb and lower limb, with VRR used on its own or as an adjunct to traditional rehabilitation approaches or other technology-based rehabilitation approaches such as treadmills or computer-assisted rehabilitation tools.7-9
This can be beneficial to occupational therapy practice as it enables the creation of person-relevant and environment-relevant scenarios to support a return to occupational performance of daily tasks in real life. Features of VRR include simulated environments and gamification features to reward effort and engagement, and the ability to tailor the VRR activities to the individual patient. 10 An earlier pilot study demonstrated that these features of immersive VRR can enhance upper limb rehabilitation. 11 However, as this study was conducted with individuals who had burn injuries, some participants had difficulty holding the controllers. Indeed, fine motor skills and grasping are common challenges encountered in studies using VRR in participants with upper limb impairment. 12 Therefore, the need to explore controller—free technology was indicated.13,14
The Oculus hand-tracking Software Development Kit was released in December 2019.15,16 Immersive VRR with hand-tracking technology can lead to movements which are more reflective of real-world actions. A study by Coox et al 17 evaluated the usability and comfort of a VRR application using hand-tracking technology. This study created an application programming interface rehabilitation library which contained various hand interactions to replicate daily activities such as gesturing. 17 This immersive VRR technology and the application programming interface rehabilitation library was low cost and had the potential to be used as an adjunct to occupational therapy upper limb rehabilitation.
There is limited published evidence for the use of a novel hand-tracking tool in upper limb rehabilitation. Existing studies mostly rely on external input devices such as sensor systems and motion tracking gloves, and report technical challenges that do not support routine adoption in clinical settings, especially if considering the use of VR headsets alone presents challenges in terms of staff confidence and lack of technical support availability. 13 Therefore, there is a need to explore the use of self-contained VR systems that can be used across diverse groups of individuals receiving occupational therapy upper-limb rehabilitation. The primary aim of this study was to determine patient experience using VRR as an adjunct to upper-limb rehabilitation.
Methods
Study Design
This study was cross-sectional in design across two sites—acute care setting and subacute setting.
Ethical Approval
Ethical approval was granted by the local hospital and health service (which has governance for both acute and subacute services) human research ethics committee HREC/2021/QRBW/76983.
Design and Development of VRR Game
This stage involved OTs, engineers and an artist codesigning a VRR environment using the Unity game engine.
18
Key components identified by the research team were:
The game had to be relevant to real work tasks to allow for the transfer of learning The game had to increase engagement and use positive reinforcement to encourage improvement and for repetitions to be meaningful.
In a previous study, we identified that the design of VR environments for clinical use should account for cultural and geographic relevance, context of use, appropriate levels of agency and interactivity.
19
Two environmental designs were therefore considered (a) an artist's studio and (b) a smoothie bar. The smoothie bar was chosen as meal preparation was deemed to be a more universal activity of daily living. The environment was modified to allow the user to reach for assets while remaining stationary. Interactable assets were added to the bench top. Figure 1. The narrative for the game was then added using a custom Unity Game Object called “Game Sequence.” Gameplay mechanics were used to create the interface and rules for the game.

Blender asset.
Participants
Individuals attending occupational therapy upper limb therapy at an acute tertiary public hospital which includes services for burns, neurology, neurosurgery, orthopedic surgery, plastics and reconstructive surgery and rheumatology were invited to participate in the study. Individuals receiving inpatient rehabilitation at a public health facility were also invited to participate in the study. Individuals were not eligible for the study if the VR interventions occurred on the day of their first occupational therapy session at both facilities.
Inclusion Criteria
Age 18–85 years (male or female)
Attending occupational therapy services for treatment of an upper limb injury/trauma
Activity tolerance for 20 min of upper-limb movement while standing or sitting as determined by their treating OT providing their usual care
Existing right, left, or bilateral upper limb impairment
Sufficient cognitive function and communication to follow instructions and participate in the session.
Exclusion Criteria
Facial burns which were unhealed
History of dizziness or nausea likely to interfere with the ability to use the virtual reality headset for more than 30 min
History of epilepsy or seizures which may be triggered by the virtual reality headset
Participants who wear large, framed glasses that did not fit inside the virtual reality headset
Upper-limb amputations
Severe binocular vision abnormalities, blindness, significant visual perceptual deficits.
Recruitment
Potential participants were identified by OTs nominated for this study. All eligible participants were invited to participate in the study and given a participant information and consent form by the research team, who were not the primary OTs providing usual care. Participants were only enrolled in the study upon receipt of written informed consent.
Procedure
Stage 1: Presession (5 min)
Participants were given a description of the VR Task Protocol, prior to commencement of the session.
The OT clinician selected the VR game that was most suitable to compliment the participant's existing rehabilitation plan. This was aligned with their preexisting therapy goals which is standard practice at both facilities. The treating OT selected one VRR task configuration for each participant. VRR task configuration could be a bilateral task or a unilateral task. This was recorded on a Presession Data Collection Form (Appendix 1).
Stage 2: VR Task (20 min)
Participants were seated at the center of a dedicated virtual reality space. The Oculus Quest VR headset was given to the participant and fitted as per the safety and instruction manual. The participants confirmed that the head strap was secure and comfortable and that the VR display was focused.
A member of the research team read the written script to guide the participant to minimize variations in instructions while the OT independently modulated the session.
Participants completed a series of VR games, which included the following details:
Initially, from within the virtual smoothie bar environment, the participant was guided through practice tasks to understand the basic hand motions required within the game, such as pinching and grasping with different fingers. All task instructions included visual cues to direct them on how to use their hands to interact with virtual objects. Objects were within reaching distance and did not require the participant to mobilize while wearing the headset.
Participants were then instructed to move specific virtual objects (fruit) of various sizes from one location to another (from kitchen bench to blender) with precision and speed to create a smoothie. The emphasis was placed on the precision metric by communicating to the participant a higher reward (eg points) for precision. The Oculus Quest VR headset does not provide direct access to the headset cameras or the computer vision system. Instead, it provides the wrist pose, poses of each hand position, and tracking confidence. 18 Within each frame, the system reports whether that data is “invalid,” “low confidence,” or “high confidence.” The development team implemented additional filtering and extrapolation methods to improve the system's accuracy and reliability. 18 The first preprocessing layer was designed to further assess tracking quality when the estimated confidence is low. Thresholds of maximum position, velocity, acceleration, and rotation change were used to empirically evaluate the error. 18 To reduce jitter, a first-order low-pass filter was applied to the position and orientation of all the hands, and an exponentially weighted moving average filter was employed. 18 When tracking is lost or data becomes unusable for a short period of time, ie either the systems report the data is “invalid” or our method determines a “low confidence” data is invalid, the hand position is extrapolated for a short time based on its previous velocity. 18 Only the velocity of the entire hand is interpolated, while hand rotation and individual joints are kept static. This was done to prevent extrapolation into invalid hand poses, such as ones beyond the average amplitude of motion of the fingers or wrist. 18
They were also instructed to grasp the virtual objects in a specific manner via visual hand demonstrations shown in the virtual environment.
After the completion of the VR tasks, the participant removed the headset with the assistance of a member of the research team. The headset was cleaned prior to use with the next participant as per local infection control guidelines.
Stage 3: Postsession Questionnaire (5–7 min)
Participants were asked to fill out a deidentified questionnaire (Appendix 1) on their experience immediately after completing the VR task. This included questions pertaining to the VR headset and experience with the VR game. This questionnaire was developed for this study to determine experience with the new VR game. This included open questions and Likert-scale questions. No identifying data was captured. Data from the questionnaires and performance metrics recorded by the VR system was deidentified and uploaded to the secure UQ Research Data Manager system.
Data Analysis
Descriptive statistics was used to quantitatively describe and summarize outcomes from the data collection. SPSS statistical package (Version 28.0, IBM, NY, USA) was used to analyze numerical data from the Likert Scales.
Results
Twenty-three potential participants were identified for inclusion in this study. Three individuals chose not to participate; therefore, the participation rate was 87%. Twenty participants were enrolled in this study, n = 12 were in the acute care setting and n = 8 were in the subacute setting. The mean age of the participants was 49.5 years (range 19-81 years). Fifty-five percent of participants had not used VR prior to enrolling in the study. Table 1 provides a summary of participant diagnoses.
Diagnosis.
Ninety-five percent of individuals who participated in this study were right-hand dominant. Figure 2 outlines the limb side which was impacted by the participants’ diagnosis.

Limb(s) affected by diagnosis.
The bilateral task configuration option was chosen for 19 participants with 37% of participants assigned to target both upper limbs, 37% of participants assigned to target the left upper limb only and 26% of participants assigned to target the right upper limb only. One participant was assigned the unilateral configuration targeting the left upper limb only due to the limited range of motion in their right upper limb which was not requiring intervention.
Participants were asked to provide feedback regarding the Oculus Quest headset during the study period. Table 2 outlines the feedback provided using a Likert scale pertaining to comfort, image quality, sound quality and movement limitations.
Feedback on VR Headset.
Abbreviation: VR: virtual reality.
Participants were asked questions pertaining to the levels of enjoyment, satisfaction, and sense of task meaningfulness during VR. Table 3 outlines a number of participants’ responses to each question.
Number of Responses to Questions Pertaining to Enjoyment, Satisfaction, and Use of VR.
Abbreviation: VR: virtual reality.
A free text box was available at the end of the questionnaire to allow participants to provide subjective feedback. Six participants provided feedback in this section. My injured hand had difficulty to open up to release objects might be good to have several tolerance levels depending on subject’s ability
Fun exercise, strange without the tactile feedback though
Fun
Had difficulty viewing the instructions screen through the blender, hard movements were difficult at times
Text on screen requires lifting head to remove blurriness, instruct user prior could be helpful
Would be good for more complex tasks to reflect stages of rehab
Discussion
The results of this study found participants found the VR tasks were fun and engaging and there was positive support to include this as if their ongoing rehabilitation interventions. All participants were pleased with their experience of the virtual setting chosen for the VR rehabilitation tasks. Satisfaction during rehabilitation is a part of therapeutic interventions and improves the overall experience with healthcare providers.5,20 Interruptions to movement and functions of the upper limb as a result of injury, illness or a condition can have many negative impacts on an individual's participation in meaningful activities across the domains of self-care, productivity, and leisure. 20 The impacts can be both physical and psychological with individual experiences being varied and lasting different lengths of time. 1 Sequala as a result of their injury, illness or conditions such reduced power, pain or hypersensitivity impact on participation in occupational therapy upper limb rehabilitation.1,5 However as occupational therapy is a client-centered profession, clinicians strive to use rehabilitation interventions which are realistic and purposeful to the individuals receiving therapy.2,5 VRR has been successfully used in upper limb rehabilitation post subacute and chronic stroke21,22 prosthetic training, 23 spinal cord injuries, 24 acquired brain injuries 25 and burn injuries.26,27 Seventy-five percent of participants in this study reported that VR would be helpful to include as part of their rehabilitation programme.
However, there are many upper limb injuries, illnesses, and conditions where VRR has not been used as a therapy tool or as an adjunct to traditional occupational therapy practice. The results of this study show positive support for to use of a VR experience as an adjunct to their rehabilitation. The participants in this study had a wide range of upper limb injuries, illnesses and conditions which is reflective of individuals receiving occupational therapy interventions. 28 The inclusion of a more diverse group of individuals in this study suggested that future studies in each specific population are required.
VR has become more commonplace in daily life and with this uptake, the graphics and features have become more realistic, 29 as a result, there is an opportunity to use VR to stimulate real-life daily activities. As occupational therapy practice is founded on participation in everyday tasks 20 this was an ideal opportunity to create a VRR experience which had wide applicability, for a range of individuals. The design of the VR experience in this study was achieved through collaboration between the engineering team and the occupational therapy clinicians to create a VR experience that could be meaningful to real-life tasks. 18 While the activity chosen was to prepare a smoothie, this is an inclusive activity which could be used for a broader range of individuals and may be beneficial for use with older children. While some participants had previous experiences with VR, 95% of participants in this study reported the VR tasks to be fun and engaging. Individuals in this study presented with both unilateral and bilateral upper limb impairments increasing the generalizability of the findings of this study. The VR experience could be tailored by the OT for each individual, to target the side of upper limb dysfunction or to encourage bilateral function with all participants reporting they were pleased with the virtual setting chosen.
The results of this study demonstrate that the VR headsets and the quality of sounds and images were acceptable to the participants with no perceived restrictions in their movement during VRR. These findings are more positive compared to Mosadeghi et al study where many participants found the headsets uncomfortable. 30 Yan et al found in their study that the weight of the headset also impacted comfort levels, 31 however this was not reported by any participants in the current study. Only 5% of the participants in this study disagreed that VRR was fun and engaging, this is similar to the findings of the Cimolino et al study in the spinal cord injury cohort. 32 Positive engagement in activities increases an individual's inclination to participate in upper limb rehabilitation.1,20,27 The findings in the current study report that 75% of the participants indicated that the VR tasks would be helpful to include as part of their rehabilitation programme.
Previous publications have highlighted some of the difficulties with using VR in clinical settings such as issues with following instructions, not having sufficient hand function to hold controllers and technical issues. 33 Some of these issues occurred during this study as discussed in our results however this did not impact the levels of enjoyment experienced during the VRR session. There is an opportunity to use VRR in multiple populations where pain, anxiety, or fear may limit individuals’ levels of participation in upper limb rehabilitation. The subjective comments provided by participants in the free text box demonstrate the benefits of VRR but also how it is possible to grade and improve the tasks as their recovery progresses.
Limitations
A limitation of this study was the small sample size therefore we are unable to generalize the results. Another limitation was the diverse range of conditions included in this study therefore further specific cohort studies are indicated to determine the benefits for each population.
Conclusion
The participants in this study had a wide range of upper limb impairments because of many different illnesses, injuries, and conditions. This is the first study the research team is aware of which investigated the use of VRR in such a diverse group of individuals receiving upper limb occupational therapy rehabilitation. The results of this study found that participants enjoyed their experience of VRR as a method of providing upper limb occupational therapy rehabilitation as an adjunct to current therapy provisions. Future studies are warranted to explore options of the VR experience to grade and increase the complexity of the activities included across multiple patient cohorts and to further evaluate the experiences of using VR over the course of their recovery.
Footnotes
Appendix 1. Participant Feedback Questionnaire
Acknowledgments
Brooke Colby, Occupational Therapist, Tara Courtney, Occupational Therapist and Lexi Townsend 3D and VR artist, QUT.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
Statement of Contributions
All authors have made substantial contributions to all of the following: (1) the conception and design of the study AMK, MD, APB, BZ, SL, and GT (2) acquisition of data—AMK, MD, APB, BZ, SL, and GT, (3) analysis and interpretation of data AMK, MD, APB, BZ, SL, and GT (4) drafting the article and revising it critically for important intellectual content—AMK, MD, APB, BZ, SL, and GT and (5) final approval of the version to be submitted AMK, MD, APB, BZ, SL, and GT.
