Sage Journals: Discover world-class research

Abstract

People suffering from rheumatoid arthritis are normally prescribed with physical exercises. The realization of these exercise routines should be supervised to prevent wrongly adopted postures or bad movements that end up harming the affected articulations. The motivational component associated with a supervised session is also important. On the contrary, having access to qualified supervision is not always possible due to different reasons, such as tight schedules, limited mobility, or economic reasons. This article presents a system for at-home rehabilitation, based on Microsoft Kinect, that resembles the role played by the physiotherapist during the supervision of an exercise session. The system not only supervises the realization of exercises but it also collects and manages information about the exercise-session history. Additionally, and despite affecting young people, rheumatoid arthritis is a disease that mainly affects seniors. For this reason, the system provides a natural interface to ease the interaction. Finally, the performance of the proposed system has been evaluated using a three-dimensional dynamic measurement system as a gold standard to validate the accuracy of the measurements. Obtained results demonstrate the potential of the Kinect-based system for at-home rehabilitation exercise routines.

Keywords

At-home rehabilitation Microsoft Kinect rheumatoid arthritis space-time interest points three-dimensional dynamic measurement system natural interface

Introduction

Rheumatoid arthritis (RA) is one of the most common musculoskeletal disorder affecting up to 1% of the general population worldwide.¹ This disease causes joints to be swollen leading to ankylosis. This systematic inflammation not only leads to morbidity and long-term disability but also to an important burden in terms of societal and economic cost to both patients and health services.²

RA patients tend to bend damaged joints, adopting a more comfortable and less painful position. However, after being bent for long periods, returning them to their original state might not be possible due to muscle shrink. To avoid this situation, RA patients should maintain an adequate level of physical activity. In this sense, RA patients are normally prescribed with a pharmacological treatment to deal with pain as well as an exercise routine that will help them to reduce pain, stay more energetic, and reduce weakness while preventing accidents due to their lack of agility.³

Exercise routines are normally prescribed and supervised by a physical therapist. Daily routines are unfeasible for many patients due to the economic cost, time constraints, and the impossibility to travel to a health center on a daily basis. Nevertheless, unsupervised realization of exercise routines is neither a good solution since it could be ineffective or unsafe.⁴ Unsupervised activities could lead patients to adopt inappropriate corporal poses that might damage joints. There is, therefore, a trade-off between the need for supervision and the comfort and convenience of exercising at home. Despite the demonstrated benefits derived from physical therapy,⁵ patients continuation of this type of treatments dramatically drops with time, especially when they are performed in an unsupervised manner.⁶

The use of computer-aided rehabilitation systems has attracted much attention from the research community.^7–12 Not so long ago, the main inconvenience that had to be addressed when automating any aspect related to human kinetics was the high cost involved in the required hardware and the participation of trained staff. This barrier has been eliminated to some extent since Microsoft Kinect was launched for the video game industry in 2010 and the release of a software development kit (SDK) that opened up the possibility of developing new applications based on it. Moreover, wearable technologies are also opening new lines of research, such as those based on foot pressure sensor analysis,¹³ which could be used for body posture analysis in combination with depth and RGB images.

The work of Hondori and Khademi¹⁴ reviews the impact of Kinect on physical therapy and rehabilitation. Regarding skeletal tracking, despite the increase in accuracy when compared with systems based on RGB images, Kinect struggles under occlusions or noisy environments, as pointed out by Hondori and Khademi.¹⁴ Still, same authors conclude that Kinect accuracy is acceptable for rehabilitation purposes. Similarly, the work in Webster and Celik¹⁵ presents a review of the use of Kinect for elderly care and stroke rehabilitation, paying special attention to the spatial accuracy, rehabilitation methods, and limitations. Webster and Celik¹⁵ conclude that, in general, Kinect has a significant potential to be used as a cost-effective solution for motion capture systems in the context of rehabilitation.

Several applications based on Kinect have been presented in recent years. A system for gait assessment in people with a stroke is presented in Clark et al.¹⁶ The work in Galna et al.¹⁷ is intended for people with Parkinson’s disease and proposes a rehabilitation method based on a video game to enhance motivation among users, while providing a preliminary assessment of user’s safety. Postural stability is shown to be enhanced in individuals with Parkinson’s disease according to the results presented in Shih et al.¹⁸ after an exergaming intervention using Kinect.

Despite the promising results obtained from state-of-the-art systems for rehabilitation support, therapists cannot be totally relegated from the process once the exercise routine has been prescribed. On the contrary, they have to be kept in the loop to monitor and assess the patient performance and motivation as well as to prescribe appropriate changes in the routine and assess the advances.¹² Motivation plays an essential role not only on determining adherence to the rehabilitation routines¹⁹ but also on its outcome.²⁰ In fact, identified barriers to treatment adherence²¹ evidence the importance of therapist supervision in the early identification of such barriers. The work in Colombo et al.²² refers to the awareness of the performance obtained and the quantity and quality of the feedbacks as elements that can influence the patient motivation. In Cameirão et al.,²³ strategies based on gaming and coaching are analyzed for at-home stroke rehabilitation.

Motivated by the need to combine the benefits of at-home rehabilitation and therapist supervision, as well as trying to maximize the performance awareness and the feedback provided to patients, this work proposes a system, known as ArthriKin, based on Microsoft Kinect, a low-cost depth camera. The system is intended to supervise and provide real-time feedback, similarly to what a therapist would do during a session. In addition, the proposed system keeps records of the patient activity, enabling therapists and doctors to assess their progression over time.

The ArthriKin system

ArthriKin is intended to supervise the execution of exercise routines while providing real-time corrections when deviations in trajectories or body poses are detected. The proposed system, as depicted in Figure 1, comprised a set of modules that can be analyzed from the main stages they intervene in, which are known as

The data acquisition module, which uses the depth camera (in this case Microsoft Kinect) to record body movement.

The natural interface module, employed for user interaction and for guiding patients through the exercise routine, correcting them when deviations are detected.

The activity data logging module, in charge of recording the patient performance during the exercise session, enabling afterwards the medical staff to supervise patient performances.

Figure 1.

Stages involved in the ArthriKin architecture.

Data acquisition

This module is responsible for recording the different body postures adopted during the execution of the exercise routine. The implementation of this module has been achieved by resorting to the Microsoft Kinect SDK to extract the skeletal features of the captured posture for each frame. These features, as depicted in Figure 2, consists in the three-dimensional (3D) points representing the position of each one of the considered skeleton joints.

Figure 2.

Skeleton representation. The circles in the skeleton represent joints.

Pose representation

ArthriKin is based on the skeleton information provided by the Microsoft Kinect SDK. This information, however, has to be conveniently parsed and stored so that representative features can be easily extracted. XML files are employed to track the position and orientation of the 20 joints comprising the skeleton. Moreover, additional information such as the frame number, the detected user ID, and the timestamp are also recorded.

The execution of a certain exercise will generate a collection of XML files that should be conveniently stored. A NoSQL database, namely eXist DB (http://www.exist-db.org/), has been employed for persistence purpose.

Calibration

During the calibration process, different measures are carried out in order to adapt the exercise routine to the particular body characteristics and the environment. This process is especially important for patients performing the exercise routine from a wheelchair due to the distortions introduced in the body posture in those cases.

The calibration is carried out in a two-stage process. First, the camera records a set of frames during the initial 5 s. From each frame, the body pose is represented as a skeleton and parsed in an XML file. After this initial stage, the following points, as depicted in Figure 3, are computed:

Inner point. It refers to the internal point of the limb. If the exercise involves an arm, this refers to the shoulder. If it involves a leg then, the inner point refers to the hip.

Middle point. In this case, the middle point of the arm refers to the elbow, whereas for a leg it refers to the knee.

External point. This refers to the hand in the case of an arm and the foot in the case of a leg.

Floor. This measure identifies the floor position based on the foot position.

Figure 3.

Lower and upper limb points.

This stage results in four different vectors of points, corresponding to each tracked limb. From every vector, the mean point is calculated and considered as the reference one. Considering more than one measure helps overcoming periods of loss of skeletal tracking, the presence of noise, or small movements.

During the second stage, the limb length is calculated. Some users might find it difficult to take their limb to a complete extension, due to the effects of the disease. Therefore, the length $(L)$ is calculated by adding the Euclidean distance between the inner point $(I)$ and the middle point $(M)$ and between the middle and the external point $(E)$

d (I, M) = \sqrt{{(I_{x} - M_{x})}^{2} + {(I_{y} - M_{y})}^{2} + {(I_{z} - M_{z})}^{2}}

(1)

d (M, E) = \sqrt{{(M_{x} - E_{x})}^{2} + {(M_{y} - E_{y})}^{2} + {(M_{z} - E_{z})}^{2}}

(2)

L = d (I, M) + d (M, E)

(3)

Determination of the areas of relevance

The type of exercises prescribed to RA patients mainly involve those related to the range of motion³ (see Figure 4). These exercises are intended to improve and preserve the health of the joint as well as its range of movement. Every prescribed exercise is therefore going to focus on a specific limb or part of the body. Based on that, we define an area of relevance as the space range in which a certain joint is expected to be constrained during the exercise execution. In order to calculate these limits, during the training stage, the physical therapist is recorded while performing a correct execution of the exercises. The data gathered are post-processed to identify, in first place, the limbs or body parts involved in the exercise and the boundaries within which a safe and correct execution of the exercise is achieved.

Figure 4.

Exercises considered in ArthriKin.

The identification of the limbs or body parts involved in an exercise is carried out implementing an approach based on the space-time interest points (STIP) proposed in Laptev²⁴ that has been extensively used in computer vision systems for scene recognition. Similarly to what the STIP algorithm proposes, our system observes the skeleton joint positions during the exercise execution. From the obtained data, those joints showing a high variation in any or several dimensions ( $x$ , $y$ , or $z$ ) are considered joints of relevance and therefore subject to a further analysis to determine the area of relevance where their movement should be constrained.

Figure 5 represents the hand position in coordinates $x$ , $y$ , and $z$ for the shoulder abduction exercise. It can be easily spotted how, for every repetition, the position in the $x$ and $y$ axes significantly changes. From this analysis, a maximum and minimum point can be identified in the $x$ and $y$ axes.

Figure 5.

Hand position in $xyz'$ coordinates during the shoulder abduction exercise (see Figure 12 for the definition of the coordinate axes xyz).

As it can be observed from the $z$ axis position (i.e. depth dimension) in Figure 5, even though the position should be ideally kept constant, some degree of freedom should be allowed, referred to as the depth factor. The area of relevance of the exercise in the $z$ axis establishes a safe range in which the joint can move along.

Despite dealing with 3D points, ArthriKin provides feedback to users in a two-dimensional (2D) space, by means of an arc that represents the trajectory (with the maximum and minimum points) that the patient should stick to (see Figure 6). This decision was adopted for the sake of feedback simplicity. For this reason, depending on the exercise, the patient should adopt a position facing the camera or sideways to it.

Figure 6.

Trajectory to be followed by the patient represented as an arc.

The shoulder abduction exercise, depicted in Figure 6, for instance, has to be carried out facing the Kinect, which means that the area of relevance to be controlled is parallel to the device (in the $x$ and $y$ planes). Still, the depth factor in the $z$ dimension has to be controlled to avoid unwanted deviations that could end up harming the patient’s articulations.

Exercise routine supervision therefore relies on having identified the limits that fall into a normal and correct execution of the exercise. The maximum and minimum point calculation should also be normalized so that they can be adapted to different users (i.e. different heights or arm lengths). This adaption process is especially important for those users in wheelchairs. Depending on the exercise, areas are extended to consider the space taken by the wheels. In the case of the shoulder abduction exercise, for example, the minimum point is adapted to consider the armrest of the chair.

Control of the exercise execution

The supervision of the exercise execution depends not only on the identification of the area of relevance for a given exercise, but also on performance (P) achieved. The area of relevance states the exercise safety limits, including the maximum and minimum point that should not be overpassed. On the other hand, the performance is used to monitor the degree of achievement and guide the user through the exercise execution by providing visual feedback using a color code (see Figure 10) or counting repetitions, whenever a trajectory described by the limb reaches a certain minimum. Figure 7 graphically identifies different levels of performance for the shoulder abduction exercise. Performance for the axes of interest is calculated as follows

P_{x} (%) = \frac{E_{x} - I_{x}}{L} \times 100

(4)

P_{y} (%) = \frac{E_{y} - I_{y} + L}{2 L} \times 100

(5)

where, for a given limb, such as an arm or a leg, the inner $(I)$ and external $(E)$ points can be identified. These points, for the arm case, correspond to the shoulder and the hand, respectively, whereas for the leg they refer to the hip and the foot. $I = (I_{x}, I_{y})$ and $E = (E_{x}, L_{y})$ are the x and y coordinates of both points in the corresponding limb, whose length is defined as the distance $L$ defined in equation (3).

Figure 7.

Performance in $x$ and $y$ coordinates, $P_x$ and $P_y$ , respectively.

Yet, there are some exercises in which the performance depends on the angle formed by the extremity and the body torso, like in the shoulder or hip extension exercises (see Figure 8). The performance, although also stated in %, should be calculated considering that angle, as follows

P_{θ} (%) = \frac{θ}{π} \times 100

(6)

where $θ$ is the angle, in radians, formed by the extremity and the body torso, and it can be calculated as follows

θ = ta n^{- 1} \frac{E_{x} - I_{x}}{E_{y} - I_{y}}

(7)

Figure 8.

Performance calculation $P_{θ}$ for the shoulder extension exercise.

The control of the exercise execution also involves the repetition counter. There are several issues that have to be addressed in order to count repetitions, such as dealing with situations in which low span of movements is achieved and assuring that a correct posture has been maintained throughout the execution of the exercise. Particularly challenging are those patients that workout from a wheelchair, mainly due to the armrest position. Moreover, it has to be checked that limbs are extended and performance in $x$ and $y$ is increasing with respect to the starting position.

The determination of the exercise performances is very important when monitoring the progression of the patient over time. The analysis of the resulting performance values can provide relevant information to detect patients going through an RA²⁵ and when assessing the benefits of certain exercise routines.

Natural interface interaction

RA is a disease whose prevalence increases with age, with an average onset at the age of 55.²⁶ This means that the majority of potential users of the proposed system will unlikely be familiar or confident with technology. In this sense, if this system is to be used without supervision, we must pursue users’ self-sufficiency when interacting with the system.

In this sense, ArthriKin has paid special attention to ease system interaction and keep the user experience as close as possible to interacting directly with a therapist. The natural interface interaction provided by the system is based on three main aspects: gesture control, voice commands, and visual representation of exercise trajectories.

Gesture control

The system is designed to be operated without any classical peripheral device (mouse or keyboard). The user will control the application by using a natural interface, thanks to the gesture control implemented in ArthriKin.

Hands are our natural interface when operating with an object. Similarly, the Graphical User Interface (GUI) of ArthriKin has been designed to be manually operated using the depth camera to detect when a hand is simulating a pressing action over a GUI button. Rather than using the mouse arrow to represent the location of the pointer, a hand is employed. The hand is gradually filled in purple as the hand pushes forward pretending to click a GUI button, as observed in Figure 9.

Figure 9.

Natural interaction with the GUI by pressing buttons with the hand.

Voice commands

The system provides the user with voice command feedback, pretending to be a therapist. The following commands will guide the user through the exercise set up and execution:

Starting calibration. This message announces the start of the 5-s calibration process during which body and context features are gathered to adapt the exercise-execution model to the user.

Starting exercise. After this message, the user can proceed with the performance of the exercise. A visual representation of the articulation trajectory will also appear on the screen guiding the user through the execution.

Please, try to keep your arm parallel to the body. This message indicates that a deviation from the execution model has been detected. In order to prevent any harm, this message prompts the user to correct the body posture.

Please, try to keep your leg parallel to the body. Similar to the previous command, this one is intended to correct the position of the leg with respect to the body.

Repetition counter. Once a repetition is completed, a message is issued indicating the repetition number (i.e. one, two, three, etc.).

Exercise finished. Every exercise is configured to complete a number of repetitions. Once the repetitions have been achieved, a message announces the end of the routine.

Visual representation

The system provides real-time feedback to the user about the safe maximum and minimum points as well as the trajectory the arm or leg has to follow for a given exercise. In Figure 6, a green semicircle can be observed, with three bullet points indicating the maximum and minimum points and the current position of the hand.

It is worth mentioning that a color legend is also employed to provide the user with visual feedback about the performance or the deviation degree. Figure 10 shows the use of this color legend. The performance degree is represented using a color scale that ranges from red (at the minimum point) to green (at the maximum point). Then, the yellow color is employed to warn about a deviation from the area of relevance of the exercise. Similarly, the black color indicates that the exercise limits have been overpassed (either by the maximum or minimum points).

Figure 10.

Color legend devised to provide feedback about the performance achieved.

Activity record

ArthriKin is not only intended to support the RA patients in implementing a daily and safe exercise routine at home but it is also aimed at collecting, storing, and processing data about those routines. Relevant information can be inferred from that data, and it can eventually help the therapist and the doctor to adjust and personalize the treatment prescribed to every patient.

The system records information about patients, exercises, and detailed information about daily routine executions. XML files are employed to record information so that more advanced searches and data processing can be eventually supported. Currently, the system implements statistical functions that provide information about the maximum or minimum points reached or the average level of performance.

The system also offers the possibility of obtaining graphical representation of the stored information from where relevant conclusions can be easily spotted, such as a sudden reduction in performance due to an RA crisis or the steady improvement on flexibility based on the maximum achieved points, for example. Figure 11 shows the maximum, minimum, and mean point achieved during the execution of the shoulder abduction exercise in five different days. The graphical representation of results offers a flexible configuration from where to choose the exercise type and daily records of different patients.

Figure 11.

Representation of the activity record history.

Future work is intended to explore machine-learning techniques to determine whether some correlation exists between the data, the progression of the disease, and some other factors related to the patient state (stress, style of life, etc.).

System evaluation

The benefits derived from a continuous exercise program in people with RA have already been reported in previous works.^5,27 Despite the fact that the proposed system has been initially devised for people with RA and the list of exercises is intended for RA rehabilitation, its application has the potential to be extended to supervise other types of rehabilitation routines. The preliminary approach adopted to evaluate the proposed tool presented in this article has therefore considered a more general scope than just evaluating the benefits of the intervention in people with RA.

The initial working hypothesis was that a computer-based tool with motion capturing, as ArthriKin, can be an efficient tool for at-home rehabilitation and remote assistance and supervision by a physiotherapist. To validate this hypothesis, we have analyzed the system suitability and efficacy on the basis of three different dimensions:

The physiotherapist motivation to use the system and be assisted by a computer-based tool.

The system accuracy when tracking the movements of the body during exercises.

The system usability when people use the tool.

Motivation evaluation

The physiotherapist motivation to use the proposed system is measured using a selection of 26 items from the Intrinsic Motivation Inventory (IMI) questionnaire.²⁸ This instrument has been previously used in rehabilitation contexts.²² The original questionnaire comprised 45 items organized into seven subscales: interest/enjoyment, perceived competence, effort/importance, pressure/tension, perceived choice, value/usefulness, and relatedness. According to Ryan and Deci,²⁸ the intrinsic motivation per se is measured by the interest subscale although the perceived competence is considered to be related to it. The perceived choice is also considered a positive predictor of both self-report and behavioral measures of intrinsic motivation; however, pressure/tension is handled as a negative predictor. Effort is used in some questions, if relevant, so it is a separated variable. The value/usefulness subscale is related to the idea that people internalize and observe those experiences that they find useful for themselves.²⁹ Finally, the relatedness subscale is used in studies considering interactions between people.

Previous studies mention that the effects of the order of items appear to be negligible, and the inclusion or exclusion of specific subscales does not seem to have any impact on the others. Another important issue is the redundancy, because items within subscales overlap notably, although the fact of randomizing their presentation makes it less relevant for most participants.

In order to evaluate the intrinsic motivation, we proposed an IMI questionnaire of 26 items to 11 physiotherapists who were previously introduced to the tool. The number of items of the each of the six subscales considered is shown in Table 1 (discussed later).

Table 1.

Subscale findings of the Intrinsic Motivation Inventory questionnaire.

Subscales (n = 11)	Number of items	Score (mean ± SD)
Perceived competence	7	6.09 ± 0.94
Value/usefulness	2	5.84 ± 0.59
Interest/enjoyment	2	5.58 ± 0.68
Effort/importance	2	5.23 ± 0.75
Perceived choice	4	4.84 ± 0.65
Pressure/tension	2	3.95 ± 0.93

Accordingly to the authors of self-determination theory, we randomized the IMI items. Each item was evaluated in a range between 1 (not at all true) and 7 (very true). There were 6 reverse score items, so if the physiotherapist for instance assigned a score equal to 1, the score was subtracted from 8 and therefore transformed into 7.

Accuracy evaluation

The evaluation of the proposed system based on Kinect has been carried out by comparing the obtained measurements with a high-accuracy 3D measurement system that is used as a gold standard. In this case, a Krypton K-400 dynamic measuring machine manufactured by Metris N.V. (nowadays Nikon Metrology N.V.)³⁰ has been used for this purpose. The main characteristics of this system are shown in Table 2.

Table 2.

Krypton K-400 (Metris M.V.) characteristics.

Parameter	Value
Maximum and minimum sampling frequency	$10$ and $150 Hz$
Maximum number of markers	$50$
Measurement volume	$6 m^{3}$
Accuracy	$< 0.1 mm$
Measurement uncertainty	$90 μ m + 10 μ m / m$
Measurement resolution	$2 μ m$

Four different exercises were proposed by a physiotherapist, considering the most commonly used exercises during a person-to-person rehabilitation session. These exercises, as depicted in Figure 4, were recorded simultaneously by the Kinect- and the Krypton-based systems.

The Krypton-based system employed three infrared (IR) cameras and IR light emitting diode (LED) markers (LED1, LED2, and LED3) placed in the measurement points. The measurement setup is depicted in Figure 12, showing the relative position of the Krypton and the Kinect systems, as well as the coordinate systems associated with them ( $x' y' z'$ and $xyz$ , respectively).

Figure 12.

Measurement setup (position of the Kinect system in x′y′z′ coordinates: x′ = −374 mm, y′ = 171 mm, and z′ = −1714 mm).

Three IR LED markers were used for each exercise. Two of them (LED2 and LED3) were placed in the joints of interest for each exercise, according to Table 3. The remaining marker was placed in the origin of the Kinect coordinate system to capture the relative position between coordinate systems for post-processing purposes.

Table 3.

IR LED marker positions for each exercise.

Exercise	LED1	LED2
Shoulder abduction (Ex_sa)	Left hand	Left shoulder
Shoulder extension (Ex_se)	Left hand	Left shoulder
Elbow flexion (Ex_ef)	Left hand	Left elbow
Hip separation (Ex_hs)	Left hip	Left knee

IR LED: infrared light emitting diode.

Once the measurements were captured for each exercise, the data obtained with the Krypton and Kinect needed to be post-processed in order to obtain the position errors. The post-processing stage was been carried out using Matlab and it consisted of the following steps:

Parsing of the Kinect XML files and the Krypton mat files.

Segmentation and synchronization of recordings.

Resampling to 150 Hz as common sampling frequency (Kinect works at 30 fps).

Transformation to a common coordinate system (Krypton coordinates $x' y' z'$ ).

Error computation.

Usability evaluation

The guidelines established by ISO/IEC 9126-4 have been used to evaluate the usability of the system. The ISO 9241-11 standard provides a guide to understand usability. This standard defines usability as “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.” As we can see in the definition, usability is not a one-dimensional magnitude, but rather a combination of different measures or factors. The ISO/IEC 9126-4 standard recommends that usability metrics should include the following three properties:

Effectiveness, defined as “the accuracy and completeness with which users achieve specified goals.”

Efficiency, which refers to “the resources expended in relation to the accuracy and completeness with which users achieve goals.”

Satisfaction, defined as “the comfort and acceptability of use.”

The effectiveness ( $ϵ$ ) is calculated by means of the completion rate

ϵ = \frac{N_{t s}}{N_{t u}} \times 100 (%)

(8)

where $N_{ts}$ is the number of task completed successfully and $N_{tu}$ is the total number of task undertaken.

In addition, the number of errors that the participants had in the completion of the set of tasks was counted. Any omission, slips, or mistake in the execution of the task was counted as an error.

Regarding the efficiency assessment, it requires measuring the time that has been consumed to complete the task. Efficiency can be calculated based on time or as the overall relative efficiency.

Time-based efficiency $(η_{t})$ is obtained as follows

η_{t} = \frac{\sum_{j = 1}^{R} \sum_{i = 1}^{N} \frac{n_{ij}}{t_{ij}}}{NR}

(9)

where $N$ is the total number of tasks (goals), $R$ is the number of users, $n_{ij}$ is the result of task i undertaken by user j (if the user successfully completes the task, then $n_{ij} = 1$ , else $n_{ij} = 0$ ), and $t_{ij}$ is the time spent by user j to complete task i (if the task is not successfully completed, then the time is measured till the moment the user quits the task).

The overall relative efficiency $(η_{r})$ is obtained with the following expression

η_{r} = \frac{\sum_{j = 1}^{R} \sum_{i = 1}^{N} n_{i j} t_{i j}}{\sum_{j = 1}^{R} \sum_{i = 1}^{N} t_{i j}} \times 100 (%)

(10)

Finally, satisfaction was measured by questionnaires that are completed after each task (Task Level Satisfaction) and/or after the usability session (Test Level Satisfaction). In our case, we selected the Single Ease Question (SEQ) to evaluate each task and the System Usability Scale (SUS) to evaluate the user’s satisfaction with the system.

In order to evaluate the usability of the system, 20 volunteers with ages between 13 and 52 years participated in the study. Among the volunteers, there were 7 women and 13 men with different levels of education and different professions, each of whom had different experiences with technology. Each volunteer was asked to complete 15 tasks, and for each task, it was recorded the time needed to complete the task, the errors made, and whether they completed the task successfully. Volunteers were also asked at the end of each task about the level of difficulty they experienced completing it (SEQ). Finally, the participants were asked to complete a final satisfaction test (SUS).

Results and discussion

Motivation

The IMI questionnaire with 26 questions was completed by 11 physiotherapists and the mean values and standard deviations of the scores in each of the 6 subscales selected were calculated. These values are shown in Table 1.

As it can be seen, the standard deviation of each of the subscales is low, which means that a high level of agreement is found in the scores given by the different physiotherapists. Furthermore, it can be observed that all subscales are above 4, which corresponds the medium value of the scoring scale that goes from 1 to 7, except for the Pressure/Tension subscale which has a mean value of 3.95.

The best-rated subscale is perceived competence, indicating that physiotherapists felt they would be very capable of using the tool. Value/Usefulness also scored high, so it can be inferred that they find the tool very useful for the rehabilitation purpose. Furthermore, the score obtained in Interest/Enjoyment indicates that the use of the tool is interesting for them.

The score obtained in Effort/Importance and Perceived Choice subscales suggests that the professionals who took part in the study would make an effort to set up the tool and its use, in some cases for their own interest, but in other cases they would need some additional stimulus to adopt the new tool. This result is not surprising, since as in other sectors there are many people who are reluctant to use new technologies and embrace the changes they entail.

Finally, the Pressure/Tension subscale scored low, meaning that professionals believe they would not experience moments of pressure and tension while using the tool. In order to clearly illustrate the aforementioned results, Figure 13 shows a box plot of the motivation results by subscales.

Figure 13.

Box plot with the IMI results.

Accuracy

As an illustrative example, Figure 14 depicts the performance of the proposed system, in terms of accuracy, for the shoulder abduction exercise obtained by both systems (Kinect and Krypton) after the post-processing stage.

Figure 14.

Left hand position measurements with Kinect and Krypton systems for the shoulder abduction exercise.

The assessment of the performance obtained with the Kinect-based system is based on the statistical analysis of the measurements and errors obtained from the comparison with the Krypton-based system. This statistical analysis is divided into the calculation of basic descriptive statistic values (root-mean-square error, mean absolute error, standard deviation) and the agreement between the two measurement techniques and reproducibility, which is evaluated by means of Pearson’s correlation coefficient and Lin’s Concordance correlation coefficient. This analysis is suitable for the validation of the Kinect-based system according to the study presented in Springer and Seligmann³¹ for gait assessment. The results obtained for the different exercises are summarized in Table 4.

Table 4.

Statistical analysis (measures in mm).

Exercise-marker-axis	RMS error	Mean error	STD	$r$	$r_{c}$
Ex_sa-hand-x	27.80	−10.02	25.93	0.991	0.985
Ex_sa-hand-y	37.40	−0.47	37.40	0.996	0.995
Ex_sa-hand-z	13.38	−1.70	13.28	0.969	0.948
Ex_sa-shoulder-x	18.84	3.69	18.48	0.926	0.642
Ex_sa-shoulder-y	6.53	1.01	6.46	0.983	0.982
Ex_sa-shoulder-z	40.46	39.83	7.14	0.770	0.004
Ex_se-hand-x	18.32	−4.69	17.71	0.998	0.995
Ex_se-hand-y	29.65	−0.50	29.65	0.959	0.937
Ex_se-hand-z	24.10	23.53	5.23	0.958	0.522
Ex_se-shoulder-x	14.35	−5.01	13.45	0.704	0.522
Ex_se-shoulder-y	19.46	0.06	19.46	0.215	0.065
Ex_se-shoulder-z	69.32	68.30	11.79	0.045	0.001
Ex_ef-hand-x	29.00	−7.99	27.89	0.936	0.921
Ex_ef-hand-y	38.49	−2.60	38.42	0.980	0.979
Ex_ef-hand-z	17.82	13.51	11.62	0.967	0.911
Ex_ef-elbow-x	29.85	−29.11	6.60	0.904	0.197
Ex_ef-elbow-y	12.69	0.09	12.70	−0.137	−0.047
Ex_ef-elbow-z	33.19	32.58	6.35	0.829	0.086
Ex_hs-hip-x	70.20	−69.98	5.60	0.626	0.008
Ex_hs-hip-y	10.79	−0.96	10.75	0.821	0.767
Ex_hs-hip-z	52.59	52.00	7.88	0.157	0.003
Ex_hs-knee-x	60.55	−15.69	58.51	0.743	0.728
Ex_hs-knee-y	80.84	−3.99	80.78	0.658	0.621
Ex_hs-knee-z	77.85	77.31	9.12	0.418	0.009

RMS: root-mean-square; STD: standard deviation.

The overall mean value of the root-mean-square (RMS) errors for all the exercises, joints, and dimensions is 34.93 mm. Correlation values for $r$ and $r_{c}$ are significant, showing a strong correlation, for those movements with a large span (e.g. hand position for exercise Ex_sa). For those joints with very little movement during the execution of exercises, correlation values are low (e.g. the elbow in Ex_ef) as expected. Nevertheless, errors are sufficiently low and acceptable for the purpose of the application presented. Overall, the performance of Kinect-based system has been positively validated in the evaluation carried out by comparing the gold standard system.

In general, our results are in agreement with the work presented in Webster and Celik,¹⁵ where the spatial accuracy of Kinect-based rehabilitation solutions is reviewed, and it is concluded that “the Kinect has been found, in general, to carry significant potential for a cost-effective motion capture system for rehabilitation.” Our results support the conclusions in Webster and Celik:¹⁵“Kinects ability to accurately capture upper extremity movements is consistently reported as sufficient for clinical use with regards to the elbow and wrist joint tracking; however, mixed results have been reported for the shoulder.” Similarly we have obtained less accurate results regarding the position of the hip. It should also be noted that the accuracy levels required depend on the type of exercise to be done as Zhao et al.³² suggests, and the minimum accuracy level for a certain exercise has to be established with the participation of experts.

Usability

During the usability tests, each of the volunteers completed a set of 15 tasks with the ArthriKin tool to determine the usability of the system.

First of all, in order to assess the effectiveness of the system, it was analyzed for each proposed task whether it was completed or not and how many errors were made during completion. All users completed the proposed tasks, so the completion rate was 100%. Nevertheless, if we look at the error rate to be more accurate in terms of efficiency, 75% of users had less than 25% errors when performing all tasks, 20% of users obtained between 25% and 50% errors, and only one user was above 50% errors. Overall, an 18% error rate was achieved for all the tasks, which is a considerably low rate, and in spite of these errors, users were not prevented from successfully finishing all the tasks.

Regarding efficiency, it was necessary to know how long it took users to complete each task, if completed. The time-based efficiency calculation indicates the rate of goals achieved per second. This measurement is useful in the context of incorporating improvements to the tool and assessing the benefits in terms of efficiency. It is important to see that as Figure 15 shows, most users completed the assigned tasks in a similar amount of time. Figure 15 also illustrates the difficulties encountered by some users to complete certain tasks. This is visible by observing the number of outliers (e.g. task 4) and how far are their time values from the median (e.g. task 5). The overall relative efficiency is 100% in all tasks because all users managed to complete all tasks.

Figure 15.

Box plot with the time per task measurements.

Finally, to complete the usability study, users were asked to complete two satisfaction tests to find out their opinions on the tool. Through the answers in the SEQ tests, it can be generally observed that users rated the tasks performed with high marks, which means that most of the tasks were easy for them to complete as Figures 16 and 17 illustrate.

Figure 16.

Box plot with the SEQ results by user.

Figure 17.

Box plot with the SEQ results by task.

If we analyze the evaluations of the results of the SUS, we see that the results are also positive since 60% consider the application as Excellent, 30% as Good, 5% as OK, and the other 5% as Poor, as it can be extracted from Table 5.

Table 5.

Individual’s SUS score.

Participant ID	SUS score	Adjective rating
10	100	Excellent
12	100	Excellent
14	100	Excellent
15	100	Excellent
1	97.5	Excellent
7	95	Excellent
19	95	Excellent
2	92.5	Excellent
11	92.5	Excellent
4	90	Excellent
8	87.5	Excellent
16	87.5	Excellent
6	80	Good
9	80	Good
18	75	Good
3	72.5	Good
5	70	Good
20	70	Good
13	62.5	OK
17	55	Poor

SUS: System Usability Scale.

Conclusion

This article has presented ArthiriKin, a rehabilitation system based on computer vision for patients suffering RA. The system is aimed for at-home rehabilitation, providing supervision and feedback for the user and a friendly natural interface for interaction. Furthermore, the performance during exercise routines is recorded to allow the remote supervision of physiotherapists. Calibration and performance metrics are adapted to the user, providing online feedback to the user while performing the exercises. The resulting system has been evaluated by means of a high-accuracy measurement system that is used as a gold standard: the Krypton K-400 dynamic measuring machine (Metris M.V.). The accuracy obtained with the Kinect-based system is considered to be sufficient considering the computing errors and correlation indexes obtained. Motivation and usability have also been analyzed. Motivation has been analyzed from the perspective of therapists to determine their willingness to use and recommend this tool. Overall, the tool is highly perceived by professionals (highly considered in terms of usefulness) who show medium–high interest in the tool and therefore in their willingness to adopt it as a working tool. In terms of usability, the high rate of completion (100%) and low error rate (18%) lead us to conclude that the proposed system is effective and also efficient because of its high efficiency rate (100%). In terms of user satisfaction, most of the participants (90%) are positively satisfied with the system usability and easiness of use.

This work therefore demonstrates the viability of a system based on a low-cost depth camera to support at-home rehabilitation for RA patients, but it can easily be extrapolated to any other condition benefiting from daily exercise. The system assessment yields a sufficiently low error for supervising the correct execution of the most commonly used exercises for this type of rehabilitation. Particularly important is the performance of the supervising algorithm, capable of working in real time, instantaneously detecting and notifying deviations.

These evidences lead us to conclude that the proposed system could have a potential impact in improving the quality of life of people suffering from RA. Patients count on an external supervisor detecting wrong positions or incorrect exercise performance in real time. Besides, performance data is collected assisting the specialists in the evaluation of the progression of the patient as well as the detection of crisis.

Footnotes

Handling Editor: Xing Chen

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work has been partly funded by the Spanish Ministry of Economy and Competitiveness under project PLATINO (TEC2017-86722-C4-4-R) and by the Regional Government of Castilla-La Mancha under project SYMBIOT (SBPLY/17/180501/000334) which is partially funded by the European Social Fund. Mr Cantarero is funded by the Spanish Ministry of Education, Culture and Sport within the framework of the National FPU Program (ref. FPU 16/06205). Xavier del Toro Garcia has received financial support from the European Regional Development Fund (Fondo Europeo de Desarrollo Regional, FEDER).

ORCID iD

Maria J Santofimia

References

Firestein

. Chapter 65: etiology and pathogenesis of rheumatoid arthritis. In: Firestein

, et al. (eds) Kelley’s textbook of rheumatology. Philadelphia, PA: Elsevier/Saunders, 2013, pp.1035–1086.

Cooper

. Economic burden of rheumatoid arthritis: a systematic review. Rheumatology (Oxford) 2000; 39(1): 28–33.

Nolte

van Rensburg

DCJ

. Exercise prescription in the management of rheumatoid arthritis. S Afr Fam Pract 2013; 55(4): 345–349.

Chiang

Huang

. Kinect-enabled home-based rehabilitation system using dynamic time warping and fuzzy logic. Appl Soft Comput 2014; 22: 652–666.

Baillet

Payraud

Niderprim

, et al. A dynamic exercise programme to improve patients’ disability in rheumatoid arthritis: a prospective randomized controlled trial. Rheumatology (Oxford) 2009; 48(4): 410–415.

Campbell

Evans

Tucker

, et al. Why don’t patients do their exercises? Understanding non-compliance with physiotherapy in patients with osteoarthritis of the knee. J Epidemiol Community Health 2001; 55(2): 132–138.

González-Ortega

Daz-Pernas

Martnez-Zarzuela

, et al. A Kinect-based system for cognitive rehabilitation exercises monitoring. Comput Methods Prog Biomed 2014; 113(2): 620–631.

Brennan

Lum

Uswatte

, et al. A telerehabilitation platform for home-based automated therapy of arm function. In: 2011 annual international conference of the IEEE engineering in medicine and biology society, EMBC, Boston, MA, 3 September 2011, pp.1819–1822. New York: IEEE.

Metsis

Jangyodsuk

Athitsos

, et al. Computer aided rehabilitation for patients with rheumatoid arthritis. In: 2013 international conference on computing, networking and communications (ICNC), 2013, pp.97–102, https://ieeexplore.ieee.org/document/6504061

10.

Xia

Chen

Aggarwal

. Human detection using depth information by Kinect. In: IEEE computer society conference on computer vision and pattern recognition workshops (CVPRW) 2011, Colorado Springs, CO, 20–25 June 2011, pp.15–22. New York: IEEE.

11.

Ruiz-Fernandez

Marín-Alonso

Soriano-Paya

, et al. eFisioTrack: a telerehabilitation environment based on motion recognition using accelerometry. Sci World J 2014; 2014: 495391.

12.

Reinkensmeyer

Boninger

. Technologies and combination therapies for enhancing movement training for people with a disability. J Neuroeng Rehabil 2012; 9: 17.

13.

Guo

Deng

Bebek

, et al. Personal inertial navigation system assisted by MEMS ground reaction sensor array and interface ASIC for GPS-denied environment. IEEE J Solid-St Circ 2018; 53(11): 3039–3049.

14.

Hondori

Khademi

. A review on technical and clinical impact of Microsoft Kinect on physical therapy and rehabilitation. J Med Eng 2014; 2014: 84651411.

15.

Webster

Celik

. Systematic review of Kinect applications in elderly care and stroke rehabilitation. J Neuroeng Rehabil 11(1): 108.

16.

Clark

Vernon

Mentiplay

, et al. Instrumenting gait assessment using the Kinect in people living with stroke: reliability and association with balance tests. J Neuroeng Rehabil 2015; 12: 15.

17.

Galna

Jackson

Schofield

, et al. Retraining function in people with Parkinson’s disease using the Microsoft Kinect: game design and pilot testing. J Neuroeng Rehabil 2014; 11: 60.

18.

Shih

Wang

Cheng

, et al. Effects of a balance-based exergaming intervention using the Kinect sensor on posture stability in individuals with Parkinson’s disease: a single-blinded randomized controlled trial. J Neuroeng Rehabil 2016; 13(1): 78.

19.

Maclean

Pound

Wolfe

, et al. The concept of patient motivation: a qualitative analysis of stroke professionals’ attitudes. Stroke 2002; 33(2): 444–448.

20.

Maclean

Pound

Wolfe

, et al. Qualitative analysis of stroke patients’ motivation for rehabilitation. BMJ 2000; 321(7268): 1051.

21.

Jack

McLean

Moffett

, et al. Barriers to treatment adherence in physiotherapy outpatient clinics: a systematic review. Man Ther 2010; 15(3): 220–228.

22.

Colombo

Pisano

Mazzone

, et al. Design strategies to improve patient motivation during robot-aided rehabilitation. J Neuroeng Rehabil 2007; 4: 3.

23.

Cameirão

Smailagic

Miao

, et al. Coaching or gaming? Implications of strategy choice for home based stroke rehabilitation. J Neuroeng Rehabil 2016; 13(1): 18.

24.

Laptev

. On space-time interest points. Int J Comput Vision 2005; 64(2–3): 107–123.

25.

Aung

MSH

Alquaddoomi

Hsieh

, et al. Leveraging multi-modal sensing for mobile health: a case review in chronic pain. IEEE J Sel Top Signa 2016; 10(5): 962–974.

26.

Van Schaardenburg

. Rheumatoid arthritis in the elderly. Drugs Aging 1996; 7(1): 30–37.

27.

Siegel

Tencza

Apodaca

, et al. Effectiveness of occupational therapy interventions for adults with rheumatoid arthritis: a systematic review. Am J Occup Ther 2017; 71(1): 7101180050p1–7101180050p11.

28.

Ryan

Deci

. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am Psychol 2000; 55(1): 68–78.

29.

Deci

Eghrari

Patrick

, et al. Facilitating internalization: the self-determination theory perspective. J Pers 1994; 62(1): 119–142.

30.

Krypton Electronic Engineering n.v. Krypton help pages on K400/K600 hardware & software guide, 2003, https://manualzz.com/doc/7167896/krypton-help-pages-on-k400-k600-hardware-and-software-guide

31.

Springer

Seligmann

. Validity of the Kinect for gait assessment: a focused review. Sensors (Basel) 2016; 16(2): 194.

32.

Zhao

Espy

Reinthal

, et al. A feasibility study of using a single Kinect sensor for rehabilitation exercises monitoring: a rule based approach. In: 2014 IEEE symposium on computational intelligence in healthcare and e-health (CICARE), Orlando, FL, 9–12 December 2014, pp.1–8. New York: IEEE.