Abstract
Objective
This study aims to evaluate the feasibility of an integrated multi-domain telerehabilitation (TR) system in stroke patients and to observe whether there are changes in the quality of life (QoL) levels of patients undergoing TR treatments.
Methods
Patients were enrolled for a longitudinal multicentric pilot study conducted in six Italian research hospitals (IRCCS). The primary outcome was the feasibility of an integrated TR system, assessed by calculating treatment adherence and by collecting data from the Technology Acceptance Model and the System Usability Scale (SUS). Information on time and travel distance savings was also collected. As secondary outcomes, we evaluated changes in QoL levels with the EuroQol 5-dimensions (EQ-5D) and the Short Form-36 (SF-36) and in caregiver burden through the Zarit Burden Inventory.
Results
We enrolled 84 patients. Our system turned out to be feasible (treatment adherence = 85%), usable (SUS = 73.36/100, classifying it as a ‘good’ system) and well accepted by patients. Quality of life levels improved significantly from pre- to post-treatment (EQ-5D:
Conclusions
Telerehabilitation has been confirmed to be a feasible, usable and acceptable solution to guarantee continuity of care and improve accessibility to rehabilitation treatments to post-stroke patients. Furthermore, the strength of TR is in the possibility to improve patients’ QoL, which in turn could impact on functioning.
Introduction
Stroke is a cerebrovascular disease representing the second cause of death and a major cause of disability worldwide. 1 People who suffered a stroke commonly experience a wide range of symptoms, including impairment of motor functions, speech disturbances, swallowing difficulties, cognition deficits, impaired vision and sensation disorders. 2 Recovery phases after stroke are defined as acute (1–7 days), subacute (7 days to 6 months) and chronic (>6 months), with clinical improvement diminishing in accordance with time from stroke onset, 3 even though it is still possible years after stroke, especially for cognitive domains like language. 4 Nevertheless, long-term therapy is hardly accessible for stroke survivors, and patients who suffered from stroke often have several comorbidities that make it more difficult to reach a rehabilitation centre. 5 In this context, telerehabilitation (TR) emerges as a viable solution to ensure continuity of care for rehabilitation of post-stroke symptoms while optimising timing, intensity and sequencing of interventions that are likely to produce the greatest functional outcome for the patient. 6 It is defined as the delivery of rehabilitation services via information and communication technologies. 7 The use of TR is particularly useful in a context such as Italy, where in some regions, there are geographical barriers that make access to rehabilitation sites difficult for patients with motor impairments and developing TR systems feasible and easy to use is fundamental to guarantee a certain continuity of care and equity to rehabilitative services access. At the same time, one of the main strengths of TR relies on the possibility to provide treatment in the patients’ own home. Indeed, treatment conducted within patients’ social, educational and vocational environment can lead to improved functional outcomes and enhanced family and community integration. 8 The possibility for the patient to have treatment at home therefore has positive effects not only in terms of time and cost related to the travels to the rehabilitation centres but also in terms of quality of life (QoL), 9 which in turn could have positive effects on functional improvements. However, current literature supports the use of TR systems for motor,10,11 cognitive,12,13 speech–language14,15 and swallowing impairments 16 in different pathologies, as well as the use of telecare consultation for nursing management, 17 most of them focusing and referring to TR as ‘non-inferior’ to conventional face-to-face treatments. Given the fact that TR brings with it many strengths that are outside the purely functional improvements, but are closely related to them, it is worth asking: is it correct to say that TR is non-inferior to in-person treatment? Therefore, we developed a novel TR system that integrates virtual reality scenarios for the treatment of motor, cognitive and linguistic functions. The present study, designed as a pilot one, aims to evaluate and present pilot findings on the feasibility of this integrated multi-domain TR system targeting motor, language and cognitive impairments in stroke patients. Furthermore, we wanted to observe whether there are changes in the QoL levels of patients undergoing TR treatments.
Materials and methods
Study design
The present study is part of a pilot, multicentric, longitudinal cohort study. A sample of 84 post-stroke patients was recruited between March 2018 and December 2021 by six recognised Italian research hospitals [Istituti di Ricovero e Cura a Carattere Scientifico (IRCCS)] of the Italian Neuroscience and Rehabilitation Network (https://www.reteneuroscienze.it/en/profilo/): IRCCS San Camillo Hospital (Venice), IRCCS Fondazione Don Carlo Gnocchi Onlus (Milan), IRCCS Centro Neurolesi Bonino-Pulejo (Messina), IRCCS Fondazione Istituto Neurologico Casimiro Mondino (Pavia), IRCCS San Raffaele (Rome), and Istituto Clinico Humanitas (Milan). Patients were included in the study during their hospitalisation and initiated the treatment upon discharge, returning to their homes. Subjects were enrolled at the end of their hospitalisation period if they met the following inclusion criteria: (a) single ischemic stroke documented by CT/MRI; (b) time elapsed from stroke event ranged from 2 to 18 months from the event; (c) age ranged from 18 to 80 years; and (d) signed informed consent for the participation to the study. The inclusion criteria were selected to ensure a homogeneous study population that accurately reflected the target demographic for this pilot study, focusing on patients with single ischemic strokes within a specific recovery timeframe to assess the intervention's effectiveness without confounding factors. Patients were excluded if they had (a) presence or history of cognitive decline (i.e. MoCA score < 17.54), (b) recent fractures, (c) history of psychiatric disorders and (d) presence or history of other neurological and/or internal diseases that could have interfered with the outcomes. This study builds upon previous research that has analysed the clinical effects of integrated treatment approaches for post-stroke rehabilitation, providing complementary insights and enhancing our understanding of patient outcomes. 18
The study was registered at ClinicalTrials.gov (NCT05703906). The study was conducted in accordance with the principles of the Helsinki Declaration and was approved by the IRCCS San Camillo Hospital Ethics Committee (Coordinator Center, Prot. 2017.16) and by all other Ethical Committees from Participant Centres. Written and informed consent was obtained for all participants.
Intervention protocol
The intervention consisted of a TR treatment for motor, cognitive and/or language domains, taking place for 1 h/session, five times a week, for four weeks, for each treatment domain. The TR intervention was based on patients’ impairment, so that a patient could have conducted sessions for a single impaired domain (e.g. only motor or only language domain) or for all the three domains (i.e. one motor session, one cognitive session and one language session, resulting in 3 h of treatment per day). The exercises were performed at home by the patient using a dedicated virtual reality rehabilitation system home tablet kit in a synchronous TR modality, with remote supervision by the therapists (i.e. physiotherapists, speech–language therapists and neuropsychologists, based on patients’ needs). For motor training, patients had to move the end-effector (i.e. object or limb) following the trajectory of the corresponding virtual activity displayed on the screen. Cognitive exercises were focused on attention and executive functions, by asking patients, for example, to find a square within a matrix with images of various shapes. Language rehabilitation was based on exercises for the interpretation and production of written and oral words. In this case, the exercises proposed were based on lexical retrieval and patients were asked to write, letter by letter, the targeted item displayed on the screen. The therapists involved in the study customised the protocol of exercises in TR mode according to the characteristics and needs of the subject, and they were able to monitor patients’ progress thanks to a report that could be provided by the TR system. Therapists who performed the treatments were trained for the use of the system and for delivering the programme. If technical problems emerged, therapists contacted the company's engineers to resolve them.
Outcomes assessment
The primary outcome for this study was the feasibility of a multi-domain TR programme, assessed by calculating the adherence to treatment (i.e. percentage of training sessions completed by the patients) and by collecting information from the Technology Acceptance Model (TAM), a specific questionnaire used to quantify the likelihood of technology acceptance, and from the System Usability Scale (SUS).
Secondary outcomes included measures of QoL levels of participants, assessed with the Short-Form-36 (SF-36) Health Survey and the EuroQol 5-dimensions (EQ-5D) and changes in caregiver burden [assessed with the Zarit Burden Interview (ZBI)].
Evaluations and clinical data collection were performed by trained physiotherapist, speech–language therapist or neuropsychologist in relation to the domain needed to be assessed, at the beginning of the treatment programme (T0) and at the end of 20 sessions of TR (T1). QoL and customer satisfaction measures were assessed using patient-reported outcomes. This approach ensured that each domain was assessed by an appropriate specialist, enhancing the validity of our findings.
Statistical analysis
Missing data were found to be present for some of the variables. Where the percentage of missing data was <25%, the choice was made to impute data using the multivariate imputations by chain equations method.
The demographic characteristics and clinical data of the sample were described using descriptive statistics (i.e. mean, standard deviation [SD], frequency and percentage).
The statistical significance level was set at
Analyses for each questionnaire were conducted and are described below, according to the instructions of the authors of the scales.
Feasibility
System Usability Scale
At the end of the treatment programme, patients were presented with the SUS questionnaire that provides a reliable tool for measuring the usability of a system. It consists of 10 questions, with five positive and five negative statements. 20
To calculate the SUS score, we considered the contribution of each item (with a score from 1 to 5). Specifically, for Items 1, 3, 5, 7, and 9, the contribution to the score was the scale position minus 1. For Items 2, 4, 6, 8, and 10, the contribution was 5 minus the scale position. The contributions to the score for each item were then summed and multiplied by 2.5 to obtain the final score. 20 Final SUS scores can range from 0 to 100, where higher scores indicate better usability. The overall average score considered on the SUS scale is set at 68. A higher SUS score is therefore considered above average, while a lower score is considered below average. 21 We assessed the reliability of the SUS by calculating Cronbach's alpha for the entire scale.
To analyse the results of the SUS questionnaire, we chose to evaluate them in different ways: raw score, percentile rank, grading, adjectives and acceptability. All these markers give us a judgement on the usability of the system by using different descriptors. For a detailed description, please refer to the relevant references our analyses were based on Sauro and Lewis and Bangor et al.21–23 We found correspondences, based on the study by Lewis, 2018, between SUS score, grading, adjectives and acceptability for the interpretation of the SUS and presented them in Table 1. 24
Correspondence between SUS score, grading, adjectives and acceptability SUS.
Technology Acceptance Model: theoretical framework
The TAM 25 argues that the adoption rate of a product does not depend on the features it has, but rather on the experience that the user has (i.e. user experience). The TAM assumes that there are two factors that determine the acceptance of a particular technology by its potential users: (a) perceived usefulness (PU) and (b) perceived ease of use (PEU).
Perceived usefulness is defined as the degree to which a person believes that the use of a particular system would improve their performance. Perceived ease of use, on the other hand, is defined as the degree to which a particular system is easy to use. To determine the PU and the PEU of TR systems, users are asked to rate the product on a 12-item questionnaire.
Our study questionnaire consists of 36 items divided into eight theoretical dimensions: attitude (ATT), compatibility (COM), PEU, PU, habits (HAB), facilitators (FAC), influence (INF) and intention to use (IU). These dimensions can be grouped in turn into three main dimensions, following the adaptation of Gagnon et al. 26 : the individual context, the technological context and the organisational context. The proposed theoretical framework is adapted from Chau and Hu's telemedicine acceptance model 27 and presented in Figure 1. For a detailed description of the dimensions and the proposed theoretical model, please refer to Appendix A of the Supplementary Materials.
Adapted theoretical model.
The scores of the theoretical variables were examined by calculating mean and SD. The validity of the theoretical model was assessed using item correlation analysis. In particular, the correlation between the IU variable and the other theoretical variables was investigated using the Spearman correlation test. Furthermore, the internal consistency of the instrument was assessed by calculating Cronbach's alpha values for each theoretical variable.
Travel time and distance
With the purpose to definitely quantify patients’ time and travel savings, patients were asked to quantify the average travel time and distance (in kilometres) saved, as compared to a hypothetical condition in which they would have had to conduct outpatient or inpatient in-person treatment.
For both questions, responses were grouped into categories and analysed by considering percentage and absolute frequencies.
Treatment adherence
Finally, we analysed adherence to TR treatment by calculating the percentage of completed sessions out of the targeted number of sessions (i.e. 20 sessions).
Quality of life
EuroQol 5-dimensions
The EQ-5D 28 is a standardised instrument for use as a measure of health outcome, which is divided into two distinct sections: 5-dimensions subscale and Visual Analogue Scale (VAS).
The questionnaire was administered before (T0) and after treatment (T1) and the answers were analysed by calculating the absolute frequencies and percentages for each of the two time points.
With the VAS score, the health status of the subjects was calculated by means of the mean score with relative SD. To check the trend during the observation period, a test for comparison (i.e. Student's
Short form-36
The SF-36
29
is a questionnaire comprising 36 questions that cover eight domains of health (i.e. physical functioning, role limitations due to physical health, role limitations due to emotional problems, energy/fatigue, emotional well-being, social functioning, pain and general health). To assess whether there was a change in scores before and after therapy, scores from the pre-treatment (T0) were compared to the post-treatment (T1) assessments using Student's
Zarit Burden Interview
The ZBI is a questionnaire consisting of 22 items for the assessment of the consequences the care burden has on the caregiver, with the aim of identifying whether the caregiver is at risk of depression and/or whether further assessment and intervention are necessary. The total scoring is calculated by summing the scores for the individual items and ranges from 0 (i.e. no care burden) to 88 (i.e. highest level of care burden). A score above 24–26 identifies those caregivers for whom further investigation and possible support intervention would be indicated. 30
To identify the level of caregivers’ burden in our population, the mean and SD of the total scores and the maximum and minimum values were calculated.
Sample size
As a pilot study, to evaluate the reliability of our study's results, a retrospective (post hoc) power analysis was conducted using G*Power 3.1.9.6 software (Heinrich-Heine-University Düsseldorf, Germany). With a two-tailed Wilcoxon signed-rank test, a moderate effect size (Cohen's
Results
Sample characteristics
Out of 100 participants enrolled, 16 of them dropped out. Demographic characteristics are summarised in Table 2 for the whole sample (
Characteristics of the participants at baseline.
Customer satisfaction analysis
System Usability Scale
Cronbach's alpha for the 5-point SUS scale for the 10 questions was 0.83, indicating good reliability.
Raw score: The overall average score of the questionnaire was 73.36, with the lowest score of 10 and the highest score of 100. Results showed that most users found the TR system easy to use (73% agreed or strongly agreed) and well integrated (50% agreed or strongly agreed) and reported that they would be confident in using it (60% agreed strongly) and would like to use it (48% agreed or strongly agreed) (Table 3).
Results of the System Usability Scale items for all participants.
The mean of standardised responses per question and the percentages of patients’ responses are shown in Figure 2 and Table 4.
Percentages of patients’ responses per question.
Mean of standardised responses per question.
Percentile rank: The SUS scores obtained are converted into percentiles as reported in Table 1. Overall, the SUS score showed relatively high scores in the third quartile (75th percentile) and the second quartile (50th percentile: median) of 85 and 75, respectively, with an interquartile range of 22.5 and a minimum score of 10, suggesting that the subjects were interested in using TR. In this study, the average SUS score was 73.36, which, when combined with the overall SUS score on the percentile ranking curve, indicates that the score is above average overall (Figure 3).
Percentile rankings of SUS scores.
Considering the scale of adjectives, results indicated that the TR system we used was classified as a ‘good’ system. When we associated the average SUS score with the grade scale, it could be seen that the score is in Grade B (Table 1), which indicates the score is above average. In relation to the acceptability scoring, the average SUS score obtained is classified as ‘acceptable’. The use of the TR system can therefore be accepted and used in general by the patients.
Technology Acceptance Model: theoretical framework
Table 5 shows the mean and SD of the theoretical variables. In addition, the table shows the correlations between the theoretical constructs and IU variable. The results indicated that all constructs were well correlated with the variable IU, except for the construct facilitators (rho = 0.03;
Descriptive analysis of the variables of the model.
Table 6 presents the adaptation of the original TAM to our technology acceptance questionnaire (Appendix A, Supplementary materials). Furthermore, the internal consistency of the instrument was assessed by calculating the Cronbach's alpha values for each theoretical variable. Except for facilitators and habits, Cronbach's alpha values were acceptably high (≥0.7) for the remaining theoretical constructs (Table 6).
Internal consistency of each theoretical variable of the TAM instrument.
Travel time
Figure 4 presents the percentage frequencies related to the average travel time saved by patients undergoing TR, compared with a hypothetical condition in which they would have had to carry out outpatient or inpatient in-person treatment.
Average time saved by the patients.
Most respondents (
Travel distance
On average, most patients (
Average distance (in km) to the hospital.
Treatment adherence
In our study, we found that out of 20 sessions prescribed to patients, the average attendance was 17 sessions (SD = 7.5), accounting for 85% of total sessions.
Quality of life
EuroQol 5-dimensions
Data from the EQ-5D questionnaire showed that subjects at T0 most frequently reported having some or severe problems in the domains of movement skills (59%) and activities of daily living (66%); conversely, the domain with the lowest percentage of subjects referring problems was that of anxiety or depression (66%). As for the scores at T1, the ratings did not change compared to T0, confirming as the most problematic the domains of the ability to move (53%) and of daily activities (56%), while the domain of anxiety or depression (64%) was the less problematic.
With regard to the VAS subscale, the average score was 61 (15.6) at T0 and 68 (15.5) at T1. The Wilcoxon test for comparison revealed a significant difference between the values at T0 and T1 (
EuroQol 5-dimensions (EQ-5D) Visual Analogue Scale (VAS). Differences between values at T0 and T1.
Short form-36
When administering the SF-36 questionnaire before and after therapy, there was a significant difference only for the subdomain of ‘general health’ (
Short Form-36 (SF-36) outcomes.
Values are expressed as mean ± SD. *
role lim phy health, role limitations due to physical health; role lim emo, role limitations due to emotional problems.
Zarit Burden Interview
For the ZBI questionnaire, the lowest grade was 1, and the highest grade was 49. The average of the total sum of the scores to the individual items was 19.03 (SD = 13.7), indicating that caregivers perceive little or no significant care burden.
Discussion
This study aimed to evaluate the feasibility, acceptability and usability of an integrated multi-domain TR system in stroke patients by performing a customer satisfaction analysis, which is one of the main strengths of this study. Furthermore, we observed whether there are changes in the perceived QoL levels of patients undergoing TR treatments.
In relation to the usability of the system, our results showed that the majority of users found the TR system easy to use and well integrated, demonstrating confidence in using it and a willingness to continue using TR, as opposed to other research in which technology barriers and lack of computer literacy prevailed as a major issue in successfully implementing telemedicine. 31 Indeed, barriers to the delivery of TR for stroke patients reported were frequently related to ease of use, which was influenced by the difficulty of using the system, as well as connectivity and technical issues. 32 In our study, we tried to overcome this issue by providing a dedicated system for TR, with a period of training on the use of the TR system for patients before hospital discharge. In this way, we were able to deliver TR treatments even to older and cognitively impaired patients, with reduced patients’ dissatisfaction and minimal technical issues. One of the main reasons for the high acceptability levels reported may be attributable to the possibility to continue a rehabilitation programme even after hospitalisation with a high reduction of time spent travelling. Indeed, from the results obtained from the TAM questionnaire, we can observe that most of the participants declared to save more than 1 h of travel time per session by using TR, compared with rehabilitation in a hospital setting. To our knowledge, this is the first study in the field quantifying time and travel distance savings when using TR. This brings to light one of the greatest strengths of TR, that is, allowing patients not only to receive treatment at home but also to be able to save travel time and to allow patients to access rehabilitation interventions at appropriate frequency and intensity, even in those case where comorbidities and motor impairments restrict access to rehabilitation centres, especially in areas where geographical barriers are not fixable (e.g. Venetian territory). This is highly related to one of the main topics of healthcare management, that is to guarantee and improve accessibility to rehabilitation services to post-stroke patients, even in their chronic phase.
All these results taken together may have positively influenced the treatment adherence of our sample that was 85% of the total sessions. Previous studies have shown that adherence to the treatment programme can range from 14% to 70%,33,34 placing our results well above the average reported. The adoption rate of TR and the adherence to treatment can be depended on the users’ experience we observed from the results of the TAM questionnaire. Indeed, we found that the higher the PU and the PEU are, the more there is IU the technology and, in turn, to complete the treatment programme.
All these results therefore showed us that TR is feasible and well accepted by patients, which also highlighted beneficial effects in terms of perceived QoL levels. In fact, we observed an increase in perceived QoL levels between the beginning and the end of the treatment programme. Certainly, a central role in this outcome is played by the fact that the patient is at home, in a familiar environment shared with close caregivers and relatives members. In addition, the possibility to undergo treatment at home also makes it easier to reintegrate the person in own social and working life, facilitating the individual's inclusion within the International Classification of Functioning, Disability and Health (ICF) framework that includes, in addition to body functions, also activities and participation. 35 At the same time, TR could be beneficial also in reducing caregivers’ burden, as suggested by the positive response we received.
At this point, it is worth asking whether it is really correct to look at TR only in terms of non-inferiority to conventional treatments, since, as we have seen from our study, from a service perspective, there are many benefits that this treatment modality delivers to people with stroke, overall incorporating improvement of perceived QoL, reduction of travel time and distance and also reduction of caregivers’ burden, together with the ease of use of the system. Future research is needed to quantitatively investigate if there is an association between improvements in perceived QoL and clinical outcomes.
Limitations
This study is a pilot study with a single experimental group. Therefore, the absence of a control group poses a challenge in establishing the intervention's effectiveness, especially in relation to the perceived QoL, as it becomes difficult to differentiate the impact of the intervention from external variables or potential confounding factors. Another limitation arises from the lack of long-term follow-up, preventing the assessment of the intervention's enduring effects on perceived QoL.
Conclusions
Telerehabilitation treatment is feasible, usable and well accepted by patients, even with comorbidities and cognitive deficits. The use of well-designed and tailored TR systems can allow continuity of treatment and accessibility to rehabilitation treatments, while saving time and increasing perceived QoL levels, which in turn could impact on person functioning.
Supplemental Material
sj-docx-1-dhj-10.1177_20552076241308617 - Supplemental material for Feasibility, acceptability, usability and quality of life levels in post-stroke patients undergoing telerehabilitation: Results from a multicentric pilot study
Supplemental material, sj-docx-1-dhj-10.1177_20552076241308617 for Feasibility, acceptability, usability and quality of life levels in post-stroke patients undergoing telerehabilitation: Results from a multicentric pilot study by Luisa Cacciante, Sara Federico, Lorenza Maistrello, Pawel Kiper, Roberto De Icco, Tommaso Milanesi, Cristina Tassorelli, Roberto Gatti, Johanna Jonsdottir, Marco Franceschini, Michela Goffredo, Rocco Salvatore Calabrò, and Andrea Turolla onbehalf of The Italian Neuroscience and Rehabilitation Network (RIN) in DIGITAL HEALTH
Footnotes
Acknowledgments
The authors would like to acknowledge the members of the Italian Neuroscience and Rehabilitation Network (RIN) and all the participants who took part to the study.
Contributorship
All authors conceived the study and were involved in protocol development, gaining ethical approval and patients’ recruitment. LC, SF, LM, AT and RDC were involved in data analysis. LC wrote the first draft of the manuscript. All authors reviewed and edited the manuscript and approved the final version of the manuscript.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Ethical approval
The study was approved by the IRCCS San Camillo Hospital Ethics Committee (Coordinator Center, Prot. 2017.16).
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded by the Italian Ministry of Health (Ricerca Corrente).
Guarantor
LC.
Data availability statement
Data are available upon request to the corresponding author.
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
