Sage Journals: Discover world-class research

Abstract

Objective

This scoping review aimed to identify the design and usability testing of a telerehabilitation (TR) system, and its characteristics and functionalities that are best-suited for rehabilitating adults with chronic diseases.

Methods

Searches were conducted in PubMed, EBSCO, Web of Science, and Cochrane library for studies published between January 2017 and December 2022. We followed the Joanna Briggs Institute guidelines and the framework by Arksey and O’Malley. Screening was undertaken by two reviewers, and data extraction was undertaken by the first author. Then, the data were further reviewed and discussed thoroughly with the team members.

Results

A total of 31 results were identified, with the core criteria of developing and testing a telerehabilitation system, including a mobile app for cardiovascular diseases, cancer, diabetes, and chronic respiratory disorders. All developed systems resulted from multidisciplinary teams and employed mixed-methods research. We proposed the “input-process-output” framework that identified phases of both system design and usability testing. Through system design, we reported the use of user-centered design, iterative design, users’ needs and characteristics, theory underpinning development, and the expert panel in 64%, 75%, 86%, 82%, and 71% of the studies, respectively. We recorded the application of moderated usability testing, unmoderated testing (1), and unmoderated testing (2) in 74%, 63%, and 15% of the studies, respectively. The identified design and testing activities produced a matured system, a high-fidelity prototype, and a released system in 81.5%, 15%, and 3.5%, respectively.

Conclusion

This review provides a framework for TR system design and testing for a wide range of chronic diseases that require prolonged management through remote monitoring using a mobile app. The identified “input-process-output” framework highlights the inputs, design, development, and improvement as components of the system design. It also identifies the “moderated-unmoderated” model for conducting usability testing. This review illustrates characteristics and functionalities of the TR systems and healthcare professional roles.

Keywords

Design usability testing chronic diseases telerehabilitation framework

Introduction

The “input-process-output” (IPO) framework provides researchers and professionals from several disciplines, particularly healthcare, with an inclusive roadmap for designing and usability testing the telerehabilitation (TR) system for chronic diseases. It also establishes a future detailed framework in this field. The components, flow, and basic recommendations to adopt the IPO were highlighted to facilitate the TR system design and usability testing, enabling healthcare professionals (HCPs) to better deliver rehabilitation services to their patients.

Despite the recurring,¹ long duration, and slow progression nature of chronic diseases, their behavioral related risk factors can be modified using rehabilitation programs,² including self-management support.^3–5 Conventional rehabilitation aims at opposing disease progression and improving health outcomes as well as function.⁴ However, the inadequate and ineffective interventions create the current gap, suggesting a critical need for new advances toward self-management. TR has the potential to fulfil this gap.^4,6,7 Rehabilitation of chronic diseases is complex and described as multidisciplinary,⁸ multistage, and multi-component.^3,5,7,9 It has different pillars, including screening and assessment, setting a rehabilitation plan, exercise training, self-management support, psychological and social support, and maintenance and follow-up.⁵ The complexity of noncommunicable disease rehabilitation as well as its inadequacy in fulfilling the gaps in rehabilitation necessitate searching for alternatives to do so. TR is a good replacement that provides a comfortable and safe environment, in addition to increasing access, maintenance, and follow-up.^10,11 It also facilitates collaborative care and self-management in controlling behavioral risk factors.^12–14 TR involves providing rehabilitation services remotely by using Information and Communication Technologies (ICT) while patients are physically presented in different environments.¹⁵ It motivates and empowers the patients to take control of their management^15,16 and provides the HCPs with easy access to patients’ records collected through the telerehabilitation systems.¹⁷

User-centered design (UCD) is the most commonly used system design approach that maps and guides design activities. UCD focuses on end users and stakeholders in iterative research, design, and evaluation of services or systems¹⁸ to identify and prioritize their needs.¹⁹ From the healthcare perspective, UCD implies translating evidence-based practices from research into the industry. It can be utilized alone or with integration with other approaches.^20–22 Iteration, also known as agile development, is associated with UCD, which refers to the repetitive, incremental, and responsive-to-change design, development, and testing, allowing for quick learning and product modification.²³ Agile is recommended to improve system features and reduce usability issues early in the design process, allowing the inclusion of more features and modules to satisfy the end users.²⁴ Agile adapts well to the design changes²⁵ and allows making the most of the specified budget and timeline.²³

Theory underpinning development and multidisciplinary team are crucial inputs for a system design. Behavioral change theory (BCT) is critical to translate theoretical constructs into technological outputs as exemplified in features, modules, and functionalities. It is crucial for chronic disease rehabilitation to support long-lasting maintenance of the changed behaviors.^26,27 However, a more organized practice is required to standardize the translation process and design techniques and match the final output with the underlying construct.^26,28 Working in multidisciplinary team from health, engineering, and computer sciences,^29,30 or variable health disciplines,³¹ is essential to improve the quality of research work, output, and publications.³² Early engagement of team members is imperative to overcome the reported challenges of working in MDT.³³

Usability testing is “the extent to which a system, product or service can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.”³⁴ This definition highlights five aspects of usability testing: (1) the system, (2) the task, (3) the user, (4) the goal and usability metrics, and (5) the environment, which the technology readiness level would best measure. Generally, usability testing is conducted to measure the systems’ effectiveness and efficiency, and the patients’ satisfaction.³⁵

The basic TR system comprises at least a camera, allowing HCPs to visualize and monitor patients through videoconferencing while advanced TR system comprises sensors and provides more functionalities for managing multiple pathologies.¹⁷ TR system can be categorized into three groups: (1) the one connected with a wearable device, (2) the Kinect-based system connected with a tracking sensor, such as an accelerometer, and (3) more complex rehabilitation robotics.^17,36 Regardless of its classification, the basic TR system architecture should include three components: patient, therapist, and communication HUB components.³⁷ Figure 1 illustrates TR system architecture, components, functionalities, and flow.^16,17,37,38 Basically, a minimum of three modules should be included in any TR system: the data processing, therapy evaluation, and interface module.³⁹

Figure 1.

Telerehabilitation system: basic architecture and flow.^16,17,37,38 Basic architecture of the TR system includes patient component, communication hub component, and therapist component. Numbers indicate the flow of the TR system, which starts from the patient's component with: (1) data collection. At the communication hub (2), data processing takes place through three modules: data processing, therapy evaluation, and interface modules. At the therapist component, the process continues through (3) tailored intervention, and (4) tracks therapy progress. Then, the (5) input data, which convey the required information back to the communication hub, go through (6) data processing by the three modules. Finally, the data from the communication hub notify the patients to (7) identify abnormalities, (8) and give feedback and reminders.

An effective TR system allows the therapist to select appropriate therapies, evaluates them while performed by patients, and manages the overall therapies remotely.¹⁷ The fundamental characteristics of TR system that should be considered during system design to meet the 10 e's criteria are efficiency, enhancing quality, evidence-based, empowerment, encouragement, education, enabling, extending, ethics, and equity. It would also be imperative to develop an easy-to-use, entertaining, and exciting TR system to enhance engagement and avoid boredom.⁴⁰

The maturity of a TR system's output could be measured by the technology readiness level (TRL), which consistently compares such maturity across various components of the technology as it develops.^41,42 It can be utilized to guide usability testing studies and identify its context.⁴¹ Therefore, moderated and unmoderated testing models are identified^35,43 to describe the context of usability testing and provide recommendations for flexibility and experimentation.¹⁸ Other available frameworks focus on professional judgment versus the user experience. However, they are less utilized in usability research.⁴⁴

The objective of the system design framework is to map and guide the overall design process and help drive progress. Despite the variability of the available design frameworks, it includes the predesign stage, actual design, preliminary development, and enhanced development.⁴⁵ The biodesign process is one of the available and recent frameworks that describe the phases, stages, and activities of medical technologies design. It describes the design process into three distinct yet interrelated phases, each with subsequent two stages: (1) Identify: needs finding and screening; (2) Invent: concept generation and screening; and (3) Implement: strategy development and business planning.⁴⁶

Physiotherapists^17,39 and HCPs increasingly participate in the TR system design and evaluation as they best identify and fulfil patients’ needs,^17,47 thus increasing the demand to enhance their knowledge and skills in this field. Therefore, this review ought to provide them with a framework for developing and testing the usability of telerehabilitation systems and mHealth from healthcare perspectives, thus effectively and efficiently delivering rehabilitation services to the patients.

To the best of our knowledge, there is a lack of a framework that helps HCPs effectively participate in the design and evaluation of TR system. The available literature in this field provides either design⁴⁸ or usability testing frameworks.^49–51 Some literature focuses on specific diseases^18,30,52 or symptom management⁵³ rather than the noncommunicable diseases. A review of the overall process targeting the noncommunicable diseases is missing.

This review was conducted to: (1) identify the approaches and components of the design of a TR system, (2) identify the approaches and stages of conducting usability testing of a TR system, (3) identify characteristics and functionalities of the TR system that are best-suited to rehabilitate adults with noncommunicable diseases, and (4) identify a comprehensive framework for design and usability testing of TR system for chronic diseases.

Methods

This review was conducted according to the guidelines provided by the Joanna Briggs Institute⁵⁴ on scoping reviews and the framework provided by Arksey and O’Malley,⁵⁵ with improvements suggested by Levac et al.⁵⁶ and Daudt et al.⁵⁷ In addition, we adopted the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Extension for Scoping Reviews.⁵⁸

Stage one: identifying research questions

This review was designed to answer the main research question: “What are the available frameworks for designing and usability testing a telerehabilitation system for chronic diseases?” Afterward, we intended to answer the subsequent sub-questions:

What are the approaches and components in designing a telerehabilitation system?

What are the approaches and stages of conducting usability testing of a telerehabilitation system?

What are the characteristics and functionalities of the telerehabilitation system that are best-suited for chronic disease rehabilitation?

What are the components and flow of the framework for design and usability testing of TR system for chronic diseases rehabilitation?

Stage two: identifying relevant studies

A comprehensive search was performed to retrieve studies on the design and usability testing of TR systems for chronic disease from January 2017 to December 2022. This period was selected because the literature illustrated that the publication of usability studies reached its peak around 2017,⁵¹ and further increased in 2018–2019.⁴³

The electronic searches were done on PubMed, Web of Science, EBSCO, and Cochrane Library electronic databases. In addition, we conducted a manual hand search on key electronic journals, such as SAGE Digital Health and the Journal of Medical Internet Research in addition to the search in grey literature to maximize the search coverage.

Searching on grey literature included reports, book chapters, guidelines, academic theses, and dissertations.

The search strategy comprised of four keywords: design, usability testing, chronic disease, and telerehabilitation. Synonymous keywords were identified using a MeSH search on PubMed. Two researchers (SG and AFML) developed a list of keywords synonymous to capture potential studies in the resources, as summarized in Table 1.

Table 1.

List of keywords and synonyms generated as search terms.

Design	Usability testing	Chronic disease			Telerehabilitation
Medical device design	Testing usability	Noncommunicable disease	Non-infectious disease	Obesity	Remote rehabilitation	Virtual rehabilitation
Equipment design	User-centered design	Overweight	Diabetes	Cardiovascular	Telehealth	Telemedicine
Device design	–	Cancer	Dyslipidemia	Mortality	Mobile health	mHealth
Framework	–	Morbidity	Chronic disorders	Pulmonary diseases	Telemedicine	–
–	–	Chronic obstructive	COPD	–	–	–

Search terms were connected using Boolean operators to produce search strings. Subsequently, we used (Design OR Medical device design OR Equipment design OR Device design OR framework) AND (Usability testing OR Testing usability OR User Centered Design) AND (“Chronic diseases” OR “Noncommunicable Disease” OR “Non-infectious Diseases” OR “Metabolic syndrome” OR Obesity OR Overweight OR Diabetes OR Cardiovascular OR Cancer OR Dyslipidemia OR Mortality OR Morbidity OR “Chronic disorders” OR Pulmonary diseases OR Chronic obstructive OR COPD) AND (“Telerehabilitation” OR “Remote Rehabilitation” OR “Virtual Rehabilitation” OR “telehealth” OR “Telemedicine” OR “Mobile health” OR “eHealth” OR “mHealth”) for the article retrieving.

Stage three: study selection

The third stage focused on identifying studies to be included in the review. The screening procedure consisted of two stages: (1) a title and abstract and (2) full-text screening. The PRISMA flow chart was deployed to guide the study's selection procedure. The research team used Rayyan, the intelligent systematic review website, to perform the following two stages of the review. In the first stage, two reviewers (SG and AFML) independently screened the titles and abstracts of the articles. In the second stage, SG reviewed the full text of the included studies based on the eligibility and exclusion criteria detailed in Table 2, and then reviewed and discussed the findings thoroughly with AFML.

Table 2.

Eligibility criteria for study inclusion and exclusion.

Criteria	Inclusion criteria	Exclusion criteria
Date of publication	2017–2022 full-text studies	Studies before 1 January 2017 Conference Proceedings Abstracts
Chronic diseases Noncommunicable diseases	Diabetes, CVD, cancer, and COPD	Infectious diseases
Chronic disease rehabilitation (8 functions)	Rehabilitation for the chronic disease includes one or more of the following: 1. Screening 2. Development of a rehabilitation plan 3. Assessment (initial, progress, and discharge) 4. Self-management support (including health education, smoking cessation, and nutrition) 5. Psychological and social support 6. Supervised exercise training 7. Advance care planning 8. Maintenance and follow-up	Studies limited to mental health Studies limited to medication adherence Studies limited to cognitive behavioral changes
Telerehabilitation system	Mobile app and\hardware, cloud	Software; website app only Interactive messaging SMS
Study design	Design and usability testing Case studies, piloting, intervention studies	Studies limited to system design Studies limited to system evaluation Study protocol Review studies
Population	adult aged 18–60 years	A healthy adult requires behavioral changes, such as smokers and pregnant
Researchers	Multidisciplinary team, including healthcare researchers	–

Eligibility criteria

The eligibility criteria for chronic disease, chronic disease rehabilitation, telerehabilitation system, design and usability testing, study design, and population are summarized in Table 2. The list of chronic diseases was derived from the definition provided by Bernell and Howard¹ to include conditions that recur or repeatedly occur for a long time, and the description provided by the WHO² to encompass the mentioned four clusters. However, the team decided to include cardiovascular diseases instead of the mentioned heart diseases. The concept of chronic disease rehabilitation was derived from the Agency for Clinical Innovation (ACI) that focused on restoring and maintaining an optimal level of one's functional ability.⁵ The definition, architecture, and essential components of the TR system were based on Brennan et al.,¹⁵ Anton et al.,¹⁷ Vukićević et al.,¹⁶ and Tsutsui et al.⁵⁹ The design and usability testing of the TR system was obtained from Harst et al.⁶⁰ and Kip et al.⁴⁸ The significance of multidisciplinary team approach was established according to Aji et al.³⁰ and Shah et al.²⁹

Stage four: data charting process and data items

The charting process was done by SG by charting forms and developing a priori based on a toolkit,⁶¹ theories,⁶² and previous studies,^22,60,63,64 and it was discussed thoroughly with AFML. The charting forms encompassed four main sections:

The first section described the study characteristics, including location, year of publication, length of time to conduct research activities, description of the research team, design, and data collection tool. Additionally, this section described the chronic disease type and participants’ characteristics, focusing on their roles.

The second section identified the design of the TR system, which has been incorporated as separate independent procedures divided into several steps.

The third section identified the usability testing of the TR system, and it was divided into three independent phases.

The fourth section described the developed and tested TR system in terms of name, features and components, system functionalities, and maturity of the output.

The components of chronic disease rehabilitation provided by the TR system that should be commensurate with the eight core components of chronic disease rehabilitation⁵ was added by SG, and reviewed and discussed with AFML.

To explore the clarity and suitability of the charting sections, the charting form was piloted using one of the included papers by SG and AFML before data extraction started. Then, SG further read and extracted data into the charting forms before reviewing and having a thorough discussion with AFML, and further discussed with other members (DKAS, KBG, NMA) to agree on the final extraction. Any ambiguity or uncertainty was discussed across the multidisciplinary research team. According to the scoping review guidelines, a critical appraisal was not undertaken.⁵⁸

Stage 5: collating, summarising and reporting the results

Collation and summary of the data based on the objectives were performed by SG, and reviewed and discussed thoroughly with AFML. Then, the charted data were reviewed, classified, and clarified with the sources by SG, and reviewed and discussed with AFML. These summaries were consulted to identify the most appropriate way of presenting the results, and then were sent to the team for review and discussion at a team meeting (SG, AFML, DKAS, KBG, and NMA). Data were presented descriptively using percentages and numbers.

The characteristics of the included studies are illustrated in Table 3, consisting of geographic location, year of publication, MDT, length of time conducting research activities, type of chronic disease, and participant's characteristics. Variable approaches, phases of system design and usability testing, and characteristics of the TR system were discussed in depth by all (SG, DKAS, KBG, NMA, and AFML), and then further summarized in Supplemental Tables 1–3, respectively.

Table 3.

Study characteristics.

Paper	Location	Chronic disease	Multidisciplinary team (researchers or developers)		Study participants			Length of time of research activities	Mixed-methods research (MMR)
Paper	Location	Chronic disease	Medicine & health	Engineering & IT	Patient	Expert	Caregivers	Length of time of research activities	Mixed-methods research (MMR)
Adu et al. 2020	Australia	Diabetes	√	√	√	–	–	• Need assessment: 8 months • Phase1 Usability: 7 days	√
Agher et al. 2020	France	CVD	√	√	√	–	–	• 2 months	√
Baek et al. 2018	Republic of South Korea	CVD	√	√	√	√	–	• Survey: 17 days • Usability: 2 weeks	√
Castensøe-Seidenfaden et al. 2017	Denmark	Diabetes	√	√	√	√	√	• Need assess workshop: 1 month • 6 Iterative cycles: 7 months • Feasibility study: 5 weeks	√
Davies et al. 2020	UK	COPD	√	√	√	√	√	• Phase 2 usability: 3 months	√
Deng et al. 2021	China	COPD	√	√	√	–	–	• Preliminary test: 8 weeks • Assessment test: 12 weeks	√
Duan et al. 2020	China	Hypertension	√	√	√	–	–	• Real-life usability: 2 months	√
Duff et al. 2018	Ireland	CVD	√	√	√	–	–	Not mentioned	√
Faiola et al. 2019	USA	Diabetes	√	√	√	–	–	Not mentioned	√
Fico et al. 2020	Spain	Diabetes	√	√	√	√	–	• 8 months	√
Hou et al. 2020	Taiwan	Cancer	√	√	√	–	–	Not mentioned	√
D. Inupakutika, et al. 2020	USA	Cancer	√	√	√	√	–	Not mentioned	√
Eunjoo Jeon and Hyeoun-Ae Park. 2018	Republic of South Korea	Diabetes	√	√	√	–	–	Not mentioned	√
Kim et al. 2022	Republic of South Korea	PAD	√	√	√	√	–	• Need assessment Survey: 4 months.	√
Korpershoek et al. 2020	Netherlands	COPD	√	√	√	√	–	Not mentioned	√
Kwon et al. 2018	Republic of South Korea	COPD	√	√	√	–	–	Not mentioned	√
LeBaron et al. 2022	USA, UK, Nepal	Cancer pain	√	√	√	√	–	• 3 months	√
Monteiro-Guerra et al. 2020	Ireland, Spain	Cancer	√	√	√	–	–	Not mentioned	√
Morse et al. 2021	USA, Tanzania	Cancer	√	√	√	√	–	• 22 months	√
Wu et al. 2019	China, Australia	Stroke	√	√	√	√	√	Not mentioned	√
Nicklas et al. 2020	USA	CVD	√	√	√	–	–	• The three (4-week pilots): 8 months.	√
O’Malley et al. 2020	USA	Cancer	√	√	√	√	–	• Phase 1 usability: 5 months	√
Petersen and Hempler. 2017	Denmark	Diabetes	√	√	√	–	–	• 13 months	√
Tundjungsari et al. 2018	Indonesia	CVD	√	√	√	√	–	Not mentioned	√
Velardo et al. 2017	UK	COPD	√	√	√	–	–	Not mentioned	√
Welch et. Al. 2021	USA, Canada	Cancer	√	√	√	–	–	• UCD, iterative evaluation and 2 field testing: 18 months	√
Yee et al. 2021	USA	Diabetes	√	√	√	√	–	• 2 FGDs: 15 months	√

COPD: chronic obstructive pulmonary disorder; CVD: cardio vascular disease; FGDs: focus group discussions; MMR: mixed-methods research; PAD: peripheral artery disease; UCD: user-centered design; √ means involvement; – means no involvement.

Results

Searches on databases yielded 466 results (PubMed = 36, Web of Science = 225, EBSCO = 108, and Cochrane library = 97), as shown in Figure 2. The research team performed the following phases of the review: (1) 110 articles were removed for duplication among databases. (2) Two researchers (SG and AFML) independently reviewed 356 titles and abstracts according to the inclusion and exclusion criteria, which resulted in 144 potentially relevant studies for full-text review and the exclusion of 212 articles. The results of this review were discussed with the team afterward. (3) During the full-text review phase, SG read the full text of the 144 studies and, afterward, reviewed and discussed thoroughly with AFML. Both reviewers agreed to exclude 116 studies due to non-eligible study design (56), non-eligible population (17), non-eligible publication type (24), non-eligible both publication type and population (5), non-eligible publication type and study design (13), and non-eligible design and non-eligible population (2). The team also added four reports of previous versions of the reviews, as shown in Figure 2, whereas the needs assessment studies were conducted and reported in previous studies.^65–68 Article retrieving ended up with 31 studies, in which the needs assessments of four TR system were published in separate studies. We grouped together according to the original study and reported as one. We finally ended up with 27 studies.^69–72

Figure 2.

PRISMA flow diagram for review phases, including results identified and excluded and reasons for exclusion.

Study characteristics

The 31 included studies have resulted in 27 TR systems either by multi-national collaboration or as uni-national projects. Five of the developed systems resulted from the cooperation of multi-national institutions,^72–76 mainly from the United States (USA), the United Kingdom (UK), Canada, Australia, and countries from Africa and China. Additionally, cooperation across European countries has been recorded as well.

Regarding the uni-national projects, the USA has recorded the highest number of five projects,^71,77–80 followed by South Korea that developed four TR systems,^81–84 followed by China,^85,86 Denmark,^47,87 and UK^88,89 that developed two TR systems independently. The remaining seven TR systems were created in Australia,⁶⁹ France,⁹⁰ Netherlands,⁹¹ Ireland,⁹² Spain,⁷⁰ Taiwan,⁹³ and Indonesia.⁹⁴

Cardiovascular disorder (CVD) was a chronic disease to be targeted the most by the intervention, as eight TR systems were developed,^{75,78,81,83,86,90,92,94} followed equally by diabetes^{47,69,70,77,80,82,87} and cancer,^{71–74,76,79,93} with seven TR systems targeting each of them. Finally, COPD was targeted by five TR systems.^{84,85,88,89,91}

All studies were conducted by multidisciplinary teams from healthcare, engineering, and information technology (IT), either as researchers or developers. Regarding participants’ characteristics, only their role was considered in the analysis. Role analysis showed that 13 studies had HCPs and developers as participants, and all studies included patients.

Since the design, development, and evaluation of the TR system was multistage, complex, and resource-consuming, and the reported “length of time of the research activities” was inconsistent in analysis, the team decided to report the time for conducting such studies by describing the raw data as shown in Table 3. About 55.5% of the studies roughly documented the time required to conduct specific research activities, while the timeline of the entire study was described in detail in one study.⁷⁴ All studies conducted mixed-methods research (MMR), deploying a quantitative and qualitative research methodology. The characteristics of the included 27 studies are illustrated in Table 3.

System design: phases and approaches

The identified system design model, as shown in Figure 3, encompassed four phases: inputs, ideation, development, and improvement. The overall model was guided by adopting a design approach. Supplemental Table 1 outlines the system design phases and components, as part of the IPO framework; we identified that 55.5% of the studies clearly articulated the adoption of the UCD approach, which was a pillar to drive the system design. In comparison, two studies separately adopted each human-centered design and design thinking. Only one study adopted the goal-directed design and participatory design, separately. Three studies integrated two or more of the UCD, participatory design, design thinking, and co-design to guide their design process. One study relied on usability testing principles to guide their system design, and the remaining two did not mention adopting any design approach.

Figure 3.

The system design model based on the 27 TR systems.

Design inputs represent system design requirements, including theory-informed development, multidisciplinary team, review of literature and guidelines, users’ needs and characteristics, context analysis, expert panel, secured funds, and other inputs that were inconsistently presented in the studies.

Theory underpinning development is imperative in translating theoretical constructs into technological outputs, exemplified in features, modules, and functionalities. Almost 82% of the studies clearly articulated the underpinning theory while the remaining five did not mention any groundwork. The self-management construct was embraced the most (10 studies), followed by the BCT and its derivatives, which were adopted in six studies. Notably, six projects merged two or more theories from the self-management construct and the BCT or its derivatives. Working in multidisciplinary team is essential in TR system design and testing, and was clearly illustrated in all studies. Notably, it was considered a concrete phase of the system design process in one study.

The literature review is a foundation work to be enclosed in the system design process. Similarly, guidelines and evidence-based reviews are the groundwork for interventional studies. Regardless of being combined in one item, the specifications of the guideline review were described in the data collection tool. Literature or guideline review was not identified in six studies; those were fellow publications in the system development pathway.

The users’ needs and characteristics are essential to inform system design, whether users are identified as patients, caregivers, or HCPs. The users’ needs were clearly charted in 23 studies. Furthermore, it was clear how researchers identified the users’ needs and characteristics through surveys, interviews, workshop discussions, FGDs, or personal development. The remaining four studies relied on literature and guideline review, feature analysis of related apps, and theory embracing.

Context analysis was slightly described as a design requirement. Only three studies described the current practice analysis, existing services, and environmental considerations in their design process. In contrast, the expert panel was illustrated as a core design requirement in 19 studies. Their role was clearly identified to review and validate the conceptualization, navigation and interface, and mock-ups. Additionally, the panel reviewed and validated the created contents of the TR systems and verified rounds of iterations. In the remaining eight studies, the research team, HCPs, and designers carried out such responsibilities, but the process was vaguely described, and the undertaken roles were doubtfully identified. Securing funds is a prerequisite to launch a TR system design and testing. Generally, 100% of the included studies have secured funds, either through governmental or association projects, research grants, or international research grants.

Other design inputs were inconsistently but clearly illustrated in the studies. The feature analysis of related apps was identified in four studies while the security of the users’ data, technical specifications’ documents, adopting a specific theoretical model such as the IMBS, patients’ self-monitoring tasks, and case scenarios were identified in two studies separately. We also identified a single use of the benchmark of app interface design, benchmark of individualized goals, review of two FDA products, comparison of two versions of the developed mobile app, and heuristic interface design principles.

The design process consisted of three other phases: ideation, development, and improvement. The ideation phase was arranged into four parts: conceptualization, navigation and interface, visuals, and mock-ups. The conceptualization part was illustrated in 26 studies, either through workshop discussion, brainstorming and card sorting method, persona and case scenario creation, interviews, focus group discussions (FGDs), affinity wall method, or expert panel recommendations. The “navigation and interface part” was identified in 21 studies. The “Navigation and interface part” was performed by sketching the layout, mapping content, and creating flow chart. The visual part was missed in 17 studies, suggesting a knowledge and skill gap among the MDT. Mock-ups, as a final part of the design phase, was identified in 15 studies; they were created as paper prototypes, low-fidelity prototypes, or wireframes.

The development phase encompassed content development, coding, and prototype creation. Despite being consecutively arranged, those parts were inconsistently identified across the included studies. Content creation was well articulated in 16 studies, and coding was clearly illustrated in 19 studies. Finally, prototyping was clearly identified in 24 studies; it was missed in two studies and was unclear in one study. The three parts of the development phase were clearly identified in 17 studies while one of the three parts was well articulated in three studies.

The improvement phase included two parts whereas iteration and verification were consecutively performed to refine and enhance system features. Iteration was identified in 21 studies, and the number of iteration rounds was mentioned in two studies while it was recommended to keep the number of iterations flexible.¹⁸ Verification of the iterative development was well identified in 17 studies, and both parts of improvements were articulated in 14 studies.

Tools for data collection are vital for researchers since working with identifiable tools would facilitate research activities and, in turn, facilitate the system design process. Across the 27 included studies, researchers utilized common data collection tools, such as interviews, FGD, surveys, structured questionnaires, interactive workshops and brainstorming, guideline principles reviews, scoping, and systematic reviews and meta-analysis. However, a set of uncommon tools were identified in this review, such as the persona and case scenario development, card sorting method, affinity wall method, Delphi technique, and logic modeling.

In summary, we identified the essential phases of the design model according to the system design process, as shown in Figure 3. The overall design model was guided by adopting a design approach, basically UCD. The design input identified self-management and behavioral change theories as primary constructs to inform system design. Almost all projects have secured funds and involved multidisciplinary teams. Identifying the users’ needs and characteristics was a cornerstone to inform the subsequent design activities and was performed by needs assessment studies, persona development, analysis of relevant apps, and literature and guideline reviews. In the ideation phase, conceptualization and “navigation and interface” were essential in 96%, and 78% of the studies, while visual design and mock-up creation were missed in 63% and 44.5% of the studies, respectively. The development phase was upside down, as the prototyping and coding were considered more frequently than the content creation, indicating a loss of the foundation work. The research teams considered iterative development in 78% of the developed systems.

Usability testing: phases and methods

Usability measures imply how system performance is measured, and generally, researchers measure the system's effectiveness, efficiency, and patient satisfaction. The usability testing model, as shown in Figure 4,^18,41,43,95 categorized usability into three phases: moderated testing, unmoderated testing (1), and unmoderated testing (2), each with specified context. These are outlined in Supplemental Table 2. The moderated testing suggested a controlled testing procedure, aiming at concept testing refinement within a controlled environment, mostly in laboratory settings. The unmoderated testing (1) aimed to integrate the system into the relevant environment and test the developed TR system's functionalities (effectiveness). The unmoderated testing (2) tested the TR system in a realistic environment to demonstrate the developed system's effectiveness in measuring clinical outcomes through routine use. The TRL would measure the maturity of the resulting TR systems. Whereas the moderated usability produced TRL4, the unmoderated usability (1) produced TRL6, and the unmoderated usability (2) produced TRL7.⁴¹

Figure 4.

The usability testing model based on the 27 TR systems. Maturity of the developed TR system is measured by TRL. TRL4 is equivalent to Concept Refinement, TRL6 is equivalent to Technology Development, and TRL7 is equivalent to System Development and Demonstration.⁴¹

The “moderated-unmoderated” usability testing model implies the consecutive testing of the technology, starting from laboratory settings until being tested in an unmoderated environment, assuming that the developed technology should be tested in a moderated setting, refined, tested in a relevant environment, and then further in a real environment.⁴³ Here, we identified 20 studies that have articulated the utilization of moderated usability testing; six of them reported dual moderated usability testing, however, in variable methodologies. We also mapped the use of unmoderated usability (1) in 17 studies, four of which have conducted dual unmoderated usability (1). Lastly, the unmoderated usability (2) was mapped in four studies, and two studies have conducted three consecutive usability testing stages.^85,89

It was unclear from the literature whether one, two, or three consecutive usability testing is satisfactory to produce a matured technology.^18,53 We identified the execution of three consecutive stages of usability testing in two studies, and of two consecutive stages of usability in 10 studies. In contrast, one stage of usability testing was identified in 15 studies. A single one-stage usability testing was conducted in nine studies.

The 27 studies have evaluated their TR systems in concrete 51 usability testing. Here, we mapped the execution of 26 moderated usability, 21 unmoderated usability (1), and four unmoderated usability (2). Usability methods are essential to effectively and efficiently assess usability. We mapped the utilization of the questionnaire in 51% usability testing studies; specifically, the structured usability questionnaire was used in 11 usability studies, and the Mobile App Rating Scale – user version and the System Usability Questionnaire each were used in five usability studies. We also identified the use of the Post-Study System Usability Questionnaire in three studies. We identified a single use of the Single Each Question Scale, the User Experience Questionnaire, the Unified Theory of Acceptance and Use of Technology, the modified Technology Acceptance Model scale, the app behavior changes scale, an app notification content review survey, and a satisfaction scale. In four usability studies, free space was added to collect qualitative data from the mentioned scales. Lastly, clinical outcome measures were used in four usability studies. We also mapped the use of other data collection tools, such as interviews in 16 studies, FGDs in four studies, workshop discussions in four studies, Talk\think-Aloud in nine studies, heuristic assessment in four studies, compliance calculation in two studies, feedback in 10 studies, usage scenario in three studies, self-monitoring symptoms in three studies, and guideline and technical document in seven studies. Each online observation and cognitive task analysis was used only once.

Effectiveness and efficiency are tested objectively by assessing how users achieve certain goals with the technology as they perform specific tasks. Meanwhile, satisfaction is a subjective measure that can be captured in attitude measures, implying what the user thinks about the technology and its components.⁹⁶ Both effectiveness and efficiency are measured indirectly by measuring system functionality, ease of use, problem or inconvenience use, content quality, usefulness, aesthetics, compliance, and performance measures.

Here, we mapped a clear description of measuring functionality in 36 tests, the ease of use in 22 tests, and problem or inconvenience use in 20 tests. We also charted the identification of content quality in 14 tests, usefulness in eight tests, and aesthetics in 13 tests. Satisfaction and acceptance were identified in 10 and six tests, respectively. Lastly, we separately mapped the compliance calculation and performance measures in six tests.

Characteristics and functionalities of the telerehabilitation systems

The eight core components of chronic disease rehabilitation,⁵ supposed to be delivered by the TR system, have been categorized into five components, as shown in Supplemental Table 1.3. Here, we mapped the clear articulation of evaluation in 24 studies, self-management support in 25 studies, and psycho-social support in seven studies. We also identified exercise supervision in 11 studies and follow-up in three studies.

HCPs play an integral role in providing rehabilitation services through the TR system; their role was adopted from a previous study,²⁷ with modification, and included: active intervention, monitoring, coordinated care, or no intervention, where the system provided auto-reminders only. In this review, we charted the clear articulation of HCP's role as active intervention, monitoring, and coordinated care in 26, 15, and nine studies, respectively. The TR system provided auto-reminder in 12 studies.

The TR system consisted of three components: the patient, the therapist, and the communication hub. These components were transformed technically into software and hardware; the latter might include one or more sensors or a connected monitoring device. The software components might include a mobile app, a web-based service, and a cloud. All TR system of the mapped studies consisted of a mobile app since this was one of the identified inclusion criteria. We identified a clear description of the web-based service in nine studies, the hardware in 10 studies, and the cloud in five studies. Therefore, in eight studies, the TR system included three components: mobile app, hardware, and web-based service or a cloud.

The maturity of the outputs was evaluated in the context of the (IPO) framework and the TRL. Here, we identified the production of fully functional and high-fidelity prototypes in 22 and four studies, respectively. Only one study has released the TR system. The system's scalability indicated planning for future technology integration, which was articulated in two studies only.^71,97

In summary, the level of adoption of the design, development, and testing of the 27 TR systems are graphically represented in Figure 5. The TR systems’ functionalities, maturity levels, and role of HCPs are also described.

Figure 5.

The level of adoption of the design and testing processes of the 27 TR systems according to the IPO framework. Each cluster of bars represents one design and usability testing phase. S-M indicates self-management; CC indicates coordinated care.

The “Input-Process-Output” framework

In this review, we identified the results of the 27 studies related to system design and usability testing. Based on these results, we identified the “input-process-output” framework to guide the TR system design, and usability testing as shown in Figure 6. This framework will enable physiotherapists and HCPs to participate effectively in the design and evaluation research and to enhance the provided rehabilitation services.

Figure 6.

The “input-process-output” framework (IPO) for TR system design and usability testing.

The identified framework illustrates the two main phases of a TR system: design and usability testing. System design named the inputs and identified their subsequent processes: ideation, development, and improvement. The IPO framework linked the mock-up, prototype, and matured system as products of ideation, development, and improvement. It also linked the three stages of usability testing to the design phases. Iteration was a cornerstone of this framework, as researchers and developers could repeatedly modify the design and testing procedures to enhance the system's features. Additionally, crucial recommendations to adopt the IPO framework were included, and the resulting outputs were classified based on their level of maturity and named by the TRLs.

Discussion

The IPO framework highlights system design and usability testing, focusing on its components, flow, and recommendations. The IPO is expected to enable physiotherapists and HCPs to improve rehabilitation services through their influential roles in system design and testing.

Chronic diseases

According to the eligibility criteria of this review, the four clusters of chronic diseases,² along with their rehabilitation programs,^3–5 were included, regardless of their specific characteristics. Therefore, the identified IPO framework illustrates the system design and usability testing for chronic diseases with no particular considerations to differences among them. While this review specifies the TR systems for chronic diseases, in general, the focus on specific differences among chronic diseases is out of the scope of this review. It may inform future work, as different chronic diseases may require a more specific and detailed framework.

In this review, we charted a comparable level of focus on the four chronic disease clusters, with the highest emphasis on CVD, followed by diabetes and cancer, and finally, COPD. To a certain extent, this distribution is commensurate with the worldwide consequences of noncommunicable diseases. The WHO stated that CVDs account for the most deaths, followed by chronic respiratory diseases, cancer, and finally, diabetes.⁹⁸ These findings would identify the direction of future research and policies, prioritizing the CVDs, chronic respiratory diseases, cancer, and diabetes, respectively.

System design

A novel finding of this review is the participants’ role and research timeline. While previous studies described primary participants’ demographics, we classified the users by their roles. All charted studies included patients as study participants. Notably, more than 50% of the studies have included HCPs and designers as participants. Researchers should consider this to adapt the participants’ tasks and data collection tools accordingly. We also mapped an imprecise reporting of research activities’ timeline in nearly 55% of the studies while the remaining studies have skipped such part. It was essential to set a research timeline in a priori, specifically for complex and multistage research activities.⁹⁹ It would be recommended to develop a time plan for system design, development, and testing research activities.

This review highlighted a high level of collaboration across national and international institutions. Systems have been developed globally, with a high concentration in western countries, specifically USA, Europe, Canada, and UK. Considerable projects from south Korea and China were identified as well. The presence of the developing countries is negligible, except for collaboration within international projects, mainly with the USA. It seems compatible with the reported fact that the USA and Europe are the global leaders in medical technology innovation.⁴⁶ However, the other side of this fact is that business and profits are the main drivers of this industry. Inventors play a crucial role in gathering ideas, translating them into medical products and services, and incorporating them into patient care. On the other hand, medical device sales in developing countries are overgrowing.⁴⁶ It would be essential to highlight the significance of localizing technology and enhancing international collaboration in developing medical technologies rather than relying on importing them.^72,74

We have also mapped the UCD, agile iteration, and the users’ needs and characteristics as the most frequently used methods in conducting system design, all of which has been emphasized in the literature.^20,22 Furthermore, a previous review developed a more specific guideline for applying UCD to identify and prioritize the users’ needs, which is essential to inform mHealth development.¹⁰⁰ This guideline identified qualitative methodologies, using focus and discussion groups, organized over four sessions of variable durations,¹⁰⁰ and ought to guide the design and testing of the intended mHealth.

We have charted the self-management and behavioral change theories as an essential design input to guide the development of features, modules, and functionalities.²⁶ Theories were deployed either solely or in combination in 83% of the studies, which promoted self-management toward modifying behavioral risk factors.² However, the unstandardized, random practice of the deployment process^26,28 made the effectiveness of theory deployment questionable. It would therefore be recommended to utilize standardized models to guide theory deployment and technology development, such as the intervention mapping, behavioral change model, logic model, and control-oriented model. These models are recommended by healthcare²⁶ and engineering¹⁰¹ disciplines.

In this review, we identified significant evidence that securing funds and working in MDTs are essential and clearly understood among research teams. However, the reported challenges of working with professionals from different mindsets necessitate building up a common language among them, launching frequent discussions, and exchange of practical knowledge and technical skills.²⁹ Other recommended practices to facilitate working in multidisciplinary team included close collaboration, sharing data, and using standard methodologies, such as case scenarios discussion.²⁹ Besides that, setting up an effective plan for researchers’ early engagement greatly influences enhancement of collaboration.³³

Undoubtedly, visual design is the primary responsibility of the IT members of the team. However, the reported significant loss in the visual design and the mock-up creation indicated either a skill gap or a collaboration gap among the MDT. Similarly, we charted an order gap in the development phase; the foundation works have been conducted less frequently than the output work prototyping, suggesting a knowledge and skill gap in this field. It would be essential to bridge these gaps through capacity building and enhanced cooperation programs, enhancing user engagement and use adherence.¹⁰⁰ It is recommended to identify visuals, interface contents, and flow before design.¹⁰⁰

Another novel finding of this review is that we mapped the administration of MMR across all included studies. Conducting an MMR with a specific focus on the integration approach¹⁰² could result in a whole greater than the sum of the individual parts (1 + 1 = 3).¹⁰² It would also help produce more publication opportunities and facilitate working in teams. Finally, there are numerous opportunities for integrating qualitative and quantitative methods in MMR,¹⁰³ facilitating to conduct the methodology, particularly for developing mHealth and medical technologies. The iterative convergent mixed-methods design¹⁰⁴ provides a guideline for conducting MMR along with an iterative design that eventually produces an mHealth technology that is satisfactory for users and researchers.¹⁰⁴ In parallel, we mapped the administration of common data collection tools. However, a set of uncommon tools has also been mapped and required for consideration.

Usability testing

In this review, we mapped the usability testing as the “moderated-unmoderated” process. The identified results triggered three essential questions that might inform future work: (1) should we start usability testing studies from the moderated laboratory settings and proceed to the wild testing, or can we invest more resources in the design to diminish the need for several usability testing? (2) Should we conduct the three consecutive stages of usability testing? (3) What are the benefits of conducting dual moderated or dual unmoderated usability. Here, we argue that the charted data of this review and the available literature have not yet provided a satisfactory answer or evidence for these questions despite the provided data from a previous study that have spotlighted a similar issue.⁵¹ We also wonder whether a single moderated or single unmoderated usability has been reported as insufficient for the usability goals. In short, we argue that usability testing studies are conducted randomly. At the same time, it would be feasible to launch a standard of conduct to specify goals, users, context, and metrics.

What is unique about the IPO framework

Here, we have mapped the “input-process-output” framework (IPO), which encompasses four phases of the design process; inputs, conceptualization, development, and improvement, and three consecutive phases of usability testing, while other frameworks provide different perspectives. Therefore, it would be essential to illustrate our (IPO) framework in comparison with the other frameworks, the biodesign process,⁴⁶ “input-mechanism-output”¹⁰⁵ (IMO) model, and the “User-Centered Health App Design,”¹⁰⁰ to highlight its significance. Here, our IPO framework focused on the scientific perspective to design, and tested medical technologies. At the same time, the biodesign process⁴⁶ provided a more comprehensive framework that would benefit educators, researchers, entrepreneurs, medical practitioners, practical engineers, policymakers, inventors, companies, and economists. It was adopted in developing health technologies for CVD¹⁰⁶ and orthopaedic surgeries.¹⁰⁷ It was influential in transforming patients’ needs into medical technologies that had the potential to fulfil patients’ needs and advance the field. On the other hand, our IPO framework provided an overview of the system inputs, design, and testing until producing the output, with specific consideration of the theory as a focal design requirement. The IMO model highlighted the theory as either an input requirement or a mechanism of system development to guide the overall proceeding activities. Yet, this model has missed essential design requirements, main process, and evaluation in its illustration. Lastly, our framework provided an overall view for researchers, keeping the door open to apply all research methodologies and tools while the “User-Centered Health App Design” applied only to qualitative research, using the focus and discussion groups.

Future research

Since scoping reviews often inform future systematic reviews and empirical studies, this review mapped the design and usability testing of the TR system for chronic diseases. It highlighted the basics of system design: UCD and iteration, self-management and behavioral change theories, and the users’ need identifications via surveys, persona development, guideline reviews, analysis of relevant apps, and expert panels. We also mapped the execution of MMR with variable flows. Therefore, it would be advisable to evaluate and compare the effectiveness of these basics in facilitating system design and producing high-quality and effective interventions (TR system). Such studies would prioritize the best evidence of TR system design, and evaluation toward high-quality, efficient, and effective outputs.

Study strengths

To the best of our knowledge, this is the first scoping review that systematically maps TR system's design and usability testing through the “input-process-output” journey. It is the first to cover chronic diseases as well. In this review, we mapped study locations, types of chronic diseases, and characteristics of research teams and study participants. Moreover, we mapped the four phases of system design, the three consecutive phases of usability testing, and their stages. Lastly, we identified characteristics and functionalities of the developed TR systems that served chronic disease rehabilitation.

Study limitations

This review covered all stages of system design and usability of the four clusters of chronic diseases. Such a wide range of searching triggered the team to focus on the database search over the last 5 years, thus capturing the most relevant and recent publications. While we suspected that our methodology might lead to missing some publications, the literature showed that the publication of usability studies peaked around the selected study period.^43,51 Nevertheless, those publications focused only on usability testing while this review covered system design and usability.

Conclusion

Over the last 5 years, the developed TR systems were charted according to the “input-process-output” framework. The system design was conducted in four stages: inputs, ideation, development, and improvement. Usability testing of the TR system was conducted in three consecutive phases: moderated usability, unmoderated usability (1), and unmoderated usability (2), however, in a random manner. We also identified functionalities of the developed TR system to provide evaluation, self-management, and therapy supervision, where HCPs actively participated in delivering telerehabilitation, therapy supervision, and monitoring progress. The identified functionalities of the TR systems and the roles of HCPs would address the recurring, long duration, and slow progression of chronic diseases and would manipulate their behavioral risk factors. This review is expected to facilitate TR system design and usability testing for chronic diseases, highlighting their main components and flow and providing recommendations for application. In addition, this review is expected to trigger HCPs to work in multidisciplinary teams and transform their patients’ needs into functionalities of the TR systems, thus, enhancing rehabilitation services.

Supplemental Material

sj-docx-1-dhj-10.1177_20552076231191014 - Supplemental material for A framework for design and usability testing of telerehabilitation system for adults with chronic diseases: A panoramic scoping review

Supplemental material, sj-docx-1-dhj-10.1177_20552076231191014 for A framework for design and usability testing of telerehabilitation system for adults with chronic diseases: A panoramic scoping review by Suad J Ghaben, Arimi Fitri Mat Ludin, Nazlena Mohamad Ali, Kok Beng Gan and Devinder Kaur Ajit Singh in DIGITAL HEALTH

Footnotes

Availability of data and materials

All data generated or analyzed during this study are included in this published article.

Contributorship

SG and AFML conceived and planned the review. SG and AFML screened the data. SG extracted the data. SG and AFML contributed to the development of search strategy, and SG conducted the searches. DKAS, NMA, and KBG undertook initial title review, and titles/abstracts. SG drafted the manuscript, and full-text review was undertaken by AFML, DKAS, NMA, and KBG.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Suad J Ghaben

Arimi Fitri Mat Ludin

Devinder Kaur Ajit Singh

Supplemental material

Supplemental material for this article is available online.

References

Bernell

Howard

. Use your words carefully: what is a chronic disease? Front Public Health 2016; 4: 159.

(WHO) WHO. Noncommunicable diseases, https://www.who.int/health-topics/noncommunicable-diseases#tab=tab_1 (2022).

Mulligan

Wilkinson

Chen

, et al. Components of community rehabilitation programme for adults with chronic conditions: a systematic review. Int J Nurs Stud 2019; 97: 114–129.

Richardson

Franklin

Moy

, et al. Advances in rehabilitation for chronic diseases: improving health outcomes and function. Br Med J 2019; 365: 1–15.

Agency for Clinical Innovation. Rehabilitation for chronic conditions framework. 2017. New South Wales, Australian State.

Lahham

Holland

. The need for expanding pulmonary rehabilitation services. Life 2021; 11: 1236.

Isernia

Pagliari

Bianchi

LNC

, et al. Characteristics, components, and efficacy of telerehabilitation approaches for people with chronic obstructive pulmonary disease: a systematic review and meta-analysis. Int J Environ Res Public Health 2022; 19: 15165.

(WHO) WHO. Rehabilitation Competency Framework. 2020. Geneva.

Heine

Derman

Hanekom

. Towards a framework for the scale-up of rehabilitation for patients with non-communicable disease in low-resource settings. medRxiv 2022.

10.

Cox

Dal Corso

Hansen

, et al. Telerehabilitation for chronic respiratory disease. Cochrane Database Syst Rev 2021; 1: 1–10.

11.

Chapel

Alexandre

Heraud

, et al. Standardization of the assessment process within telerehabilitation in chronic diseases: a scoping meta-review. BMC Health Serv Res 2022; 22: 1–15.

12.

Vernooij

Willson

Gagliardi

. Characterizing patient-oriented tools that could be packaged with guidelines to promote self-management and guideline adoption: a meta-review. Implement Sci 2015; 11: 1–13.

13.

Borries

Dunbar

Bhukhen

, et al. The impact of telemedicine on patient self-management processes and clinical outcomes for patients with types I or II diabetes mellitus in the United States: a scoping review. Diabetes & Metabolic Syndrome: Clinical Research & Reviews 2019; 13: 1353–1357.

14.

Qudah

Luetsch

. The influence of mobile health applications on patient-healthcare provider relationships: a systematic, narrative review. Patient Educ Couns 2019; 102: 1080–1089.

15.

Brennan

Mawson

Brownsell

Telerehabilitation: enabling the remote delivery of healthcare, rehabilitation, and self management. In: Riva, G, Gaggioli, A, Keshner, EA, Weiss, PL. (eds.) Advanced technologies in rehabilitation. Amsterdam: Amsterdam: IOS Press, 2009, pp.231–248.

16.

Vukićević

Stamenković

Murugesan

, et al. A new telerehabilitation system based on internet of things. Facta Universitatis Series: Electronics and Energetics 2015; 29: 395–405.

17.

Anton

Berges

Bermúdez

, et al. A telerehabilitation system for the selection, evaluation and remote management of therapies. Sensors 2018; 18: 1459.

18.

Cornet

Toscos

Bolchini

, et al. Untold stories in user-centered design of mobile health: practical challenges and strategies learned from the design and evaluation of an app for older adults with heart failure. JMIR Mhealth Uhealth 2020; 8: e17703.

19.

Williams

User-centered design, activity-centered design, and goal-directed design: a review of three methods for designing web applications. In: Proceedings of the 27th ACM international conference on design of communication, 2009, pp.1–8.

20.

Dopp

Parisi

Munson

, et al. A glossary of user-centered design strategies for implementation experts. Transl Behav Med 2019; 9: 1057–1064.

21.

Wallisch

Sankowski

Krause

, et al. Overcoming fuzzy design practice: revealing potentials of user-centered design research and methodological concepts related to user involvement. In: 2019 IEEE international conference on engineering, technology and innovation (ICE/ITMC), 2019, pp.1–9. IEEE.

22.

Farao

Malila

Conrad

, et al. A user-centred design framework for mHealth. PloS one 2020; 15: e0237910.

23.

Jacobs

Graham

. Iterative development and evaluation methods of mHealth behavior change interventions. Curr Opin Psychol 2016; 9: 33–37.

24.

Gharajeh

. Waterative model: an integration of the waterfall and iterative software development paradigms. Database Syst J 2019; 10: 75–81.

25.

Balaji

Murugaiyan

. Waterfall vs. V-Model vs. Agile: a comparative study on SDLC. International Journal of Information Technology and Business Management 2012; 2: 26–30.

26.

Rabbi

Philyaw-Kotov

, et al. Translating behavioral theory into technological interventions: case study of an mHealth app to increase self-reporting of substance-use related data. arXiv preprint arXiv:200313545 2020.

27.

Clarkson

Stephenson

Grimmett

, et al. Digital tools to support the maintenance of physical activity in people with long-term conditions: a scoping review. Digital Health 2022; 8: 20552076221089778.

28.

Chib

Lin

. Theoretical advancements in mHealth: a systematic review of mobile apps. J Health Commun 2018; 23: 909–955.

29.

Shah

Yang

Demo

, et al. Technical guidance for clinicians interested in partnering with engineers in mobile health development and evaluation. JMIR Mhealth Uhealth 2019; 7: e14124.

30.

Aji

Gordon

Stratton

, et al. Framework for the design engineering and clinical implementation and evaluation of mHealth apps for sleep disturbance: systematic review. J Med Internet Res 2021; 23: e24607.

31.

Woods

Cummings

Duff

, et al. Partnering in digital health design: engaging the multidisciplinary team in a needs analysis. Stud Health Technol Inform 2018; 252: 176–181.

32.

Molina Recio

Garcia-Hernandez

Molina Luque

, et al. The role of interdisciplinary research team in the impact of health apps in health and computer science publications: a systematic review. Biomed Eng Online 2016; 15: 53–97.

33.

Williams

Hayes

Guo

, et al. HCI And mHealth wearable tech: a multidisciplinary research challenge. In: Extended abstracts of the 2020 CHI conference on human factors in computing systems, 2020, pp.1–7.

34.

Standardization. IOf. Ergonomics of human-system interaction-Part 11: Usability: Definitions and concepts, https://www.iso.org/obp/ui/#iso:std:iso:9241:-11:ed-2:v1:en (2018, 2023).

35.

Yen

P-Y

Bakken

. Review of health information technology usability study methodologies. J Am Med Inform Assoc 2012; 19: 413–422.

36.

Calvaresi

Schumacher

Marinoni

, et al. Agent-based systems for telerehabilitation: strengths, limitations and future challenges. In: Montagna S, Abreu PH, Giroux S, Schumacherz MI. (eds.) Agents and multi-agent systems for health care. Switzerland: Springer, 2017, pp.3–24.

37.

Simonetti

Zollo

Vollero

, et al. A modular telerehabilitation architecture for upper limb robotic therapy. Advances in Mechanical Engineering 2017; 9: 1687814016687252.

38.

Pereira

Almeida

Martins

, et al. Evaluation of complex distributed multimodal applications: evaluating a telerehabilitation system when it really matters. In: International conference on human aspects of IT for the aged population, 2015, pp.146–157. Springer.

39.

Antón

Goñi

Illarramendi

, et al. Kires: a kinect-based telerehabilitation system. In: 2013 IEEE 15th international conference on e-health networking, applications and services (healthcom 2013), 2013, pp.444–448. IEEE.

40.

Eysenbach

. What is e-health? J Med Internet Res 2001; 3: e833.

41.

Olechowski

Eppinger

Joglekar

Technology readiness levels at 40: a study of state-of-the-art use, challenges, and opportunities. In: 2015 Portland international conference on management of engineering and technology (PICMET), 2015, pp.2084–2094. IEEE.

42.

Mankins

JC.

Technology readiness levels: A white paper, http://www hq nasa gov/office/codeq/trl/trl pdf (1995).

43.

Wang

Zhou

Liu

, et al. Usability evaluation of mHealth Apps for the elderly: A scoping review. 2022.

44.

Cheng

Mustafa

A reference to usability inspection methods. In: International colloquium of art and design education research (i-CADER 2014), 2015, pp.407–419. Springer.

45.

Medina

Kremer

GEO

Wysk

. Supporting medical device development: a standard product design process model. J Eng Des 2013; 24: 83–119.

46.

Denend

Biodesign. UK: Cambridge University Press, 2015.

47.

Castensoe-Seidenfaden

Husted

Teilmann

, et al. Designing a self-management app for young people with type 1 diabetes: methodological challenges, experiences, and recommendations. JMIR Mhealth Uhealth 2017; 5. DOI: 10.2196/mhealth.8137.

48.

Kip

Keizer

da Silva

, et al. Methods for human-centered eHealth development: narrative scoping review. J Med Internet Res 2022; 24: e31858.

49.

Bradway

Gabarron

Johansen

, et al. Methods and measures used to evaluate patient-operated mobile health interventions: scoping literature review. JMIR Mhealth Uhealth 2020; 8: e16814.

50.

Slater

Campbell

Stinson

, et al. End user and implementer experiences of mHealth technologies for noncommunicable chronic disease management in young adults: systematic review. J Med Internet Res 2017; 19: e8888.

51.

Maramba

Chatterjee

Newman

. Methods of usability testing in the development of eHealth applications: a scoping review. Int J Med Inf 2019; 126: 95–104.

52.

Cruz-Martínez

Wentzel

Asbjørnsen

, et al. Supporting self-management of cardiovascular diseases through remote monitoring technologies: metaethnography review of frameworks, models, and theories used in research and development. J Med Internet Res 2020; 22: e16157.

53.

Shah

UEM

Chiew

. A systematic literature review of the design approach and usability evaluation of the pain management mobile applications. Symmetry (Basel) 2019; 11: 400.

54.

Aromataris

Bjerrum

. JBI Manual for Evidence Synthesis. JBI 2020. DOI: 10.46658/JBIMES-20-01.

55.

Arksey

O’Malley

Scoping studies: towards a methodological framework. Int J Soc Res Methodol 2005; 8: 19–32.

56.

Levac

Colquhoun

O’Brien

. Scoping studies: advancing the methodology. Implement Sci 2010; 1: 1–9.

57.

Daudt

van Mossel

Scott

. Enhancing the scoping study methodology: a large, inter-professional team’s experience with Arksey and O’Malley’s framework. BMC Med Res Methodol 2013; 13: 1–9.

58.

Tricco

Lillie

Zarin

, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med 2018; 169: 467–473. 2018/09/05.

59.

Tsutsui

Gerayeli

Sin

. Pulmonary rehabilitation in a post-COVID-19 world: telerehabilitation as a new standard in patients with COPD. Int J Chron Obstruct Pulmon Dis 2021; 16: 379.

60.

Harst

Wollschlaeger

Birnstein

, et al. Evaluation is key: providing appropriate evaluation measures for participatory and user-centred design processes of healthcare IT. Int J Integr Care 2021: 21: 1–18.

61.

Johnson

Johnston

Crowley

, et al. EHR usability toolkit: a background report on usability and electronic health records. Rockville, MD: Agency for Healthcare Research and Quality, 2011. 2018.

62.

Michie

West

Campbell

, et al. ABC of behaviour change theories, an essential resource for researchers, policy makers and practitioners. Sutton, UK: Silverback, 2014.

63.

Bevan

Carter

Earthy

, et al. New ISO standards for usability, usability reports and usability measures. In: International conference on human-computer interaction, 2016, pp.268–278. Springer.

64.

Gorini

Mazzocco

Triberti

, et al. A P5 approach to m-health: design suggestions for advanced mobile health technology. Front Psychol 2018; 9: 2066.

65.

Fico

Cancela

Arredondo

, et al. User requirements for incorporating diabetes modeling techniques in disease management tools. In: 6th European conference of the international federation for medical and biological engineering, 2015, pp.992–995. Springer.

66.

Chalela

Munoz

Inupakutika

, et al. Improving adherence to endocrine hormonal therapy among breast cancer patients: study protocol for a randomized controlled trial. Contemporary Clinical Trials Communications 2018; 12: 109–115.

67.

Adu

Malabu

Malau-Aduli

, et al. Users’ preferences and design recommendations to promote engagements with mobile apps for diabetes self-management: multi-national perspectives. PLoS One 2018; 13: e0208942.

68.

LeBaron

Adhikari

Bennett

, et al. A survey of cancer care institutions in Nepal to inform design of a pain management mobile application. BMC Palliat Care 2021; 20: 1–21.

69.

Adu

Malabu

Malau-Aduli

, et al. The development of my care hub mobile-phone app to support self-management in Australians with type 1 or type 2 diabetes. Sci Rep 2020; 10: 1–10.

70.

Fico

Martinez-Millana

Leuteritz

J-P

, et al. User Centered Design to Improve Information Exchange in Diabetes Care Through eHealth: results from a Small Scale Exploratory Study. 2020; 44. DOI: 10.1007/s10916-019-1472-5.

71.

Inupakutika

Kaghyan

Akopian

, et al. Facilitating the development of cross-platform mHealth applications for chronic supportive care and a case study. J Biomed Inform 2020; 105: N.PAG-N.PAG.

72.

LeBaron

Horton

Adhikari

, et al. A global collaboration to develop and pilot test a mobile application to improve cancer pain management in nepal. Frontiers in Pain Research 2022; 3. DOI: 10.3389/fpain.2022.910995.

73.

Monteiro-Guerra

Signorelli

Tadas

, et al. A Personalized Physical Activity Coaching App for Breast Cancer Survivors: design Process and Early Prototype Testing. JMIR Mhealth Uhealth 2020; 8. DOI: 10.2196/17552.

74.

Morse

Lambden

Quinn

, et al. A Mobile App to Improve Symptom Control and Information Exchange Among Specialists and Local Health Workers Treating Tanzanian Cancer Patients: Human-Centered Design Approach. JMIR CANCER 2021; 7. DOI: 10.2196/24062.

75.

Gong

Wang

, et al. A smart and multifaceted Mobile health system for delivering evidence-based secondary prevention of stroke in rural China: design, development, and feasibility study. JMIR Mhealth Uhealth 2019; 7: e13503.

76.

Welch

Solk

Auster-Gussman

, et al. User-centered development of a smartphone application (Fit2Thrive) to promote physical activity in breast cancer survivors. Transl Behav Med 2022; 12: 203–213.

77.

Faiola

Vatani

Agarwal

. Hypoglycemic Detection by Human Breath: A Mobile Health App that Alerts Diabetics of Low Blood Glucose. EAI Endorsed Transactions on Ambient Systems 2019; 6. DOI: 10.4108/eai.23-3-2018.162220.

78.

Nicklas

Leiferman

Lockhart

, et al. Development and modification of a mobile health program to promote postpartum weight loss in women at elevated risk for cardiometabolic disease: single-arm pilot study. JMIR formative Research 2020; 4: e16151.

79.

O’Malley

Davis

Devine

, et al. Development and usability testing of the e-EXCELS tool to guide cancer survivorship follow-up care. Psychooncology 2020; 29: 123–131.

80.

Yee

Leziak

Jackson

, et al. Patient and provider perspectives on a novel mobile health intervention for low-income pregnant women with gestational or type 2 diabetes mellitus. J Diabetes Sci Technol 2021; 15: 1121–1133.

81.

Baek

Kang

Hwang

, et al. Enhancing user experience through user study: design of an mhealth tool for self-management and care engagement of cardiovascular disease patients. J Med Internet Res 2018; 20. DOI: 10.2196/cardio.9000.

82.

Jeon

Park

. Development of the IMB model and an evidence-based diabetes self-management mobile application. Healthc Inform Res 2018; 24: 125–138.

83.

Kim

Choi

. Mobile health platform based on user-centered design to promote exercise for patients with peripheral artery disease. BMC Medical Informatics & Decision Making 2022; 22: 1–12.

84.

Kwon

Lee

Jung

, et al. An mhealth management platform for patients with chronic obstructive pulmonary disease (efil breath): randomized controlled trial. 2018; 6: e10502. DOI: 10.2196/10502.

85.

Deng

Sheng

Jiang

, et al. A home-based pulmonary rehabilitation mHealth system to enhance the exercise capacity of patients with COPD: development and evaluation. BMC Med Inform Decis Mak 2021; 21. DOI: 10.1186/s12911-021-01694-5.

86.

Duan

Wang

, et al. Using goal-directed design to create a mobile health app to improve patient compliance with hypertension self-management: development and deployment. JMIR Mhealth Uhealth 2020; 8. DOI: 10.2196/14466.

87.

Petersen

Hempler

. Development and testing of a mobile application to support diabetes self-management for people with newly diagnosed type 2 diabetes: a design thinking case study. BMC Med Inform Decis Mak 2017; 17. DOI: 10.1186/s12911-017-0493-6.

88.

Davies

Mueller

Hennings

, et al. Recommendations for developing support tools with people suffering from chronic obstructive pulmonary disease: co-design and pilot testing of a mobile health prototype. JMIR HUMAN FACTORS 2020; 7. DOI: 10.2196/16289.

89.

Velardo

Shah

Gibson

, et al. Digital health system for personalised COPD long-term management. 2017; 17: 19. DOI: 10.1186/s12911-017-0414-8.

90.

Agher

Sedki

Despres

, et al. Encouraging behavior changes and preventing cardiovascular diseases using the prevent connect Mobile health app: conception and evaluation of app quality. J Med Internet Res 2022; 24: N.PAG-N.PAG.

91.

Korpershoek

YJG

Hermsen

Schoonhoven

, et al. User-centered design of a mobile health intervention to enhance exacerbation-related self-management in patients with chronic obstructive pulmonary disease (Copilot): mixed methods study. J Med Internet Res 2020; 22. DOI: 10.2196/15449.

92.

Duff

Walsh

Malone

, et al. MedFit App, a Behavior-Changing, Theoretically Informed Mobile App for Patient Self-Management of Cardiovascular Disease: User-Centered Development . 2018, pp.1–1.

93.

Hou

I-C

Lan

M-F

Shen

S-H

, et al. The development of a mobile health app for breast cancer self-management support in Taiwan: design thinking approach. JMIR Mhealth Uhealth 2020; 8: e15780.

94.

Tundjungsari

Yugaswara

Putra

ATD

, et al. Development of mobile health application for cardiovascular disease prevention. International Journal of Advanced Computer Science and Applications 2018; 9: 543–550.

95.

Mankins

. Technology readiness assessments: a retrospective. Acta Astronaut 2009; 65: 1216–1223.

96.

Klaassen

van Beijnum

Hermens

. Usability in telemedicine systems—A literature survey. Int J Med Inf 2016; 93: 57–69.

97.

Nicklas

Leiferman

Bull

, et al. Beta-testing a mobile health program designed to increase postpartum weight loss in women at elevated risk for cardiometabolic disease. 2016; 22: A140-. DOI: 10.1089/acm.2016.29003.abstracts.

98.

(WHO) WHO. Fact sheets-Noncommunicable diseases, https://www.who.int/news-room/fact-sheets/detail/noncommunicable-diseases (2023).

99.

Brown

Framing the research. In: Opie C, Brown D. (eds.) Getting started in your educational research: design, data production and analysis. London: Sage, 2019, pp.39–54.

100.

Molina-Recio

Molina-Luque

Jiménez-García

, et al. Proposal for the user-centered design approach for health apps based on successful experiences: integrative review. JMIR Mhealth Uhealth 2020; 8: e14376.

101.

Martin

Rivera

Hekler

, et al. Development of a control-oriented model of social cognitive theory for optimized mHealth behavioral interventions. IEEE Trans Control Syst Technol 2018; 28: 331–346.

102.

Fetters

Freshwater

The 1+1=3 integration challenge. Los Angeles, CA: Sage Publications Sage CA, 2015, pp.115–117.

103.

Fetters

. Six equations to help conceptualize the field of mixed methods. Los Angeles, CA: Sage Publications Sage CA, 2018, pp.262–267.

104.

Alwashmi

Hawboldt

Davis

, et al. The iterative convergent design for mobile health usability testing: mixed methods approach. JMIR Mhealth Uhealth 2019; 7: e11656.

105.

Chib

van Velthoven

Car

. Mhealth adoption in low-resource environments: a review of the use of mobile healthcare in developing countries. J Health Commun 2015; 20: 4–34.

106.

Schwartz

Kumar

Azagury

, et al. Needs-based innovation in cardiovascular medicine: the Stanford biodesign process. JACC: Basic to Translational Science 2016; 1: 541–547.

107.

Dharmawan

HHM

, et al. Implementing the biodesign process for medical device innovation in head and neck surgery. Surg Innov 2020; 27: 653–658.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.09 MB