Sage Journals: Discover world-class research

Abstract

Background

Excessive mucus secretion is a serious issue for patients with chronic obstructive airway disease (COAD), which can be effectively managed through postural drainage and percussion (PD + P) during pulmonary rehabilitation (PR). Home-based (H)-PR can be as effective as center-based PR but lacks professional supervision and timely feedback, leading to low motivation and adherence. Telehealth home-based pulmonary (TH-PR) has emerged to assist H-PR, but video conferencing and telephone calls remain the main approaches for COAD patients. Therefore, research on effectively assisting patients in performing PD + P during TH-PR is limited.

Objective

This study developed a mobile-based airway clearance care for chronic obstructive airway disease (COAD-MoAcCare) system to support personalized TH-PR for COAD patients and evaluated its usability through expert validation.

Methods

The COAD-MoAcCare system uses a mobile device through deep learning-based vision technology to monitor, guide, and evaluate COAD patients’ PD + P operations in real time during TH-PR programs. Medical personnel can manage and monitor their personalized PD + P and operational statuses through the system to improve TH-PR performance. Respiratory therapists from different hospitals evaluated the system usability using system questionnaires based on the technology acceptance model, system usability scale (SUS), and task load index (NASA-TLX).

Results

Eleven participant therapists were highly satisfied with the COAD-MoAcCare system, rating it between 4.1 and 4.6 out of 5.0 on all scales. The system demonstrated good usability (SUS score of 74.1 out of 100) and a lower task load (NASA-TLX score of 30.0 out of 100). The overall accuracy of PD + P operations reached a high level of 97.5% by comparing evaluation results of the system by experts.

Conclusions

The COAD-MoAcCare system is the first mobile-based method to assist COAD patients in conducting PD + P in TH-PR. It was proven to be usable by respiratory therapists, so it is expected to benefit medical personnel and COAD patients. It will be further evaluated through clinical trials.

Keywords

Airway clearance care chronic obstructive airway disease home-based pulmonary rehabilitation postural drainage mobile-based system deep learning-based vision usability testing

Introduction

Mucus secretions are crucial in the respiratory tract's defense and innate immunity.¹ Factors like aging, smoking, and air pollution can lead to excessive mucus production, which can obstruct the respiratory tract lumen. Chronic obstructive airway disease (COAD), including chronic obstructive pulmonary disease (COPD), asthma, and bronchiectasis, is characterized by excessive mucus secretions, which cause difficulties with expectoration and adversely impact patients’ quality of life.² Airway clearance therapy aims to minimize the devastating effects of airway obstruction, infection, and inflammation due to mucus stasis in the conducting airways and lung parenchyma.³

Medical devices can assist patients with respiratory issues in hospitals. However, after discharge, patients must learn physical therapy techniques like postural drainage (PD), chest percussion, and effective coughing for clearing sputum, particularly to be performed at home. PD exploits the force of gravity to facilitate the sliding of the mucus from the periphery toward the central airways.^4–6 Percussion is applied to specific segments to be treated while the patient breathes at tidal volume, and it consists of a rhythmic succession of rapid and light strokes performed with cupped hands on the patient's chest wall.^6,7 Some studies showed that a combination of PD and percussion, hereafter called PD + P, can improve sputum production.^8,9

Pulmonary rehabilitation (PR) is helpful for COAD patients, and PR programs can be both hospital-based and home-based. Hospital-based programs are time-consuming and costly. One clinical study showed that only 42% of patients with COPD completed PR training¹⁰ due to limitations such as the cost of medical care, the availability of trained professionals, transportation, mobility of the population, distance, and training location.^11,12

Home-based PR is a viable alternative to hospital-based PR, particularly for COPD patients who have difficulty accessing hospital-based rehabilitation.^13–16 The coronavirus disease 2019 (COVID-19) pandemic further emphasized the convenience of home-based options.¹⁵ Standard home-based PR training lasts 8–12 weeks and requires long-term maintenance training to ensure a satisfactory PR effect.¹⁷ However, patient motivation and adherence tend to be low, leading to increased time and costs of rehabilitation. The lack of real-time professional supervision and timely feedback in the home environment negatively affects patient motivation and adherence.^18–20 Therefore, determining how to use remote patient monitoring to provide the necessary guidance for rehabilitation to improve PR outcomes at home has become an important issue.

Research statistics showed that most telehealth home-based PR systems, hereafter called TH-PR, for COAD patients, currently use video conferencing and telephones, along with home-based self-management exercise programs.¹⁵ To improve outcomes and because of their convenience, mobile devices (smartphones and tablets) are increasingly being used to assist COAD patients during TH-PR.^21,22 Several existing studies developed and used mobile applications (apps) to provide educational and self-management assistance for COAD patients during TH-PR.^12,23–26 However, monitoring and feedback for physical exercise have so far not been provided.

In recent years, deep learning-based vision technologies have been developed, such as OpenPose²⁷ and MediaPipe,²⁸ which can be used with a standard camera to estimate human poses. Studies^29,30 demonstrated the potential of these technologies in estimating 3D human poses by integrating depth information captured from depth-sensing cameras.^31–33 However, for COAD patients, PD and percussion (P) airway clearance techniques play essential roles in the PR process,⁹ but there seems to be a lack of research on how to effectively assist patients with PD + P and provide necessary remote monitoring and timely feedback to improve the effectiveness of TH-PR.

Therefore, to improve the convenience and flexibility of TH-PR for COAD patients, in this study, we developed a mobile-based airway clearance care for chronic obstructive airway disease (COAD-MoAcCare) system to support personalized TH-PR for COAD patients. Performing PD + P by patients and caregivers can be estimated and analyzed in real time with a smartphone using deep learning-based vision technology, i.e. MediaPipe,²⁸ based on expert-defined PD + P standard models. Consequently, the system can monitor, guide, and remind patients to perform PD + P correctly in real time, and the results can be evaluated and reports provided for patients (and caregivers) for reflection, thereby improving their familiarity with and effectiveness of PD + P.^9,34 Medical personnel can customize PD + P according to a patient's condition through the system's medical functions. The patient's PD + P process can be recorded and fed back to the system for management, providing medical personnel with an assessment of the patient's TH-PR condition, allowing timely advice and care to the patient, enhancing the patient's motivation and adherence,^18–20 and ultimately improving the overall effectiveness of TH-PR for COAD patients.

The present study thus aimed to evaluate the usability of the COAD-MoAcCare system in assisting medical personnel in conducting PD + P for COAD patients during TH-PR. A pilot study was conducted through expert validation, where professional respiratory therapists were invited to assess the system usability using the system's questionnaires based on the technology acceptance model (TAM), system usability scale (SUS), and task load index (NASA-TLX).

Methods

Study design

The present study was conducted in two phases. Phase I involved the development of the system, while phase II focused on pilot usability testing. In phase II, we evaluated the functional effects of the system when conducting PD + P for COAD patients during TH-PR through an expert validation method. It assessed the perceptual feedback of medical personnel regarding the system using self-reported measurement questionnaires. The Research Ethics Committee of a medical university hospital approved the study (TMU-JIRB-N202104005), and written informed consent was obtained from all participants.

Participants

Participants for the pilot usability testing phase were eligible for inclusion if they were registered respiratory therapists and had experience conducting PD + P for COAD patients based on purposeful sampling. Inclusion criteria were (a) agreed to consent to participate, (b) had sufficient PD + P experience, and (c) owned a mobile phone that could execute the COAD-MoAcCare system. Exclusion criteria were (a) refused to participate, (b) had insufficient PD + P experience, and (c) had no qualified mobile phone that could execute the COAD-MoAcCare system.

Procedures

Figure 1 presents the full experimental procedures. Qualified registered respiratory therapists were invited to participate in this pilot usability testing of the COAD-MoAcCare system. After providing consent, participants joined the test process. Participants self-learned how to use the system through the system manual to assess the usefulness of the system manual and the user-friendliness of the system. No formal training was required to ensure that participants could independently use the system.

Figure 1.

Experimental procedures.

After they were familiar with the system, participants began conducting system function tests based on predefined test cases for 1 month. For example, participants issued a COAD prescription and set daily PD + P goals for a patient based on their condition using medical personnel functions of the COAD-MoAcCare system. Subsequently, they used the mobile patient app (MPA) to perform PD + P during TH-PR to test the system's functionality for patients and caregivers. Participants needed to fulfill specified criteria for each test case to consider it a valid test. For instance, the accuracy of PD + P operations needed to exceed 90%, meaning that the system should detect at least 108 or 132 valid operations per minute based on a standard of 120 operations per minute.

Participants had the flexibility of conducting the system usability tests at a time and location of their choosing. After completing all of the test cases, they filled out system questionnaires to provide feedback on their experience with the COAD-MoAcCare system. The feedback we received underwent system record checking to ensure its validity.

Data collection

Data were collected at two distinct time points. Once participants comprehended the study's objectives, they provided consent. After completing system usability testing encompassing all test cases, participants filled out the system questionnaires and offered feedback. The validity of their feedback was confirmed after verifying test records from the system side, ensuring that they had fulfilled the test requirements. Demographic information, such as age, gender, clinical experience, and mobile device familiarity, was gathered and is presented in Table 1.

Table 1.

Demographics of participants.

	Participants (N = 11)
Age	n (%)
20–29 years	7 (63.6%)
30–39 years	2 (18.2%)
40–49 years	1 (9.1%)
50–59 years	1 (9.1%)
Over 60 years	0 (0.0%)
Gender
Female	7 (63.6%)
Male	4 (36.4%)
Clinical experience
Yes	8 (72.7%)
No	3 (27.3%)
Mobile device familiarity
Good	6 (54.5%)
Very good	5 (45.5%)

Scheme of the COAD-MoAcCare system

The scheme of COAD-MoAcCare (Figure 2) included (a) a web-based patient management system (WPMS) for medical personnel (physicians and therapists) and (b) a mobile patient app (MPA) for COAD airway clearance cases. In the WPMS, physicians can use Portfolio Management to view a patient's rehabilitation status (via Report Checking) and issue a COAD prescription based on the patient's current condition. Therapists can use the COAD prescription and PD + P setting to set daily PD + P goals for the patient, e.g. lateral, upper, middle, and lower lobe operations, and use the Location Setting to set the personal location of the lobes based on the patient's sputum mucus condition.

Figure 2.

Scenario and scheme of the mobile-based airway clearance care for chronic obstructive airway disease (COAD-MoAcCare) system.

Patients and caregivers can use the MPA to conduct PD + P during TH-PR. The MPA provides “Location Checking” to confirm the position of each PD + P and “Descriptions” to understand proper usage. Using deep learning-based vision technology, the MPA conducts a “Motion Analysis” through a standard camera on a mobile device to provide “Postural Drainage Guidance” to assist patients in getting into a correct PD posture before percussion. Real-time “Percussion Monitoring” and “Operation Prompting,” i.e. “Visual Prompts” and “Voice Prompts,” assist patients and caregivers in effectively performing PD + P during the percussion process.

Each PD + P operation automatically generates a report according to the percussion situation through “Operation Analysis.” Patients (caregivers) can easily view this report on their smartphone via “Web Check,” without needing to use the MPA. This allows them to understand the status and potential problems with PD + P and make corrections as needed to ensure effective and correct PD + P operation. Accordingly, they can become increasingly skilled in PD + P.

The execution status and report of a patient's daily PD + P are sent back to the WPMS. This allows medical personnel to use “Portfolio Management” to view and track the patient's rehabilitation status (via “Report Checking”) in real time. They can provide necessary care reminders and adjustments to the PD + P plan as needed. Therefore, the COAD-MoAcCare scheme can assist COAD patients in conducting effective self-care with PD + P during TH-PR.

Process of the MPA in COAD-MoAcCare

The MPA in COAD-MoAcCare is integrated with MediaPipe, a deep learning-based vision open-source tool,²⁸ to perform real-time human pose estimations (HPEs). With PD + P, some operations can be performed by the patient alone (e.g. lateral and upper lobes), while others require the assistance of a caregiver (e.g. middle and lower lobes). Therefore, the MPA provides different PD guidance and percussion monitor instructions to help patients (and caregivers) perform correct PD and P operations according to their personalized PD + P plan.

In Figure 3, when a patient performs PD + P alone (i.e. done by the patient), the MPA initiates “Body Pose Initialization” in the PD Guidance stage to capture the patient's body skeletal coordinates, i.e. (x,y)_orig. During the patient's PD operation, “Pose Monitor” obtains pose coordinates at each time point (t_i), i.e. (x,y)_ti, and applies “Stabilization” to ensure stability. “Coordinate Normalization” converts the coordinates for “PD Pose Evaluation,” i.e. (x,y)_norm, compares them to standard posture coordinates, i.e. (x,y)_std, and provides visual prompts (i.e. a simple stick figure design to help patients clearly see the visual guidance displayed on the small screen of the mobile device) and voice prompts (e.g. too close, too far, okay, move your body, lie back, and lie on your side) for correct posture. In the Percussion Monitor stage, “Action Monitor and Stabilization” captures real-time pose coordinates, which are normalized and evaluated against the standard percussion action and target area coordinates. Visual prompts (i.e. a red hand indicates incorrect operation) and voice prompts (e.g. too slow, too fast, wrong position, and great) guide the patient in effective percussion operation.

Figure 3.

Postural drainage and percussion (PD + P) process of the mobile patient app (MPA) in the mobile-based airway clearance care for chronic obstructive airway disease (COAD-MoAcCare) system.

A two-stage approach is needed to ensure accurate detection and recognition of actions during caregiver-assisted PD + P (done by the caregiver). In step 1, the patient adjusts their position and posture based on visual and voice prompts in PD Guidance, within the range recognized by the MPA. In step 2, the caregiver follows the MPA's guidance to enter the recognition range and confirm the percussion position. Once both steps are completed, they can proceed to percussion operation, allowing the MPA to effectively analyze the caregiver's percussion actions and provide guidance for successful caregiver-assisted PD + P. Figure 4 demonstrates the system screen process, which guides the patient and caregiver using visual and voice prompts.

Figure 4.

Caregiver-assisted postural drainage and percussion (PD + P) process of “Done by Caregiver” in the mobile patient app (MPA).

Process of the WPMS in COAD-MoAcCare

Figure 5 shows WPMS screenshots. Medical personnel can create personalized PD + P plans, view operation records (Figure 5(a)), and generate reports (Figure 5(b)). The WPMS records and manages a patient's results returned from the MPA, allowing the personnel to review statistical charts (Figure 5(c)) and generate time-specific reports (Figure 5(d)) for the effective monitoring and adjustment of a patient's activity tasks.

Figure 5.

Screenshots of the web-based patient management system (WPMS) including (a) postural drainage and percussion (PD + P) management, (b) a PD + P report, (c) a PD + P statistical chart, and (d) a time-specific report.

Measurement instruments

SUS

System usability measures users’ perceptions of a product or service through the SUS.³⁵ The SUS questionnaire (Table 2) is a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree), for which the internal consistency (Cronbach's alpha α) was 0.8.³⁶ Cronbach's alpha for the overall scale was 0.837 in this study. A SUS score of >70 is typically regarded as acceptable, and a score of 74.1 to 77.1 suggests good usability.³⁷

Table 2.

Results of the system usability scale (SUS) questionnaire (total score range: 0–100).

Survey item	Mean (SD)
Q1. I think that I would like to use this system frequently.	4.3 (0.8)
Q2. I found the system unnecessarily complex.	2.3 (1.5)
Q3. I thought the system was easy to use.	2.9 (1.6)
Q4. I would need the support of a technical person to be able to use this system.	2.2 (1.4)
Q5. I found that the categorized results in this system were very sensible.	4.0 (0.6)
Q6. I thought that there was too much inconsistency in this system.	1.8 (0.8)
Q7. I would imagine that most people could learn to use this system very quickly.	3.7 (0.6)
Q8. I found the system very cumbersome to use.	2.0 (1.0)
Q9. I felt very confident using the system.	4.5 (0.7)
Q10. I needed to learn a lot of things before I could get going with this system	1.5 (1.0)
Final SUS Score (after the SUS score calculation)	74.1 (16.9)

SD: standard deviation; SUS: system usability scale.

NASA-TLX

The NASA-TLX³⁸ is a widely accepted and validated tool³⁹ used for measuring and determining the mental workload experienced by participants while performing a task (Table 3). It evaluates the workload across five 7-point scales: mental demand, physical demand, temporal demand, individual performance, effort, and frustration level, resulting in 21 gradations. Scoring ranges 0 to 100, where a higher score indicates a more significant perceived task load. The internal consistency of the NASA-TLX questionnaire in this study was reliable (α = 0.725).

Table 3.

Results of the NASA-task load index (NASA-TLX) questionnaire (total score range: 0–100).

Survey item	Mean (SD)
Q1. How mentally demanding was the task?	25.5 (23.0)
Q2. How physically demanding was the task?	26.4 (23.8)
Q3. How hurried or rushed was the pace of the task?	29.1 (14.5)
Q4. How hard did you have to work to achieve your level of performance?	26.4 (22.9)
Q5. How successful were you in accomplishing what you were asked to do?	34.5 (6.9)
Q6. How insecure, discouraged, irritated, stressed, and annoyed were you?	38.2 (20.9)
Final NASA-TLX score	30.0 (13.8)

SD: standard deviation; TLX: task load index.

Perception of system use

To evaluate system usability, participants completed a system questionnaire to obtain feedback on their experience with the system. The system questionnaire was applied based on the TAM.^40–42 The TAM is a widely used theoretical framework explaining computer usage behavior⁴⁰ and considers perceived usefulness and ease of use as factors influencing technology acceptance. TAM2 adds social influence and cognitive instrumental processes,⁴¹ while TAM3 integrates determinants of perceived ease of use.⁴²

This study also evaluated the effect of the COAD-MoAcCare system by examining seven expert-defined scales (assistance for patient care, assistance for medical care, professionality, convenience, comparison, recommendation, and satisfaction). The system questionnaire in Table 4 presents each of the ten measurement scales. These scales utilize a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). Cronbach's alpha values of the system questionnaire and the expert-defined scales were 0.973 and 0.899, respectively, indicating that the internal consistency was reliable in this study.

Table 4.

Definitions of questionnaire measurements (5-point Likert scale: 5 = strongly agree to 1 = strongly disagree).

Measurement scale	Description
Assistance for Patient Care (Assistance-PC)	Feedback on the effectiveness of the system in assisting patients (caregivers) in TH-PR, including PD + P guidance, real-time feedback, and analysis reports.
Assistance for Medical Care (Assistance-MC)	Feedback on the effectiveness of the system in assisting medical personnel in performing TH-PR for patients, including PD + P planning and operation recording.
Professionality	The professionality of the system in assisting TH-PR in conducting PD + P.
Convenience	The convenience of the system in assisting TH-PR in conducting PD + P.
Comparison	The assistance effects of the system compared to those of other systems that have been used in terms of usefulness, convenience, and enjoyment.
Recommendation	Willingness to recommend the use of this system to other relevant personnel.
Satisfaction	Satisfaction is the attitude toward using technology.
Intention to use	These scales are defined based on the TAM model.⁴⁰ Intention-to-use is the behavioral intention to use the COAD-MoAcCare system in the future.
Usefulness
Ease of use
Output quality and enjoyment	The output quality and enjoyment are respectively defined by TAM2⁴¹ and TAM3⁴².

COAD-MoAcCare: mobile-based airway clearance care for chronic obstructive airway disease; PD + P: postural drainage and percussion; TAM: technology acceptance model; TH-PR: telehealth home-based pulmonary rehabilitation.

Permission of measurement instruments

Regarding the SUS permission, in Brooke's article,³⁵ the author described that “SUS has been made freely available for use in usability assessment, and has been used for a variety of research projects and industrial evaluations; the only prerequisite for its use is that any published report should acknowledge the source of the measure.” Regarding the NASA-TLX permission, the NASA-TLX website⁴³ states that “The NASA TLX tool is ‘open source,’ and is therefore available for immediate use by any person or organization (worldwide). It is not necessary to obtain permission from NASA to use TLX, nor is permission needed to make modifications to the tool itself (i.e. language translation).” Moreover, this study has obtained permission to use the TAM questionnaire from Professor Davis for TAM⁴⁰ and Professor Venkatesh for TAM2⁴¹ and TAM3.⁴²

Data analysis

Descriptive statistics were utilized to show the demographic characteristics of participants. Continuous variables are expressed as the mean and standard deviation (SD), and categorical variables are expressed as a frequency (percentage). The Mann–Whitney U test (non-parametric statistical analysis) was used to compare differences between the two small groups. The Spearman correlation coefficient, r_s, was employed to evaluate the strength and direction of the monotonic relationship between two variables, ranging from 0.6 to 0.8, indicating a moderately strong correlation.^44,45

Results

Analysis of demographic data

In total, 15 registered respiratory therapists were invited; 11 of them from four hospitals and one medical university participated. Table 1 presents participants’ demographics for expert validation of system usability testing.

Analysis of SUS and NASA-TLX

Tables 2 and 3 show each survey item's final and mean scores in the SUS and NASA-TLX questionnaires. The SUS score of 74.1 indicates good usability (within the range of 74.1 to 77.1).³⁷ The final NASA-TLX score of 30 suggests a lower perceived mental workload when using the system.³⁸

Analysis of perception of system use

Figure 6 presents a statistical histogram of measurement scales from the system questionnaire completed by 11 participants. Results indicated that all scales were ≥4.1, with the satisfaction scale reaching 4.5. These findings suggested a high level of perceptual satisfaction with the COAD-MoAcCare system. Moreover, to investigate differences in gender and age, the analytical results in Table 5 show that none of the scales exhibited a significant difference between gender and age, implying that the use of the system did not vary according to gender or age.

Figure 6.

Statistical histogram of measurement scales from the satisfaction questionnaire.

Table 5.

Differences between gender and age using the Mann-Whitney U-test.

	Total	Gender (A: male; B: female)			Age (A: >30, B: 20∼29 years)
Scale	N = 11 Mean (SD)	Group A (N = 4) Mean (SD)	Group B (N = 7) Mean (SD)	p value (z-value)	Group A (N = 4) Mean (SD)	Group B (N = 7) Mean (SD)	p-value (z-value)
Assistance-PC	4.5 (0.5)	4.7 (0.1)	4.4 (0.6)	0.527 (−0.779)	4.6 (0.4)	4.5 (0.5)	0.927 (−0.195)
Assistance-MC	4.6 (0.5)	4.6 (0.5)	4.6 (0.5)	0.927 (−0.104)	4.4 (0.5)	4.8 (0.4)	0.230 (−1.462)
Professionality	4.6 (0.5)	4.8 (0.5)	4.6 (0.5)	0.648 (−0.565)	4.5 (0.6)	4.7 (0.5)	0.648 (−0.678)
Convenience	4.2 (1.0)	4.5 (0.5)	4.0 (1.2)	0.648 (−0.610)	4.3 (0.5)	4.1 (1.2)	0.788 (−0.305)
Comparison	4.3 (0.8)	4.4 (0.4)	4.3 (1.0)	0.927 (−0.100)	4.4 (0.4)	4.3 (1.0)	0.927 (−0.100)
Recommendation	4.6 (0.5)	4.5 (0.6)	4.7 (0.5)	0.648 (−0.678)	4.3 (0.5)	4.9 (0.4)	0.109 (−1.920)
Satisfaction	4.5 (0.5)	4.6 (0.1)	4.4 (0.6)	0.927 (−0.191)	4.4 (0.3)	4.5 (0.6)	0.315 (−1.147)
Enjoyment	4.5 (0.7)	4.8 (0.3)	4.3 (0.8)	0.527 (−0.801)	4.5 (0.4)	4.4 (0.8)	0.788 (−0.400)
Usefulness	4.4 (0.5)	4.5 (0.2)	4.3 (0.6)	0.927 (−0.193)	4.3 (0.2)	4.4 (0.6)	0.412 (−0.963)
Ease of use	4.1 (1.1)	4.6 (0.6)	3.8 (1.3)	0.412 (−0.972)	4.4 (0.7)	3.9 (1.3)	0.788 (−0.389)
Intention to use	4.4 (0.5)	4.6 (0.1)	4.3 (0.6)	0.412 (−0.958)	4.4 (0.3)	4.4 (0.7)	0.927 (−0.096)
Output Quality	4.3 (0.3)	4.5 (0.0)	4.2 (0.4)	0.164 (−1.671)	4.4 (0.3)	4.3 (0.4)	0.648 (−0.627)
SUS	74.1 (16.9)	70.6 (17.5)	76.1 (17.6)	0.527 (−0.759)	66.9 (13.9)	78.2 (18.0)	0.164 (−1.424)
NASA-TLX	30.0 (13.8)	38.8 (16.8)	25.0 (9.9)	0.164 (−1.519)	37.9 (18.0)	25.5 (10.0)	0.230 (−1.234)

PC: Patient Care; MC: Medical Care; SUS: system usability scale, NASA-TLX: NASA task load index.

Analysis of correlations

A Spearman correlation analysis revealed significant correlations between various scales. Notably, the SUS showed a strong positive correlation with Satisfaction (r_s = 0.848, p = 0.001), Assistance-MC (r_s = 0.664, p = 0.026), Comparison (r_s = 0.615, p = 0.044), and Usefulness (r_s = 0.688, p = 0.019).

Analysis of accuracy of PD + P operations

This study calculated the accuracy of PD + P operations by comparing the effective PD + P counts monitored by the MPA (Number_by_System) to those assessed by experts (Number_by_Expert) using the formula: Accuracy = (|Number_by_Expert - Number_by_System| / Number_by_Expert) × 100%. The mobile device should have a minimum specification of CPU, Qualcomm Snapdragon 650; GPU, Adreno 510; and OS, Android 7.0, or equivalent specifications to an iPhone 8, for stable execution of the COAD-MoAcCare system. Table 6 presents results of three test sessions, indicating that the COAD-MoAcCare system achieved a high level of accuracy, with an average of 97.5% and a range of 95.6%–99.4%. However, several PD + P types, such as the lower lobes, exhibited slightly lower accuracy but was still >95%.

Table 6.

Results of the accuracy of postural drainage and percussion (PD + P) operations using the COAD-MoAcCare system.

PD + P type	1st	2nd	3rd	Average (%)
Upper lobes
Left side	96.7	99.1	96.7	97.5
Right side	98.3	100	96.7	98.3
Lateral lobes
Left side	98.3	99.2	100	99.2
Right side	95.0	99.2	95.0	96.4
Middle lobes (caregiver-assisted)
Left side	100	100	98.3	99.4
Right side	96.7	100	91.7	96.1
Lower lobes (caregiver-assisted)	98.3	96.7	91.7	95.6
Average	97.6	99.2	95.7	97.5

Discussion

It has been proven that using software and hardware systems to assist COAD patients in TH-PR can be effective, e.g. PR education and self-management.^{12,15,21–26} However, to the best of our knowledge, the COAD-MoAcCare system presented in this study is the first system to aim to use mobile and deep learning-based vision technology to assist COAD patients in performing PD + P during TH-PR.

Similar to our idea, Latreche et al.⁴⁶ utilized the MediaPipe tool to create a website to remotely monitor the rehabilitation status of patients. They also demonstrated the reliability and validity of MediaPipe for telerehabilitation. That study suggested that this approach eliminates the need for additional devices, such as wearable inertial sensors^47,48 and depth-sensing cameras (e.g. Kinect),^31–33 providing convenient and cost-effective rehabilitation for patients. However, they only focused on estimating simple shoulder poses, and their future research objective will be how to address more complex movement types, e.g. reach-to-grasp movements. The PD + P operations in our study were much more complex than theirs.

Research has shown that PD + P requires teaching and a long learning time for patients and caregivers,⁹ and incorrect PD + P operation can actually harm the health of patients,³⁴ although the effect of TH-PR is similar to that of hospital-based PR.^13,15 In addition, the lack of records and supervision makes it difficult for medical personnel to understand the execution status of patients.²⁰ Therefore, the COAD-MoAcCare system allows medical personnel to manage and set personalized PD + P for patients through the WPMS and view the execution status and effectiveness of TH-PR so that they can understand the patient's condition in real time.^18–20

The positive feedback given by 11 respiratory therapists (participants) after using the system included the following: (1) the connection between the web and mobile device was very fast and convenient, (2) it was convenient to view details of a patient's history records at the same time, and (3) this application is very advanced. Once the app can assist in expectorating sputum, it will help many people after the system's improvement. Latreche et al.⁴⁶ also pointed out that involved medical personnel gave similar positive feedback.

Consequently, based on the questionnaire analysis, participants were highly satisfied with the COAD-MoAcCare system (Satisfaction = 4.5 ± 0.5 out of 5.0) and believed that the system had Professionality (4.5 ± 0.5) and provided better results compared to existing rehabilitation models (Comparison = 4.3 ± 0.8). Therefore, they highly agreed that the system was useful (Usefulness = 4.4 ± 0.5) and enjoyable (Enjoyment = 4.5 ± 0.7) and had the intention to use it (4.4 ± 0.5). Most of them were willing to recommend (4.6 ± 0.5) this system as an assistance of PD + P for COPD patients during TH-PR and therefore highly agreed that the COAD-MoAcCare system was useful for helping medical personnel conduct PD + P during TH-PR (Assistance-MC = 4.6 ± 0.5).

The questionnaire analysis also showed that Ease of Use (4.1 ± 1.1) was the lowest item among all scales. Convenience was inferred to be affected by Easy-to-Use, so it was slightly lower at 4.2 ± 1.0. Further analysis focusing on Ease of Use (Table 5) showed that the male group rated it higher (4.6 ± 0.6) compared to the female group (3.8 ± 1.3). In contrast, the senior group rated it higher (4.4 ± 0.7) than the younger group (3.9 ± 1.3). However, these differences were not statistically significant. Previous research indicated that gender plays a role in technology adoption, with males being more technologically adept than females.^49,50 The SUS and NASA-TLX results showed the opposite pattern, although these differences were not statistically significant. Therefore, average results of the SUS (74.1 ± 14.9) and NASA-TLX (30 ± 13.8) can be used to indicate that the COAD-MoAcCare system has good usability and low user mental workload when using the system.

Additionally, the present system mainly evaluated the usability of the COAD-MoAcCare system through expert validation. Although clinical trials with COAD patients have not yet been conducted, respiratory therapists also highly agreed that the COAD-MoAcCare system seemed helpful in assisting COAD patients in performing PD + P during TH-PR (Assistance-PC = 4.5 ± 0.5) according to feedback from the questionnaire. In this regard, the positive feedback included the following: (1) it is a new project that is more convenient and faster to assist therapy, (2) the operational interface of the MPA is simple and easy to understand, and (3) it is necessary to read the manual repeatedly when first used, but it can be mastered after frequent use. In addition, suggestions for improvement included the following: (1) for elderly patients, the usability of the app may need to be improved, (2) the accuracy of real-time monitoring of PD + P operations can be improved, and (3) there are high requirements for mobile phone hardware specifications.

Currently, the accuracy of PD + P operations was high, at an average of 97.5% (Table 6). However, there were slightly lower accuracy rates for certain types of PD + P operations, such as lower lobes (95.6%), middle lobes (right) (96.1%), and lateral lobes (right) (96.4%), which was related to HPE accuracy for these specific poses in the system. Existing research also analyzed and compared the accuracy of HPEs.^46,51 Therefore, based on participant feedback, it is necessary to further improve the accuracy of specific types of PD + P operations to enhance the system's usability.

Limitations and future work

Although feedback from participants proved the usability of the COAD-MoAcCare system, it still has limitations that need to be improved in the near future, as described below.

First, the present study was a pilot usability test. Second, while we invited professional respiratory therapists from different hospitals to participate, the small sample size in our study may have introduced potential bias to the results. Third, the short-term usability testing primarily focused on immediate perceptual feedback from experts. To validate the long-term effectiveness of the system, a large-scale, long-term follow-up clinical study is necessary.

Fourth, the HPE tool greatly influenced the accuracy of PD + P operations in the COAD-MoAcCare system. Currently, the system utilizes 2D coordinates output by the MediaPipe tool for calibration and action comparison analyses. Therefore, the accuracy and performance of HPEs in the MPA of the COAD-MoAcCare system mainly depend on MediaPipe.

Latreche et al.⁴⁶ conducted tests on the accuracy of MediaPipe in conducting HPEs and found that it provided results close to those of clinical goniometer and angle ruler instruments. Chung et al.⁵¹ analyzed the performance and accuracy of four deep learning-based vision technologies, including OpenPose, PoseNet, MediaPipe, and MoveNet, for 2D single-person HPEs. Among these, only MediaPipe and MoveNet could be executed on mobile devices.

Results indicated that MoveNet exhibited superior performance. For videos and data with inappropriate camera positions or self-occlusion, MediaPipe ranked second, while OpenPose performed the least accurately. Therefore, MoveNet would be a promising HPE tool for improving the accuracy and performance of our system in the future.

However, MoveNet currently only provides 2D-based HPE, which means it lacks 3D coordinates. This limitation could lead to unstable accuracy, especially when a patient has an incorrect posture. On the other hand, although MediaPipe may not match MoveNet in terms of performance, its BlazePose⁵² offers predictive 3D coordinates for HPEs. Therefore, adoption of 3D-based HPE using MediaPipe will be implemented to improve the recognition accuracy of our system.

Nevertheless, incorporating depth (z-value) calculations by MediaPipe will increase the hardware requirements of mobile devices. For instance, currently, 3D-based HPEs using MediaPipe achieve an execution performance of 49 FPS for the lite model, 40 FPS for the full model, and 19 FPS for the heavy model using TFLite GPU on Pixel 3 (CPU, Qualcomm Snapdragon 845, and GPU, Adreno 630),⁵³ whose hardware specifications are higher than the 2D-based HPE used by our system.

While deploying the HPE tool on a server can reduce the hardware specifications required for mobile devices, as Latreche et al.⁴⁶ suggested, unstable network transmission may result in high latency, making it challenging to achieve a real-time response. Therefore, they recommended implementing HPEs on mobile devices to enhance performance, which aligns with the approach taken in our study.

Fifth, the limitation of the single-person HPE method restricts its convenience in practical use. Currently, MediaPipe only supports single-person HPEs, which reduces the flexibility and ease of use for patients and caregivers during caregiver-assisted PD + P (Figures 3 and 4). Therefore, MoveNet, which can provide multiple-person HPEs and high accuracy on mobile devices, remains a promising solution to improve this issue, although it cannot support 3D-based HPEs like MediaPipe.

Sixth, recognition using MediaPipe may be unstable and mistaken due to the patient's clothing (short sleeves are better) and color (a clear difference from the background color is better) and background conditions (a clear and uncluttered background is better). Latreche et al.⁴⁶ also mentioned similar concerns in their study. In the present study, a system manual was provided to remind users of important considerations during operation, aiming to enhance recognition accuracy. Results confirmed that this approach improved the operational effectiveness. Therefore, further investigation is needed to explore how to enhance the stability of recognition and analysis by improving system use instructions and post-processing techniques.

Finally, although the internal consistency of the expert-defined scales was 0.899, there is still room for improvement in enhancing reliability. Future studies can consider exploring more suitable and reliable existing measurement tools to further enhance the reliability of these scales.

Conclusions

In this study, we developed the COAD-MoAcCare system to intelligently assist COAD patients and medical personnel in performing PD + D during TH-PR. By using the convenience of mobile devices and deep learning-based vision technology, the COAD-MoAcCare system can monitor and guide personalized PD + P plans for patients and provide real-time analysis and feedback to enable medical personnel to understand the TH-PR status of patients and further make corrections and suggestions for improving the TH-PR performance. After assessment by 11 professional respiratory therapists for usability, they were highly satisfied, believed that the COAD-MoAcCare system would be helpful for medical personnel to conduct PD + P during TH-PR, and anticipated its potential benefits for COAD patients. The 97.5% overall accuracy of PD + P operations using the COAD-MoAcCare system further demonstrated the high usability of the system. In the near future, efforts will be made to optimize the system's accuracy, stability, and usability by further conducting clinical trials on COAD patients.

Supplemental Material

sj-docx-1-dhj-10.1177_20552076231207206 - Supplemental material for A mobile-based airway clearance care system using deep learning-based vision technology to support personalized home-based pulmonary rehabilitation for COAD patients: Development and usability testing

Supplemental material, sj-docx-1-dhj-10.1177_20552076231207206 for A mobile-based airway clearance care system using deep learning-based vision technology to support personalized home-based pulmonary rehabilitation for COAD patients: Development and usability testing by Jun-Ming Su, Kuan-Yuan Chen, Sheng-Ming Wu, Kang-Yun Lee and Shu-Chuan Ho in DIGITAL HEALTH

Footnotes

Acknowledgments

We thank the respiratory therapists for their assistance and guidance in this research, and appreciate the anonymous reviewers and editors for their valuable feedback, which helped to improve this paper.

Contributorship

JMS: conceptualization, methodology, system development, researched literature, formal analysis, and manuscript drafting. KYC: researched literature, data curation, and analyzed data set. SMW: researched literature, data curation, and analyzed data set. KYL: conceptualization, supervision, project administration, and manuscript review. SCH: conceptualization, study design, supervision, project administration, and manuscript finalization.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical approval

The Ethics Committees of the Taipei Medical University-Joint Institutional Review Board (no: N202104005) reviewed and approved the study protocol.

Funding

This study was supported by grants from the National Science and Technology Council (NSTC 112-2622-B-038-003), Ministry of Science and Technology (MOST 111-2314-B-038-090-MY3, MOST 111-2314-B-038-082, and MOST 111-2410-H-024-002-MY2), the Ministry of Education of Taiwan (DP2-111-21121-01-T-01-05), and Taipei Medical University (110TMU-SHH-05).

Guarantor

S-CH.

ORCID iDs

Jun-Ming Su

Shu-Chuan Ho

Supplemental material

Supplemental material for this article is available online.

References

Rogers

Barnes

. Treatment of airway mucus hypersecretion. Ann Med 2006; 38: 116–125.

Anandan

Nurmatov

van Schayck

, et al. Is the prevalence of asthma declining? systematic review of epidemiological studies. Allergy 2010; 65: 152–167.

Volsko

. Airway clearance therapy: finding the evidence. Respir Care 2013; 58: 1669–1678.

Nelson

. Postural drainage of the lungs. Br Med J 1934; 2: 251–255.

Pryor

. Physiotherapy for airway clearance in adults. Eur Respir J 1999; 14: 1418–1424.

Reid

Chung

. Airway clearance techniques. In: Clinical management notes and case histories in cardiopulmonary physical therapy. NJ: Slack Incorporated, 2004, p.105e111.

Yernault

Bohadana

. Chest percussion. Eur Respir J 1995; 8: 1756–1760.

Gallon

. Evaluation of chest percussion in the treatment of patients with copious sputum production. Respir Med 1991; 85: 45–51.

Belli

Prince

Savio

, et al. Airway clearance techniques: the right choice for the right patient. Front Med (Lausanne) 2021; 8: 544826.

10.

Jones

Taylor

Gowler

, et al. Systematic review of interventions to improve patient uptake and completion of pulmonary rehabilitation in COPD. ERJ Open Res 2017; 3: 00089–2016.

11.

Rochester

Vogiatzis

Holland

, et al. An official American Thoracic Society/European Respiratory Society policy statement: enhancing implementation, use, and delivery of pulmonary rehabilitation. Am J Respir Crit Care Med 2015; 192: 1373–1386.

12.

Jiang

Liu

Guo

, et al. Evaluating an intervention program using WeChat for patients with chronic obstructive pulmonary disease: randomized controlled trial. J Med Internet Res 2020; 22: e17089.

13.

Holland

Mahal

Hill

, et al. Home-based rehabilitation for COPD using minimal resources: a randomised, controlled equivalence trial. Thorax 2017; 72: 57–65.

14.

Roberto

Jennifer

Johanna

, et al. Feasibility of a health coaching and home-based rehabilitation intervention with remote monitoring for COPD. Respir Care 2021; 66: 960–971.

15.

Michaelchuk

Oliveira

Marzolini

, et al. Design and delivery of home-based telehealth pulmonary rehabilitation programs in COPD: a systematic review and meta-analysis. Int J Med Inf 2022; 162: 104754.

16.

Lee

Cecins

Hill

, et al. The effects of pulmonary rehabilitation in patients with non-cystic fibrosis bronchiectasis: protocol for a randomised controlled trial. BMC Pulm Med 2010; 10: 5.

17.

Alison

McKeough

Johnston

, et al. Australian and New Zealand pulmonary rehabilitation guidelines. Respirology 2017; 22: 800–819.

18.

Bassett

Prapavessis

. Home-based physical therapy intervention with adherence-enhancing strategies versus clinic-based management for patients with ankle sprains. Phys Ther 2007; 87: 1132–1143.

19.

Miller

Porter

DeBaun-Sprague

, et al. Exercise after stroke: patient adherence and beliefs after discharge from rehabilitation. Top Stroke Rehabil 2017; 24: 142–148.

20.

Liao

Vakanski

Xian

, et al. A review of computational approaches for evaluation of rehabilitation exercises. Comput Biol Med 2020; 119: 103687.

21.

Van der Heijden

Lucas

PJF

Lijnse

, et al. An autonomous mobile system for the management of COPD. J Biomed Inform 2013; 46: 458–469.

22.

Sánchez-Ramírez

Pol

Loewen

, et al. Effect of telemonitoring and telerehabilitation on physical activity, exercise capacity, health-related quality of life, and healthcare use in patients with chronic lung diseases or COVID-19: a scoping review. J Telemed Telecare 2022: 1–19. DOI: 10.1177/1357633X221122124.

23.

Rassouli

Boutellier

Duss

, et al. Digitalizing multidisciplinary pulmonary rehabilitation in COPD with a smartphone application: an international observational pilot study. Int J Chron Obstruct Pulmon Dis 2018; 13: 3831–3836.

24.

Spielmanns

Boeselt

Huber

, et al. Impact of a smartphone application (KAIA COPD app) in combination with activity monitoring as a maintenance program following pulmonary rehabilitation in COPD: the protocol for the AMOPUR study, an international, multicenter, parallel group, randomized, controlled study. Trials 2020; 21: 636.

25.

Huang

Lin

Dang

, et al. Effect of internet-based self-management on pulmonary function rehabilitation and living quality in patients with chronic obstructive pulmonary disease. Am J Transl Res 2021; 13: 5224–5231.

26.

Deng

Sheng

Jiang

, et al. A home-based pulmonary rehabilitation mHealth system to enhance the exercise capacity of patients with COPD: development and evaluation. BMC Med Inform Decis Mak 2021; 21: 325.

27.

Cao

Hidalgo

Simon

, et al. Openpose: realtime multi-person 2D pose estimation using part affinity fields. IEEE Trans Pattern Anal Mach Intell 2021; 43: 172–186.

28.

Lugaresi

Tang

Nash

, et al. Mediapipe: a framework for building perception pipelines. arXiv 2019; arXiv:1906.08172.

29.

Martini

Boldo

Aldegheri

, et al. Enabling gait analysis in the telemedicine practice through portable and accurate 3D human pose estimation. Comput Methods Programs Biomed 2022; 225: 107016.

30.

Baranyi

Melício

BCDS

Gaál

, et al. AI Technologies for machine supervision and help in a rehabilitation scenario. Multimodal Technol Interact 2022; 6: 48.

31.

Mangal

Tiwari

. A review of the evolution of scientific literature on technology-assisted approaches using RGB-D sensors for musculoskeletal health monitoring. Comput Biol Med 2021; 132: 104316.

32.

Triantafyllidis

Filos

Buys

, et al. Computerized decision support for beneficial home-based exercise rehabilitation in patients with cardiovascular disease. Comput Methods Programs Biomed 2018; 162: 1–10.

33.

Belotti

Bonfanti

Locatelli

, et al. A tele-rehabilitation platform for shoulder motor function recovery using serious games and an Azure Kinect device. Stud Health Technol Inform 2022; 293: 145–152. PMID: 35592974.

34.

Jette

Warren

Wirtalla

. The relation between therapy intensity and outcomes of rehabilitation in skilled nursing facilities. Arch Phys Med Rehabil 2005; 86: 373–379.

35.

Brooke

. SUS-A quick and dirty usability scale. In: Patrick

Thomas

McClelland

Weerdmeester

(eds) Usability evaluation in industry. 1996. London: Taylor & Francis, 1996, pp.189–194.

36.

Gao

Kortum

Oswald

. Multi-language toolkit for the system usability scale. Int J Hum–Comput Interact 2020; 36: 1883–1901.

37.

Bangor

Kortum

Miller

. Determining what individual SUS scores mean: adding an adjective rating scale. J Usability Stud 2009; 4: 114–123.

38.

Hart

Staveland

. Development of NASA-TLX (task load index): results of empirical and theoretical research. Adv Psychol 1988; 52: 139–183.

39.

Said

Gozdzik

Roche

, et al. Validation of the raw national aeronautics and space administration task load index (NASA-TLX) questionnaire to assess perceived workload in patient monitoring tasks: pooled analysis study using mixed models. J Med Internet Res 2020; 22: e19472. PMID: 32780712; PMCID: PMC7506540.

40.

Davis

. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q 1989; 13: 319–340.

41.

Venkatesh

Davis

. A theoretical extension of the technology acceptance model: four longitudinal field studies. Manage Sci 2000; 46: 186–204.

42.

Venkatesh

Bala

. Technology acceptance model 3 and a research agenda on interventions. Decis Sci 2008; 39: 273–315.

43.

NASA-TLX. Permission description of NASA TLX tool, https://humansystems.arc.nasa.gov/groups/tlx/tlxpaperpencil.php (retrieved 2023).

44.

Akoglu

. User's guide to correlation coefficients. Turk J Emerg Med 2018; 18: 91–93.

45.

Chan

. Biostatistics 104: correlational analysis. Singapore Med J 2003; 44: 614–619.

46.

Latreche

Kelaiaia

Chemori

, et al. Reliability and validity analysis of MediaPipe-based measurement system for some human rehabilitation motions. Measurement (Mahwah N J) 2023;214:112826. ISSN 0263-2241.

47.

Collin

Davidson

Kirkko-Jaakkola

Inertial sensors and their applications. In: Bhattacharyya

Deprettere

Leupers

, et al. (eds) Handbook of signal processing systems. Cham: Springer, 2019. DOI: 10.1007/978-3-319-91734-4_2.

48.

Walshe

Roche

RAP

Ward

, et al. Comparable walking gait performance during executive and non-executive cognitive dual-tasks in chronic stroke: a pilot study. Gait Posture 2019; 71: 181–185.

49.

Goswami

Dutta

. Gender differences in technology usage—a literature review. Open J Bus Manag 2016; 4: 51–59.

50.

Morris

Venkatesh

Ackerman

. Gender and age differences in employee decisions about new technology: an extension to the theory of planned behavior. IEEE Trans Eng Manage 2005; 52: 69–84.

51.

Chung

Ong

Leow

. Comparative analysis of skeleton-based human pose estimation. Future Internet 2022; 14: 380.

52.

Bazarevsky

Grishchenko

Raveendran

, et al. Blazepose: on-device real-time body pose tracking. arXiv preprint 2020; arXiv:2006.10204.

53.

MediaPipe Pose. Pose landmark detection guide, https://developers.google.com/mediapipe/solutions/vision/pose_landmarker/index (retrieved 2023).

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB