Effectiveness,usability,and patient satisfaction of an mHealth application with an integrated ePRO system following lumbar degenerative spinal surgery: A quasi-experimental study

Abstract

Introduction

There is a lack of comprehensive clinical research to assess potential benefits of mHealth solutions in post discharge follow-up care after spinal surgery.

Purpose

This quasi-experimental study evaluated the effectiveness, usability, and patient satisfaction of an mHealth pathway with an electronic Patient-Reported Outcome (ePRO)-based post-discharge nurse-led intervention for patients undergoing surgery for lumbar spine degenerative disorders, compared to standard care.

Methods

Conducted at a Danish tertiary spine center, this study represents the final stage of a three-phase participatory design. The primary outcome was patient quality of recovery, measured by the Quality of Recovery-15 (QoR-15) questionnaire. Secondary outcomes included patient-perceived usability, assessed with the Danish System Usability Scale (SUS). To capture additional patient insights, an open-ended feedback question was included at the end of the survey.

Results

Data from 150 patients (77 women and 73 men) were analyzed, with 104 in the intervention group and 46 in the comparison group. Both groups showed significant improvement over time, but no significant difference between groups. Of 154 potential SUS respondents, 110 participated. Analysis revealed 48 statements, categorized into five themes: (1) Usability and functionality of the mHealth solution, (2) Feedback on the QoR-15 questionnaire, (3) Safety and support, (4) Missing functions and suggestions for improvements, and (5) Patient satisfaction.

Conclusion

No significant differences in effectiveness between the mHealth pathway and standard care were found for post-surgery recovery in lumbar spine patients. Usability and patient satisfaction were generally high, though suggestions for improvements were noted. The study highlighted the importance of post-discharge support, suggesting that the intervention provided crucial security and support, potentially demonstrating compassion through nurse-led care.

Keywords

mHealth degenerative lumbar spine surgery electronic patient-reported outcome quasi-experimental design post-discharge recovery after surgery

Introduction

Surgery can be an option for patients with severe, debilitating conditions such as lumbar disc herniation, stenosis, and spondylolisthesis who have not benefited from conservative treatment.¹ An average of 6000 lumbar operations are performed and registered annually in the Danish spine surgery registry (DaneSpine).² Although programs to improve patient recovery in the immediate post-operative period while the patient is still in the hospital have been implemented,^3–9 programs to enhance the patient's recovery after discharge, when they are at home, have not been as extensively developed as needed.^10,11

Utilizing a mobile health (mHealth) application to support the patient at home is feasible and enhances satisfaction for patients and healthcare providers.¹² It can potentially help in early complication detection, reduce readmissions, and improve adherence to postoperative instructions, ultimately optimizing recovery quality and potentially lowering healthcare costs.¹³ However, there is a need for further clinical research studies to quantitatively and qualitatively evaluate mHealth solutions for post-discharge follow-up.¹² The ‘My Hospital’ app¹⁴ serves patients in the Region of Southern Denmark, providing access to structured plans outlining the sequence of healthcare information and interventions (care pathways) for various medical needs, including surgeries. Currently, it has 40,000 active users engaged in 375 pathways. The app intends to foster improved communication and engagement between healthcare providers and their patients.¹⁴

In cooperation with relevant stakeholders, we developed a digital pathway in the ‘My Hospital’ app tailored to patients who had surgery for degenerative conditions in the lumbar spine. Details of the developed pathway can be found in the Content of the mHealth pathway section. A tailored workflow was also designed to ensure seamless integration of the app and its functionalities into clinical practice.

Objective

To conduct an exploratory assessment of the effectiveness, usability, and patient satisfaction with an mHealth pathway integrating an electronic Patient-Reported Outcome (ePRO)-based post-discharge nurse-led intervention designed specifically for patients who underwent surgery for degenerative disorders in the lumbar spine, in comparison to standard care.

Methods

Reporting follows the Transparent Reporting of Evaluations with Non-randomized Designs (TREND) statement guidelines.¹⁵

Study design

This study is a quasi-experimental design using a cohort with pre- and post-test comparisons.¹⁶ The study is the last stage of a three-phase participatory study design (Figure 1) aimed to optimize the post-discharge patient pathway following surgery for degenerative disorders in the lumbar spine. This last phase focuses on testing and evaluating a technological solution and tailored organization^17,18 for addressing the needs of patients during the early post-discharge phase of their recovery.

Figure 1.

Participatory design process.

Setting and participants

The study was conducted at a tertiary spine center, within a medium-sized hospital in Denmark. Patients were enrolled consecutively through purposive sampling¹⁹ by nurses during their pre-admission visit at the spine surgical outpatient clinic. Patients aged 18 and above, scheduled for surgery due to degenerative disorders in the lumbar spine, including discectomies, spinal decompressions, and spinal fusions, were eligible for inclusion. Patients were excluded if they were unable or declined to utilize the mHealth solution or complete digital Danish questionnaires.

Intervention

The intervention under study was an mHealth pathway “Back Surgery” specifically designed for patients who had surgery for degenerative disorders in the lumbar spine including a nurse-led post-discharge care intervention based on ePRO data. The proposal for the intervention was informed by prior studies that identified patients’ needs and wishes during the early discharge period at home after spine surgery and further refined through workshops, a widely employed method within participatory design studies.^18,20–23 Head nurses and appointed key stakeholders, in both the outpatient clinic and the surgical ward, played crucial roles, serving as intermediaries between the research team and the mHealth users, ensuring successful implementation and integration.

Content of the mHealth pathway

The mHealth pathway “Back Surgery” (Table 1), included the menu items: “How to use the app,” “Functions in the app,” “Before surgery,” “After surgery,” “Mostly for relatives,” and “Contact,” which are available to all individuals and do not require the patient to be registered or affiliated to the spinal surgery department. Significant attention has been given to the development of the “After surgery” section within the mHealth pathway. The app included menu items such as “My appointments,” “My materials,” “Messages,” “My data,” and “Questionnaire.” Access to these functions required the patients to register with the spine surgical department. Under “My appointments,” patients could view their surgical date as well as scheduled appointments across all hospital departments, allowing patients to view appointments related to other treatment pathways.

Table 1.

Content structure of the mHealth pathway.

How to use the app

II.

Functions in the app a.

Invite your relative

III.

Before surgery a.

Acute warning signs before surgery

Information concerning your surgery

IV.

After surgery a.

Knowledge and expectations i.

The body's reaction 1.

Fatigue a.

Fatigue and exhaustion i.

Pain

ii.

Side effects of medication

iii.

Nervousness and worries

iv.

Your healing body

Good advice

Pain a.

Back pain

Leg pain / nerve pain

Pain relief i.

Pain-relieving medication

Mental reactions a.

Worries

Loneliness and boredom

Reduced appetite and nausea a.

What to do

Antiemetics

ii.

Acute warning signs

iii.

Activity 1.

Recommendations a.

General recommendations

Herniated disc and spinal stenosis

Lumbar fusions

Everyday activities a.

Gradually resume usual activities

Balancing activity and rest

Walking and cycling a.

Walking

Cycling

Exercise routines a.

Exercises after hospital discharge

Training program i.

Exercises, descriptions, photos, and videos

Pain-relieving stretches a.

Stretching exercises i.

Stretching exercises, descriptions, photos, and videos

Other pain-relieving strategies

Pain treatment i.

Medicine 1.

Who is responsible for your medicine?

Medicine card

Opioid tapering plan

Information concerning your medicine

Prescription renewal

ii.

Opioid tapering 1.

Opioid tapering a.

Gradually tapering

Discomfort associated with opioid tapering a.

Withdrawal symptoms

Increasing pain a.

Recommendations

Wounds and dressing i.

How to care for wounds 1.

Wound care

Sticking plaster

The sticking plaster is waterproof

Bathing and swimming

Absorbable sutures

Avoid the sun

Bowel problems i.

Constipation 1.

Opioid-induced constipation

Constipation symptoms

How to prevent or alleviate constipation

Laxatives

Food and drink i.

Food and drink 1.

Dietary advice related to your surgery a.

Link to a brochure with dietary advice

V. Mostly for relatives a.

Relatives and My Hospital

How to take care of your loved ones

VI.

My appointments

VII.

My materials

VIII.

Messages

IX.

My data

X. Questionnaire

XI.

Contact

Under “My materials,” patients could access relevant information tailored to their needs. During their pre-admission visit, nurses distributed pertinent digital brochures concerning the specific surgical procedure, anesthesia, and relevant medication information. Nurses from the ward regularly added relevant articles if patients were started on a new medication and removed articles from the menu if a medication was discontinued. “Messages” allowed patients to send and receive messages to and from the department, targeting specific recipients such as medical secretaries, nurses before or after surgery, or physiotherapists. The patients also received information on available questionnaires and messages from the ward nurses through the messaging module. This feature included the possibility to submit photos. Via the “My data” function patients could share information with the healthcare professionals (HCPs) regarding blood pressure measurements.

In particular to this study, an electronic patient-reported outcome (ePRO) system²⁴ using the Danish version of the Quality of Recovery-15 (QoR-15),²⁵ Appendix A, was included in the menu item “Questionnaire.” The patients’ responses provided the hospital ward nurses with access to identify patients who required professional instructions and guidance to self-manage their symptoms. Finally, the app included a notification feature, which required patients to opt in, allowing them to receive reminders for new messages or available questionnaires.

Patients, included in the intervention group, were introduced to and registered as users of the mHealth solution during their pre-admission visit with the nurse in the outpatient clinic on the day they were scheduled for surgery. Once registered, patients had access to all of the menu items. The patients were informed about the timing of the questionnaires in the app. The surgical ward nurses repeated this information at the time of hospital discharge.

Nurse-led post-discharge care intervention based on ePRO data

To categorize patients’ ePRO responses, the severity classification of the QoR-15 ²⁶ was considered. Within the clinician interface of the mHealth app, each response was automatically assigned a color indicating whether the individual patient's QoR-15 total score ≥122, corresponding to “good or excellent recovery” (green), 96–121 indicating “moderate recovery” (yellow), or ≤95 indicating “poor recovery” (red).

Responses assigned to the “good or excellent” recovery category were automatically archived, and nurses were not required to take further action. In cases indicating “moderate” or “poor” recovery, a notification appeared on the clinician interface, calling for the ward nurses’ attention. Subsequently, the responses were reviewed and assessed by a nurse. In line with our findings from previous research,²⁷ the intervention particularly focused on supporting patients with poor sleep and experienced pain. Thus, responses with item scores for Items 4 (Have had a good sleep), 11 (Moderate pain), or 12 (Severe pain) < 5, triggered a message from the nurse notifying and informing the patients that they would be contacted on the same day.

Patients who had prior interactions with HCPs post-discharge, and had an established supplemental recovery plan, did not receive this message and accompanying telephone call. Instead, they received the same message as the other patients, who were stratified as either yellow or red, but who had an item score of ≥5 in items 4, 11, or 12. In these cases, a message from a nurse urged them to contact the department if they felt that they needed advice or guidance from a nurse or physiotherapist. Regardless of whether a phone call was initiated by patients or HCPs, the HCPs provided a care intervention consisting of support, guidance, and counseling based on the individual situation.

Comparison group

The comparison group, derived from a previous study,²⁷ underwent assessment using the QoR-15 scale and received standard care during the post-discharge recovery period. Standard care involved paper-based informational material concerning the specific surgical procedure, anesthesia, and relevant medication information. During discharge, patients were invited to contact the hospital ward's nursing staff or physiotherapists by telephone, allowing participants to seek guidance and advice from the inpatient unit as needed. Thus, post-discharge contact between the patient and the HCPs occurred solely at the patients’ initiative. Data were collected, using an mHealth application. However, this technology exclusively incorporated questionnaires without any additional features, and data were simply recorded and utilized for statistical analysis.

Data collection and data sources

A flow diagram of the QoR-15 data collection is shown in Figure 2. Data collection occurred at two distinct time points in this study. Initially, data from the comparison group were gathered between December 2021 and September 2022, predating the design and development of the intervention. Following this phase, the intervention was implemented and tested, with data collected from the intervention group spanning from August 2023 to March 2024. This sequential approach ensured that data from the comparison group represented a baseline before the introduction of the intervention, while data from the intervention group were collected during the implementation and evaluation of the intervention itself. Patient demographics and perioperative information were obtained from DaneSpine, the Danish database of spine surgery.^2,28,29 Frequency and distribution of postoperative contacts in intervention and comparison groups was determined using a journal audit.

Figure 2.

Flow diagram of the QoR-15 data collection.

Primary outcome measurements

The primary outcome measure of this study was the assessment of patient quality of recovery over time using the validated Danish version of the QoR-15 questionnaire,²⁵ Appendix A. Each of the 15 items of the questionnaire was answered on an 11-point numerical rating scale from 0 to 10. The total score ranges from 0 to 150, with a higher score indicating a better quality of recovery.²⁵ We performed measurements using the QoR-15 at four separate time points (see Figure 3). The questionnaire was sent out to the patients 4 days before surgery (baseline) and on day 1 (D1), day 7 (D7) and day 14 (D14) after discharge. A digital reminder was sent if the patients had not answered the primary questionnaire within 24 h.

Figure 3.

Timeline of patient pathways and data collection in the comparison group and intervention group.

Sample size

The sample size was calculated to detect a difference in the improvement of QoR-15 scores between the interventional and comparison groups over time. Specifically, the aim was to detect a difference between groups in the change from pre-operative to post-operative QoR-15 scores. The sample size was estimated using the formula for a comparative clinical interventional study.³⁰ The Minimal Clinically Important Difference (MCID) for the QoR-15 score was set at 8.0,³¹ and the average standard deviation (SD) of the pre-operative scores was approximately ±18.3.²⁷ Consequently, for this study, a minimum of 82 participants, 41 in each cohort was required.

S a m p l e s i z e = \frac{2 S D^{2} {(\frac{Z α}{2} + Z β)}^{2}}{d^{2}}

S a m p l e s i z e = \frac{2 \times {18.3}^{2} {(\frac{Z_{0.05}}{2} + Z_{0.20})}^{2}}{8^{2}}

S a m p l e s i z e = \frac{2 \times {18.3}^{2} {(1.96 + 0.84)}^{2}}{8^{2}} \approx 82

Secondary outcomes

Secondary outcome measures included the evaluation of patient-perceived usability utilizing the validated Danish version of the System Usability Scale (SUS),³² Appendix B in the online supplementary materials.

The original SUS instrument³³ is designed to assess the usability of a system, product, or service. It is composed of 10 statements related to the user's perception of the system's usability, such as ease of use and user satisfaction, scored on a 5-point Likert scale ranging from “strongly agree” to “strongly disagree.” The final scores for the SUS range from 0 to 100, where higher scores indicate better usability. A total score of 68 corresponds to an average experience, while a score of 80 corresponds to a good experience.^34,35

In this study, the SUS survey was converted into an electronic format, compatible with the Survey Xact software package, by the authors [Marianne Dyrby Lorenzen and Casper Friis Pedersen]. We utilized a secure email platform that allowed recipients to read and respond via the website or the platform's application. The SUS questionnaire was distributed to those patients who completed the baseline QoR-15 questionnaire.

A general open-ended and non-directive feedback question³⁶ was integrated into the survey's conclusion to supplement the closed questions within the SUS to uncover any issues not addressed within the survey itself and to empower the participants by giving them a voice, which is in line with the participatory design methodology.¹⁷

Data were exclusively collected from the intervention group, and the survey was distributed to each participant within two weeks after the finishing of the individual series of QoR-15 questionnaires (Figure 3).

The necessary permissions were obtained from the respective copyright holders for the use of the Danish versions of the QoR-15 and SUS questionnaires in this study.

Statistical analysis

We used IBM SPSS Statistics for Windows (Version 28.0, IBM Corp., Armonk, NY, USA) to analyze the data. Unless otherwise stated, results are presented as means ± standard deviations (SD) in both text and graphs. A p-value of less than 0.05 was considered statistically significant.

Frequency and distribution of postoperative contacts in intervention and comparison groups

A chi-squared test for independence was performed to determine whether the total count of phone calls significantly differed from the expected counts based on group size.

Repeated measures of quality of recovery (QoR-15) and heatmap analysis

To assess changes in QoR-15 sum scores over time, a within-subjects and between-subjects longitudinal repeated measures analysis was employed. Estimated marginal means for each measurement time point were calculated and displayed in a profile plot with 95% confidence interval error bars. Pairwise comparisons were used to evaluate the significance of changes over time.

An independent samples t-test was conducted to compare the change in QoR-15 (Delta QoR-15) between the control group and the intervention group. Cohen's d was applied to assess the effect size.

We created and compared heatmaps of the QoR-15 item scores to identify patterns, differences, or similarities between the data from the intervention group and the comparison group, respectively. This assisted us in visualizing if any different factors affected the QoR-15 sum score between the two groups. This two-dimensional visualization combines numeric values with color to represent their magnitude. Red (RGB: 248, 105, 107) indicates a value of 0, yellow (RGB: 255, 235, 132) represents a median value of 5, and green (RGB: 99, 190, 123) indicates a value of 10. Variations between 0 and 10 are reflected by both the hue and intensity of the color.

System usability scale

The responses of the SUS were converted to SUS scores following the guidelines set by Digital Equipment Corporation.³³ The average SUS score (mean), its variability (standard deviation), and distribution across the population (percentiles) were reported. Additionally, a histogram was created to visually represent the spread of SUS scores.

Analysis of the feedback comments

The feedback comments were analyzed through a quantitative assessment and qualitatively using an inductive coding strategy and content analysis inspired by Decorte et al.³⁶ Two independent reviewers [MDL] and [LN] analyzed the comments individually, allowing for diverse perspectives and minimizing bias, and coded the comments using a line-by-line technique. The reviewers compared their findings and reached a consensus on the identified themes and codes through a thorough academic discussion. The comments were translated from Danish to English after the data analysis. All quotations were included in Appendix C to ensure transparency and rigor.

Ethical considerations

The study complies with the ethical standards from the Declaration of Helsinki.³⁷ All participants received written and oral information and gave their consent before participation. The processing of personal data was submitted to and approved by the Region of Southern Denmark and listed in the internal record (File no. 21/32183). Ethical approval was obtained from the local institutional review board. The study was presented to The Regional Committees on Health Research Ethics for Southern Denmark but was deemed to be exempt from review (File no. S-20212000-95 and Acadre 21/209).

Results

Baseline characteristics of the participants

A flowchart of the data collection is presented in Figure 2. Two hundred and twenty-two of the possible 259 patients (86%) completed the pre-operative QoR-15 baseline questionnaire. The analysis of the QoR-15 repeated measures included data from a total of 150 patients, consisting of 77 women and 73 men, with 104 in the intervention group and 46 patients in the comparison group, who completed the QoR-15 score at baseline and at all three post-discharge time points. Baseline demographics are presented in Table 2, indicating that the two groups, despite the difference in the number of participants, are comparable across all mentioned parameters, including age, gender distribution, smoking status, BMI, levels of surgery, duration of symptoms, and patient-reported outcomes that account for patients’ self-assessed of leg and back pain, level of function, and quality of life.

Table 2.

Baseline demographics for the QR-15 participants.

Characteristics	Intervention group	Control group	Dif.	p
Number of patients, n	104	46
Age, years, mean (SD)	54.1 (15.4)	50.1 (11.3)	4.0	.101^a
Gender, females, n (%)	57 (54.8)	20 (43.5)	11.3	.201^b
Smoker, n (%)	14 (13.5)	7 (15.2)	1.7	.775^b
BMI, mean (SD)	28.1 (4.2)	27.9 (3.7)	0.2	.867^a
Previously operated, n (%)	21 (20.2)	8 (17.4)	2.8	.689^b
Surgery at no. levels: one level, n (%)	90 (88.2)	39 (84.8)	3.3
Two levels, n (%)	11 (10.8)	6 (13.0)	2.2	.551^c
Three levels, n (%)	1 (1.0)	1 (2.2)	1.2
Leg pain duration: no pain, n (%)	1 (1.0)	1 (2.2)	1.2
<3 months, n (%)	8 (7.7)	8 (17.4)	9.7
3–12 months, n (%)	52 (50.0)	21 (45.7)	4.3	.159^c
1–2 years, n (%)	21 (20.2)	9 (19.6)	0.6
>2 years, n (%)	22 (21.2)	7 (15.2)	6.0
Back pain duration: no pain, n (%)	5 (4.8)	3 (6.5)	1.7
<3 months, n (%)	9 (8.7)	5 (10.9)	2.2
3–12 months, n (%)	35 (33.7)	15 (32.6)	1.1	.723^c
1–2 years, n (%)	21 (20.2)	8 (17.4)	2.8
>2 years, n (%)	34 (32.7)	15 (32.6)	0.1
Functional impairment (ODI), mean (SD)	41.9 (14.7)	38.1 (14.6)	3.8	.141^a
VAS: Leg pain, mean (SD)	63.6 (22.3)	59.8 (27.3)	3.8	.749^a
Back pain, mean (SD)	51.3 (26.2)	49.3 (29.4)	2.0	.718^a
Quality of life (EQ-5D-3L), mean (SD)	0.454 (0.284)	0.474 (0.304)	0.02	.416^c
SF-36: Physical Component Score, mean (SD)	31.1 (5.9)	33.0 (7.4)	1.9	.091^a
Mental Component Score, mean (SD)	43.6 (11.3)	48.1 (12.2)	4.5	.038^a
Complications: Dural tears, n (%)	3.0 (2.8)	2.0 (4.5)	1.0	.642^a
Nerve root injury, n (%)	0.0 (0.0)	0.0 (0.0)	0.0	1.000^a
QoR-15 sum score: Preoperative, mean (SD)	100.9 (19.8)	105.5 (18.3)	4.6	.186^a
Day 1 mean (SD)	106.2 (17.9)	108.1 (19.2)	1.9	.717^a
Day 7, mean (SD)	119.7 (17.2)	118.5 (17.4)	1.2	.682^a
Day 14, mean (SD)	121.1 (20.5)	120.7 (20.9)	0.4	.961^a

Note. ODI: Oswestry Disability Index; SD: standard deviation; SF-36: 36-Item Short Form Survey; VAS: Visual Analogue Scale.

Independent t-test.

Chi-squared test.

Mann–Whitney U test.

Baseline comparisons between QoR-15 completers and non-completers within the intervention group (see Table 3) indicate no significant differences, with the exception of the duration of back pain and EQ-5D-3L scores. Non-completers did not report higher levels of pain, functional impairment, or poorer scores on the SF-36 mental component. However, a higher proportion (13.2%) of smokers was observed among non-completers.

Table 3.

Baseline comparisons for QoR-15 completers and non-completers in the intervention group

Characteristics	Responders	Non-responders	Dif.	p
Number of patients, (n)	104	50
Age, years, mean (SD)	54.1 (15.4)	53.9 (15.8)	0.2	.879^a
Gender, females, n (%)	57 (54.8)	20 (44.0)	10.8	.209^b
Smoker, n (%)	14 (13.5)	13 (26.0)	13.2	.060^b
BMI, mean (SD)	28.1 (4.2)	28.3 (4.7)	0.2	.757^a
Previously operated, n (%)	21 (20.2)	6 (12.0)	8.2	.211^b
Surgery at number of levels: one level, n (%)	90 (88.2)	41 (82.0)	6.2
Two levels, n (%)	11 (10.8)	9 (18.0)	7.2	.374^c
Three levels, n (%)	1 (1.0)	0 (0.0)	1.0
Leg pain duration: no pain, n (%)	1 (1.0)	2 (4.0)	3.0
<3 months, n (%)	8 (7.7)	2 (4.0)	3.7
3–12 months, n (%)	52 (50.0)	31 (62.0)	12.0	.303^c
1–2 years, n (%)	21 (20.2)	9 (18.0)	2.2
>2 years, n (%)	22 (21.2)	6 (12.0)	9.2
Back pain duration: no pain, n (%)	5 (4.8)	0 (0.0)	4.8
<3 months, n (%)	9 (8.7)	0 (0.0)	8.7
3–12 months, n (%)	35 (33.7)	27 (54.0)	20.3	.027^c
1–2 years, n (%)	21 (20.2)	7 (14.0)	6.2
>2 years, n (%)	34 (32.7)	16 (32.0)	0.7
Functional impairment (ODI), mean (SD)	41.9 (14.7)	44.4 (13.8)	2.5	.324^a
VAS: leg pain, mean (SD)	63.6 (22.3)	64.9 (21.7)	1.3	.736^a
Back pain, mean (SD)	51.3 (26.2)	59.2 (22.8)	7.9	.072^a
Quality of life (EQ-5D-3L), mean (SD)	0.454 (0.284)	0.329 (0.321)	0.125	.016^c
SF-36: Physical Component Score, mean (SD)	31.1 (5.9)	31.0 (6.7)	0.1	.938^a
Mental Component Score, mean (SD)	43.6 (11.3)	40.9 (12.9)	2.7	.210^a
QoR-15 sum score: Preoperative, mean (SD)	100.9 (19.8)	103.5 (19.0)	2.5	.563^a

Note. ODI: Oswestry Disability Index; SD: standard deviation; SF-36: 36-Item Short Form Survey; VAS: Visual Analogue Scale.

Independent t-test

Chi-squared test

Mann–Whitney U test

Frequency and distribution of postoperative contacts in intervention and comparison groups

In the intervention group, 50 out of 104 patients (48.08%) had postoperative contact with the clinical nurses, resulting in a total of 81 phone calls, with an average of 0.779 calls per patient. Of these, 53 calls (65.4%) were initiated by the clinical nurses as part of the intervention, while 28 calls (34.57%) were made at the patients’ initiative. Notably, nine of the nurse-initiated calls (16.98%) were classified as unnecessary, as the patients did not require advice or guidance. The number of calls per patient ranged from 0 to 4. Conversely, in the comparison group, participants contacted the nurses by phone a total of 28 times after discharge, with the number of calls per participant ranging from 0 to 5, averaging 0.687 calls per participant. This represents a 27.91% higher average number of calls per patient in the intervention group compared to the comparison group. However, this difference was not statistically significant, as indicated by a p-value of .08, based on the results of a chi-squared test for independence.

Completion rate of the QoR-15 questionnaires

The QoR-15 score was assessed at all four time points in 150 of 222 possible participants. This corresponds to a completion rate of 67.6%.

Repeated measures of quality of recovery (QoR-15)

There was a significant improvement across time points in both groups (p < .001), but no differences between groups (p = 0.442) (Table 4 and Figure 4). Specifically, the preoperative QoR-15 mean sum scores were 100.9 (SD = 19.8) for the intervention group and 105.5 (SD = 18.3) for the comparison group. On Day 1, the intervention group had a mean score of 106.2 (SD = 17.9) compared to 108.1 (SD = 19.2) for the comparison group, reflecting a difference of 1.9. By Day 7, the mean QoR-15 sum score for the intervention group was 119.7 (SD = 17.2), while the comparison group had a mean score of 118.5 (SD = 17.4), showing a difference of 1.2. On Day 14, the intervention group's mean score was 121.1 (SD = 20.5) and the comparison group's mean score was 120.7 (SD = 20.9), resulting in a difference of 0.4.

Figure 4.

Repeated measures of quality of recovery (QoR-15). Cohort 1: Comparison group, Cohort 2: Intervention group. Time 1: Preoperative baseline, Time 2: Day 1, Time 3: Day 7, Time 4: Day 14.

Table 4.

Outcomes, repeated measures QoR-15

	Intervention group (n = 104)	Control group (n = 46)	Dif.	p
QoR-15 sum score
Preoperative, mean (SD)	100.9 (19.8)	105.5 (18.3)	4.6	.442^a
Day 1 mean (SD)	106.2 (17.9)	108.1 (19.2)	1.9
Day 7, mean (SD)	119.7 (17.2)	118.5 (17.4)	1.2
Day 14, mean (SD)	121.1 (20.5)	120.7 (20.9)	0.4

Note. SD: Standard deviation.

Repeated measures multivariate test.

The intervention group had a mean QoR-15 change of 19.99 (SD = 22.68), while the comparison group had a mean change of 14.93 (SD = 19.18). The t-test for equality of means showed no statistically significant difference between the two groups (p = .190). Additionally, effect sizes were computed, yielding a Cohen's d of 0.233 (95% CI: −0.115 to 0.581), indicating a small effect size.

Heatmap analysis

The comparison of the heatmaps between the two groups (Figure 5) revealed no discernible differences, indicating similarity in item scores and distributions across the two groups.

Figure 5.

Comparison of QoR-15 item heatmaps.

System usability scale

A total of 110 out of 154 possible respondents (71.4%) participated in the SUS survey. Seventy-two patients (65.5%) were included in the QoR-15 repeated measures analysis and the remaining 38 patients (34.5%) were among the participants who were lost to follow-up.

The distribution of the SUS responses is presented in Figure 6 and the descriptive results are presented in Table 5. The SUS scores ranged from 47.5 to 100. The mean SUS score was 84.682 (SD 12.86). Based on the results, 25% of participants scored below 76.9, 50% scored below 87.5, and 75% scored below 96.6.

Figure 6.

Distribution of system usability scale responses.

Table 5.

System usability scale

N	Valid	110
	missing	0
Mean		84.682
Median		87.500
Mode		100.0
SD		12.8600
Minimum		47.5
Maximum		100.0
Percentiles	25	76.875
	50	87.500
	75	95.625

Findings from the patients’ feedback comments

Among the 110 respondents in the SUS survey, 39 patients, approximately 35% of the sample, provided feedback comments. However, three of these comments were deemed incomprehensible and were excluded from the analysis. The length of the comments varied from 2 to 91 words before translation into English. The analysis identified 48 statements, which were classified into five themes; (1) Usability and functionality of the mHealth solution, (2) Feedback on the QoR-15 questionnaire, (3) Safety and sense of support, (4) Missing functions and suggestions for improvements, and (5) Patient satisfaction, Appendix C.

Usability and functionality of the mHealth app

The participants expressed satisfaction with the mHealth app's performance, emphasizing qualities and conveniences. For instance, one participant stated: “The app was very clear and easy to use,⁵” while another appreciated the app's centralized resources, noting: “Nice that pamphlets are gathered in one place, and my arthritis treatment is on the same app.”⁴

The participants also commended the app's multimedia features, such as instructional videos, with one saying: “I also used the app when doing exercises, and here it was nice to have the opportunity to watch them on video.”¹⁴ However, some participants encountered issues, such as errors in certain tabs: “Several tabs had errors. Or it said that an error had occurred.”²⁷ Despite these challenges, users valued the app's guidance and communication channels, as indicated by one participant's feedback: “Very good idea and a quick and easy way to seek answers and communicate.”³¹ Overall, the feedback underscores the app's effectiveness in providing clear guidance, facilitating communication with HCPs, and offering quick access to relevant information, which contributed positively to the overall user experience.

Feedback on the QoR-15 questionnaire

Some of the feedback comments concerned the QoR-15 questionnaire, and these comments underscored several critical issues. Some of the participants were confused about the question formulation, with one commenting: “It was a bit confusing that some questions are formulated as ‘negative’… and some as positive.”⁷ Furthermore, the participants suggested a text field to elaborate on their experiences of symptoms. As one participant stated, “The questionnaire… could have a text field where one could explain their checkbox selections.”⁸ Ambiguity in questions led to inaccurate responses, as another participant noted, “Some questions were in a bit of a gray area… I answered incorrectly because it was just the opposite of the questionnaire, you sent previously.”¹⁰ Moreover, participants highlighted the questionnaire's lack of specificity and comprehensiveness, noting its inability to capture the full spectrum of postoperative challenges.

Safety and sense of support

The participants’ comments indicated the significance of feeling safe and supported post-discharge, with the mHealth solution playing a central role. As one of the patients expressed: “I felt safe and in good hands after my operation by staying in touch with ‘My Hospital.’”¹ This highlights the reassurance gained from staying connected to healthcare providers by using the app. Moreover, the expression, “It provides safety, and that is invaluable when you just had surgery”³ captures the importance of feeling safe during the post-discharge recovery period. The ability of the app to offer constant monitoring and support is further emphasized by comments such as, “I felt monitored and assessed via frequent questionnaires, which resulted in safety after discharge.”¹⁴ Additionally, the tailored nature of the app, specifically designed for “Back Surgery,” is emphasized for its relevance and impact: “It is reassuring that it is relevant to exactly my operation.”²⁶ This tailored approach extends to the training program is effective and beneficial, which was noted as: “The training program for use after discharge was good.”²⁶

Missing functions and suggestions for improvements

The participants identified various critical areas that call for enhancement within the mHealth app. Specifically, for patients who preferred to use the mHealth solution via their computer instead of a smartphone or tablet, the system lacked email notifications: “For those of us who are more computer-minded than app-minded, it would be nice to receive notifications by email.”⁷

Participants encountered functions within the app that proved difficult to utilize or operate correctly, despite its availability, such as the inability to submit their medical data like blood pressure measurements: “Just before my operation, I had to submit blood pressure measurements, which was not possible to submit via the app, so I had to call in the measurements to you.”¹² In addition, one expressed frustration over missing notifications, leading to incomplete engagement with app features like questionnaires: “I missed getting notifications when there was something new in the app, such as questionnaires after different numbers of days. This meant that I did not answer some of them.”¹⁰

Several specific suggestions for improvement were proposed, including integrating artificial intelligence into rehabilitation planning to tailor programs to individual needs effectively: “It could be a good thing if it became more visual, interactive, and personal, e.g., by utilizing AI along with data about me to plan the rehabilitation.”²⁹

Furthermore, the participants suggested a more simple notification setup and also the implementation of a search function within the app, particularly for post-discharge information. This indicates a need for user-friendly customization options and enhanced navigational tools, as emphasized by: “Maybe an easier way to set up notifications, so you get notified when you need to reply to something”³⁰ and “A search function in the app is desirable, especially under ‘After discharge’, so that one can easily find what they are looking for.”³⁶

Patient satisfaction

The feedback comments indicated a high level of patient satisfaction with the mHealth solution and nurse-led intervention. Patients expressed benefits derived from the app, using terms like “greatly benefited,”¹¹ “all in all, a really good experience,”¹⁴ and “absolutely ingenious… high praise.”¹⁵ Even in cases where patients encountered technical challenges, satisfaction remained evident, and the responsive support from healthcare staff enhanced patient trust and satisfaction with the app's functionality: “Apart from the fact that the app has ‘kicked me off’ a couple of times, I have been very satisfied with it and also with the response time from the staff, all in all, a good and easy experience.”¹⁹

Discussion

This study assessed the effectiveness, usability, and patient satisfaction of an mHealth pathway including an ePRO-based post-discharge nurse-led intervention designed specifically for patients who had surgery for degenerative disorders in the lumbar spine compared to standard care.

On the question of effectiveness, this study found that there were no statistically significant differences between the intervention group and the comparison group based on the mean sum score of the repeated measures of quality of recovery (QoR-15). Similarly, the two heatmaps depicting the individual QoR-15 items also revealed no discernible differences, indicating similarity in item score distributions across the two groups.

The feedback comments regarding the QoR-15 questionnaire highlighted significant issues to consider. The most striking finding is that the participants emphasized the questionnaire's lack of specificity and comprehensiveness in capturing the range of postoperative challenges. This finding can contribute to questioning the applicability and reliability of the QoR-15 questionnaire in assessing the post-discharge recovery experience for patients who had surgery for degenerative disorders in the lumbar spine. An alternative to the QoR-15 could be a tailored questionnaire that includes more specific questions covering a wider range of symptoms and experiences related to post-discharge recovery. This was done by Jaensson et al.,³⁸ who developed and psychometrically evaluated a 31-item Swedish Web Version of Quality of Recovery Questionnaire to day surgery patients to become a part of a digital solution called “Recovery Assessment by Phone Points (RAPP).” They incorporated contributions from two existing questionnaires; The Swedish version of QoR-24 ³⁹ and Postoperative Recovery in Children⁴⁰ together with the Postoperative Recovery Profile,⁴¹ and finally a study examining pain assessment in day surgery patients.⁴² Furthermore, Jaensson and her colleagues incorporated a question:

“Do you want to be contacted by a nurse?” making it possible for patients to request a follow-up call within 24 h on weekdays.^38,43 Similarly, Higgins et al.⁴⁴ created a mobile app recovery indicators for patients who had anterior cruciate ligament (ACL) reconstruction using the QoR-9 questionnaire, a visual analog scale for pain, and some surgeon-specific questionnaires.

Regarding usability and patient satisfaction, an important result in this study was the mean score of SUS of approximately 85, indicating an ‘above average’ user experience. According to Lewis et al.,³⁴ a SUS of 80 and above reflects a ‘good experience’ of using the mHealth solution. However, the percentiles from the statistical analysis indicated that there were variations in usability satisfaction levels among the respondents, suggesting the need for further investigation and potential improvements to aim for consistently high satisfaction rates. Many of the feedback comments supported the positive user experience. However, this study also indicates recommendations for a more user-friendly interface, incorporating a search function, and offering a more personalized experience, preferably through AI integration, to guide and strengthen ongoing endeavors to enhance the solution.

Based on the feedback comments the significance of feeling safe and supported post-discharge, facilitated by the mHealth solution, became evident. The quotes underscored how technology, in the form of the “Back Surgery” mHealth pathway and the companying nurse-led intervention, not only provided a sense of security but also seemed to foster a supportive environment crucial for post-discharge recovery. This also prompts us to explore the intersection of compassion within this context. Durkin et al.⁴⁸ emphasize that nurses can express an emotional connection with the patient and use complex communication to build an understanding of the patient's needs which in turn is an expression of compassion. Compassion begins with one person recognizing or noticing another person's suffering, which then motivates them to act and provide assistance. The actions taken are appropriate for the situation and are demonstrated through presence, behavior, and words. This involves a deliberate effort to engage with the suffering of another person to ease their pain.^48,49 The patients’ feedback comments in this current study advocate that the nurse-led intervention in this study based on ePRO data can be used as a way for nurses to demonstrate compassion in response to patients’ suffering during their post-discharge recovery. However, exploring alternative methods to uncover the patients’ poor recovery (suffering) such as tailored questionnaires seems necessary to enhance the assessment of post-discharge recovery experiences more effectively.

Strengths and limitations

Concerning the research method of quasi-experimental design, some limitations need to be acknowledged. The lack of random assignment is the major weakness of this study type, which potentially influenced the internal validity of our study concerning confounding variables, and any events occurring concurrently with intervention.¹⁶ However, this study was strengthened by comprehensive baseline demographic data that showed no significant differences between the two groups despite the groups’ different sample sizes. The generalizability of our findings is limited because this study was performed as a single-center study, which potentially influences the external validity.⁵⁰ In addition, the severity of the disease and the invasiveness of the surgery were similar across both cohorts. Thus, we were unable to evaluate the usability and applicability of the mHealth app in patients with severe disease or those undergoing more complex surgeries. Due to the limited number of participants in our cohorts, it was not possible to conduct meaningful subgroup analysis to assess the effect of the mHealth pathway in relation to disease severity or surgical complexity. It is therefore recommended that future studies with larger populations investigate whether an mHealth pathway offers specific benefits for patients with more complex conditions or those requiring multiple interventions.

In our study, the completion rate for the QoR-15 was 67.6%, corresponding to an attrition rate of 32.4%. The absence of comparable studies with a similar method of data collection and measurement intervals, specifically those including four time points (preoperative and three postoperative), complicates the assessment of whether this attrition rate is acceptable. For comparison, Vlieger et al.⁴⁵ reported a completion rate of 78% with one preoperative baseline and two follow-up assessments by mail, while Kim et al.⁴⁶ achieved a completion rate of 97% with a preoperative and a 24-h in-hospital postoperative measurement. The lower completion rate reflect a common challenge faced by electronic surveys, which generally exhibit lower response rates compared to traditional methods like postal or telephone surveys.⁴⁷ The increased attrition rate in our study may be due to the inclusion of four measurement points, which could have affected participant retention.

In interpreting our findings, it is crucial to consider the temporal context of the COVID-19 pandemic. The comparison group in our study was assessed during a more acute phase of the pandemic, potentially influencing their outcomes compared to the intervention group assessed later. This temporal difference may have affected the observed differences in outcomes between the groups. Therefore, caution is warranted when generalizing these findings to other phases of the pandemic or different healthcare settings.

A strength of our study is its focus on patients’ recovery in the early phase after discharge, highlighting patient-reported outcomes during this critical period. However, the follow-up period was relatively short, which may limit our ability to capture the long-term effects and outcomes of the intervention. Future research should consider incorporating a longer follow-up period to provide a more comprehensive understanding of the intervention's impact over time.

Postoperative follow-up mHealth app solutions have the potential to contribute significantly to health equity by providing accessible and tailored healthcare services.⁵¹ However, we did not ensure that our solution was designed and implemented with a focus on addressing healthcare disparities and meeting the diverse needs of patients of various social groups. This can ultimately result in disparities in access to and utilization of healthcare.⁵² Therefore, issues concerning health equity will be taken under consideration when the solution is further developed.

Additionally, a limitation of this study is the lack of consideration for socio-economic differences among participants. Exploring how socio-economic status influences the effectiveness of the mHealth pathway is crucial. Patients from lower socio-economic backgrounds might derive greater benefits from nurse-initiated assistance via mHealth, compared to more educated patients who may already possess sufficient health literacy. Future research should therefore prioritize stratifying participants based on socio-economic status to discern potentially significant insights into the impact of mHealth interventions across diverse demographic groups. This approach could enhance the applicability and effectiveness of healthcare strategies aimed at improving patient outcomes.

Conclusion

In conclusion, this study found no significant differences in effectiveness between the mHealth pathway and standard care for post-surgery recovery in lumbar spine patients, as assessed by the QoR-15 questionnaire. However, participants’ feedback highlighted shortcomings in questionnaire specificity, suggesting a need for tailored alternatives. Usability and patient satisfaction were generally high, but variation suggestions for improvements. The study underscored the importance of feeling supported post-discharge, indicating that the intervention facilitated a sense of security and support crucial for recovery, potentially embodying compassion through nurse-led interventions.

Supplemental Material

sj-docx-1-dhj-10.1177_20552076251324687 - Supplemental material for Effectiveness, usability, and patient satisfaction of an mHealth application with an integrated ePRO system following lumbar degenerative spinal surgery: A quasi-experimental study

Supplemental material, sj-docx-1-dhj-10.1177_20552076251324687 for Effectiveness, usability, and patient satisfaction of an mHealth application with an integrated ePRO system following lumbar degenerative spinal surgery: A quasi-experimental study by Marianne Dyrby Lorenzen, Casper Friis Pedersen, Line Nielsen, Mikkel O. Andersen, Jane Clemensen and Leah Y. Carreon in DIGITAL HEALTH

Supplemental Material

sj-pdf-2-dhj-10.1177_20552076251324687 - Supplemental material for Effectiveness, usability, and patient satisfaction of an mHealth application with an integrated ePRO system following lumbar degenerative spinal surgery: A quasi-experimental study

Supplemental material, sj-pdf-2-dhj-10.1177_20552076251324687 for Effectiveness, usability, and patient satisfaction of an mHealth application with an integrated ePRO system following lumbar degenerative spinal surgery: A quasi-experimental study by Marianne Dyrby Lorenzen, Casper Friis Pedersen, Line Nielsen, Mikkel O. Andersen, Jane Clemensen and Leah Y. Carreon in DIGITAL HEALTH

Supplemental Material

sj-docx-3-dhj-10.1177_20552076251324687 - Supplemental material for Effectiveness, usability, and patient satisfaction of an mHealth application with an integrated ePRO system following lumbar degenerative spinal surgery: A quasi-experimental study

Supplemental material, sj-docx-3-dhj-10.1177_20552076251324687 for Effectiveness, usability, and patient satisfaction of an mHealth application with an integrated ePRO system following lumbar degenerative spinal surgery: A quasi-experimental study by Marianne Dyrby Lorenzen, Casper Friis Pedersen, Line Nielsen, Mikkel O. Andersen, Jane Clemensen and Leah Y. Carreon in DIGITAL HEALTH

Footnotes

Acknowledgments

The authors thank the study participants for their contributions. Additionally, our gratitude goes to the nurses who assisted in the inclusion and data collection, whose support was crucial for this research.

Contributorship

MDL, LYC, JC, and MOA were involved in the design of the study. MDL collected the data and wrote the initial draft of the article, excluding the section on statistical analysis. CFP analyzed the quantitative data and wrote the statistical analysis section. MDL and LN analyzed the qualitative data. LYC, JC, and MOA reviewed and provided comments in the draft, which MDL subsequently revised. All authors read and approved the final manuscript. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted. The authors affirm that the manuscript is an honest, accurate, and transparent account of the study being reported.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Ethical approval

Ethical approval was obtained from the local institutional review board. The study was presented to The Regional Committees on Health Research Ethics for Southern Denmark but was deemed to be exempt from review (File no. S-20212000-95 and Acadre 21/209).

Guarantor

MDL.

ORCID iD

Marianne Dyrby Lorenzen

Supplemental material

Supplemental material for this article is available online.

References

Danish Health Authority. Anbefalinger for tværsektorielle forløb for mennesker med kroniske lænderygsmerter. Sundhedsstyrelsen 2017; 1.1: 4–37.

Andersen

Nielsen

Bech-azeddine

. Rygkirurgi i Danmark Årsrapport 2021. Dansk Rygkirurgisk Selsk. 2021, Available from: http://drks.ortopaedi.dk/wp-content/uploads/2021/06/Årsrapport-DRKS-2020.pdf

Corniola

Debono

Joswig

, et al. Enhanced recovery after spine surgery: review of the literature. Neurosurg Focus 2019; 46: E2.

Debono

Corniola

Pietton

, et al. Benefits of enhanced recovery after surgery for fusion in degenerative spine surgery: impact on outcome, length of stay, and patient satisfaction. Neurosurg Focus 2019; 46: E6.

Brusko

Kolcun

JPG

Heger

, et al. Reductions in length of stay, narcotics use, and pain following implementation of an enhanced recovery after surgery program for 1- to 3-level lumbar fusion surgery. Neurosurg Focus 2019; 46: E4.

Cui

Wang

, et al. Comparison of perioperative outcomes in frail patients following multilevel lumbar fusion surgery with and without the implementation of the enhanced recovery after surgery protocol. Front Surg 2022; 9: 997657.

Tong

Fernandez

Bendo

, et al. Enhanced recovery after surgery trends in adult spine surgery: a systematic review. Int J Spine Surg 2020; 14: 623–640.

Kurnutala

Dibble

Kinthala

, et al. Enhanced recovery after surgery protocol for lumbar spinal surgery with regional anesthesia: a retrospective review. Cureus 2021; 13: e18016.

Kehlet

Wilmore

. Evidence-based surgical care and the evolution of fast-track surgery. Ann Surg 2008; 248: 189–198.

10.

Kehlet

. Fast-track surgery - status and perspectives. Ugeskr Laeger 2021; 183: 1–7.

11.

Kehlet

. Enhanced recovery after surgery (ERAS): good for now, but what about the future? Can J Anesth Can d’anesthésie 2015; 62: 99–104.

12.

Baniasadi

Ghazisaeedi

Hassaniazad

, et al. Surgical patients follow-up by smartphone-based applications: A systematic literature review. Stud Health Technol Inform 2020; 23: 85–92.

13.

Hou

Yang

, et al. The effectiveness and safety of utilizing mobile phone-based programs for rehabilitation after lumbar spinal surgery: multicenter, prospective randomized controlled trial. JMIR mHealth UHealth 2019; 7: e10201.

14.

CIMT. My hospital. Available from: https://cimt.dk/gb/mit-sygehus/ (accessed 1 June 2023)

15.

Des Jarlais

Lyles

Crepaz

. Improving the reporting quality of nonrandomized evaluations of behavioral and public health interventions: the TREND statement. Am J Public Health 2004; 94: 361–366.

16.

Harris

McGregor

Perencevich

, et al. The use and interpretation of quasi-experimental studies in medical informatics. J Am Med Inform Assoc 2006; 13: 16–23.

17.

Simonsen

Robertson

. Routlegde International Handbook of Participatory Design. London: Routledge, 2013.

18.

Clemensen

Larsen

Kyng

, et al. Participatory design in health sciences: Using cooperative experimental methods in developing health services and computer technology. Qual Health Res 2007; 17: 122–30.

19.

Moser

Korstjens

. Series: practical guidance to qualitative research. Part 3: sampling, data collection and analysis. Eur J Gen Pract 2018; 24: 9–18.

20.

Spinuzzi

. The methodology of participatory design. Tech Commun 2005; 52: 163–174.

21.

Jensen

Overgaard

Wiil

, et al. Bridging the gap: A user-driven study on new ways to support self-care and empowerment for patients with hip fracture. SAGE Open Med 2018; 6: 2050312118799121.

22.

Trettin

Danbjørg

Andersen

, et al. Development of an mHealth app for patients with psoriasis undergoing biological treatment: participatory design study. JMIR dermatology 2021; 4: e26673.

23.

Østervang

Lassen

Schmidt

, et al. Development of a health information system to promote emergency care pathways: a participatory design study. Digit Heal 2022; 8: 20552076221145856.

24.

Tsang

Lee

Richards

, et al. Electronic collection of patient-reported outcomes following discharge after surgery: systematic review. BJS Open 2021; 5: zraa072.

25.

Kleif

Edwards

Sort

, et al. Translation and validation of the Danish version of the postoperative quality of recovery score QoR-15. Acta Anaesthesiol Scand 2015. doi:https://doi.org/10.1111/aas.12525

26.

Kleif

Gögenur

. Severity classification of the quality of recovery-15 score-An observational study. J Surg Res 2018; 225: 101–7. doi:https://doi.org/10.1016/j.jss.2017.12.040

27.

Lorenzen

Pedersen

Carreon

, et al. Measuring quality of recovery (QoR-15) After degenerative spinal surgery : a prospective observational study. Brain and Spine [Internet] 2024; 4: 102767. doi:https://doi.org/10.1016/j.bas.2024.102767

28.

Andersen

Nielsen

Bech-azeddine

, et al. Rygkirurgi i Danmark Årsrapport 2022. Dansk Rygkirurgisk Selsk. 2023, Available from: https://drks.ortopaedi.dk/wp-content/uploads/2023/06/Aarsrapport-DRKS-2022-Godkendt-version.pdf

29.

Simony

Hansen

Ernst

, et al.Implementation of the Danish national database Danespine for spinal surgery. Ugeskr Laeger 2014; 176: 1878–1880.

30.

Charan

Biswas

. How to calculate sample size for different study designs in medical research? Indian J Psychol Med 2013; 35: 121–126.

31.

Myles

Galagher

, et al. Minimal clinically important difference for three quality of recovery scales. Anesthesiology 2016; 125: 39–45.

32.

Hvidt

JCS

Christensen

Sibbersen

, et al. Translation and validation of the system usability scale in a Danish mental health setting using digital technologies in treatment interventions. Int J Hum Comput Interact 2020; 36: 709–716.

33.

Booke

. SUS: a “quick and dirty” usability scale. In: Jordan

Thomas

Weerdmeester

McClelland

(eds) Usability evaluation in industry. London: Taylor & Francis, 1996, pp.189–194.

34.

Lewis

Sauro

. Item benchmarks for system usability scale. J Usability Stud 2018; 13: 158–167.

35.

Hyzy

Bond

Mulvenna

, et al. System usability scale benchmarking for digital health apps: meta-analysis. JMIR mHealth UHealth 2022; 10: e37290.

36.

Decorte

Malm

Sznitman

, et al. The challenges and benefits of analyzing feedback comments in surveys: lessons from a cross-national online survey of small-scale cannabis growers. Methodol Innov [Internet] 2019; 12: 2059799119825606.

37.

World Medical Association. Declaration of Helsinki, ethical principles for scientific requirements and research protocols. Bull World Health Organ 2013; 79: 373. Available from https://www.wma.net/policies-post/wma-declaration-of-helsinki-ethical-principles-for-medical-research-involving-human-subjects/.

38.

Jaensson

Dahlberg

Eriksson

, et al. The development of the recovery assessments by phone points (RAPP): a mobile phone app for postoperative recovery monitoring and assessment. JMIR mHealth uHealth 2015; 3: e86.

39.

Idvall

Berg

Unosson

, et al. Assessment of recovery after day surgery using a modified version of quality of recovery-40. Acta Anaesthesiol Scand 2009; 53: 673–677.

40.

Bramhagen

Eriksson

Ericsson

, et al. Self-reported post-operative recovery in children: development of an instrument. J Eval Clin Pract 2016; 22: 180–188.

41.

Allvin

Ehnfors

Rawal

, et al. Development of a questionnaire to measure patient-reported postoperative recovery: content validity and intra-patient reliability. J Eval Clin Pract 2009; 15: 411–419.

42.

Nilsson

Idvall

. Pain assessments in day surgery patients. J Clin Nurs 2010; 19: 2942–2943.

43.

Dahlberg

Jaensson

Nilsson

. “Let the patient decide”: person-centered postoperative follow-up contacts, initiated via a phone app after day surgery: secondary analysis of a randomized controlled trial. Int J Surg 2019; 61: 33–37.

44.

Higgins

Semple

Murnaghan

, et al. Mobile web-based follow-up for postoperative ACL reconstruction: a single-center experience. Orthop J Sport Med 2017; 5: 2325967117745278.

45.

de Vlieger

JCN

Luiting

Lockyer

, et al. Validation of the Dutch translation of the quality of recovery-15 scale. BMC Anesthesiol 2022; 22: 43.

46.

Kim

Yeo

. Translation and validation of the Korean version of the postoperative quality of recovery score QoR-15. Biomed Res Int 2020; 2020: 3456234.

47.

von Allmen

Tinner

Schmidli

, et al. Randomized controlled comparison of cross-sectional survey approaches to optimize follow-up completeness in clinical studies. PLoS One 2019; 14: e0213822.

48.

Durkin

Usher

Jackson

. Embodying compassion: a systematic review of the views of nurses and patients. J Clin Nurs 2019; 28: 1380–1392.

49.

Nielsen

Clemensen

Callesen

, et al. Who is supporting the parents during their child’s cancer treatment? A qualitative study through the lens of compassion. Eur J Oncol Nurs Off J Eur Oncol Nurs Soc 2024; 70: 102534.

50.

Bellomo

Warrillow

Reade

. Why we should be wary of single-center trials. Crit Care Med 2009; 37: 3114–3119.

51.

Patel

Thind

. Usability of Mobile health apps for postoperative care: systematic review. JMIR Perioper Med 2020; 3: e19099.

52.

Willis

Neblett

. Developing culturally-adapted mobile mental health interventions: a mixed methods approach. mHealth 2023; 9: 1.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.01 MB

0.10 MB