Interobserver Agreement and Satisfaction With the use of Telemedicine for Evaluating low Back Pain: A Primary,Observational,Cross-Sectional,Analytical Study

Abstract

Study design

A primary, observational, cross-sectional, analytical study.

Objective

The development of a framework for systematic telemedicine (TM) for orthopedic physicians in frequent clinical care may increase agreement in diagnosis and satisfaction among users of TM. Therefore, this study aimed to estimate the agreement in the diagnosis of low back pain (LBP) between TM, systematized by a self-completed digital questionnaire, and face-to-face (FF) care in patients with LBP.

Methods

This study included adults up to 75 years of age with LBP for more than 6 weeks. They were evaluated at 2 independent time points (TM and FF) by different orthopedists with 3 different levels of expertise. Professionals evaluated the sample without prior knowledge of the diagnosis, and each orthopedist provided a diagnosis. Diagnostic agreement was the primary outcome. Secondary outcomes were the duration of the visit and satisfaction among healthcare professionals.

Results

A total of 168 participants were eligible, of whom 126 sought care through TM and 122 sought FF care (mean age, 47 years [range, 18-75 years]; 66.4% women). The agreement among professionals regarding the diagnosis was moderate (kappa = .585, P = .001). TM was faster than FF (11.9 minutes (standard deviation = 4.1) vs 18.6 (SD = 6.9), P < .001). Professional satisfaction was higher among spine specialists than among orthopedic residents and orthopedists who were not specialists in spine surgery.

Conclusion

Agreement in diagnosis was moderate for TM, with a 30% shorter visit duration than FF. Satisfaction varied by professional expertise and was higher among spine specialists than among professionals with other expertise.

Keywords

telemedicine virtual care low back pain orthopedic diagnosis

Introduction

The pandemic caused by severe acute respiratory syndrome coronavirus 2 boosted the growth of remote healthcare, with greater use of digital equipment and technology to practice social distancing, as encouraged by renowned medical associations.¹ Consequently, the total number of telemedicine visits among all complaints related to spinal diseases increased from 10% to approximately 40%, with a practitioner’s perception that TM increases access to care and reduces costs.¹ TM has increased on all continents, especially in North America, where its use increased from 7% to more than 60%. The use of video call technologies was higher in South and North America, whereas in Europe and Africa, preference was given to telephone calls.²

The use of TM is advantageous because it allows greater access to residents of remote areas or those with limited mobility.³ TM is also a safe method for diagnosis and treatment in initial orthopedic care,^4,5 with low cost-effectiveness⁶ and high user satisfaction.⁷ Even at the end of the pandemic, these advantages support the continued use of digital care as a complementary healthcare method. However, the limitations of TM include disparity in access by race, language, and form of funding,⁸ lack of formal regulation in some countries, lack of awareness, digital illiteracy, and difficulty in performing physical examinations.^3,9

Initiatives to make the use of TM more reproducible and reliable have been described, emphasizing the systematization of care to the detriment of physical examinations occurring in face-to-face (FF) care.^10-12 In a comparison between TM and FF care for the diagnosis of orthopedic complaints at first visits, there was an agreement among TM diagnoses of 81.4%, with a shorter duration than that for FF visits at an outpatient orthopedic clinic.¹³ Nevertheless, TM presented a lower degree of satisfaction than FF care, especially for the ability to examine the patient.¹³

Low back pain (LBP) is one of the most common complaints of patients seeking medical assistance¹⁴ even for diagnosis and treatment, and there is an assumption that the development of systematic TM for LBP would increase agreement among orthopedists in the diagnosis and satisfaction among users of TM. Therefore, this study aimed to estimate the agreement of diagnosis obtained through systematized TM and FF care in patients with LBP and to secondarily quantify the duration of care and satisfaction of orthopedists with systematized TM.

Materials and Methods

Ethics Statements

This primary, observational, cross-sectional, analytical study was approved by the local ethics committee (number: CAAE 53634821.5.0000.5133), and this report was written in accordance with the STROBE statement.¹⁵ All participants provided informed consent.

Study Design and Population

The eligible participants were adults with LBP lasting 6 weeks or more, those who were up to 75 years of age, and those who were awaiting an appointment for their first orthopedic care visit at a local referral center. Individuals with suspected emergency situations, symptoms other than LBP, or those without digital resources were excluded. After eligible participants consented to participant, their demographic data were collected (Appendix I). At this stage, recommendations for TM were provided, such as a quiet and noise-free location, use of a cellular phone or notebook, and distancing and positioning during the video call, and then a link to the questionnaire regarding clinical data of LBP (Appendix II) was sent to participants for self-completion through the messaging application WhatsApp^®.

All participants who completed the questionnaire were randomly scheduled to receive TM or FF care on different dates and by different professionals to ensure concealment of the diagnosis between visits. All appointments occurred between March 2022 and August 2022. The visits were performed by 17 medical professionals with 3 different degrees of expertise: 5 orthopedic residents, 9 orthopedists who were not specialists in spine surgery, and 4 orthopedists who specialized in spine surgery. All the professionals voluntarily participated in the study.

The TM visits were performed in an adapted room with a desktop computer with high-speed internet access equipped with a camera (Lehmox Webcam LEY-233), microphone, and speakers (multilaser flat3w RMS). The Microsoft Teams^® platform (Redmond, Washington, USA) was used for all the video calls. Access to the calls was granted through a link created by the main researcher and shared with professionals and participants. The FF visits were performed at the orthopedic outpatient clinic of our institution. The digital questionnaire completed by the participants (Appendix II) was made available to physicians during the TM and FF visits.

Outcome Measures

The primary outcome was the agreement between the diagnoses obtained through TM and those determined through FF care. Possible LBP diagnoses were coded according to the International Classification of Diseases, 10th Revision (ICD-10) and presented in an electronic form to be completed at the end of the evaluation (Appendix III). The secondary outcomes were the visit duration (for TM care, it was obtained as available via Microsoft Teams; for FF care, the time disposed by the institution’s digital medical record program was used) and physicians’ perceptions, as measured according to Buvik et al⁹ with 4 domains of satisfaction: patient cooperation in providing information during the medical interview, examination/evaluation, ease in providing information to the patient, and treatment performance. Five degrees of satisfaction were assigned to each domain (very good, good, neither good nor bad, bad, or very bad) (Appendix IV). Physicians’ perceptions were classified as satisfied (very good or good) or dissatisfied (neither good nor bad, bad, or very bad). Records from each professional with complementary information to that contained in the self-completed questionnaire (Appendix III) used for the evaluation of LBP were also obtained. At the end of each visit, the physicians answered questions in the respective questionnaires by accessing the links provided in Appendices III and IV.

Statistical Analysis

The sample size was calculated using statistics. co.uk and was based on an increase of at least 20 percentage points in the satisfaction of professionals in conducting TM visits. The data available in the literature were used to assess the satisfaction of professionals with TM¹⁶ in addition to the feasibility of the study, which allowed the evaluation of an effect size of .90. Considering two-tailed tests, 95% confidence, and 80% power, the calculation indicated the need for 106 participants. After estimating a loss of 20%, a final sample size of 122 participants was calculated.

Descriptive statistics were performed by calculating the absolute (n) and relative (%) frequencies of the categorical variables and the mean ± standard deviation of the quantitative variables. Descriptive analyses were used to evaluate the percentage of concordant diagnoses. The Cohen kappa coefficient was used to classify the agreement between TM and FF, and the results were interpreted as follows: ≤0 indicates no agreement, .01-0 .20 indicates none to slight agreement, .21-.40 indicates fair agreement, .41-.60 indicates moderate agreement, .61-.80 indicates substantial agreement, and .81-1.00 indicates almost perfect agreement.¹⁷ Additionally, the odds of the TM diagnosis agreeing with the FF 1 was calculated by the ratio between the number of concordant diagnoses by the non-concordant 1. Differences in visit duration were calculated using the Student paired t-test, and the chi-square test was used to test for differences in physicians' perception of TM vs FF care and perception of professionals by expertise. Statistical analyses were performed using STATA software, version 15.1 with a P-value of <.05 indicating statistical significance.

Results

Of the 168 eligible patients, 126 received TM care and 122 underwent FF care (Figure 1).

Figure 1.

Study flowchart shows participants that initiated and completed this study.

The sample was composed of 66.4% women, and 8.2% of participants had a university degree. The mean body mass index indicated that this population was overweight; only 20.4% reported frequent physical exercise, and 18% of total patients were unmotivated and unhappy (Table 1).

Table 1.

Demographic Profile of the Participants (n = 122).

Variables	Values
Age, years	47.8 ± 13.1
Sex, n (%)
Female	81 (66.4)
Body mass index, kg/m²	28.5 ± 5.9
Current activity, n (%)
Active employee	33 (27.0)
Unemployed	24 (19.7)
Retired	21 (17.2)
Receives assistance	17 (13.9)
Homemaker	15 (12.3)
Not reported	12 (9.8)
Education
High school	68 (55.7)
Elementary and middle school	34 (27.9)
Higher education	10 (8.2)
Illiterate	1 (.8)
Not reported	9 (7.4)
Physical exercise
Yes	25 (20.5)
Not reported	3 (2.5)
Degree of motivation in life
Neither happy nor sad	51 (41.8)
Happy and with normal motivation	35 (28.7)
Unhappy and unmotivated	22 (18.0)
Very happy and motivated	12 (9.8)
Not reported	2 (1.6)

Data expressed as the mean ± standard deviation and number of participants (percentage).

The diagnostic agreement between TM and FF care was 79.5% (Table 2), and the probability of the diagnosis by TM agreeing with the diagnosis by FF care was 3.88 times. According to the Cohen kappa coefficient, the agreement between TM and FF care was moderate (k = .585, P < .001).

Table 2.

Diagnostic agreement between TM and FF care (n = 122).

	FF
	M54.5	M54.4	S-39	M46.9	M41
TM
M54.5	65 (53.3)	4 (3.3)	1 (.8)	1 (.8)	0 (0)
M54.4	15 (12.3)	31 (25.4)	0 (0)	0 (0)	1 (.8)
S-39	2 (1.6)	0 (0)	0 (0)	0 (0)	0 (0)
M46.9	1 (.8)	0 (0)	0 (0)	0 (0)	0 (0)
M41	0 (0)	0 (0)	0 (0)	0 (0)	1 (.8)

M54.5 = low back pain - degenerative without neural injury of any nature; M54.4 = low back pain with sciatica - degenerative with neural injury of any nature; S-39 = traumatic spinal injuries; M46.9 = inflammatory spondylopathies; M41 = other scoliosis – deformities.

The TM visit duration was shorter (mean 6.7 minutes) than the FF visit duration (11.9 ± 4.1 vs 18.6 ± 6.9 minutes, P < .001) (Figure 2).

Figure 2.

Duration between TT and FF care (n = 122). *P < .001.

Physicians’ perceived satisfaction with TM and FF care is shown in Figure 3. More professionals were satisfied than dissatisfied with both methods. When comparing the 2 methods, there were differences in all domains, except for patient cooperation during the medical interview.

Figure 3.

Satisfaction physicians’ perception - four domains.

The perceptions of professionals according to expertise are presented in Figures 4 and 5. The satisfaction rate was >89.3% for all domains of satisfaction among spine surgeons who provided care through TM, with a significant difference in patient evaluation and ability to perform treatment.

Figure 4.

Satisfaction criteria by categories of physicians at telemedicine.

Figure 5.

Satisfaction criteria by categories of physicians at a face-to-face care.

Complementary information to that contained in the questionnaire used for the evaluation of LBP was required in TM and FF care and accounted for 48,4% in physical evaluation and 35,2% in medical interview of TM cases (Figure 6).

Figure 6.

Need for additional information requested by professionals in addition to that contained in a questionnarie.

Discussion

The main finding of the study was that there was 79.5% agreement in LBP diagnoses between TM and FF care. In accordance with our findings, Cotrel et al evaluated the agreement between FF care and TM in the diagnosis of musculoskeletal diseases by physical therapists and obtained 83.3% similarity.¹⁸

Nevertheless, the present study showed more than 89.3% satisfaction for all domains among spine surgeons regarding the treatment recommendations for TM, which is higher than the values among the other professionals. Lovo et al compared the recommendations for the treatment or referral of patients with LBP through TM with a nursing team and physical therapist vs FF care with a physical therapist; they reported an agreement of 74.1%.⁵

The time required to perform a systematic TM evaluation was 6 minutes faster, on average, than FF care, and this finding contradicts data indicating that TM visits are longer than¹⁹ or at least as long as⁹ FF care sessions. It is likely that the introduction of an evaluation methodology prior to TM was associated with a 30% shorter visit duration than that for FF. Nonetheless, as LBP is highly prevalent worldwide, especially in low-income countries (almost 21%),²⁰ there might be a positive impact on healthcare with shortened care time.

Regarding the satisfaction of professionals, the expected barrier of TM, ie, no physical examination by direct contact, was evident, with 57.4 points out of 100 possible.¹³ However, there was an increase in professional satisfaction with the implementation of systematic TM compared with the results reported in a previous study¹³ conducted under the same conditions and team, achieving a gain of 30 percentage points.

The aim of orthopedic medical care is to obtain an accurate diagnosis, even in the absence of a physical examination, and to achieve results similar to those in the literature, with satisfaction rates for professionals and patients above 90%.^4,6,21-23 For spine surgeons who report satisfaction greater than 89.3% in all domains, systematized TM can be used in daily practice. However, TM must be adapted to achieve a higher level of satisfaction among all physicians. Hence, the objective of systematizing TM is to facilitate its implementation in a setting where professionals with different levels of expertise work with patients with LBP.

The following study limitations were identified: a lack of training for professionals in completing the data collection forms; supplying professionals with ICD-10 diagnoses, which may have impacted the agreement of diagnoses; technical issues regarding the TM, such as a noisy environment and inadequate lighting; and the unrestricted use of cellular phones by the participating patients in most TM visits, making it difficult to conduct TM visits and thus potentially impacting the satisfaction of professionals and failure to assess patient satisfaction.

Future research could develop tools to improve TM satisfaction for both professionals and patients by creating explanatory material to guide users in the correct use of TM. In this scenario, the ideal framework for systematized TM can modify the diagnosis-treatment process, accelerating the care of those who need it safely and efficiently.

In conclusion, systematic TM showed moderate overall agreement of diagnosis, with an average visit duration that was 30% shorter than that of FM care. Additionally, the degree of satisfaction varied among orthopedic specialists, with it being higher among spine surgeons than among other specialists.

Supplemental Material

Supplemental Material - Interobserver Agreement and Satisfaction With the use of Telemedicine for Evaluating low Back Pain: A Primary, Observational, Cross-Sectional, Analytical Study

Supplemental Material for Interobserver Agreement and Satisfaction With the use of Telemedicine for Evaluating low Back Pain: A Primary, Observational, Cross-Sectional, Analytical Study by Jair Moreira Dias, Adriano Fernando Mendes, Eduardo Pestana de Aguiar, Luan Costa Silveira, Maria Fernanda Noel Dias, and Nádia Rezende Barbosa Raposo in Global Spine Journal

Footnotes

Acknowledgments

The authors are immensely grateful to the head of the outpatient clinic at the University Hospital of the Federal University of Juiz de Fora, Mrs Maria Helena Barcellar, her entire team of attendants, and PhD student Luciana Angélica da Silva de Jesus for her support, assistance with the statistical calculations, and great expertise in this field. Lastly, we acknowledge all the patients who participated in the study.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Ethics Statements

This study was approved by the Ethics Committee of the University Hospital of the Federal University of Juiz de Fora (number: CAAE 53634821.5.0000.5133). All participants provided informed consent.

ORCID iD

Jair Moreira Dias

Supplemental Material

Supplemental material for this article is available online.

References

Riew

Lovecchio

Samartzis

, et al. Spine surgeon perceptions of the challenges and benefits of telemedicine: An international study. Eur Spine J. 2021;30(8):2124-2132. doi:10.1007/s00586-020-06707-x

Riew

Lovecchio

Samartzis

, et al. Telemedicine in spine surgery: Global Perspectives and practices. Global Spine J. 2023;13(5):1200-1211. doi:10.1177/21925682211022311

Makhni

Riew

Sumathipala

. Telemedicine in orthopaedic surgery: Challenges and opportunities. J Bone Joint Surg Am. 2020;102(13):1109-1115. doi:10.2106/JBJS.20.00452

Shah

. Use of telemedicine for initial consultations in elective orthopaedics—Results from a large volume centre. Foot Ankle Orthop. 2017;1(3):2. doi:10.1177/2473011417S000366

Lovo

Harrison

O’Connell

Rotter

Bath

. A physical therapist and nurse practitioner model of care for chronic back pain using telehealth: Diagnostic and management concordance. J Telemed Telecare. 2022;12:1-9. doi:10.1177/1357633X221098904

Haider

Aweid

Subramanian

, et al. Telemedicine in orthopaedics and its potencial applications during COVID-19 and beyond: A systematic review. J Telemed Telecare. 2020;0(0):1-13. doi:10.1177/1357633X20938241

Chaudhry

Nadeem

Mundi

. How satisfied are patients and surgeons with telemedicine in orthopaedic care during the COVID-19 pandemic? A systematic review and meta-analysis. Clin Orthop Relat Res. 2021;479(1):47-56. doi: 10.1097/CORR.0000000000001494

Xiong

Greene

Lightsey

, et al. Telemedicine use in orthopaedic surgery varies by race, ethnicity, primary language, and insurance status. Clin Orthop Relat Res. 2021;479(7):1417-1425. doi:10.1097/CORR.0000000000001775

Buvik

Bugge

Knutsen

Småbrekke

Wilsgaard

. Quality of care for remote orthopaedic consultations using telemedicine: A randomised controlled trial. BMC Health Serv Res. 2016;16(1):483. doi:10.1186/s12913-016-1717-7

10.

Tanaka

Martin

Berkson

. Telemedicine in the Era of COVID-19: The virtual orthopaedic examination. J Bone Joint Surg Am. 2020;102(12):e57. doi:10.2106/JBJS.20.00609

11.

Satin

Lieberman

. The virtual spine examination: Telemedicine in the era of COVID-19 and beyond. Global Spine J. 2021;11(6):966-974. doi:10.1177/2192568220947744

12.

Wahezi

Duarte

Yerra

, et al. Telemedicine during Covid-19 and beyond: A practical guide and best practices multidisciplinary approach for the orthopedic and neurologic pain physical examination. Pain Physician. 2020;23(4S):S205-S238.

13.

Dias

Jr Mendes

Jr . Telemedicine effectiveness in the first elective orthopedic care compared to a standard face-to-face visit. Rev Bras Ortop. 2022;0(0):0. http://www.thieme-connect.de/DOI/DOI?10.1055/s-0042-1756324

14.

Knezevic

Candido

Vlaeyen

JWS

Van Zundert

Cohen

. Low back pain. Lancet. 2021;398(10294):78-92. doi:10.1016/S0140-6736(21)00733-9

15.

Von Elm

Altman

Egger

, et al. The strengthening the reporting of observational studies in epidemiology (STROBE) statement: Guidelines for reporting observational studies. Lancet. 2007;370(9596):1453-1457. doi:10.1016/S0140-6736(07)61602-X

16.

Ajrawat

Young Shin

Dryan

, et al. The use of telehealth for orthopedic consultations and assessments: A systematic review. Orthopedics. 2021;44(4):198-206. doi:10.3928/01477447-20210621-08

17.

McHugh

. Interrater reliability: The kappa statistic. Biochem Med. 2012;22(3):276-282. doi:10.11613/BM.2012.031

18.

Cottrell

O’Leary

Swete-Kelly

, et al. Agreement between telehealth and in-person assessment of patients with chronic musculoskeletal conditions presenting to an advanced-practice physiotherapy screening clinic. Musculoskelet Sci Pract. 2018;38:99-105. doi:10.1016/j.msksp.2018.09.014

19.

Melian

Frampton

Wyatt

Kieser

. Teleconsultation in the management of elective orthopedic and spinal conditions during the COVID-19 pandemic: Prospective cohort study of patient experiences. JMIR Form Res. 2021;5(6):e28140. doi:10.2196/28140

20.

Maher

Underwood

Buchbinder

. Non-specific low back pain. Lancet. 2017;389(10070):736-747. doi:10.1016/S0140-6736(16)30970-9

21.

Buvik

Bergmo

Bugge

Smaabrekke

Wilsgaard

Olsen

. Cost-effectiveness of telemedicine in remote orthopedic consultations: Randomized controlled trial. J Med Internet Res. 2019;21(2):e11330. doi:10.2196/11330

22.

Abdulwahab

Zedan

. Factors affecting patient perceptions and satisfaction with telemedicine in outpatient clinics. J Patient Exp. 2021;8:23743735211063780. doi:10.1177/23743735211063780

23.

Al-Kulabi

Mansour

Thahir

. The orthopaedic experience of COVID-19: A literature review. J Perioperat Pract. 2021;31(3):102-107. doi:10.1177/1750458920971506

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.16 MB