Abstract
Background:
There is a lack of randomized controlled trials focusing on orthopedic telemedicine (TM). The objective of this research was to compare the diagnostic accuracy and pattern of TM consultations of low-risk orthopedic patients performed by general practitioners (GPs) with those of face-to-face evaluations by orthopedists at an emergency department (ED).
Methods:
This randomized, single-center study was conducted between October 2021 and November 2022 on patients at an ED. Inclusion criteria were age >18 years, low back pain, extremity contusion, ankle sprain, or neck pain. Eligible patients were randomized 1:1 for TM consultations by generalist physicians with subsequent face-to-face orthopedic evaluations (TM–ED group) or face-to-face evaluations by orthopedic physicians (ED group). Primary outcomes were syndromic diagnosis, physical examination, and tests ordered. Secondary analysis included a satisfaction survey.
Results:
A total of 99 patients were enrolled; mean age was 41 ± 10.1 years, and 62.6% were female. The most common conditions were foot contusion (28.3%), ankle sprain (27.3%), hand contusion (19.2%), low back pain (19.2%), and neck pain (6.1%). Syndromic diagnosis showed no difference between groups (p = 0.231). In the TM–ED group (n = 51), self-examination demonstrated moderate to good agreement with face-to-face evaluations in several areas. Both groups showed similar tests practices. Patient satisfaction was higher in the TM–ED group across multiple measures.
Conclusion:
TM consultations for low-risk orthopedic patients by GPs are not inferior to face-to-face specialist evaluations at the ED. Virtual assessments are associated with higher patient satisfaction. Clinical Trial Identifier: NCT04981002.
Introduction
Telemedicine (TM) is a critical resource in the health care system in which a multimodal medical assessment is provided to the population through immediate and low-cost actions. 1 TM has good accuracy in clinically evaluating several conditions, including orthopedic conditions. 2,3 However, randomized controlled trials on TM lack support for medical management and thereby lack a high level of evidence. 4
Patients with orthopedic conditions account for approximately 20% of all emergency department (ED) visits worldwide (including nonorthopedic causes) and are frequently evaluated by general practitioners (GPs). 5 Three-quarters of these orthopedic patients present with complaints of low-complexity situations, such as low-complexity trauma and pain in the hands, feet, ankles, or spine. These low-complexity conditions are usually diagnosed via only a direct physical examination and managed with multimodal pain relief strategies. 6
ED overcrowding usually occurs due to the high volume of low-complexity presentations. ED overcrowding is associated with a worsening prognosis for all patients who experience emergencies and results in dissatisfaction among patients and their families.
An alternative for low-complexity orthopedic patients is TM following specific guidelines. 7 TM can help patients with low-complexity orthopedic complaints follow the correct guidelines at home and improve the referral process. 8 Nonetheless, direct-to-consumer TM diagnostic accuracy for these types of orthopedic conditions is still neither well studied nor is the time spent on the evaluation, test ordering, and general guidance provided.
TM has been mainly utilized post-pandemic for diagnosis and treatment. 9 –11 However, no study has analyzed the precise diagnosis of a TM consultation for low-complexity orthopedic patients with acute complaints compared to an onsite orthopedic evaluation and compared with GP TM consultations. Thus, we hypothesized that the diagnostic accuracy of a direct-to-consumer TM consultation (even when performed by a guideline-directed GP) is not inferior to that of a face-to-face orthopedic evaluation at the ED.
In this study, we aimed to randomly compare the diagnoses made via TM evaluations by GPs and consultation profiles with face-to-face evaluations in patients with low-risk orthopedic conditions who spontaneously sought evaluations at an ED.
Methods
From October 2021 to November 2022, 101 adult patients (≥18 years of age) with at least one acute orthopedic symptom (neck pain, low back pain, contusion of extremities, or ankle sprain) who requested a spontaneous face-to-face evaluation at the ED were enrolled in a single-center trial with a randomized, noninferiority, open-label design. All of the patients consented to participate in this prospective study designed according to the guidelines of the Ethics Committee for Analysis of Research Projects on Human Experimentation at the study institution (CAAE 46603821.9.0000.007) and registered at ClinicalTrials.gov (Identifier: NCT04981002).
After enrollment, patients were randomly assigned to receive a medical consultation via TM with a GP, followed by a subsequent in-person assessment from an orthopedist (TM group) or a direct in-person assessment from an orthopedist (IP group). Two patients withdrew consent before the randomization. In the TM group, five patients were excluded (one patient for having connection problems on their computer and four patients for deciding to switch to an in-person assessment). The analysis was based on the intention-to-treat design (Fig. 1).
Participant flow diagram.
All of the evaluations were performed in a quaternary JCI-certified private general hospital. Orthopedists hired by the ED performed in-person assessments of patients with orthopedic complaints who were first screened by the nursing staff. GPs from the same institution’s TM center, which functions as a virtual emergency room with independent professionals and its specific methodology, performed the TM assessments.
TRIAL PROCEDURES
Eligible patients were low risk and represented most of those individuals who arrived at the emergency room and for whom TM was most indicated. They were randomly assigned to receive a TM consultation followed by an in-person assessment (TM group) or a standard in-person assessment (IP group) at a 1:1 ratio via an app-based automated randomization system (Randomizer 1.2, Darshan Institute of Engineering and Technology, India). The app provided the random allocation sequence immediately after inclusion in the study. Based on institutional protocols and clinical judgment, a face-to-face medical assessment (with all of the necessary resources) was considered to be the gold standard and was performed for all of the patients. None of the patients in the TM group were discharged before the onsite orthopedic evaluation (Fig. 2).
Trial Procedures diagram.
Patients who were assigned to receive TM first (the TM group) were evaluated in the triage room immediately after the initial protocols of nursing and enrollment by using a digital tablet connected to an institutional wireless network (Apple iPad, CA, USA). Each patient then entered a predefined virtual waiting room, and the TM center team performed the visit according to TM center protocols by using a licensed HIPAA-compliant platform (InTouch Health, CA, USA). The evaluation was recorded in the institutional electronic records but was not made readily available to the local ED medical team, who remained blinded to the test randomization. In the TM group, care was provided by a team of GPs with experience in emergencies and care protocols. Each patient underwent an orthopedic self-examination guided by a TM physician. The TM consultation was interrupted when the doctor made the diagnostic hypothesis, which the patient was not aware of. The aim of this arrangement was to partially blind the assessors and reduce bias associated with sequential face-to-face patient care. The remote consultation data were tabulated through specific checkboxes for orthopedic complaints and free text according to clinical judgment (Table 1).
Protocol for TM Evaluations of Orthopedic Conditions Assessed by General Practitioners (Yes and No Answers)
Furthermore, orientations, drug prescription suggestions, test ordering, and possible referrals were described in distinct fields. After the evaluation, the GPs filled out suggestions for prescription requests for tests, which were used exclusively for this study, as these patients were seen in person by orthopedists who provided patient management. The objective of this study was to compare the results of these treatments with those of other treatments.
The institutional protocol for the remote evaluation of orthopedic conditions could be easily consulted in an electronic tab during the consultation. According to the diagnostic hypothesis, a decision support system was also integrated into the electronic system. Immediately after the TM consultation, the patient was always taken to the typical face-to-face care flow at the ED, which was performed by an orthopedist. No patients were exclusively discharged via TM.
The diagnostic hypothesis was based on direct anamnesis, inspection, palpation, and range of motion assessments. The questions asked by the GP are listed in Table 1. Possible diagnoses included neck pain, low back pain, ankle sprain, and hand-foot contusion.
All of the study participants were assigned to receive standard face-to-face evaluations according to the institutional care flow in the ED without any contact with the TM center. Even though the same patient in the TM group was also evaluated in person (thus allowing for comparisons), we chose to have a control group (IP group) of patients who received standard face-to-face service, thus minimizing systematic errors. All of the patients in both arms of the study had their final diagnosis and were treated via face-to-face care at the ED. The ED physician knew about the protocol only if informed by the patient him/herself, which could not be measured. The final diagnosis made at all of the institutional ED or TM assessments was completed by using the International Classification of Diseases (ICD)-10 code before discharge to home or admission. Notably, none of the patients required hospitalization. Tests were ordered, discharge home instructions were provided, and medical prescriptions were compiled in the electronic medical records.
The diagnostic accuracy of the self-examination guided by TM was defined by the proportion of patients with concordant diagnoses obtained during TM care by a GP and in-person evaluations with orthopedic care.
Before leaving the hospital, patients responded to a satisfaction questionnaire, which included questions about the care they received, the service’s ability to meet their needs, the information provided about the disease, the equipment used, coherence between the sought and received treatment, waiting times, assistance received for disease care, recommendations of service to friends, whether they would seek the same service in the future, and their overall satisfaction with their health. Five options were provided to evaluate satisfaction: very satisfied, satisfied, indifferent, not satisfied, and very dissatisfied. Patients were asked whether they would recommend TM. The response to waiting time was categorized as fast or slow, and patients were also asked if they would utilize this type of service in the future.
There was no follow-up, and patient participation ended at ED discharge. The study ended after the previously estimated sample size was achieved.
Initially, we compared demographic and diagnostic data between the TM and IP groups to assess whether the two groups were similar. Afterward, we evaluated the diagnostic accuracy of the TM assessment by analyzing the proportion of patients with concordant diagnoses that were obtained between the TM and in-person evaluations. We defined noninferiority as an observed proportion of at least 70%. Finally, we evaluated patient satisfaction.
PRIMARY AND SECONDARY OUTCOMES
The primary outcomes were as follows: syndromic diagnosis, physical examination profile regarding inspection, palpation data, red flag identification, beyond-order tests, suggested medications, and proposed destinations. The secondary outcome was patient satisfaction.
STATISTICAL ANALYSIS
The noninferiority of diagnostic accuracy was investigated by using the proportion test (H0: p ≤ 70% vs. H1: p > 70%) when considering an expected accuracy of 90% and a noninferiority margin of 20%. In a previous study on the accuracy of physical examinations in diagnosing knee injuries compared to surgical diagnosis, the authors observed agreement rates between 69% and 90% depending on the injury. 12 We estimated the diagnostic accuracy of a self-examination guided by TM to be 90%, as these are clinical diagnoses of pathologies that TM can monitor.
The sample size was obtained to demonstrate the noninferiority of the diagnostic accuracy of TM, with a clinically acceptable variation of up to 20% less in the estimated accuracy (noninferiority margin of 20%). Sample size estimation calculations were performed by using the PASS program, with a one-sided significance level of 2.5%, as well as by varying the power estimate and using an exact test for a proportion. We calculated that 37 patients per group would provide a power of approximately 85% to detect a noninferiority margin in the primary analysis, with a unilateral significance level of 2.5%. We used IBM-SPSS for Windows version 22.0 software for the statistical calculations.
Continuous variables are expressed as the means and standard deviations, and categorical variables are presented as counts and percentages. There were no missing data. The demographic analysis was performed by using the chi-square test and Student’s t-test; Student’s t tests were used for responses to the satisfaction questionnaire and numerical variables, and chi-square tests or Fisher’s exact tests were used for categorical variables. Cohen’s kappa coefficient was used to measure the interrater reliability of the ICD-10 codes between the groups. Values (p < 0.05) were considered to indicate statistical significance, and the established confidence interval (CI) was 95%.
Results
The demographic distribution and diagnoses are presented in Table 2. The population was predominantly female in the fifth decade of life. The distribution of diseases was statistically similar between the groups (p = 0.231), and foot contusions were the most frequent reason for an evaluation.
Demographics and Diagnosis
Student’s t-test.
Chi-square test.
Fisher’s exact test.
n, number of patients; SD, standard deviation; y, years.
PRIMARY OUTCOME
The diagnostic agreement between the GP’s diagnoses in the TM assessment and in-person assessment was 98.0% (95% CI: 89.6% to 100.0%), as presented in Table 3. Only one patient (2.0%) who was diagnosed with a foot injury in the TM evaluation had a final diagnosis of ankle sprain in the in-person evaluation. The rate of actual ankle sprains observed was greater than 70%. Therefore, we conclude that TM assessments are not inferior to ED assessments.
Diagnostic Correlation between TM and IS
Concordance: 50/51 = 98.0%; 95% CI: 89.6% to 100.0%. Ratio comparison test (70%): p > 0.999.
The agreement between the findings of the self-examination guided by TM and the in-person assessments of the patients is presented in Table 4. We observed good agreement between the in-person physical examination and the TM examination.
Agreement between Findings of the Self-Examination Guided by TM and in-Person Assessments
Thirty-nine patients diagnosed with hand contusion, foot contusion, or ankle sprain.
Fifty-one patients diagnosed with hand injury, foot injury, ankle sprain, neck pain, or low back pain.
Forty patients diagnosed with hand injury, foot injury, ankle sprain, or neck pain.
AC1, Gwet AC1 agreement coefficient.
We obtained similar results for complementary examinations, such as radiographs and tomography, for all of the diagnoses. Regarding ankle sprains, we observed a difference in the recommended immobilization with the Roboffot (Table 5).
Telemedicine and in-Person Medical Conduct
SECONDARY OUTCOMES
The patient satisfaction results are presented in Table 6. In various aspects, we observed a greater level of satisfaction among patients in the TM group than in the IP group. The TM group showed greater satisfaction with the care received (TM: 94.1% vs. IP: 75%; p = 0.008), coherence between the sought and received treatment (TM: 90.2% vs. IP: 62.5%; p = 0.001), waiting time (TM: 94.1% vs. IP: 25%; p < 0.001), assistance received for managing the disease (TM: 90.2% vs. IP: 56.3%; p < 0.001), and overall health (TM: 96.1% vs. IP: 72.9%; p = 0.001).
Patient Satisfaction
Chi-square test.
Fisher’s exact test.
Discussion
This study constitutes a groundbreaking initiative aimed at systematically assessing the diagnostic accuracy of TM in evaluating patients with orthopedic concerns. This research focused on a prevalent presentation in ED settings (particularly during the pandemic) primarily serving low-risk individuals, thereby underscoring the substantial potential for TM utilization. Furthermore, this study delved into the effectiveness of remote orthopedic self-examinations conducted through TM, thus contributing insights into diagnosing and managing orthopedic conditions.
The findings of this study demonstrated the noninferiority of TM compared to face-to-face consultations, particularly in addressing the most prevalent diagnoses encountered in the ED. Literature reviews have demonstrated the utilization of TM under various emergent conditions. 13 In orthopedics, the exploration of TM applications has predominantly focused on postoperative situations and low-complexity cases. 3 To test the validity of TM consultations, in-person consultations were initially conducted, followed by virtual consultations. 14 The outcomes of these investigations suggest that virtual consultations can be effectively employed for initial assessments, particularly postoperative assessments, thus enabling the swift integration of this technological approach.
The determination of a suitable patient profile for teleorthopedics remains an area of ongoing investigation. This study’s inclusion criteria were centered on patients seeking orthopedic emergency care for acute complaints. This approach aligns with the findings of some studies on trauma follow-up, thus providing further support. 15,16 This research underscores the feasibility of employing TM even in low-complexity orthopedic emergency scenarios, which is a significant consideration for expanding care in geographically large countries.
In orthopedics, a physical examination plays a crucial role in diagnosing and managing musculoskeletal conditions. The use of protocols for orthopedic physical examinations is well-established. In 1992, a study led to the development of the Ottawa Ankle Rules, based on objective physical examination criteria. 17 These criteria include a history of torsional trauma with lateral malleolar pain on bony palpation, medial malleolar pain on bony palpation, and an inability to bear weight. Our GPs and orthopedists are trained in the Ottawa Ankle Rules and recognize their significance in orthopedic practice. They routinely utilize them in the clinical setting to manage such injuries. We further apply and adapt these physical examination techniques for diagnosis and treatment via TM-guided orthopedic self-examinations.
The existing body of evidence has revealed methodologies for performing virtual examinations, 18 thus encompassing specific pathologies such as those involving the spine, knee, and hand. 19,20 Virtual examinations for elective orthopedic injuries, such as shoulder rotator cuff issues, have demonstrated comparability to in-person examinations. 21 Bradley’s study, which compared both examination types against MRI results, revealed a 70% accuracy rate for both virtual and in-person methods. 21 This study demonstrated a remarkable 95% accuracy in diagnosing common emergency care pathologies, thus underscoring the feasibility of conducting virtual physical examinations for patients in the ED. This indicates that virtual examinations are noninferior to in-person examinations for low-complexity emergent cases, thus demonstrating their potential efficacy in these emergency care scenarios. Concerning the assessment of range of motion, a limited number of studies have explored the viability of virtual examinations in trauma patients 22 and their application during the postoperative period following orthopedic surgeries. 23,24 These investigations specifically focused on differentiating between flexion and extension, which is a critical consideration for postoperative care and adjustments in physiotherapy. 23
However, rather than distinguishing between flexion and extension, our study concentrated on identifying any alterations in range of motion, thus achieving a concordance rate of 87.5% between TM evaluations and the actual range of motion (AC1 = 0.796; 95% CI: 0.608 to 0.984; p < 0.001). This approach allowed us to yield findings comparable to those in the literature. In our patients in whom trauma was conservatively managed, the range of motion served as an additional factor for guiding orthopedic management decisions. In trauma cases, any limited range of motion observed serves as a criterion warranting further investigation. We have employed and believe in the utility of this criterion for TM applications, thus recognizing its potential value in identifying issues that may require additional attention during virtual examinations.
Abel et al. conducted a study assessing edema and ecchymosis in patients post knee arthroscopy, thus confirming the feasibility of digital assessments. 25 This result aligns with the findings of our study. Similar to knee arthroscopy, the presence of edema can indicate the need for modifications in rehabilitation or changes in approach due to the presence of edema or bruising. In cases of trauma, the presence of such signs indicates the necessity for investigating potential fractures, thus forming the basis of orthopedic examinations leading to the diagnosis of fractures or contusions. We obtained a 100% agreement rate regarding edema in cases of low-complexity trauma. In addition, we observed that a remotely guided inspection achieved moderate agreement with a face-to-face evaluation in the presence of ecchymosis (AC1 = 0.557; 95% CI: 0.281 to 0.833; p < 0.001) and good agreement in the presence of deformities (AC1 = 0.785; 95% CI: 0.608 to 0.963; p < 0.001). These criteria are commonly used in orthopedic practice when fractures and contusions are suspected. We can affirm that the detection of these alterations is feasible. In low-complexity cases, palpation by a health care provider may be substituted by self-palpation of bony structures guided by TM with a GP. Such a factor could streamline patient entry into an emergency service and safely facilitate medical care for these complexities through TM.
In comparisons between in-person and TM medical management, previous studies have shown that even in a digital setting with trained medical teams following protocols, it is possible to reduce the need for complementary examinations and correctly manage low-complexity patients. 24 –26 In our study, we found similar indications for prescribing radiographical and tomographic examinations for the studied pathologies among in-person and TM physicians; however, the indications for prescribing immobilization in ankle sprains, for example, differed. We speculate that this difference may be attributed to the type of training and practice of the GP or even the occasional need for in-person emergency care, which may increase the necessity for immobilizations. In orthopedic practice, multimodal control tools are used for pain management, especially for regulating opioid prescriptions, as outlined in the Clinical Practice Guidelines for Pain Management in Acute Musculoskeletal Injury. 27 However, this approach is not widely used TM prescriptions. Although our study did not include such tools, there are promising reports on the use of TM for pain management. For instance, Benedict and colleagues have effectively used TM to recommend epidural steroid injections for managing lumbar pain. 28
Patient satisfaction with TM orthopedic care was observed to vary. A systematic review encompassing more than 1,008 patients across 12 articles revealed comparable satisfaction levels between TM and in-person care and reduced TM consultation times. 29 In contrast, our study indicated heightened patient satisfaction in TM, specifically regarding waiting times (p < 0.001), thus emphasizing the importance of patient satisfaction in the prospective expansion of this care modality. Our study was conducted in an ED setting, which underscores the potential of TM in addressing acute complaints and delivering patient care.
This study had several limitations. First, the research was conducted in the ED of a private hospital in São Paulo, which caters to a specific patient demographic. TM was utilized to diagnose low-complexity orthopedic injuries through targeted physical examinations. Notably, children and elderly patients were excluded, however future TM implementations may encompass these groups. It is important to note a potential bias in our methodology. Patients underwent teleconsultation and in-person examinations on the same day, thus potentially influencing the study outcomes. To mitigate this risk, the TM doctor was strictly instructed to conduct only anamnesis and a direct physical examination, with the final diagnosis remaining to be subsequently determined by an in-person doctor. This approach was implemented to minimize any impact on the accuracy and impartiality of the final diagnosis.
Conclusions
Diagnostic and management TM consultations of low-risk orthopedic patients performed by GPs are not inferior to face-to-face evaluations in an ED performed by orthopedics. In addition, virtual TM assessments are related to better experiences and improved patient satisfaction.
Footnotes
Acknowledgments
The authors would like to thank Hospital Israelita Albert Einstein for the information technology support provided for telemedicine.
Authors’ Contributions
N.O.F., T.A.D.A, F.T.M., C.H.S.P., K.D.A.L., J.L.S.J.: Conception, planning, analysis, and interpretation of data. R.F.V.C., F.T.M.: Data collection. N.O.F., T.A.D.A, R.A.M., K.D.A.L.: The article’s writing or its critical intellectual review. N.W.: Responsible for the final approval for publication.
Disclosure Statement
All authors declare that they have no conflicts of interest.
Funding Information
The authors received no financial support for the research.