Online Self-Triage of Ear or Hearing Concerns in a Patient Portal: Comparison of Subsequent Diagnoses and Hospitalizations to National Emergency Department and National Ambulatory Ear or Hearing Visits

Abstract

Background

Although online self-triage is easily accessible, little is known about the patients who use self-triage or their subsequent diagnoses. We compared ear/hearing self-triage subsequent diagnoses to ear/hearing visit diagnoses in emergency departments (ED) and ambulatory clinics across the United States.

Methods

We compared International Classification of Diseases version 10 (ICD10) coded diagnoses following online self-triage for ear/hearing concerns with those from national ED and ambulatory clinic samples. We used data from the Centers for Disease Control (CDC) National Hospital Ambulatory Medical Care Survey (NHAMCS) and National Ambulatory Medical Care Survey (NAMCS) for comparison. Using matched ear/hearing diagnostic categories for those aged 1 and over, we compared self-triage diagnosis frequencies with national ED and ambulatory diagnosis frequencies.

Results

Following ear/hearing self-triage, there were 1092 subsequent office visits with a primary diagnosis code. For five frequently diagnosed ear/hearing conditions (i.e., suppurative and nonsuppurative otitis media [OM], otalgia, otitis externa, and cerumen impaction), there was a strong correlation between diagnosis counts made following self-triage and estimated counts of national ED visit diagnoses (r = 0.94; CI 95% [0.37 to 0.99]; p = .016, adjusted r² = 0.85). Seven diagnoses were available to compare with the national ambulatory sample; correlation was r = 0.79; CI 95% [0.08 to 0.97]; p = .037, adjusted r²= 0.54. For ages 1 and over, estimated hospital admissions from the national ED visits for ear/hearing were 0.76%, CI 95% [0.28-2.1%]; estimated total national ear/hearing ED visits were 7.5 million (for 4 years, 2016 through 2019).

Conclusion

The strong correlation of ear-related self-triage diagnoses with national ED diagnoses and the low hospitalization risk for these diagnoses suggests that there is an opportunity for self-triage of ear/hearing concerns to decrease ED visits for these symptoms.

Keywords

triage self-triage otolaryngology patient portal NHAMCS NAMCS emergency visits self-scheduling ear hearing

Introduction

Self-triage tools are increasingly available online and are used worldwide by both national health services and by independent healthcare systems. For example, the National Health Service (NHS) in England has an online self-triage called NHS 111 online.¹ NHS 111 only gives triage advice and states: “111 online will not give you a diagnosis, but we will direct you to the best place to get help for your symptoms.”¹ Australia also has a self-triage tool called “healthdirect Symptom Checker” that gives triage recommendations with the “option of making online bookings with a general practitioner (GP) when advised to consult a health professional.”² The Netherlands has an app called “Should I see a doctor?” that “was developed as a self-triage decision support tool for acute primary care in the Netherlands.”³

Despite international use, relatively little has been published on who uses self-triage and how self-triage outcomes differ from other more traditional ways to access care. Systematic reviews in 2019 and 2022 found few studies looking at diagnostic accuracy or triage accuracy and several were based on simulated patient scenarios.^4,5 The findings suggested overall poor diagnostic accuracy and inconsistent triage accuracy, but variation in study design and results precluded firm conclusions.^4,5 The paucity of quality research on self-triage is, at least in part, due to difficulty in collecting outcome measures as this requires identifiable patient information linked to subsequent visits across the spectrum of care. In the United States (US), collection of these outcomes data is generally attainable only in multidisciplinary institutions with integrated care that can capture self-triage use along with subsequent office visits, emergency department (ED) visits, telemedicine visits, and hospitalizations. Even in a highly integrated healthcare setting, there can be difficulties gathering this information. For example, it is extremely difficult to capture outcomes for patients going to out-of-network EDs or urgent care centers.

At Mayo Clinic, we have been using a self-triage tool, “Check Symptoms” since 2020 and have been collecting self-triage uses and their outcomes. During the COVID pandemic, we were able to examine process outcomes of patients getting a self-triage recommendation to schedule a COVID test.⁶ We were also able to collect subsequent encounter data within 7 days of self-triage for ear or hearing concerns from over 80% of users.⁷ We found that 83.1% of the subsequent office visits scheduled within 7 days of self-triage had at least one associated ear, nose, or throat diagnosis.⁷ Hospitalization rate within 7 days of self-triage was 0.24%.⁷

Our aim with this study was to see how diagnoses, hospitalizations, and demographics of those using self-triage for ear/hearing concerns compare with national data on ED visits and ambulatory visits for similar concerns. Although this study focuses on comparisons with national data, additional information about specific processes and clinical outcomes associated with ear/hearing self-triage is available in this journal.⁷

Methods

Setting

Mayo Clinic is a multispecialty practice located in the US and internationally. This study took place at the Mayo Clinic Rochester (MCR) and Mayo Clinic Health System (MCHS) primary care practices in Minnesota, Iowa, and Wisconsin, USA.

Mayo Clinic uses the Epic® electronic health record (EHR). Medical information in the EHR is divided into “encounters,” which can be used to categorize events in a patient's longitudinal medical care. For this study, we were interested in the initial self-triage encounter and subsequent encounters within 7 days that had documented provider visits associated with diagnosis codes.

Self-Triage Description

Mayo Clinic patients who were registered to Mayo Clinic Patient Online Services (patient portal), and who had a Mayo Clinic primary care provider (PCP) were eligible to use online self-triage that was available 24/7. Patients could sign on to the portal and self-triage with Check Symptoms online or via mobile. The ear/hearing self-triage was just one of several that patients could select from within Check Symptoms. The Mayo Clinic self-triage tool (hereafter called self-triage) uses software algorithms to guide users through question-and-answer pathways that lead to a recommendation for care. Recommendations at the conclusion of the algorithm pathways vary according to the specific self-triage algorithm. For the ear/hearing self-triage algorithm in this study, patients were given one of the following mutually exclusive recommendations: (1) call nurse triage, (2) schedule an appointment, or (3) home care (self-care instructions combined with how to proceed if not improving or for changes in symptoms or signs). Self-triage users with a recommendation of “schedule an appointment” could choose to proceed directly to a self-scheduling page which showed provider appointment slots available for booking. Thus, patients could self-schedule 24/7 for future provider visits. Self-scheduling description, use, and efficiency have been detailed in other published work.^6,8,9 A nurse triage call center was available 24/7 for those getting a recommendation to call nurse triage. Screenshot examples from Check Symptoms are included in Supplemental Files.

Self-Triage Data Collection

From December 18, 2020, to June 22, 2022, we collected user demographics and available subsequent encounter information for 7 days following the completion of self-triage of ear/hearing concerns. The primary diagnosis code for subsequent office visits scheduled within 7 days was captured. The International Classification of Diseases, 10th revision (ICD10) primary diagnosis codes for these subsequent visits were collapsed into 10 categories corresponding to major groupings in the ICD10 coding schema.¹⁰ Ear/hearing diagnostic categories in ICD10 we used were: (1) otitis externa/ear cellulitis, (2) nonsuppurative otitis media (OM), (3) suppurative OM/unspecified OM, (4) cerumen impaction, (5) hearing loss, (6) tinnitus/other ear, (7) otalgia, (8) eustachian tube disorders, (9) tympanic membrane disorder, and (10) mastoid related. The associated specific ICD10 codes within these categories are included in Supplemental Files.

National ED and Ambulatory Visit Data Collection for Comparison

For national comparison data, we used the US Centers for Disease Control (CDC) National Hospital Ambulatory Medical Care Survey (NHAMCS) and National Ambulatory Medical Care Survey (NAMCS) data.¹¹ This national data is publicly available online in datasets formatted for statistics programs with complex survey analytic capabilities such as Stata, SAS, and SPSS.^11–13 We combined all observations from each of 4 years (2016 through 2019) of national ED survey data in NHAMCS to obtain a sample of over 75,000 ED visits with patient demographics and ICD10 diagnosis codes. We also combined all observations from national ambulatory visits from 2016 and 2018, the years that had ICD10 diagnosis codes, for a total sample of over 23,000 ambulatory visits.

Construction of Comparable National ED, National Ambulatory, and Self-Triage Datasets

Both the NHAMCS and NAMCS datasets have a field called “reason for visit” (RFV) that can be matched to our ear/hearing self-triage. The RFV field for national data was developed over 4 decades ago¹⁴ and has undergone revisions since then. Comparison of telephone triage with national ED and ambulatory data using RFV codes has been published previously.¹⁵ For matching our ear or hearing self-triage uses with national data, we searched the national data RFV codebook for reasons for visits consistent with ear or hearing concerns. The following RFV codes matched ear or hearing concerns on our self-triage symptom menu: RFV1 between and including 13450 to 13654, RFV1 = 24550, RFV1 = 24500, and RFV1 = 33600. RVF1 refers to the primary RFVs. NHAMCS and NAMCS also contain RFV2, RFV3, RFV4, and RFV5 fields that correspond to the second through fifth reasons for visit, if they existed. In this study, we only used RFV1, the primary RFV. The complete description of the RFV codes we selected for this study is included in Supplemental Files. The entire list of RFV codes used in the national data can be downloaded from the CDC National Center for Health Statistics.¹⁶

For those ear/hearing self-triage users who had office visits with diagnoses, we could compare ICD10-coded diagnoses following self-triage to those in national data. For the national ED data, we also used the binary ADMITHOS field and OBSDIS field for estimates of hospital admissions and hospital observations followed by discharge associated with ear/hearing ED visits. The NHAMCS and NAMCS datasets have sample weights and sampling units designed to generalize the sample data across the USA.

Exclusions

Self-triage (by proxy) was not available for patients under 1 year of age, so we removed those in that age group from the national comparison groups. For our diagnosis frequency comparison analysis, we excluded self-triage diagnoses with fewer than 30 observations. Also, as recommended by statisticians in the National Center for Health Statistics, we excluded national data where individual non-weighted samples were fewer than 30 observations.^17–19 The exclusions of sample sizes less than 30 for both self-triage data and national data meant that we did not have all 10 self-triage diagnoses or 10 national diagnoses to use for correlation analysis.

Statistics, Analysis, and Ethics

We used Stata® 17.0 (StataCorp, College Station, Texas) for statistical analysis. Using the national sample weights and sampling unit data in the complex survey analysis of Stata,¹⁷ we estimated total national ED and ambulatory visits and the total visits related to the 10 ear or hearing categories. Correlation analysis was used to evaluate the association of self-triage ear/hearing diagnosis counts with the national ED and ambulatory diagnosis counts. We used the complex survey analytics of Stata to calculate the estimated hospital admission and hospital observation rates of national ED patients with ear/hearing RFV.

This study met the institutional review board criteria for exemption (IRB-20-006809).

Results

We found 1092 office visits with primary diagnoses (ICD10 coded) associated with 943 self-triage encounters, from 915 unique patients. There was excellent capture of diagnostic information with 100% (1092/1092) of the subsequent office visit encounters having an associated ICD10 primary diagnosis code. Demographics of self-triage users with subsequent office visit encounters are in Table 1 along with the estimates of national ED and ambulatory visits.

Table 1.

Demographics of Ear/Hearing Self-Triage Users with Subsequent Office Visit Diagnoses Compared to Demographics of US National ED and Ambulatory Visits with Ear/Hearing as RFV.

Demographic	Self-triage ear or hearing with office visit and diagnosis, N = 915 n (%)	Estimated national ED ear or hearing visits, 4-year sample (NHAMCS), age ≥1, N = 7.51 million; n, in thousands (%)	Estimated national ED 95% confidence interval for percent of total	Estimated national ambulatory ear or hearing visits, 2-year sample (NAMCS), age ≥1, N = 31.97 million; n, in thousands (%)	Estimated national ambulatory 95% confidence interval for percent of total
Age (years)
1-17	278 (30.4)	3839 (51.1)	[41.1-61.1]	12 363 (38.7)	[28.7-48.6]
18-34	227 (24.8)	1644 (21.9)	[16.9-26.8]	1856 (5.8)	[3.5-8.1]
35-49	196 (21.4)	913 (12.2)	[9.4-14.9]	3570 (11.2)	[6.5-15.8]
50-64	144 (15.7)	743 (9.9)	[7-12.8]	7368 (23)	[14.2-31.9]
65-79	70 (7.7)	315 (4.2)	[2-6.4]	4488 (14)	[10.6-17.4]
80+	0 (0)	48 (0.6)	[0.1-1.1]	2327 (7.3)	[4-10.5]
Mean age [95%CI]	31.95 [30.6-33.3]	23.1 [21.2-25.1]		38.2 [33.9-42.5]
Sex
Female	616 (67.3)	4068 (54.2)	[44.6-63.7]	17 016 (53.2)	[42-64.4]
Race
White	870 (95.1)	4359 (58)	[45.7-70.4]	19 805 (61.9)	[48.4-75.5]
Black	9 (1)	1595 (21.2)	[15.7-26.8]	1445 (4.5)	[1.9-7.1]
Asian	9 (1)	99 (1.3)	[0.7-1.9]	791 (2.5)	[0.9-4.1]
Other	21 (2.3)	77 (1)	[0.2-1.9]	401 (1.3)	[0-2.6]
Unknown/Not disclosed	6 (0.7)	1382 (18.4)	[14-22.8]	9529 (29.8)	[20.3-39.3]
Ethnicity
Hispanic	31 (3.4)	1289 (17.2)	[12.1-22.2]	2546 (8)	[4.2-11.7]
Not Hispanic	876 (95.7)	4896 (65.2)	[52.6-77.7]	18 744 (58.6)	[46.6-70.7]
Undisclosed/Unknown	8 (0.9)	1326 (17.7)	[12.4-22.9]	10 682 (33.4)	[23.1-43.8]

Figure 1 shows the data flow for the complex survey analysis that was used to calculate the national estimates in Table 1.

Figure 1.

Flow diagram showing steps used in creating subsets of NHAMCS and NAMCS data for complex survey analysis.

Ear/Hearing Diagnostic Category Counts of Self-Triage, National ED, and Ambulatory Visits

Table 2 shows estimated national ED and ambulatory ear/hearing visit counts with self-triage subsequent visit counts by each diagnostic category. Of the 1092 office visits following self-triage, 67.4% (736/1092) had a primary diagnosis in 1 of the 10 ear/hearing categories described in Methods. Of the estimated 32 million (M) national ambulatory visits for ear/hearing (over 2 years), 70.0% (22.4 M /32.0 M) had a primary diagnosis within the 10 categories. For the estimated 7.5 M national ED visits for ear/hearing (over 4 years), 73.3% (5.5 M/7.5 M) had a primary diagnosis in the 10 ear/hearing categories.

Table 2.

Self-Triage Diagnosis Category Counts and Estimated National Ambulatory and ED Diagnosis Category Counts.

Diagnosis category of primary diagnosis	Self-triage subsequent office visit count (%)	Estimated national ED visits, age ≥1, for 4 years, 2016-2019; n, in thousands (%)	95% CI for estimated national ED visits, age ≥1, for 4 years, 2016-2019; n, in thousands	Estimated national ambulatory visits, age ≥1, for 2 years, 2016 and 2018; n, in thousands (%)	95% CI for estimated national ambulatory visits, age ≥1, for 2 years, 2016 and 2018; n, in thousands
Ear/Hearing total	1092 (100)	7512 (100)	[6313-8710]	31 971 (100)	[28 300-35 600]
Suppurative OM	204 (19)	2510 (33)	[1945-3075]	6155 (19)	[4873-7438]
Otitis externa	148 (14)	1027 (14)	[757-1297]	2709 (8)	[1655-3763]
Otalgia	97 (9)	1013 (13)	[781-1246]	2320 (7)	[1080-3560]
Nonsuppurative OM	78 (7)	357 (5)	[235-478]	1163 (4)	[529-1797]
Cerumen impacted	64 (6)	325 (4)	[203-447]	3685 (12)	[2761-4608]
Tinnitus/other	63 (6)	129 (2)^b	[46-212]	1300 (4)	[688-1912]
Eustachian	46 (4)	23 (0)^b	[1-44]	1662 (5)	[871-2454]
Hearing loss	28 (3)^a	49 (1)^b	[0-98]	2729 (9)	[1691-3766]
Tympanic membrane	8 (1)^a	104 (1)^b	[37-170]	532 (2)^b	[212-852]
Mastoid	0 (0)^a	4 (0)^b	[0-11]	112 (0)^b	[2-223]

CI = confidence interval; ED = emergency department; OM = otitis media.

Self-triage subsequent diagnosis counts not reaching a priori sample count of ≥30.

National data survey observations (counts) not reaching CDC threshold of ≥30 for estimation accuracy with complex survey analytic technique.

The top five most frequent diagnoses following self-triage for ear/hearing correlated strongly with the frequency rank of national ED diagnoses (Spearman's ρ = 1.0, test of independence p < .0001), but not with frequency rank of national ambulatory diagnoses (Spearman's ρ = 0.40, test of independence p = .50).

Correlations of Ear or Hearing Diagnoses Counts Between Self-Triage and National Data

As shown in Table 2, the office visit diagnoses of hearing loss, tympanic membrane disorder, and mastoid disorder did not meet our sample size criteria ≥30) for comparison with national data. Also shown are five national ED visit diagnoses that did not meet criteria (≥30 observations) for estimation accuracy. These were tympanic membrane (18 observations), tinnitus/other (14), hearing loss (8), eustachian tube (5), and mastoid (1).

The sample size limitations left us with outcomes comparisons of five diagnoses between self-triage and national ED visits and seven diagnoses for comparison between self-triage and national ambulatory visits. Scatterplots of self-triage diagnosis counts (x-axis) are shown against the national estimated diagnosis counts (y-axis) for both ED visits (Figure 2) and ambulatory visits (Figure 3).

Figure 2.

Scatterplot of estimated national ED visit primary diagnosis counts by self-triage primary diagnosis counts (with linear regression line). Vertical bars are estimated 95% confidence range of ED primary diagnosis counts using complex survey methodology. OM = otitis media.

Figure 3.

Scatterplot of estimated national ambulatory visit primary diagnosis counts by self-triage primary diagnosis counts (with linear regression line). Vertical bars are estimated 95% confidence range of ambulatory primary diagnosis counts using complex survey methodology. OM = otitis media.

For the five most frequently diagnosed ear or hearing conditions in self-triage (suppurative and nonsuppurative OM, otalgia, otitis externa, and cerumen impaction), there was a strong correlation of the counts of these diagnostic categories with estimated national ED visits (r = 0.94; CI 95% [0.37 to 0.99]; p = .016, adjusted r² = 0.85). Seven diagnoses were available in the national ambulatory sample to compare but they were less well correlated with self-triage outcomes (r = 0.79; CI 95%, [0.08 to 0.97]; p = .037, adjusted r² = 0.54).

Sensitivity Analysis for Correlation Statistics

The majority of the face-to-face visit primary diagnoses came from subsequent office visits. However, there were also 125 subsequent ED visits captured in our 7-day outcomes data. Only 34% (42/125) of these ED visits had a primary diagnosis in the 10 ear or hearing categories so we did not include these in our primary analysis. Although these visits represent a small number of diagnoses (42) compared to the number of office visit diagnoses with ear/hearing diagnoses (736), we did a sensitivity analysis by adding these to our office visits. With the addition of these Mayo ED visits to the subsequent office visits, the correlation of the five self-triage ear/hearing diagnoses to national ED data went from 0.944 (CI 95% 0.37 to 0.99, p = .016) to 0.934, (CI 95% 0.33 to 0.99, p = .018). For the comparison with national ambulatory data, addition of the self-triage ED visit outcome diagnoses resulted in the correlation changing from 0.785 (CI 95% 0.08 to 0.97, p = .037) to 0.779 (CI 95% 0.06 to 0.97, p = .039). By including the comparatively small number of ED diagnoses the correlation changes very little.

Hospital Admission and Hospital Observation Comparison

For national ED visits with ear or hearing reasons for visit and age ≥1, hospital admissions are estimated at 0.76%, with CI 95% [0.28%-2.1%] based on a sample size of 5 hospitalizations out of 1044 surveyed visits. Hospital observations in national ED data followed by dismissal were estimated at 0.68%, with CI 95% [0.23%-2.0%] based on 10 of 1044 surveyed visits. The ear/hearing self-triage hospitalization rate within 7 days following triage was 0.24%.⁷

Discussion

Principal Findings

Self-triage diagnosis outcome frequencies for ear or hearing are strongly correlated with those for national ED visits and somewhat less correlated with national ambulatory visits. For ear or hearing concerns, national ED patients and our self-triage users had a low risk of hospitalization.

Patient Implications

The national data shows that millions of ED visits are for patients with ear or hearing concerns that are diagnosed with OM, otitis externa, and otalgia. These are all diagnoses that are commonly managed in office visits. Online and mobile 24/7 access to self-triage and self-scheduling could potentially give patients information that would help avoid an ED visit while also scheduling an office visit or a recommendation for alternative care other than the ED. National ED data and our self-triage outcomes data show low risk for ear or hearing concerns needing subsequent hospitalization. Thus, an office visit appears to be a safe choice for most of these patients. Self-triage and self-scheduling could potentially expedite that.

Although we did not compare patient time in ear/hearing self-triage and self-scheduling with patient times getting other forms of access, studies with self-triage and self-scheduling for COVID have shown this to be very efficient way to get scheduled care. For example, at Mayo Clinic, the median time for a patient to self-triage and self-schedule COVID test appointments was less than 6 min.⁶ At University of California San Francisco (UCSF), 88% of patients being triaged for COVID testing used the online symptom checker.²⁰

Practice Implications

ED overcrowding has become a major problem, and nonurgent visits are one of the causes.²¹ Our analysis of national ED visits shows that ear or hearing concerns were estimated to account for more than 7.5 million visits to US emergency departments from 2016 through 2019. The five diagnoses that correlated extremely well with the self-triage diagnoses (suppurative and nonsuppurative OM, otalgia, cerumen, and otitis externa) accounted for 5.2 million visits or 69% of the total.

Our analysis shows that ear or hearing outcomes are usually medically nonurgent conditions based on low admission rates both in our self-triage uses as well as the national ED data. Additional analysis of the national data (not reported in Results) shows that an estimated 2.3% [CI 95%; 1.3% to 4.2%] of national ear/hearing ED visits were classified as immediate or emergent in ED triage on a 5-point ordinal scale of immediate (1) to nonurgent (5). With ED ear/hearing outcomes so closely correlated with self-triage and associated with low risk, it is possible that more general use of self-triage and self-scheduling could decrease the ear/hearing nonurgent ED visits.

Self-triage with self-scheduling has been shown in other studies to save a large amount of staff time. For example, the Mayo Clinic’s experience with online COVID self-triage and self-scheduling showed savings of an estimated 77,520 COVID nurse triage calls and 136,252 staff appointment calls.⁶ This translated to a saving of 12,822 nursing hours and 2913 appointment scheduler hours.⁶ Judson et al had similar results at UCSF where they had 13,729 uses of their self-triage and self-scheduler for COVID and saved over 4200 clinician hours.²⁰ Future studies will need to determine if self-triage and self-scheduling of ear/hearing and other symptoms can result in similar efficiencies.

Limitations

This study has several limitations. There were significant differences in age, race, sex, and ethnicity between self-triage users, and those in national ED and ambulatory visits. There were subsequent encounters in the Epic EHR within 7 days of self-triage for only 80.5% of the self-triage uses.⁷ Thus, we lack subsequent 7 day follow-up information for 19.5% of self-triage ear/hearing uses. However, it is notable that 83.1% of ear/hearing self-triage uses resulting in an office visit were associated with relevant ear, nose, and throat primary diagnoses.⁷

Our self-triage data collection does not match the same time frame of data collection for national data. The COVID pandemic during self-triage data collection could have altered the ear/hearing diagnosis frequencies. Our self-triage ear/hearing algorithm was developed by Mayo Clinic staff and is not yet generalizable to other practices. We limited our analysis to primary diagnosis codes both in our data and the national data.

Like other practices, we do not have complete interoperability to obtain outcomes when patients interacted with other healthcare systems or standalone urgent care centers. However, we limited this study to Mayo Clinic patients in more rural areas where primary care patients may be less likely to get care from a competing clinic. Standalone urgent care clinics are also not common in less populated areas where many of the study patients reside. Despite some potential advantages with data collection, having self-triage patients from less densely populated areas in North Central USA is an additional limitation. The national data is based on samples from urban, suburban, and rural populations designed to represent all of the USA. Mayo self-triage also requires portal registration and internet connectivity which is not required for inclusion in the national ED and ambulatory data.

The national data is limited to the years that we could match ICD10 diagnoses with our ICD10 diagnosis codes. National data started using the ICD10 coding in 2016 for both ED and ambulatory data. At the time of the analysis, there were four years of ICD10-coded national ED data (2016 through 2019) and only 2 years of data for the national ambulatory data (2016 and 2018). Our estimate of hospital admission percent and hospital observation percent following national ED visits was based on only 5 and 10 survey visits, respectively, below the 30 survey visit count suggested for estimation. We also have no comparison measure for the severity of disease encountered other than our comparison of hospitalizations following self-triage and national ED visits.

Conclusion

There was a strong correlation of self-triage diagnosis frequencies with national ED diagnoses for five frequently diagnosed ear or hearing conditions (suppurative and nonsuppurative OM, otalgia, otitis externa, and cerumen impaction). Both national ED visits and self-triage visits for ear or hearing had low risk for subsequent hospitalization.

Supplemental Material

sj-jpg-1-hme-10.1177_23333928231186209 - Supplemental material for Online Self-Triage of Ear or Hearing Concerns in a Patient Portal: Comparison of Subsequent Diagnoses and Hospitalizations to National Emergency Department and National Ambulatory Ear or Hearing Visits

Supplemental material, sj-jpg-1-hme-10.1177_23333928231186209 for Online Self-Triage of Ear or Hearing Concerns in a Patient Portal: Comparison of Subsequent Diagnoses and Hospitalizations to National Emergency Department and National Ambulatory Ear or Hearing Visits by Frederick North, Teresa B Jensen, Jennifer Pecina, Nathaniel E Miller, Michelle Duvall, Elissa M Nelson, Matthew C Thompson, Brenda J Johnson, Brian A Crum and Robert Stroebel in Health Services Research and Managerial Epidemiology

Supplemental Material

sj-jpg-2-hme-10.1177_23333928231186209 - Supplemental material for Online Self-Triage of Ear or Hearing Concerns in a Patient Portal: Comparison of Subsequent Diagnoses and Hospitalizations to National Emergency Department and National Ambulatory Ear or Hearing Visits

Supplemental material, sj-jpg-2-hme-10.1177_23333928231186209 for Online Self-Triage of Ear or Hearing Concerns in a Patient Portal: Comparison of Subsequent Diagnoses and Hospitalizations to National Emergency Department and National Ambulatory Ear or Hearing Visits by Frederick North, Teresa B Jensen, Jennifer Pecina, Nathaniel E Miller, Michelle Duvall, Elissa M Nelson, Matthew C Thompson, Brenda J Johnson, Brian A Crum and Robert Stroebel in Health Services Research and Managerial Epidemiology

Supplemental Material

sj-jpg-3-hme-10.1177_23333928231186209 - Supplemental material for Online Self-Triage of Ear or Hearing Concerns in a Patient Portal: Comparison of Subsequent Diagnoses and Hospitalizations to National Emergency Department and National Ambulatory Ear or Hearing Visits

Supplemental material, sj-jpg-3-hme-10.1177_23333928231186209 for Online Self-Triage of Ear or Hearing Concerns in a Patient Portal: Comparison of Subsequent Diagnoses and Hospitalizations to National Emergency Department and National Ambulatory Ear or Hearing Visits by Frederick North, Teresa B Jensen, Jennifer Pecina, Nathaniel E Miller, Michelle Duvall, Elissa M Nelson, Matthew C Thompson, Brenda J Johnson, Brian A Crum and Robert Stroebel in Health Services Research and Managerial Epidemiology

Supplemental Material

sj-jpg-4-hme-10.1177_23333928231186209 - Supplemental material for Online Self-Triage of Ear or Hearing Concerns in a Patient Portal: Comparison of Subsequent Diagnoses and Hospitalizations to National Emergency Department and National Ambulatory Ear or Hearing Visits

Supplemental material, sj-jpg-4-hme-10.1177_23333928231186209 for Online Self-Triage of Ear or Hearing Concerns in a Patient Portal: Comparison of Subsequent Diagnoses and Hospitalizations to National Emergency Department and National Ambulatory Ear or Hearing Visits by Frederick North, Teresa B Jensen, Jennifer Pecina, Nathaniel E Miller, Michelle Duvall, Elissa M Nelson, Matthew C Thompson, Brenda J Johnson, Brian A Crum and Robert Stroebel in Health Services Research and Managerial Epidemiology

Supplemental Material

sj-xlsx-5-hme-10.1177_23333928231186209 - Supplemental material for Online Self-Triage of Ear or Hearing Concerns in a Patient Portal: Comparison of Subsequent Diagnoses and Hospitalizations to National Emergency Department and National Ambulatory Ear or Hearing Visits

Supplemental material, sj-xlsx-5-hme-10.1177_23333928231186209 for Online Self-Triage of Ear or Hearing Concerns in a Patient Portal: Comparison of Subsequent Diagnoses and Hospitalizations to National Emergency Department and National Ambulatory Ear or Hearing Visits by Frederick North, Teresa B Jensen, Jennifer Pecina, Nathaniel E Miller, Michelle Duvall, Elissa M Nelson, Matthew C Thompson, Brenda J Johnson, Brian A Crum and Robert Stroebel in Health Services Research and Managerial Epidemiology

Supplemental Material

sj-xlsx-6-hme-10.1177_23333928231186209 - Supplemental material for Online Self-Triage of Ear or Hearing Concerns in a Patient Portal: Comparison of Subsequent Diagnoses and Hospitalizations to National Emergency Department and National Ambulatory Ear or Hearing Visits

Supplemental material, sj-xlsx-6-hme-10.1177_23333928231186209 for Online Self-Triage of Ear or Hearing Concerns in a Patient Portal: Comparison of Subsequent Diagnoses and Hospitalizations to National Emergency Department and National Ambulatory Ear or Hearing Visits by Frederick North, Teresa B Jensen, Jennifer Pecina, Nathaniel E Miller, Michelle Duvall, Elissa M Nelson, Matthew C Thompson, Brenda J Johnson, Brian A Crum and Robert Stroebel in Health Services Research and Managerial Epidemiology

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Frederick North

Jennifer Pecina

Nathaniel E Miller

Michelle Duvall

Elissa M Nelson

Matthew C Thompson

Brenda J Johnson

Brian A Crum

Robert Stroebel

Data Availability Statement

Data for the national emergency department and national ambulatory visits are publicly available and can be downloaded from the sources listed in references.

Supplemental Material

Supplemental material for this article is available online.

Author Biographies

Frederick North is an Associate Professor of Medicine in the Mayo Clinic College of Medicine and Science, USA.

Teresa B Jensen is an Assistant Professor of Family Medicine in the Mayo Clinic College of Medicine and Science, USA.

Jennifer Pecina is an Associate Professor of Family Medicine in the Mayo Clinic College of Medicine and Science, USA.

Nathaniel E Miller is an Assistant Professor of Family Medicine in the Mayo Clinic College of Medicine and Science, USA.

Michelle Duvall is an Assistant Professor of Family Medicine in the Mayo Clinic College of Medicine and Science, USA.

Elissa M Nelson is an Instructor in Health Care Administration in the Mayo Clinic College of Medicine and Science. She is Senior Director of Infrastructure Services in the Enterprise Office of Access Management, Mayo Clinic, USA.

Matthew C Thompson is Director of Demand and Digital Analytics in the Enterprise Office of Access Management, Mayo Clinic, USA.

Brenda J Johnson is a Business Analyst in the Systems Transformation Unit within the Enterprise Office of Access Management, Mayo Clinic, USA.

Brian A Crum is an Associate Professor of Neurology, Mayo Clinic College of Medicine and Science. He is Medical Director of the Enterprise Office of Access Management, Mayo Clinic, USA.

Robert Stroebel is an Associate Professor of Medicine in the Mayo Clinic College of Medicine and Science, Mayo Clinic, USA.

References

NHS 111 online, 2022. Accessed March 22, 2023. https://111.nhs.uk/.

Healthdirect Symptom Checker. Accessed March 22, 2023. https://about.healthdirect.gov.au/healthdirect-symptom-checker.

Verzantvoort

NCM

Teunis

Verheij

TJM

van der Velden

. Self-triage for acute primary care via a smartphone application: practical, safe and efficient? PLoS One. 2018;13(6):e0199284. doi:10.1371/journal.pone.0199284

Chambers

Cantrell

Johnson

, et al. Digital and online symptom checkers and health assessment/triage services for urgent health problems: systematic review. BMJ Open. 2019;9(8):e027743. doi:10.1136/bmjopen-2018-027743

Wallace

Chan

Chidambaram

, et al. The diagnostic and triage accuracy of digital and online symptom checker tools: a systematic review. NPJ Digit Med. 2022;5(1):118. doi:10.1038/s41746-022-00667-w

North

Nelson

Majerus

Thompson

Knutson

Crum

. Self-scheduling process efficiency and utilization of online self-scheduling of lab tests: a retrospective analysis of self-scheduled appointments for COVID testing. Health Serv Res Manag Epidemiol. 2022;9:23333928221125034. doi:10.1177/23333928221125034

North

Jensen

Stroebel

, et al. Self-triage use, subsequent healthcare utilization, and diagnoses: a retrospective study of process and clinical outcomes following self-triage and self-scheduling for ear or hearing symptoms. Health Serv Res Manag Epidemiol. 2023;10:23333928231168121. doi:10.1177/23333928231168121

North

Nelson

Buss

Majerus

Thompson

Crum

. The effect of automated mammogram orders paired with electronic invitations to self-schedule on mammogram scheduling outcomes: observational cohort comparison. JMIR Med Inform. 2021;9(12):e27072. doi:10.2196/27072

North

Nelson

Majerus

Buss

Thompson

Crum

. Impact of web-based self-scheduling on finalization of well-child appointments in a primary care setting: retrospective comparison study. JMIR Med Inform. 2021;9(3):e23450. doi:10.2196/23450

10.

International statistical classification of diseases and related health problems, 10th revision. Accessed March 26, 2023. https://icd.who.int/browse10/Content/statichtml/ICD10Volume2_en_2016.pdf.

11.

Ambulatory health care data survey methods and analytic guidelines, 2023. Accessed March 26, 2023. https://www.cdc.gov/nchs/ahcd/survey_methods.htm.

12.

Ambulatory health care data, datasets and documentation. Accessed March 26, 2023. https://www.cdc.gov/nchs/ahcd/datasets_documentation_related.htm.

13.

Using ultimate cluster models with NAMCS and NHAMCS public use files, 2004. Accessed March 26, 2023. https://www.cdc.gov/nchs/data/ahcd/ultimatecluster.pdf.

14.

Schneider

Appleton

McLemore

. A reason for visit classification for ambulatory care. DHEW Publication. No. 78;1979. DHEW Publication No. (PHS) 79-1352.

15.

North

Varkey

. How serious are the symptoms of callers to a telephone triage call centre? J Telemed Telecare. 2010;16(7):383-388. doi:10.1258/jtt.2010.091016

16.

2019 NHAMCS micro-data file documentation, 2019. Accessed March 26, 2023. https://ftp.cdc.gov/pub/Health_Statistics/NCHS/Dataset_Documentation/NHAMCS/doc19-ed-508.pdf, https://ftp.cdc.gov/pub/Health_Statistics/NCHS/Dataset_Documentation/NHAMCS/, https://www.cdc.gov/nchs/ahcd/datasets_documentation_related.htm.

17.

Understanding and interpreting the National Hospital Ambulatory Medical Care Survey (NHAMCS): key questions and answers. Accessed March 26, 2023. https://www.cdc.gov/nchs/data/ahcd/Annals_Emerg_Med_Q_and_A_NCHS_Web_Version.pdf.

18.

Sample text for describing NHAMCS in a research article. Accessed March 26, 2023. https://www.cdc.gov/nchs/data/ahcd/Sample_Text_for_Describing_NHAMCS_in_Research_Article.pdf.

19.

Hsiao

C-J

. Understanding and using NAMCS and NHAMCS data, 2010. Accessed March 26, 2023. https://www.cdc.gov/nchs/ppt/nchs2010/03_hsiao.pdf.

20.

Judson

Pierce

Tutman

, et al. Utilization patterns and efficiency gains from use of a fully EHR-integrated COVID-19 self-triage and self-scheduling tool: a retrospective analysis. J Am Med Inform Assoc. 2022;29(12):2066-2074. doi:10.1093/jamia/ocac161

21.

Kenny

Chang

Hemmert

. Factors affecting emergency department crowding. Emerg Med Clin North Am. 2020;38(3):573-587. doi:10.1016/j.emc.2020.04.001

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.08 MB

0.05 MB

0.06 MB

0.13 MB

0.01 MB

0.02 MB