Natural language processing to describe primary care requests for eConsult specialty care: A simple and practical application

Abstract

Objective: Free text fields embedded within electronic consultation (eConsult) orders serve as rich sources of descriptive information regarding common uses of this novel telehealth technology. Simple text mining and language processing may efficiently extract key insights that help inform providers and administrators. Methods: Text data from eConsult orders placed within a single academic medical center were extracted from the electronic health record and examined. N-gram frequencies were used to describe the content of eConsult clinical questions and care recommendations. Results: 18,609 eConsults were ordered, with volumes ranging from 12 to 3839 orders across 28 subspecialties. Median character length for the clinical question was 189 and 1393 for specialist response text. Frequency count for top bigram varied greatly by specialty, with a high of 190 (“thyroid nodule”) in Endocrinology and a low of 6 (“shoulder pain”) in Orthopedics for clinical questions, and a high of 3139 (“ref range”) in Endocrinology and a low of 6 (“surgical oncology”) in Medical Oncology for specialist response. Discussion: Descriptive word sequences from NLP may provide limited insight into common use cases for eConsult across many subspecialties, though pre-processing was required to generate meaningful results.

Keywords

natural language processing [L01.224.050.375.580]information science [L01]computing methodology [L01.224]practice patterns physicians’ [N05.300.625]data mining [L01.470.625]

Background and significance

Electronic consultation (eConsult) is being deployed across numerous healthcare systems to increase access to specialty care. This peer-to-peer telehealth model serves as a means for patient care coordination via asynchronous communication exchanges originating from a primary care provider (PCP) to a specialist provider regarding specific, patient-focused clinical questions.¹ Unlike traditional telemedicine, this model is not an interaction between a doctor and a patient but between two providers to support the management of specialty conditions in the primary care setting. eConsult order templates containing structured and unstructured data fields are embedded within the electronic health record (eHR). Pre-defined or structured data fields are convenient, though they are limited in their ability to tell the story of data. Clinical questions posed by PCPs can be complex; often, there are no viable options for capturing the breadth of the clinical question in a structured field. Likewise, the response, recommendations, and actions suggested by the specialist provider can be nuanced and variegated, necessitating a free-text data field to capture fully. Therefore, a great deal of potentially valuable information related to using eConsult is stored in unstructured text.

A burgeoning body of research has documented the various applications, successes, and barriers to using Natural Language Processing (NLP) in healthcare data at large. Preliminary studies have shown the feasibility of NLP to identify patients with various health conditions such as COVID-19 infection,² geriatric conditions,³ heart failure and diabetes,⁴ to differentiate lung cancer characteristics,⁵ to detect adverse drug events,⁶ to capture information on the patient’s perspective and goals of care,^7,8 as well as the capacity to meaningfully summarize large sets of eHR data.⁹ Successful application of NLP to eHR data in other areas of healthcare prognosticates the potential for this technique to inform eConsult-specific studies, which are needed to better understand the utilization of eConsult technology.

Although the usage of eConsult care modality has continued to increase, research to succinctly describe the most common use cases across various specialty care areas is still lacking. Descriptive studies aimed at understanding the content of eConsult exchanges have been mainly targeted at a single specialty,^10,11 required human coding or chart review,^12,13 and often linked content into pre-defined categories.^14,15 Research specific to the use of artificial intelligence (AI) for eConsult has demonstrated limited ability to classify eConsult question types into pre-determined categories related to diagnoses and procedures. Still, these approaches require human labeling of training data to produce results.¹⁶ Likewise, a 2022 study demonstrated success in understanding usage patterns for an eConsult system.¹⁷ However, it required pre-trained transformers and focused more on building and comparing models to accurately classify, but not necessarily describe, text related to eConsult questions. These resource-heavy, analytically complex approaches require hours of hand-coding and advanced handling of language models, which may not be realistic for healthcare systems with varying technical or analytic prowess. Healthcare providers, administrators, and policy-makers need easy and conveniently displayed information regarding the most common uses for eConsult within their systems to fully capture the potentially rich source of data available through this model and to improve decision-making.

Objective

We seek to evaluate the application of simple text mining and NLP techniques on free-text fields to describe the nature of eConsult requests and responses across all subspecialties. The primary goal of this project work is to assess the feasibility of simple language processing techniques to extract meaningful insight from raw, unlabeled, and minimally pre-processed eConsult order data. Provided that the primary goal results in meaningful content, the secondary goal is to describe the top ten clinical reasons for eConsult requests made by PCPs for each specialty area, along with the top ten recommendations or clinical actions suggested by the specialist provider.

Materials and methods

Program and setting

The Association of American Medical Colleges launched its project Coordinating Optimal Referral Experiences (CORE) in 2018 to improve the referral experience for clinicians and patients through tools that can be built into the eMR.¹⁸ The University of Colorado School of Medicine joined the CORE initiative in April 2018 by designing infrastructure and processes to connect PCPs with providers from 28 specialties.

Study design and data analysis

For this retrospective data analysis, we extracted eConsult exchanges between the PCP and specialist provider from the electronic health record (Epic). Each eConsult exchange potentially contains two free-text fields: one for the ordering PCP to describe their clinical question and one for the responding specialist to describe their recommendations based on information provided by the PCP. The responding specialist can reject or decline the eConsult request if they feel the PCP’s question is inappropriate (logistical/administrative in nature or better addressed by a different specialty provider). They also have the option to convert the eConsult to an in-person visit if they feel that the question or patient is too medically complex to be handled electronically. For this reason, not every clinical question has a specialist response. We examined all clinical questions available, regardless of whether they received a response. There is no character limit for the clinical question or specialist response free-text fields within the eHR.

Data from a 3-month ramp-up period during which providers gained hands-on experience using the eConsult technology was excluded from the analysis. All remaining data from the beginning of July 2018 through July 2023 was eligible for inclusion. Data was not filtered for race, age, gender, presenting medical problem, or other criteria. This exploratory, descriptive study used a convenience sample. As such, no sample size estimations were conducted. Data was exported from Epic “as-is” with no preprocessing performed. We expected large variations in the usage of eConsult technology across specialties. For this reason, we investigated the sample size for each specialty and excluded observations from those specialties with fewer than 100 eConsult requests from further analysis.

We used bigram analysis for each free-text field broken out by specialty to describe the nature of clinical requests and specialist responses. Basic text mining preprocessing was applied to the text to remove punctuation and numbers, transform all text to lowercase, and strip the white space. A list of common English stop words available through the tidytext package in R was used to eliminate words that do not contribute to the meaning or interpretation of the text. A complete list of stop words can be found in the Supplemental Material.¹⁹ Observations that were blank following pre-processing were excluded from further analysis. We intentionally limited the pre-processing application to evaluate this technique’s efficiency on raw data. Bigrams are presented as the raw number of times the word pair appeared in text and the relative frequency of the word pair amongst all words in the text across documents for each specialty. Histograms of the top ten bigram frequencies were used to visualize the results.

All analysis for this study was conducted using RStudio version 4.2.2.²⁰

This work was determined to be non-human subjects research/not research under the regulations by the Colorado Multiple Institute Board (Protocol #24-0911).

Results

Specialty volumes & character counts

A total of 18,609 eConsults were ordered across all 28 specialties. The frequency of eConsult usage varied greatly across specialties, with a minimum of 12 orders for Palliative Care and a maximum of 3839 orders for Endocrinology. See Table 1 for complete details. Eight specialties with a combined total of 302 observations were not analyzed due to small sample size (<100 eConsult orders each). On average, ∼81% of clinical questions across all specialties had a specialist response (the order was not converted/denied or left blank after pre-processing). 1517 observations with missing clinical questions were removed, leaving 16,790 clinical questions for analysis. 4684 observations with a blank specialist response (presumed converted or denied) were removed, resulting in 13,623 specialist responses available for analysis. The richness of the text varied greatly for both free-text data fields. Following preprocessing, the character count for the clinical question data field ranged from 2 to 26,953, with a median of 189 characters. Character count for the specialist response field ranged from 8 to 7,739, with a median character count of 1393.

Table 1.

Raw number of eConsults ordered by specialty with percent of total.

Specialty	Number of eConsults ordered	Percent of total ordered
Endocrinology	3839	20.6
Cardiology	1830	9.8
Dermatology	1650	8.9
Psychiatry	1517	8.2
Neurology	1352	7.3
Gastroenterology	1213	6.5
Pharmacy	1077	5.8
Infectious disease	1058	5.7
Nephrology/renal	750	4
Rheumatology	719	3.9
Hepatology	645	3.5
Urology	501	2.7
Ear, nose and throat	469	2.5
Hematology	365	2
Pulmonary	340	1.8
Obstetrics/gynecology	285	1.5
Sleep medicine	217	1.2
Orthopedic	167	0.9
Allergy	160	0.9
Medical oncology	153	0.8
Ophthalmology	76	0.4
Stroke	63	0.3
Pain clinic	61	0.3
Wound care	31	0.2
Survivorship	22	0.1
Urology	19	0.1
Physical medicine rehabilitation	18	0.1
Palliative care	12	0.1

Preprocessing

Text preprocessing of the clinical question revealed two issues that occurred commonly enough to warrant correction before n-gram analysis. First, if the exact two words were present in the text but ordered differently, language processing techniques count them as distinct bigrams. For example, “chest ct” would be analyzed separately from “ct chest.” Although our innate human capabilities allow us to ascertain that these bigrams represent the same clinical request, simple NLP techniques did not combine the frequency for each phrase without additional preprocessing before analysis. Second, providers frequently abbreviated the word “cancer” with “ca”, which language processing techniques will count separately without custom preprocessing. To correct these issues, we examined an initial plot of the top bigrams for each specialty and substituted text strings where needed.

A greater amount of preprocessing was required for the specialist response text. The repeated presence of certain phrases, such as the institutional liability statement, suggests that non-natural language was entered into the free-text response, likely through the use of a “dot phrase” or “quick text”. Words from statements such as “I spent a total of x minutes” or “please contact me if you have further questions” overwhelmed the specialist response bigrams. Some statements appeared to be used as section headers for different parts of the specialist’s response, for example, the words “assessment/plan,” “rationale, and/or evidence for a response.” Additionally, some providers appeared to copy/paste a restatement of the question into their free-text response. These issues decreased our ability to apply NLP to the raw specialist response text without additional, customized data prep.

Clinical question

Frequency counts for top bigrams varied greatly by specialty. The highest bigram frequencies were seen among specialties with the largest numbers of eConsults ordered. For example, the term “thyroid nodule” occurred 190 (2.4%) times among all endocrinology questions. Specialties with fewer eConsults ordered demonstrated smaller relative frequencies for top bigrams. For example, “shoulder pain” (n = 6, 0.5%) and “hip pain” (n = 4, 0.3%) were the top bigrams for the Orthopedic specialty. See Figure 1 for the top 10 bigrams of the clinical question text by specialty. Histograms of the top bigrams for clinical questions of each specialty, along with raw counts and relative frequencies, can be found in Supplemental Materials.

Figure 1.

Top 10 clinical question bigrams by specialty.

Certain bigrams appeared in the top ten list of more than one specialty. We noted the 3 most common terms across all specialties combined: “weight loss” occurred 260 (0.04%) times in the text of 15 different specialties. The phrase “abdominal pain” occurred 226 (0.04%) times across 16 specialties. “Thyroid nodule” occurred 210 times in the text of seven different specialties. See Figure 2 for the top five clinical question bigrams across specialties.

Figure 2.

Histogram of top 10 clinical question bigrams across all specialties.

Specialist response

Top bigram frequency ranged by specialty from 3139 (19%) in Endocrinology to 6 (1.3%) in Medical Oncology. The top bigram for Endocrinology was more than 5 times more frequent than the top bigram for any other specialty. As such, the top bigrams for all specialties combined were heavily influenced by Endocrinology data. See Figure 3 for details. The highest bigram frequency for any specialty other than Endocrinology was the phrase “max dose,” which occurred 624 (5.1%) times among the Psychiatry eConsults. Recommendations from specialists spanned from pharmaceutical (e.g., statin therapy, Cardiology, n = 294, 2.8%) and other therapeutic interventions (e.g., auto-titrating cpap, Sleep Medicine, n = 17, 0.9%), to diagnostic procedures (e.g., upper endoscopy, Gastroenterology, n = 65, 1.3%; mri brain, Neurology, n = 457, 4.6%) and non-invasive treatment (e.g., smell retraining, Ear/Nose/Throat, n = 19, 0.5%), to preventative actions (e.g., sun avoidance, Dermatology, n = 105, 1.1%) and additional screenings and outside resources (e.g., cdc information, Allergy, n = 18, 0.9%; Fleischner Society, Pulmonology, n = 34, 1.2%). Histograms of the top bigrams for specialist responses by specialty, containing raw counts and relative frequencies, can be found in Supplemental Materials.

Figure 3.

Histogram of top 10 specialist response bigrams across all specialties.

Discussion

Results from the current project demonstrate moderate ability of simple text mining and analysis to extract meaningful insights regarding eConsult usage. The most common bigrams from the clinical question field were often descriptive and mainly aligned with anecdotal evidence of how eConsult is being used for many specialties. In most cases, frequency counts demonstrated clear frontrunners for the most common clinical question(s) asked in any given specialty. For example, the most frequent bigram among Endocrinology eConsult orders indicates that PC providers seek specialist advice regarding thyroid nodules. This intel could be used to generate an educational checklist of best practice recommendations for diagnosing and treating thyroid nodules, which can be shared with PC providers throughout the health system. This targeted education can support appropriate diagnostics and first-line therapies within the primary care context, potentially eliminating the need for ongoing specialist involvement.

The usefulness of bigrams in the clinical question text varied and, in some cases, was not meaningful. For example, the repeated presence of the phrase “pt reports” in Psychiatric eConsult orders does nothing to further our understanding of how eConsults are being used within this specialty. It also necessitates further custom preprocessing to eliminate such banal phrases from the analysis.

Specialties with less data available for analysis demonstrated less ability to generate meaningful top-choice bigrams. For example, the top bigrams for Obstetrics/Gynecology (“pap smear”) and Orthopedics (“shoulder pain”) comprised less than 0.5% of the text among all clinical questions for those specialties. While these bigrams may be descriptive in nature, their infrequent presence may not warrant interpretation. Alternatively, some top bigrams were more frequent but contextually uninformative (e.g., “weeks ago”). Other top bigrams were frequent and descriptive but not specific enough to draw inferences from. For example, knowing that the term “abdominal pain” occurs in 3% of Gastroenterology eConsult orders may be informative but not necessarily prescriptive since many conditions can cause or generate abdominal pain.

Rich text from the specialist responses resulted in top bigrams that were often specific enough to understand the nature of advice given, though custom pre-processing was required. Obligatory statements of institutional liability and other niceties entered into the specialist response field, likely via the use of “quick-text” or “dot phrases,” led to the over-representation of certain combinations of words such as headings or subheadings, greetings, and closing regards. These pre-configured blocks of text increase the character length but do not contribute to the meaning of the text, which impairs our ability to extract meaningful insights without additional investment of analytic resources. We needed to examine the initial output and customize stop words to mitigate these issues.

Results from the current project describe eConsult interactions between two providers, which can be used to refine eConsult order templates and streamline exchanges, ultimately making eConsult technology and care more efficient. For example, some providers entered “See comments” in the clinical question field. Other PCPs entered too little information, entering the single word “dx” or “anxiety,” for example. This is not enough context for the specialist to generate a meaningful response, and in these cases, the specialist must create an additional exchange to request more information. We can use preliminary results from this study to re-educate ordering providers to be intentional and targeted with the information they include in their questions, thus potentially eliminating inefficiencies.

Lessons learned from the preprocessing challenges in this project could be used to generate a software package aimed at addressing some of the common language processing challenges for eConsults, which healthcare facilities could implement without the need for dedicated NLP scientists. The results of this study could also benefit existing clinical decision-support systems (CDSS) or support their development, particularly in developing preset drop-down fields for eConsult orders. Intel from the current analysis can guide the development of targeted statements and response options within pre-defined fields, which may, in turn, lessen the amount of free text required by the ordering provider to describe the clinical question adequately. For example, knowing that the term “thyroid nodule” was the most frequent bigram for Endocrinology could inform a new pre-defined drop-down field embedded in the eConsult order wherein the ordering provider selects whether the nodule is cancerous, non-cancerous or unknown, asymptomatic or symptomatic. This can potentially lessen the effort for ordering providers while ensuring that relevant information needed by the responding specialist is included. Likewise, a top bigram of “renal function” for Nephrology could inform a structured field embedded in eConsult orders to specify which facet of renal function (blood filtration, waste removal, fluid balancing, etc.,) needs to be addressed.

Even seemingly uninformative bigrams, such as “additional workup,” which was commonly observed among Hematology orders, may be helpful in clinical decision support. For instance, a pre-defined field where the ordering provider can select multiple labs to be completed before the eConsult order is placed would be helpful. Prompts such as this help ensure that the ordering provider submits a thorough clinical question.

Extracting valuable insights from free-text fields is critical for healthcare analytics. The techniques applied here demonstrated moderate capability to describe eConsult exchanges using minimally processed text data. In the age of big data and AI, it is essential to note that even simple, straightforward applications applied to moderately sized datasets can yield compelling results. The simple methods deployed for this pilot work were computationally inexpensive and did not have the privacy considerations of some cloud-hosted methods.

Next steps

The results from this project are informative but insufficient. We suggest further investigation to assess the feasibility and effort of simple techniques such as those reported here compared to more modern or advanced methodologies for language modeling, such as a neural network or BERTopic. Such comparisons can direct valuable analytic resources within health systems that wish to understand common use cases for eConsults.

Limitations

There are limitations to our work. As with other healthcare applications of NLP, the ability to mine meaningful information depends on the quality and robustness of the text, which can be affected by factors outside of our control. Due to time constraints or other factors, providers may be less comprehensive than preferred with their questions or response narratives. Additionally, we cannot distinguish between one provider using a word many times or many providers using a word once. Thus, we cannot definitively state whether frequent bigrams represent an overarching theme in eConsult ordering or limited use within a subset of the orders. The inclusion of pre-defined chunks of text such as “dot-phrases,” “quick-text,” or other non-natural statements interfered with our ability to mine and interpret data without the need for additional processing. This impacted our ability to assess this technique’s efficiency on raw data directly. Further, results from this project are from a single, urban, academic medical center and may not generalize to other settings.

Conclusion

Results from the current study demonstrate a moderate ability to describe the nature of eConsult exchanges through basic text mining and language processing. Custom preprocessing was required, increasing the requisite analytical effort and thereby decreasing the feasibility among lower-resourced health systems.

The clinical question text required less preprocessing than the specialist response. Not all top bigrams were meaningfully descriptive of the clinical reasons for eConsult, though many were. Extraction of information regarding the top ten recommendations or clinical actions suggested by specialty providers required more custom preprocessing and ultimately, led to mixed success.

Supplemental Material

Supplemental Material - Natural language processing to describe primary care requests for eConsult specialty care: A simple and practical application

Supplemental Material for Natural language processing to describe primary care requests for eConsult specialty care: A simple and practical application by Stephanie Grim, Anne Fuhlbrigge, John Fred Thomas and Rodger Kessler in Health Informatics Journal

Supplemental Material

Supplemental Material - Natural language processing to describe primary care requests for eConsult specialty care: A simple and practical application

Footnotes

ORCID iD

Stephanie Grim

Ethical statement

Author contributions

The authors confirm their contribution to the paper as follows: study conception and design: Stephanie Grim, John F. Thomas; data collection; analysis and interpretation of results: Stephanie Grim; draft manuscript preparation: Stephanie Grim; Supervision, Revision of Final Draft & Resource Provision: John F. Thomas, Anne Fuhlbrigge, Rodger Kessler. All authors reviewed the results and approved the final version of the manuscript.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Supplemental Material

Supplemental material for this article is available online.

References

Vimalananda

Gupte

Seraj

, et al. Electronic consultations (e-consults) to improve access to specialty care: a systematic review and narrative synthesis. J Telemed Telecare 2015; 21(6): 323–330.

Mermin-Bunnell

Zhu

Hornback

, et al. Use of natural language processing of patient-initiated electronic health record messages to identify patients with COVID-19 infection. JAMA Netw Open 2023; 6(7): e2322299.

Kharrazi

Anzaldi

Hernandez

, et al. The value of unstructured electronic health record data in geriatric syndrome case identification. J Am Geriatr Soc 2018; 66(8): 1499–1507.

Wiesmuller

Hayn

Kreiner

, et al. Natural Language processing for free-text classification in telehealth services: differences between diabetes and heart failure applications. Stud Health Technol Inf 2021; 279: 157–164.

Gould

Sakoda

Ritzwoller

, et al. Monitoring lung cancer screening use and outcomes at four cancer research network sites. Ann Am Thorac Soc 2017; 14(12): 1827–1835.

Wang

Hripcsak

Markatou

, et al. Active computerized pharmacovigilance using natural language processing, statistics, and electronic health records: a feasibility study. J Am Med Inf Assoc 2009; 16(3): 328–337.

Gonzalez-Hernandez

Sarker

O'Connor

, et al. Capturing the patient’s perspective: a review of advances in natural language processing of health-related text. Yearb Med Inform 2017; 26(1): 214–227.

Lee

Kross

Torrence

, et al. Assessment of natural language processing of electronic health records to measure goals-of-care discussions as a clinical trial outcome. JAMA Netw Open 2023; 6(3): e231204.

Bean

Kraljevic

Shek

, et al. Hospital-wide natural language processing summarising the health data of 1 million patients. PLOS Digit Health 2023; 2(5): e0000218.

10.

Keely

Rostom

Smith

, et al. A comparison of faxed referrals and eConsult questions for rheumatology referrals: a descriptive study. CMAJ Open 2021; 9(1): E38–E43.

11.

Corbetta-Rastelli

Morgan

Homaifar

, et al. Experiences in electronic consultation (eConsult) service in gynecology from a quaternary academic medical center. J Med Syst 2021; 45(5): 58.

12.

Wrenn

Catschegn

Cruz

, et al. Analysis of an electronic consultation program at an academic medical centre: primary care provider questions, specialist responses, and primary care provider actions. J Telemed Telecare 2017; 23(2): 217–224.

13.

Walker

Macdonald

Dennie

, et al. Electronic consultation between primary care providers and radiologists. AJR Am J Roentgenol 2020; 215(4): 929–933.

14.

Lowenstein

Bamgbose

Gleason

, et al. Psychiatric consultation at your fingertips: descriptive analysis of electronic consultation from primary care to psychiatry. J Med Internet Res 2017; 19(8): e279.

15.

Archibald

Stratton

Liddy

, et al. Evaluation of an electronic consultation service in psychiatry for primary care providers. BMC Psychiatry 2018; 18(1): 119.

16.

Ding

Barnett

Mehrotra

, et al. Classifying electronic consults for triage status and question type. Proc Conf Assoc Comput Linguist Meet 2020; 2020: 1–6.

17.

Ding

Barnett

Mehrotra

, et al. Classifying unstructured electronic consult messages to understand primary care physician specialty information needs. J Am Med Inf Assoc 2022; 29(9): 1607–1617.

18.

American Association of Medical Colleges . Project core: coordinating optimal referral experiences. American Association of Medical Colleges, (accessed 23 September 2023).

19.

Silge

Robinson

. Tidytext: text mining and analysis using tidy data principles in R. J Open Source Softw 2016; 1(3): 37.

20.

R Foundation for Statistical Computing . R: A language and environment for statistical computing. R Foundation for Statistical Computing, 2021. https://www.R-project.org/ (accessed 23 September 2023).

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB