Feasibility of ontology-guided structuring of stress information from narrative text: An exploratory study using large language models

Abstract

Background

Stress, arising from the dynamic interaction between external stressors, individual appraisals, and physiological or psychological responses, significantly impacts health yet is often underreported and inconsistently documented. When documented, stress-related information is often captured as unstructured narrative text, limiting systematic assessment, secondary use, and computational analysis.

Purpose

This study aimed to develop a mental stress ontology and to explore the feasibility of using a Large Language Model (LLM) to extract and structure stress-related information from narrative text in an ontology-guided manner.

Methods

Mental Stress Ontology (MeSO) was developed using Protégé by integrating theoretical frameworks on stress with concepts derived from 11 validated stress assessment instruments. MeSO was evaluated for content coverage using additional concepts collected from 58 text sources and for structural quality using the OntOlogy Pitfall Scanner! (OOPS!) and the Protégé Debugger. A mental health expert provided an overall qualitative evaluation of the ontology. Ontology-guided extraction of stress-related information was performed on 35 Reddit posts using an LLM (Claude Sonnet 4) and MeSO for six categories of stress-related information including stressor, stress response, coping strategy, duration, onset, and temporal profile. Human reviewers assessed the appropriateness of the extracted information and MeSO coverage of the identified stress concepts.

Results

The final ontology included 181 concepts across eight top-level classes. Human reviewers identified 220 extractable stress-related items from 35 Reddit posts. Ontology-guided extraction using an LLM resulted in 172 correctly extracted items (78.2%), with 27 items (12.3%) misclassified and 21 items (9.5%) missed. Of the extracted items, 22 represented numeric stress duration values and were excluded from ontology-based concept mapping. Of the remaining 150 items, 120 were successfully mapped to MeSO.

Conclusion

This study provides initial evidence that ontology-guided large language models may facilitate the structuring of stress-related information from narrative text, offering a foundation for future research toward systematic stress assessment and documentation.

Keywords

Mental stress mental stress ontology large language model structured documentation information extraction

Introduction

Significance of mental stress in human health

Mental stress is a major psychosocial factor that significantly affects both physical and mental health outcomes. It arises from the interaction among external stressors, individual cognitive appraisals, and the resulting physiological and psychological responses. While acute stress may be adaptive, prolonged or chronic stress can lead to dysregulation of nervous and endocrine systems, contributing to adverse health outcomes such as cardiovascular disease, gastrointestinal disturbances, immune system dysfunction, and reduced capacity to cope with everyday challenges.^1–4

Patients with health issues are particularly vulnerable to heightened stress due to the cumulative physical, emotional, and psychosocial burdens associated with illness and treatment.^5,6 For example, patients with cancer frequently experience persistent psychological distress related to uncertainty regarding prognosis, treatment-related side effects, and disruptions to personal identity and self-esteem.^7–12 Prior studies have reported that psychological distress is highly prevalent among breast cancer patients, as the disease often affects body image, self-concept, and self-esteem.^10–12 Elevated stress levels in breast cancer patients have been associated not only with diminished quality of life, but also with disease progression, increased risk of recurrence, and reduced survival rates.^13,14 Early identification and systematic documentation of stress-related information could enable timely interventions and potentially mitigate its detrimental effects on patient outcomes.

Challenges in documenting patient's stress experience in clinical settings

Despite the well-established impact of stress on health, patient's stress experiences remain underreported in clinical settings.¹⁵ The Electronic health record (EHR) serves as a comprehensive repository of longitudinal patient data systematically collected by healthcare providers. However, current documentation practices predominantly focus on acute medical conditions and their treatments, often lacking structured mechanisms for capturing and categorizing stress-related information.

Although various validated instruments are available to assess stress, there is limited guidance on how to consistently incorporate these tools into routine clinical practice. General stress assessment instruments such as the Perceived Stress Scale (PSS) and the stress subscale of the Depression, Anxiety, and Stress Scale (DASS) are commonly used in research settings to evaluate stress across diverse populations.^16,17 In clinical context, however, their use is largely restricted to psychiatric settings or patients with severe mental health conditions, and they are not routinely integrated into care plans for patients with chronic illnesses such as cancer, cardiovascular diseases, or diabetes. Moreover, the use of these instruments often depends on individual clinician discretion, resulting in inconsistent and incomplete stress assessment across patient populations. This gap persists despite substantial evidence that life stressors influence biological processes involved in the onset and progression of chronic diseases, including diabetes, obesity, and cardiovascular disease.^18–20

Stress-related documentation in EHRs is not only sparse but also unstructured. When stress-related information is assessed in routine care, it is most often documented within free-text narratives.^21–23 This lack of standardization limits the ability to systematically analyze, aggregate, and reuse stress-related data, and may result in an incomplete or fragmented understanding of a patient's stress profile. Furthermore, unstructured documentation can impede effective communication among healthcare providers and reduce the visibility of psychosocial factors that may influence treatment decisions, continuity of care, and individualized stress management strategies. These challenges underscore the need for systematic approaches to integrate comprehensive psychosocial assessments including mental stress into routine clinical documentation.

Generative AI-assisted nursing documentation

Recent advances in generative artificial intelligence (AI) have begun to influence clinical documentation practice. One of the most notable applications is ambient AI documentation, in which AI systems automatically capture patient–clinician conversations and convert them into written clinical notes within the EHR.²⁴ Prior studies have demonstrated that ambient AI documentation can reduce documentation time and alleviate clinician frustration associated with EHR use.^25,26 Although initially adopted for physicians, this technology is now being piloted for nursing practice.

Ambient AI documentation offers a promising opportunity to enhance the capture of stress-related information in clinical settings by reducing the manual burden of documentation. However, despite advances in converting spoken dialogue into written clinical narratives, such documentation largely remains in unstructured free-text forms. As a result, stress-related information embedded within narrative notes, whether generated through traditional documentation or ambient AI approaches, remains difficult to systematically analyze. This highlights the need for effective methods to transform narrative descriptions of patient stress into structured, clinically meaningful representations.

Ontology-based solutions for addressing documentation challenges

To address the limitations of unstructured clinical narratives, recent studies have explored the use of Large Language Model (LLM)-based approaches to extract and structure mental health information from free-text documentation. However, these methods often lack formal semantic grounding and may be vulnerable to hallucinations, raising concerns regarding clinical reliability and traceability.^27–30

Ontologies provide a formal, machine-interpretable representation of domain knowledge and have been widely used to support semantic interoperability, knowledge management, and reasoning in healthcare systems.^31–33 An ontology can also serve as a semantic foundation for ontology-guided information extraction, enabling consistent identification and structured representation of key concepts within narrative clinical text.³⁴ When combined with LLM-based extraction methods, an ontology-based approach may support more reliable and transparent extraction of stress-related information in clinical narratives.

Several prior efforts have attempted to formalize concepts related to mental stress using ontological frameworks. The Human Stress Ontology (HSO)³⁵ includes concepts related to stress measurements, causes, effects, mediators, and treatments. While HSO provides a useful overarching representation of the stress domain, it lacks the granularity required to capture patient's detailed stress experience, such as specific concepts on stressors and individualized responses. The Mental Health Ontology (MHO)³⁶ also includes stress-related concepts but primarily focuses on mental disorders, drawing on diagnostic frameworks such as the International Classification of Diseases (ICD) and the Diagnostic and Statistical Manual of Mental Disorders (DSM). As such, it is not well suited to represent stress experiences encountered in everyday living during routine clinical encounters.

These limitations highlight the need for a stress ontology specifically designed to represent key aspects of mental stress including stressors, responses, and management strategies to support systematic and structured capture of patient's stress experience. Such an approach may enable a more holistic understanding of patient stress profiles and ultimately contribute to improved patient-centered care and health outcomes.^37,38

Although ontology-based approaches are well suited for structuring stress-related information, the limited availability of detailed stress documentation in routine clinical records poses a practical challenge. To address this, we used publicly available Reddit posts, where individuals describe their stress experience in details, as an alternative source of narrative text for the preliminary evaluation of ontology-guided information extraction in this feasibility study.

Study aims

This study aimed to develop a mental stress ontology and to examine the feasibility of an ontology-guided LLM to automatically extract key stress-related information from narrative stress descriptions. The main contributions of this study are two-fold: (1) the development of a mental stress ontology designed to represent individual's everyday stress experiences, and (2) an exploratory evaluation of the feasibility of generating structured stress representations from narrative stress text using an ontology-guided LLM approach.

Methods

Development of mental stress ontology (MeSO)

Concept hierarchy construction

The Mental Stress Ontology (MeSO) was developed through an iterative ontology engineering process involving repeated cycles of concept identification, hierarchical structuring, and evaluation. These cycles were conducted collaboratively to refine concept coverage and improve clarity.

The initial backbone structure of MeSO was informed by the top-level structure of the Human Stress Ontology (HSO)³⁵ and Lazarus and Folkman's Transactional Model of Stress (TMS).39 The top-level classes of HSO provided a foundation for organizing core components of stress assessment, including Stress Measurements, Stress Causes, Stress Mediators, Stress Effects, and Stress Treatments.³⁵ Concepts related to Stress Appraisal and Stress Coping, which are central to stress perception and management in nursing care, were incorporated from TMS.³⁹

To populate concepts under the top-level classes, an online search was conducted to identify validated stress assessment instruments. Eleven instruments were selected (Table 1), yielding a total of 266 individual questionnaire items. Two reviewers with training in nursing informatics collaboratively extracted stress related keywords from these items. The extraction focused on terms related to Stress Measurements, Stress Causes, Stress Mediators, Stress Effects, Stress Treatments, Stress Appraisal, and Stress Coping.

Table 1.

Stress assessment instruments used for ontology development.

No.	Source
1	Coping Resources Inventory for Stress (CRIS)⁴⁰
2	Depression Anxiety Stress Scales-21 (DASS-21)¹⁷
3	International Stress Management Association (ISMA) UK⁴¹
4	Perceived Stress Scale-10 (PSS-10)¹⁶
5	Stress and Adversity Inventory for Adults (STRAIN)⁴²
6	Stress Appraisal Measure (SAM)⁴³
7	Stress Level Test (Self-Assessment)⁴⁴
8	The Ardell Wellness Stress Test⁴⁵
9	The Brief COPE Inventory⁴⁶
10	The Social Readjustment Rating Scale (SRRS)⁴⁷
11	The Work Stress Questionnaire (WSQ)⁴⁸

The extracted keywords were grouped based on conceptual similarities and iteratively refined into distinct concept classes by assigning preferred labels and removing redundancies. These classes were then grouped and organized into higher-level parent concepts, informed by existing standardized terminologies and relevant mental health literature. This bottom-up process of clustering and abstraction was repeated until the predefined top-level classes were reached, resulting in a hierarchical ontology structure.

Concept classes were labeled in English in singular nouns or gerunds and formatted in UpperCamelCase. MeSO was implemented using Protégé (Version 5.6.1),^49,50 an open-source ontology authoring tool. Ontology development was conducted collaboratively by the study authors (JK, HK) through multiple rounds of review and consensus-based discussion until no further structural changes were deemed necessary.

Class annotation

Each concept class in MeSO was annotated with an internal identifier assigned by the authors, as well as the Concept Unique Identifier (CUI), concept preferred name, and semantic type from the Unified Medical Language System (UMLS) Metathesaurus.⁵¹ Concept mapping to the UMLS Metathesaurus was performed manually by the authors to align MeSO concepts with existing biomedical terminologies and support semantic interoperability. Korean translations of the concept labels and the source language of originating instruments were also recorded as part of the class annotations.

Evaluation of MeSO

MeSO was evaluated for concept coverage, structural quality, and content validity.

Concept coverage evaluation using BERT

Bidirectional Encoder Representations from Transformers (BERT)⁵² was used to examine whether MeSO sufficiently covered commonly used stress-related concepts. A sample of 58 stress-related texts was collected, including abstracts of scientific publications and mental health-related web content produced by hospitals or governmental organizations. Stress-related keywords were extracted using KeyBERT, a BERT-based keyword extraction method.⁵³ For each text source, the top 10 keywords were identified at the unigram, bigram, and trigram level to capture both single and multiword stress-related expressions Following duplicate removal and normalization of lexical variants, 82 unique keywords were retained and mapped to MeSO concepts. The mapping outcomes were categorized into five levels of semantic equivalence, as defined in Table 2.

Table 2.

Semantic equivalence categories.

Category	Description
Exact Match	Keywords have the concepts with the same meaning in MeSO
Broader Match	Keywords can only relate to more general concepts in MeSO, meaning the former carry more specific meaning than the latter
Narrower Match	Keywords encompass multiple concepts in MeSO, meaning the former carry more general meaning than the latter
Partial Equivalence	Keywords and concepts in MeSO are related but nonhierarchically
No Match	Keywords have no related concepts in MeSO

Structural quality evaluation

The structural quality of MeSO was evaluated using the Ontology Debugger plug-in within Protégé⁵⁴ and OntOlogy Pitfall Scanner! (OOPS!).⁵⁵ The Ontology Debugger was used to evaluate logical consistency and coherence of the class hierarchy, and OOPS! was used to identify common ontology design pitfalls related to naming conventions, annotation completeness, and hierarchical structure.

Content validity evaluation

MeSO was revised based on the results of the coverage and structural evaluations and subsequently reviewed by a mental health expert. The expert reviewed whether MeSO captured key aspects of human stress in a clear and nonoverlapping manner, and whether the hierarchical relationships among classes were conceptually appropriate. The overall ontology development and evaluation process is summarized in Figure 1.

Figure 1.

The overall process of Mental Stress Ontology (MeSO) development and evaluation. Step 1 involves iterative concept hierarchy construction, followed by class annotation in Protégé to generate an initial version of MeSO. Step 2 extracts keywords from stress-related texts to assess concept coverage. Structural quality is then evaluated and the final version of MeSO was developed.

Extracting key stress-related information from free-text description

To evaluate the feasibility of ontology-guided extraction of structured stress-related information, a publicly available dataset of Reddit posts describing stress experiences (N = 2744) was obtained from Kaggle. Given the exploratory nature of this study, the focus was on assessing feasibility rather than evaluating large-scale model performance. Therefore, a randomly selected subset of 40 posts was used for detailed review of extraction results and ontology mappings.

During the initial configuration phase, five posts were used to develop the extraction prompt and to assess the basic suitability of three general-purpose LLMs (Qwen3, Claude Sonnet 4, and GPT-4o) for the task. This step was intended to support pragmatic model selection rather than formal model comparison. The LLM was instructed to first extract six predefined elements (Table 3): stressor, stress response, stress coping strategy, stress duration, stress onset, and temporal profile of stress. The LLM then mapped the extracted information to the corresponding MeSO concepts. When no matching concept was available, the LLM left the mapping result blank. The output of this process included both the extracted information and the mapped MeSO concepts. Zero-shot procedural prompting was used with default model parameters.

Table 3.

Stress-related information categories and description.

Category	Description
Stressor	Source or causes of stress
Stress Response	Mental, emotional, physical, or behavioral reaction to stress
Stress Coping Strategy	The methods used to manage stress
Stress Duration	How long the stress lasts
Stress Onset	The manner in which stress begins—sudden or gradual
Stress Temporal Profile	The overall pattern of stress—acute or chronic

Claude Sonnet 4 (CS4) was selected for subsequent analysis. The preliminary model assessment results are available in Supplemental Table 1. The remaining 35 posts were processed by CS4, and the extracted information and ontology mappings were independently reviewed by two reviewers (JK, HX). Discrepancies were resolved through consensus discussion involving a third reviewer (HK). The prompt used for information extraction is provided in Supplemental Table 2.

Results

Evaluation of MeSO

The initial version of MeSO comprised 102 unique concepts organized under five top-level classes representing key dimensions of stress: Stressor, Stress Mediator, Stress Appraisal, Stress Effect, and Stress Treatment.

To assess concept coverage, 82 newly prepared stress-related keywords were mapped to the stress ontology. Of these, 42 keywords (51.2%) showed exact matches and 34 (41.5%) showed broader matches. Two keywords were mapped to more specific concepts (i.e., narrower matches), while four had no corresponding matches in the ontology. The complete mapping results are provided in Supplemental Table 3.

The 40 keywords without exact matches were further reviewed by the authors to assess their relevance for ontology inclusion. Based on this review, 22 new concepts were added to MeSO, and the ontology structure was refined. As a result, the top-level classes were expanded from five to eight: Stressors, Stress Mediator, Stress Appraisal, Stress Response, Stress Intervention, Stress Coping Strategy, Stress Coping Outcome, and Stress Characteristics.

Structural validation using the Protégé Debugger plug-in confirmed that MeSO has a consistent and coherent ontology structure. Additional evaluation with the OntOlogy Pitfall Scanner! (OOPS!) identified minor issues, including missing class definitions and the absence of inverse property relationships. Because MeSO does not currently define property relationships, inverse properties were not specified.

OOPS! also suggested potential equivalence between the concepts “Restlessness” and “Impatience.” However, this suggestion was not adopted, as the two concepts represent related but distinct aspect of stress: “Restlessness” is typically a physical manifestation of mental discomfort, such as “Impatience.”

Expert review by a mental health specialist indicated that MeSO captures core concepts necessary to describe mental stress. Although not all relevant stress concepts are not yet represented, MeSO was considered to provide a solid conceptual foundation that supports future refinement and expansion.

Final version of MeSO

The finalized version of MeSO included 181 concepts structured under eight top level classes, as shown in Figure 2. An example of the class annotation was also presented in Figure 3. MeSO is publicly available in BioPortal.⁵⁶

Figure 2.

Top-level class hierarchy of MeSO.

Figure 3.

Class annotation example.

Stress information extraction and ontology mapping

The evaluation of extracted stress-related information showed a high level of inter-rater agreement between the two reviewers, with a weighted kappa score of 0.899. Discrepancies between the two primary reviewers (JK, HX) were resolved through discussion with a third reviewer (HK).

The result of stress information extraction using CS4 is summarized in Table 4. Ninety-five percent confidence intervals for the reported metrics were estimated using 5000 bootstrap resamples. Many posts contained multiple stressors, stress responses, and stress coping strategies. Whereas some posts lacked extractable information for specific categories. For example, two posts did not specify a stressor, and 21 did not mention stress onset. Because individual posts can contain any number of mentions per category, true negatives (TN) were not well-defined at the mention-level. Therefore, metrics relying on true negatives (e.g., accuracy, specificity) were not calculated.

Table 4.

Performance metric of information extraction.

Information category	TP(%)	FP(%)	FN(%)	Precision(95% CI)	Recall(95% CI)	F1 score(95% CI)
Stressor	39 (81.25)	0 (0.00)	9 (18.75)	1.00 (1.00–1.00)	0.81 (0.70–0.93)	0.90 (0.83–0.96)
Stress Response	55 (77.46)	8 (11.27)	8 (11.27)	0.87 (0.79–0.95)	0.87 (0.80–0.94)	0.87 (0.81–0.93)
Stress Coping Strategy	21 (60.00)	11* (31.43)	3 (8.57)	0.66 (0.49–0.81)	0.88 (0.75–1.00)	0.75 (0.62–0.86)
Stress Duration	19 (85.36)	3 (13.64)	0 (0.00)	0.86 (0.70–1.00)	1.00 (1.00–1.00)	0.93 (0.82–1.00)
Stress Onset	9 (64.29)	5^† (35.71)	0 (0.00)	0.64 (0.38–0.89)	1.00 (1.00–1.00)	0.78 (0.55–0.94)
Stress Temporal Profile	29 (96.67)	0 (0.00)	1 (3.33)	1.00 (1.00–1.00)	0.97 (0.89–1.00)	0.98 (0.94–1.00)
Total	172 (78.18)	27 (12.27)	21 (9.55)	0.86 (0.82–0.91)	0.89 (0.85–0.93)	0.88 (0.84–0.91)

*One case was attributable to hallucination; †Two cases were attributable to hallucination.

Extraction outcomes varied across information categories. Higher performance was observed for Stressor, Stress Response, Stress Duration, and Temporal Profile of Stress, whereas lower performance was observed for Stress Coping Strategy and Stress Onset. Across all categories, human reviewers identified a total of 220 extractable information items, of which, 172 (78.2%) were correctly extracted by CS4. Of the remaining items, 27 (12.27%) represented false positives, largely due to semantic misinterpretation, and 21 (9.5%) were not extracted (false negatives). Notably, errors in Stress Coping Strategy and Stress Onset were primarily attributable to misinterpretation rather than omission, including a small number of cases where CS4 generated stress-related information despite not being stated in the source text. The complete set of evaluated posts, extracted information, and human reviews is provided in Supplemental Table 4.

Of the correctly extracted 172 stress-related information items, 22 items were related to Stress Duration. Although duration information was correctly identified and extracted, it was often expressed as numeric quantities (e.g., “3 months,” “2 years”). Because MeSO in its current version focuses on a class-based concept hierarchy and does not model numeric quantities or datatype properties, these values were not included in ontology-based concept mapping. The remaining 150 items were evaluated for concept mapping, of which 120 were successfully mapped to MeSO.

Among the 30 unmapped items, 20 corresponded to Stressor concepts, six to Stress Response concepts, and four to Stress Coping Strategy concepts. After removing duplicates, 15 Stressor concepts, four Stress Response concepts, and four Stress Coping Strategy concepts remained. These unmapped concepts included both general expressions (e.g., “Unspecific stressors” and “Multiple unspecific stressors”) and more specific descriptions such as “political difference,” “career failure,” and “lack of motivation.” The complete list of unmapped concepts is provided in Supplemental Table 5.

Discussion

This study developed the Mental Stress Ontology (MeSO) to support more consistent and unambiguous representation of stress-related information and explored the feasibility of using an ontology-guided LLM to extract structured stress information from narrative text. In this work, MeSO guided Claude Sonnet 4 (CS4) to extract six categories of stress-related information such as Stressors, Stress Responses, Stress Coping Strategies, Stress Duration, Stress Onset, and Stress Temporal Profile from stress narratives.

Because stress is often underreported in clinical documentation and, when recorded, is typically captured as unstructured free text, methods to support a structured representation and extraction are important first step toward improving documentation consistency and downstream reuse. At the same time, the present work should be viewed as a proof-of-concept feasibility study, aimed at examining whether (1) a stress ontology can represent meaningful elements of stress experiences and (2) an LLM can extract and map those elements to the ontology with reasonable reliability.

Using Reddit posts for feasibility evaluation

For this feasibility evaluation, we used written Reddit posts describing individuals’ stress experiences as a source of rich narrative text, given the limited availability of shareable, detailed clinical stress narratives. However, Reddit posts differ from clinical encounters in tone, structure, and context. Accordingly, findings from this study should not be interpreted as evidence of clinical readiness. Rather, they provide an initial indication that ontology-guided information extraction is technically plausible and help identify where the approach is likely to struggle.

Overall extraction performance and ontology coverage

When guided by MeSO, CS4 extracted a substantial proportion of reviewer identified stress-related information items, and MeSO covered most extractable concepts aside from numeric duration values that were not presented as ontology concepts. Together, these results support the feasibility of ontology-guided structuring from narrative text, while underscoring the need for cautious interpretation given the single nonclinical data source, the small evaluation sample, and remaining gaps in ontology coverage.

CS4 performed relatively well for Stressors, Stress Responses, and Temporal Profile, and Duration. These categories of information were often expressed directly, for example, as triggering events or explicit time spans, making them more amenable to extraction. At the sample time, we observed errors caused by over-interpretation. For example, CS4 sometimes mapped frustration, distress, or low motivation to more severe responses (e.g., depression or social withdrawal), suggesting difficulty calibrating affective or behavioral cues in informal narratives.

Challenges in information extraction

In contrast, CS4 showed greater difficulty with Coping Strategy and Onset, and the observed errors point to practical limitations of the current approach. For Coping Strategies, a common failure mode was labeling venting as help-seeking. Help-seeking should be seen as proactive coping that includes an explicit request for advice or assistance (e.g., “Has anyone had a similar experience?” or “Does anyone have suggestions for dealing with this?”). Reddit narratives often blur this boundary: authors may vent, seek validation, and implicitly invite support without making a direct request. In that context, labeling venting as help-seeking is understandable, but it reduces the precision of structured extraction. Prompt refinement could emphasize this distinction, although overly prescriptive prompts may not generalize well across narrative styles. CS4 also missed some plausible help-seeking cases, which may reflect the difficulty of detecting implicit coping behaviors and the instability of performance in small samples. Another coping-related error involved labeling medication nonadherence due to side effects as a coping strategy, reflecting confusion between coping intent and associated behavior.

For Onset, incorrect extractions were often driven by implicit or underspecified temporal cues. Onset was not consistently stated explicitly, and identifying onset frequently required contextual inference. CS4 sometimes inferred onset without sufficient support and occasionally misread gradual onset as sudden. These patterns suggest that onset is a context-dependent attribute that may require (1) more explicit narrative structure as often elicited in clinical assessment, or (2) extraction approaches that explicitly reason over time-related cues before generating a final output.

Importantly, these challenges are likely influenced by the nature of informal narratives. Unlike clinical assessments, Reddit posts are not guided by clinician questions that systematically elicit key stress characteristics (e.g., onset, typical coping methods, etc.). As a result, relevant information is often omitted, implied, or expressed indirectly. Therefore, the performance patterns observed in this study should not be assumed to generalize to clinician-patient dialogue, where targeted questioning may increase the explicitness of onset and coping-related information. Nonetheless, the present findings highlight categories that will require additional methodological attention before clinical translation.

We also identified three incorrect outputs that were considered potential hallucinations (Supplemental Table 4). In these cases, CS4 extracted information that was not grounded in the original post. An example is shown in Table 5. In this case, CS4 identified the coping strategy as Participating in Social Activity, although the post does not contain any explicit or implicit indication that the writer or their partner managed stress through social activities. This example illustrates the LLM may occasionally infer information beyond what is supported by the text.

Table 5.

Example of a hallucination case.

Post	Information category	Extracted information
She was on opiates/heroin really hard but i loved her even then and just hoped she could overcome it. She did and we got together and have a beautiful 5 month old daughter. Anyone out there using you can beat that!	Stressor	Partner Conflict
	Stress Response	NA (Positive outcome described)
	Stress Coping Strategy	Participating Social Activity
	Duration	NA
	Onset	NA
	Temporal Profile	NA

Limitations

This study was designed as a feasibility evaluation and has several limitations that restrict generalizability and immediate clinical applicability. First, this study relied on written Reddit posts rather than real clinical conversations. Although speech-to-text technologies have become increasingly reliable, clinical environments introduce background noise, varied speech patterns, interruptions, and contextual complexity. These factors may affect both transcription quality and subsequent information extraction. In addition, stress-related information in clinical encounters is often embedded within broader discussions of symptoms, diagnoses, and care plans, which may further complicate the extraction task.

Second, the main extraction experiment used a single LLM (Claude Sonnet 4) and was conducted on a small sample of 35 posts. Although Claude Sonnet 4 (CS4) was selected based on preliminary comparison, this limited evaluation may not sufficiently reflect performance across various models. Future work should include larger datasets, real clinical dialogue, and more systematic multimodel comparisons, including on-premises models to address privacy and deployment requirements.

Third, MeSO was constructed by integrating theoretical frameworks with concepts drawn from 11 validated stress assessment instruments, and its coverage was evaluated using a small convenience sample of mental health-related texts. Accordingly, MeSO should be viewed as an initial ontology rather than a comprehensive representation of the mental stress domain. Some stress-related concepts identified in the Reddit narratives were not represented, underscoring the need for ongoing expansion and refinements as additional data sources are incorporated. In addition, as MeSO was designed primarily as a concept hierarchy to represent key stress-related concepts and their hierarchical relationships, the current version of MeSO does not represent numeric quantities or datatype properties. Duration, which is typically expressed as a numeric temporal value (e.g., “3 days” or “2 months”) rather than a conceptual category, was therefore excluded from the mapping step. Future enhancements of MeSO may include extending the ontology to represent quantitative temporal values by linking stress-related events or attributes to datatype properties (e.g., durationValue, durationUnit).

Implications and next steps

This study provides early evidence that ontology-guided information extraction using an LLM can produce structured representation of stress-related information from narrative text. If appropriately adapted and validated, such approach may support more consistent documentation and facilitate downstream reuse of stress-related data in clinical settings.

However, translation to clinical practice will require further work, including evaluation using real clinical dialogue, testing on larger and more diverse datasets, and refinement of prompting strategies to better align with clinical language and documentation workflow. In addition, given the subjective and context-sensitive nature of mental stress, discrepancies between LLM output and human judgment were observed even for relatively objective information categories. These findings suggest that AI-assisted information extraction and structuring should be viewed as an augmentation of existing clinical documentation practices, rather than a replacement for clinical judgment and expertise.

Conclusion

Mental stress is a complex phenomenon that affects both physical and psychological health, yet it is often inconsistently documented in routine clinical practice. This study developed a mental stress ontology and explored the feasibility of using it to guide the automated extraction of stress-related information from narrative text using a large language model. The findings provide early evidence that ontology-guided information extraction can support the generation of structured representation of stress-related data from unstructured narratives.

Although this work does not establish clinical readiness, it highlights the potential of ontology-guided approaches to improve the consistency, interpretability, and downstream reuse of patient-reported stress information if further validated in clinical settings. Future work should focus on testing this approach with real clinical dialogue, larger and more diverse datasets, and continued refinement of both the ontology and extraction strategies, with the goal of supporting more systematic documentation of mental stress across care context.

Supplemental Material

sj-docx-1-dhj-10.1177_20552076261435838 - Supplemental material for Feasibility of ontology-guided structuring of stress information from narrative text: An exploratory study using large language models

Supplemental material, sj-docx-1-dhj-10.1177_20552076261435838 for Feasibility of ontology-guided structuring of stress information from narrative text: An exploratory study using large language models by Hyeoneui Kim, Jeongha Kim, Huijing Xu, Jinsun Jung, Sunghoon Kang and Sun Joo Jang in DIGITAL HEALTH

Supplemental Material

sj-pdf-2-dhj-10.1177_20552076261435838 - Supplemental material for Feasibility of ontology-guided structuring of stress information from narrative text: An exploratory study using large language models

Supplemental material, sj-pdf-2-dhj-10.1177_20552076261435838 for Feasibility of ontology-guided structuring of stress information from narrative text: An exploratory study using large language models by Hyeoneui Kim, Jeongha Kim, Huijing Xu, Jinsun Jung, Sunghoon Kang and Sun Joo Jang in DIGITAL HEALTH

Footnotes

Acknowledgments

The authors have no acknowledgements to declare.

ORCID iDs

Hyeoneui Kim

Jeongha Kim

Huijing Xu

Jinsun Jung

Sunghoon Kang

Sun Joo Jang

Ethical considerations

This study was exempt from the Institutional Review Board (IRB) of Seoul National University because it involved the analysis of publicly available de-identified secondary data (IRB No. E2507/004-005).

Author contributions

HK and SJ contributed to the conceptualization of the study. HK and JK developed MeSO and drafted the manuscript. JJ and SK prepared the data analyzed the results. JK, HX, and HK conducted information extraction and evaluated the results. All authors contributed to critical review and revision of the manuscript. All authors agree with the final content of the manuscript.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported in part by a grant from the National Research Foundation of Korea (grant number: 2022R1A2C201136011) and the BK21 four project (Center for Human-Caring Nurse Leaders for the Future) funded by the Ministry of Education and National Research Foundation of Korea.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data availability statement

The data used in this study are openly available at . The data that supports the findings of this study are available in the supplementary material.

Supplemental material

Supplemental material for this article is available online.

References

Liu

, et al. Psychological stress in inflammatory bowel disease: psychoneuroimmunological insights into bidirectional gut-brain communications. Front Immunol 2022; 13: 1016578.

Menke

Nitschke

Hellmuth

, et al. Stress impairs response to antidepressants via HPA axis and immune system activation. Brain Behav Immun 2021; 93: 132–140.

Osborne

Shin

Mehta

, et al. Disentangling the links between psychosocial stress and cardiovascular disease. Circ Cardiovasc Imaging 2020; 13: e010931.

Monroe

Slavich

Torres

, et al. Major life events and major chronic difficulties are differentially associated with history of major depressive episodes. J Abnorm Psychol 2007; 116: 116–124.

Abuatiq

Brown

Wolles

, et al. Perceptions of stress: patient and caregiver experiences with stressors during hospitalization. Clin J Oncol Nurs 2020; 24: 51–57.

Herschbach

Britzelmeir

Dinkel

, et al.

Distress in cancer patients: who are the main groups at risk?

Psychooncology 2020; 29: 703–710.

Antoni

Moreno

Penedo

. Stress management interventions to facilitate psychological and physiological adaptation and optimal health outcomes in cancer patients and survivors. Annu Rev Psychol 2023; 74: 423–455.

Bach

Knauer

Graf

, et al. Psychiatric comorbidities in cancer survivors across tumor subtypes: a systematic review. World J Psychiatry 2022; 12: 623–635.

Demirtaş

Temircan

. Examining the relationship between depression, anxiety and stress in kidney cancer patients. J Kidney Cancer VHL 2022; 9: 19–26.

10.

Dinapoli

Colloca

Di Capua

, et al. Psychological aspects to consider in breast cancer diagnosis and treatment. Curr Oncol Rep 2021; 23: 38.

11.

Lee

T-Y

Hsing

S-C

C-C

. An improved stress-scale specifically designed to measure stress of women with newly diagnosed breast cancer. Int J Environ Res Public Health 2021; 18: 2346.

12.

Tang

W-Z

Mangantig

Iskandar

YHP

, et al. Prevalence and associated factors of psychological distress among patients with breast cancer: a systematic review and meta-analysis. BMJ Open 2024; 14: e077067.

13.

Ahabrach

El Mlili

Mafla-España

, et al. Hair cortisol concentration associates with insomnia and stress symptoms in breast cancer survivors. Int J Psychophysiol 2023; 191: 49–56.

14.

P-J

Cho

J-R

. Changes in fatigue, psychological distress, and quality of life after chemotherapy in women with breast cancer: a prospective study. Cancer Nurs 2020; 43: E54–E60.

15.

Wulsin

Sagui-Henson

Roos

, et al. Stress measurement in primary care: conceptual issues, barriers, resources, and recommendations for study. Psychosom Med 2022; 84: 267–275.

16.

Cohen

Kamarck

Mermelstein

. A global measure of perceived stress. J Health Soc Behav 1983; 24: 385–396.

17.

Lovibond

. The structure of negative emotional states: comparison of the Depression Anxiety Stress Scales (DASS) with the Beck Depression and Anxiety Inventories. Behav Res Ther 1995; 33: 335–343.

18.

Eriksson

A-K

van den Donk

Hilding

, et al. Work stress, sense of coherence, and risk of type 2 diabetes in a prospective study of middle-aged Swedish men and women. Diabetes Care 2013; 36: 2683–2689.

19.

Miao Jonasson

Hendryx

Shadyab

, et al. Social support, social network size, social strain, stressful life events, and coronary heart disease in women with type 2 diabetes: a cohort study based on the women’s health initiative. Diabetes Care 2020; 43: 1759–1766.

20.

Wang

Lê-Scherban

Taylor

, et al. Associations of job strain, stressful life events, and social strain with coronary heart disease in the women’s health initiative observational study. J Am Heart Assoc 2021; 10: e017780.

21.

Feller

Bear Don’t Walk Iv

Zucker

, et al. Detecting social and behavioral determinants of health with structured and free-text clinical data. Appl Clin Inform 2020; 11: 172–181.

22.

Han

Zhang

Shi

, et al. Classifying social determinants of health from unstructured electronic health records using deep learning-based natural language processing. J Biomed Inform 2022; 127: 103984.

23.

Weissman

Pathak

Talati

. Personal life events-A promising dimension for psychiatry in electronic health records. JAMA Psychiatry 2020; 77: 115–116.

24.

Poon

Lemak

Rojas

, et al. Adoption of artificial intelligence in healthcare: survey of health system priorities, successes, and challenges. J Am Med Inform Assoc 2025; 32: 1093–1100.

25.

Shah

Crowell

Jeong

, et al. Physician perspectives on ambient AI scribes. JAMA Netw Open 2025; 8: e251904.

26.

Stults

Deng

Martinez

, et al. Evaluation of an ambient artificial intelligence documentation platform for clinicians. JAMA Netw Open 2025; 8: e258614.

27.

Cardamone

Olfson

Schmutte

, et al. Classifying unstructured text in electronic health records for mental health prediction models: large language model evaluation study. JMIR Med Inform 2025; 13: e65454.

28.

Lawrence

Schneider

Rubin

, et al. The opportunities and risks of large language models in mental health. JMIR Ment Health 2024; 11: e59479.

29.

Guo

Lai

Thygesen

, et al. Large language models for mental health applications: systematic review. JMIR Ment Health 2024; 11: e57400.

30.

Kermani

Perez-Rosas

Metsis

. A systematic evaluation of LLM strategies for mental health text analysis: fine-tuning vs. prompt engineering vs. RAG. 2025. DOI: 10.48550/arXiv.2503.24307

31.

Gruber

. A translation approach to portable ontology specifications. Knowl Acquis 1993; 5: 199–220.

32.

Studer

Benjamins

Fensel

. Knowledge engineering: principles and methods. Data Knowl Eng 1998; 25: 161–197.

33.

Liyanage

Krause

De Lusignan

. Using ontologies to improve semantic interoperability in health data. J Innov Health Inform 2015; 22: 309–315.

34.

Ambalavanan

Snead

Marczika

, et al. Ontologies as the semantic bridge between artificial intelligence and healthcare. Front Digit Health 2025; 7. DOI: 10.3389/fdgth.2025.1668385

35.

Nasiri Khoozani

Hadzic

. Designing the human stress ontology: a formal framework to capture and represent knowledge about human stress. Aust Psychol 2010; 45: 258–273.

36.

Hadzic

Chen

Dillon

. Towards the mental health ontology. In: 2008 IEEE international conference on bioinformatics and biomedicine, 2008, pp.284–288.

37.

Ghanem

Castelo

Jimenez-Fonseca

, et al. Coping strategies and depressive symptoms in cancer patients. Clin Transl Oncol 2020; 22: 330–336.

38.

Amirshamsi

Shahrbabaki

Dehghan

. Coping strategies for stress and self-efficacy in patients with cancer and their spouses: a cross-sectional study. Cancer Nurs 2022; 45: E614–E620.

39.

Lazarus

Folkman

. Stress, appraisal, and coping. New York, NY: Springer Publishing Company, 1984.

40.

Matheny

Aycock

Curlette

, et al. The coping resources inventory for stress: a measure of perceived resourcefulness. J Clin Psychol 1993; 49: 815–830.

41.

International Stress Management Association (ISMA). Stress Management Association, https://isma.org.uk/ (accessed 1 May 2025).

42.

Slavich

Shields

. Assessing lifetime stress exposure using the stress and adversity inventory for adults (adult STRAIN): an overview and initial validation. Psychosom Med 2018; 80: 17–27.

43.

Peacock

Wong

PTP

. The stress appraisal measure (SAM): a multidimensional approach to cognitive appraisal. Stress Med 1990; 6: 227–236.

44.

Therapy Mantra. Free online stress level assessment test and screening. Therapy Mantra, https://onlinetherapymantra.com/assessments/stress-level/ (accessed 1 May 2025).

45.

Ardell

. High level wellness: an alternative to doctors, drugs, and disease. Emmaus (PA): Rodale Press, 1977.

46.

Carver

. You want to measure coping but your protocol’s too long: consider the brief COPE. Int J Behav Med 1997; 4: 92–100.

47.

Holmes

Rahe

. The Rating Social Readjustment Scale. J Psychosom Res 1967; 11: 213–218.

48.

Holmgren

Hensing

Dahlin-Ivanoff

. Development of a questionnaire assessing work-related stress in women - identifying individuals who risk being put on sick leave. Disabil Rehabil 2009; 31: 284–292.

49.

Musen

. Protégé team. The protégé project: a Look Back and a Look forward. AI Matters 2015; 1: 4–12.

50.

Gennari

Musen

Fergerson

, et al. The evolution of Protégé: an environment for knowledge-based systems development. Int J Hum Comput Stud 2003; 58: 89–123.

51.

Bodenreider

. The unified medical language system (UMLS): integrating biomedical terminology. Nucleic Acids Res 2004; 32: D267–D270.

52.

Devlin

Chang

M-W

Lee

, et al. BERT: pre-training of deep bidirectional transformers for language understanding. Epub ahead of print 24 May 2019. DOI: 10.48550/arXiv.1810.04805

53.

Grootendorst

. KeyBERT: minimal keyword extraction with BERT. Epub ahead of print 25 January 2021. DOI: 10.5281/zenodo.4461265

54.

Schekotihin

Rodler

Schmid

. Ontodebug: interactive ontology debugging plug-in for protégé. In: Ferrarotti

Woltran

(eds) Foundations of information and knowledge systems. Cham: Springer International Publishing, 2018, pp.340–359.

55.

Poveda-Villalón

Gomez-Perez

Suárez-Figueroa

. OOPS! (OntOlogy Pitfall Scanner!): an on-line tool for ontology evaluation. Int J Semantic Web Inf Syst 2014; 10: 7–34.

56.

Kim

, et al. Mental stress ontology. NCBO BioPortal, https://bioportal.bioontology.org/ontologies/STRONG (accessed 12 August 2025).