Sage Journals: Discover world-class research

Abstract

This study explored the application of meta-analysis and convolutional neural network-natural language processing (CNN-NLP) technologies in classifying literature concerning radiotherapy for head and neck cancer. It aims to enhance both the efficiency and accuracy of literature reviews. By integrating statistical analysis with deep learning, this research successfully identified key studies related to the probability of normal tissue complications (NTCP) from a vast corpus of literature. This demonstrates the advantages of these technologies in recognizing professional terminology and extracting relevant information. The findings not only improve the quality of literature reviews but also offer new insights for future research on optimizing medical studies through AI technologies. Despite the challenges related to data quality and model generalization, this work provides clear directions for future research.

Plain language summary

This study examines how advanced technologies like meta-analysis and machine learning, specifically through Convolutional Neural Networks and Natural Language Processing (CNN-NLP), can revolutionize the way medical researchers review literature on radiotherapy for head and neck cancer. Typically, reviewing vast amounts of medical studies is time-consuming and complex. This paper showcases a method that combines statistical analysis and AI to streamline the process, enhancing the accuracy and efficiency of identifying crucial research. By applying these technologies, the researchers were able to sift through thousands of articles rapidly, pinpointing the most relevant ones without the extensive manual effort usually required. This approach not only speeds up the review process but also improves the quality of the information extracted, making it easier for medical professionals to keep up with the latest findings and apply them effectively in clinical settings. The findings of this study are promising, demonstrating that integrating AI with traditional review methods can significantly aid in managing the ever-growing body of medical literature, potentially leading to better treatment strategies and outcomes for patients suffering from head and neck cancer. Despite some challenges like data quality and the need for extensive computational resources, the study provides a forward path for using AI to enhance medical research and practice.

Keywords

natural language processing convolutional neural networks meta-analysis medical literature classification normal tissue complication probability

Background

Head and neck cancer patients undergoing radiation therapy often face risks of normal tissue complications, such as dry mouth, difficulty swallowing, and mucositis.¹ Predicting these complications accurately is crucial for optimizing treatment plans and improving patient outcomes.² However, the vast and diverse medical literature makes manual review and filtering of relevant studies increasingly arduous and time-consuming.³

To address this challenge, this study aimed to enhance the efficiency and accuracy of literature reviews concerning the normal tissue complication probability (NTCP) in head and neck cancer patients following radiation therapy. This improvement is pursued through the utilization of meta-analysis (MA) and natural language processing (NLP).^4,5 The research began with statistical analyses using Python to evaluate NTCP models for conditions such as dry mouth, difficulty swallowing, and mucositis. It then advanced to optimizing the literature search process by integrating NLP with convolutional neural networks (CNNs),⁶ successfully narrowing down from 3256 articles to just 12. The CNN-NLP model developed in this study achieved a notable accuracy rate of 0.94 after 200 training epochs, with a precision of 0.95, F1-score of 0.94, recall of 0.94, and an AUC (Area Under the Curve) of 0.81. The performance results on the training set were an accuracy of 0.95, precision of 0.96, F1-score of 0.95, recall of 0.95, and an AUC of 0.83. The discrepancy between the training and test set performance is primarily attributed to the diversity of the test set data. While the training set data was directly used to optimize the model, leading to higher performance metrics, the test set included a broader and more varied range of samples that the model had not encountered before. This variation challenges the model’s ability to generalize, resulting in slightly lower performance metrics on the test set.

The decision to review this type of article was driven by its demonstration of how effectively integrating meta-analysis and advanced NLP technology can enhance the efficiency and quality of medical literature reviews. This work is crucial for understanding the potential normal tissue complications faced by head and neck cancer patients after radiation therapy and offers fresh insights into optimizing medical research through artificial intelligence technologies.

In existing research within this field, the development and validation of NTCP models are pivotal, especially for assessing the risks to head and neck cancer patients after radiation treatment.^7,8 These models are typically designed to predict the probability of specific complications based on clinical data and radiation therapy parameters. However, due to the vast scope and diversity of medical literature, manual review and filtering of relevant research findings have become increasingly arduous and time-consuming. In the study by Deng et al, literature review requires many abstracts to be manually screened, which is often the most labor-intensive and time-consuming step in systematic reviews. Using a semi-automated NLP procedure for literature screening reduced the workload by 84% compared to manual methods (2774 abstracts vs 16,941 abstracts).⁹ As a result, recent studies have employed NLP and machine learning technologies, particularly CNNs, to automate the literature review process. The application of these technologies aims to enhance the efficiency and accuracy of literature filtering, thus accelerating the advancement of medical research and enhancing clinical decision support. This work is designed for researchers conducting meta-analysis, helping them to process and analyze large volumes of medical literature more efficiently and accurately.

Meta-Analysis

Meta-analysis is a statistical technique that combines the results of multiple scientific studies to derive a more precise overall effect size or outcome. Individual studies often have different sample sizes, methods, and results. By aggregating data from multiple studies, meta-analysis can synthesize new conclusions across all studies and use statistical analysis to demonstrate the validity of these conclusions.

The process of conducting a meta-analysis includes the following key steps:

1) Systematic Literature Review: The first step is to perform a systematic literature review to identify all relevant studies on a specific topic. This includes defining inclusion and exclusion criteria, searching multiple databases, and screening studies based on titles, abstracts, and full texts.

2) Data Extraction: After identifying the relevant studies, data extraction is performed. This involves collecting information on study characteristics, methods, sample sizes, outcomes, and other relevant variables.

3) Statistical Analysis: The extracted data is then statistically analyzed. Common statistical methods used in meta-analysis include calculating effect sizes, pooling data using fixed or random-effects models, and assessing heterogeneity among studies. This helps to determine the overall effect and identify patterns or trends.

4) Bias and Quality Assessment: It is necessary to assess the quality and potential biases of the included studies. Tools such as funnel plots and statistical tests for publication bias are used to evaluate the risk of bias and the robustness of the results.

5) Interpretation and Reporting: Finally, the results of the meta-analysis are interpreted and reported. This includes discussing the implications of the findings, potential limitations, and areas for future research.

Statistical Analysis

In the process of meta-analysis, statistical analysis is used to combine the results of different studies and determine the overall effect. In this study, we used heterogeneity analysis and random-effects models for statistical analysis.

Heterogeneity Analysis: Heterogeneity analysis is used to assess the differences in results between different studies. To quantify heterogeneity, we used the I² statistic, a common measure of heterogeneity. The I² value ranges from 0% to 100%, with higher I² values indicating higher heterogeneity, which represents greater differences between study results.

Random-Effects Model: Due to differences in sample sizes, methods, and study subjects across studies, these heterogeneities may affect the integration of results. Therefore, we chose a random-effects model to handle these variations. The random-effects model assumes that the effect sizes of individual studies are randomly distributed and takes into account the heterogeneity between studies.

In this study, our statistical analysis process is as follows:

1) Data Extraction: Extract relevant data from selected studies, including effect sizes, sample sizes, and variance data.

2) Heterogeneity Assessment: Use the I² statistic to assess heterogeneity between studies to determine if significant heterogeneity exists.

3) Model Selection: Use the random-effects model for the integration of effect sizes. This model accounts for heterogeneity between studies, providing more accurate effect estimates.

Data Sources and Preparation Process

Below is a detailed description of the databases used in this study and the data preparation process:

1) Database Sources: We used two primary medical literature databases: WOS (Web of Science) and PubMed. These databases cover a large volume of medical research literature. We selected literature related to head and neck cancer radiation therapy, specifically those mentioning normal tissue complications.

2) Data Selection Criteria:

a) Inclusion Criteria: We selected head and neck cancer-related literature that was clearly annotated, containing key medical terminology and relevant research results.

b) Exclusion Criteria: We excluded literature that had incomplete data, lacked clear conclusions, or was unrelated to the research topic.

3) Data Extraction and Preprocessing:

a) Data Extraction: We extracted relevant information from the selected literature, including titles, abstracts, keywords, and research results.

b) Text Preprocessing: We performed tokenization and word embedding on the text data. Tokenization divides the text into individual words or phrases, and word embedding uses wiki.en.vec to convert these words or phrases into numerical vectors for CNN model processing.

c) Data Annotation: We annotated the data based on the content of the literature to ensure the model could learn and recognize key medical terminology.

4) Data Splitting:

The dataset consisted of a total of 512 records. We split the dataset into training, testing, and validation sets. The data was split with 70% (359 records) allocated to the training set, 20% (102 records) to the testing set, and 10% (51 records) to the validation set.

The validation set, which was selected from the testing set, was used to tune hyperparameters and prevent overfitting during the training process.

CNN-NTCP in Classifying Literature for Head and Neck Cancer

In the current field of medical research, effectively utilizing the vast resources of literature is crucial for advancing clinical practice and academic progress. This is particularly true for studies related to the treatment of head and neck cancer, where accurate predictions from NTCP models are vital for optimizing treatment plans, minimizing side effects, and improving patient quality of life.^10,11 However, with the explosive growth of medical literature, traditional literature review methods have become increasingly time-consuming and inefficient. Therefore, the development of new technologies and methods for efficiently and accurately reviewing and classifying relevant literature is particularly important.

The approach adopted in this article combines meta-analysis with CNN-NLP technology, offering a new perspective and an effective technical pathway to address these challenges. Meta-analysis, as a statistical method, can provide more reliable and comprehensive conclusions by synthesizing the results of multiple studies. The application of CNN-NLP technology leverages the powerful capabilities of deep learning in text processing and classification, and is particularly effective in handling large-scale textual data, identifying, and extracting key information.

This study initially identified key variables and parameters in head and neck cancer NTCP model research through meta-analysis, and then used the CNN-NLP model to automatically review and classify a large volume of medical literature, aiming to pinpoint studies related to these key variables and parameters. This method allowed researchers to efficiently filter valuable information from extensive datasets, thereby accelerating the literature review process and enhancing the accuracy and reliability of the research.

The results demonstrate that the combination of meta-analysis and CNN-NLP has achieved a high accuracy rate in the literature review of head and neck cancer NTCP models, significantly improving the efficiency and quality of literature filtering. This not only underscores the potential application of deep learning technology in the field of medical literature reviews but also provides new tools and methods for future related research. However, despite these positive outcomes, there are challenges and limitations in applying these technologies, such as the substantial workload required for high-quality data annotation and the significant computational resources needed for CNN-NLP model training and optimization. Moreover, the generalizability of the model and its accuracy in identifying professional terminology still require further improvement. Therefore, future research needs to innovate in data preprocessing, model design, and optimization strategies to adapt to the complexity and diversity of the medical literature.

This study has significant practical value for researchers and clinicians in the field of head and neck cancer treatment. By enabling efficient and precise literature reviews, this approach accelerates the application of new knowledge, optimizes treatment plans, and provides more personalized and effective treatment options for patients. Additionally, the methodology of this study serves as a reference for other medical research fields, illustrating the potential of artificial intelligence technology to expedite the scientific research process. In summary, this study has successfully applied meta-analysis and CNN-NLP technologies to the review and classification of medical literature, not only enhancing research efficiency and quality but also proposing new directions and ideas for future medical research. However, to fully harness the potential of these technologies, challenges such as data quality and model generalization must be addressed. With continued technological advancements and deeper research, it is anticipated that these challenges will be gradually overcome, thereby playing an increasingly significant role in medical research and clinical practice.

The application of CNN-NLP technology in medical literature classification can be achieved through the following key steps^4,12,13:

1) Model Architecture: In this study, we selected CNN as the model from various NLP techniques. The architecture combines CNNs for extracting text features with NLP techniques to understand semantics and identify medical professional terminology. The CNN model processes the text by focusing on the keywords in the articles.

2) Training Process: The training process involves using a large volume of labeled medical literature data. Preprocessing includes tokenization and word embedding to convert the text into a numerical format that the CNN can process. The CNN model learns to minimize prediction errors through parameter adjustments.

3) Identifying Medical Terminology: Word embedding technology is used to convert medical terminology into vector form, allowing CNNs to extract features from these terms. By emphasizing keywords in the medical literature, the model can precisely identify and classify professional terminology in a large volume of medical literature.

4) CNN Architecture and Optimization Methods: The CNN model used in this study comprises multiple layers, including convolutional layers, pooling layers, and fully connected layers. Each layer has specific functions for extracting and processing text features.

a) Convolutional Layers: These layers scan the text by applying multiple filters to identify key features and patterns. In this study, the filter sizes used are 3, 4, and 5, and each convolutional layer utilizes 50 filters.

b) Pooling Layers: Pooling layers reduce the size of the features through down-sampling, which decreases computational load and prevents overfitting.

c) Fully Connected Layers: These layers integrate features from the previous layers and apply activation functions (such as ReLU) to introduce non-linearity, ultimately outputting the classification results. The dimension of the hidden layer is set to 50, ensuring a balanced trade-off between model complexity and performance.

5) Optimization Methods:

a) Loss Function: We use the cross-entropy loss function to measure the discrepancy between the predicted and actual labels.

b) Optimizer: We employ the Adam optimizer to update model parameters. Adam is an adaptive learning rate method that accelerates convergence and improves model performance.

c) Regularization: To prevent overfitting, we use dropout regularization in the model. This technique randomly drops some neurons during training to enhance the model’s generalization capability.

Experimental Equipment

1) CPU: Intel(R) Core(TM) i7-10700

2) RAM: 40 GB

3) GPU: NVIDIA GeForce RTX 3070 Ti

Discussion

The CNN-NLP model was used to process medical literature related to head and neck cancer, achieving an accuracy of 0.94, precision of 0.95, F1-score of 0.94, recall of 0.94, and an AUC of 0.81 on the test set after 200 training epochs. These metrics demonstrate the efficiency and accuracy of the CNN-NLP model in literature filtering. In comparison, the LLM model also achieved high performance in medical text classification and processing. In the study by Huang et al.,¹⁴ the LLM demonstrated an average precision exceeding 0.90 across various medical text tasks, excelling in semantic understanding and natural language generation. LLM’s generative models can handle context and abstract concepts, giving them an advantage in tackling more complex medical issues. However, they require higher computational resources and longer training times.

Future Research

Future research can further enhance model performance and broaden the application of CNN-NLP technology across various cancer types and medical fields by emphasizing the importance of interdisciplinary collaboration:

1) Improving model performance

a) Advanced NLP technologies: Investigate and integrate the latest NLP technologies, such as transformer models and pretrained language models (eg, bidirectional encoder representations from transformers [BERT], generative pre-trained transformer [GPT]). These models excel in semantic understanding and context capture, thereby improving accuracy in identification and classification.¹⁵

b) Multimodal analysis: Text data were combined with other data types (such as imaging and genomic data) for analysis. Multimodal learning techniques can be used to extract and synthesize information from diverse sources, enhancing the ability of models to understand and predict complex medical issues.¹⁶

c) Model fine-tuning and optimization: The model is tailored to specific medical domains or challenges, including adjusting the model architecture, optimizing training strategies, and using expert knowledge to guide feature selection. These steps help enhance the model’s generalization capabilities and performance in targeted areas.¹⁷

2) Potential applications

a) Other cancer types or medical fields: Extend CNN-NLP technology beyond head and neck cancer research to include other types of cancer, such as breast and lung cancer, as well as non-cancerous medical fields like cardiovascular diseases and diabetes. Its applications could include literature filtering and disease risk prediction.

b) Interdisciplinary cooperation: The complexity of medical research necessitates leveraging knowledge and technology from various disciplines, including computer science, biology, and statistics. Such interdisciplinary collaboration can foster the development and application of innovative technologies, accelerating the conversion of medical research findings into practical applications.

3) Addressing Ethical and Social Challenges

a) Strengthening Legal and Regulatory Frameworks: As technology advances, it is imperative to continually update relevant laws and regulations to safeguard patient data privacy and ensure the lawful and compliant use of AI in health care.¹⁸

b) Public Education and Communication: Enhance public understanding of AI’s role in the medical field, alleviate concerns and fears about new technologies through effective communication, and engage in thorough discussions about the ethical boundaries of AI technology. This ensures that technological development aligns with societal values and ethical standards.¹⁹

4) Promoting Interdisciplinary Collaboration

a) Collaborating with Hospitals: Collaborate with hospitals and clinical research institutions to conduct joint research, utilizing clinical data and expertise to enhance the applicability and accuracy of the models.

b) Integrating Medical Expert Knowledge: Actively involve medical experts in the research process to provide professional opinions and suggestions, ensuring that research findings can be effectively applied in clinical practice.

5) Developing Specific Actionable Steps and Roadmap

a) Setting Short-term and Long-term Research Goals: Define specific short-term and long-term research goals to ensure orderly and trackable progress.

b) Clarifying Resource Requirements and Allocation: Identify the required resources (such as personnel, equipment, and funding) and allocate them reasonably to ensure smooth research progress.

c) Regular Evaluation and Adjustment of Research Strategies: Conduct regular assessments of research progress and adjust strategies based on actual conditions to ensure that the research direction aligns with the objectives.

Conclusion

By integrating advanced NLP technologies with multimodal analysis and enhancing interdisciplinary collaboration, future research can significantly improve model performance and broaden its applications in the medical field. This approach not only propels medical research forward but also lays the groundwork for achieving personalized medicine. Concurrently, addressing the challenges of interdisciplinary collaboration and focusing on ethical and social issues are crucial. Future directions should emphasize promoting knowledge sharing, cultivating multidisciplinary talent, developing AI technologies tailored to personalized medicine, and strengthening ethical and legal frameworks for AI applications. This will ensure that technological advancements align with societal values and ethical standards.

Footnotes

Authors’ Contributions

Conceptualization: P-J.C, T-F.L., S-A.Y. Data curation: Y-W.H., Y-H. L., C-H. C., J-C. S., S-H.L., Methodology: P-J C., Y-W.H., S-H.L., C-L.C. Project administration: T-F L. P-J C., S-A.Y. Writing, original draft: T-F L. Writing, final draft: T-F L. All authors reviewed the manuscript.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Science and Technology Council; 111-2221-E-992-016-MY2, 10.13039/501100020950; National Science and Technology Council; 113-2221-E-992-011-MY2.

Ethical Statement

ORCID iD

Tsair-Fwu Lee

Appendix

References

Taylor

Amdal

Bjordal

, et al. Serious long-term effects of head and neck cancer from the survivors’ point of view. Healthcare. 2023;11:906.

Araújo

ALD

Moraes

Pérez-de-Oliveira

, et al. Machine learning for the prediction of toxicities from head and neck cancer treatment: a systematic review with meta-analysis. Oral Oncol. 2023;140:106386.

Mäkitie

Alabi

, et al. Artificial intelligence in head and neck cancer: a systematic review of systematic reviews. Adv Ther. 2023;40(8):3360-3380.

Lee

T-F

Hsieh

Y-W

Yang

P-Y

, et al. Using meta-analysis and CNN-NLP to review and classify the medical literature for normal tissue complication probability in head and neck cancer. Radiat Oncol. 2024;19(1):5.

NQK

. Leveraging transformers‐based language models in proteome bioinformatics. Proteomics. 2023;23(23-24):2300011.

Zhao

Gui

Yao

NQK

Chua

MCH

. Improved prediction model of protein and peptide toxicity by integrating channel attention into a convolutional neural network and gated recurrent units. ACS Omega. 2022;7(44):40569-40577.

Gan

Langendijk

van der Schaaf

, et al. An efficient strategy to select head and neck cancer patients for adaptive radiotherapy. Radiother Oncol. 2023;186:109763.

Spiero

Schuit

Wijers

Hoebers

Langendijk

Leeuwenberg

. Comparing supervised and semi-supervised machine learning approaches in NTCP modeling to predict complications in head and neck cancer patients. Clin Transl Radiat Oncol. 2023;43:100677.

Deng

Yin

Bao

, et al. Validation of a semiautomated natural language processing–based procedure for meta-analysis of cancer susceptibility gene penetrance. Jco Clinical Cancer Informatics. 2019;3:1-9.

10.

Yorke

. Modeling clinical outcomes in radiotherapy: NTCP, TCP and the “TECs”. Med Phys. 2023;50:122-124.

11.

Roers

Rolf

Baehr

, et al. Impact of modern low dose involved site radiation therapy on normal tissue toxicity in cervicothoracic non-hodgkin lymphomas: a Biophysical study. Cancers. 2023;15(24):5712.

12.

Vuyyuru

Krishna

Mary

SSC

Kayalvili

Alsubayhay

AMS

. A transformer-CNN hybrid model for cognitive behavioral therapy in psychological assessment and intervention for enhanced diagnostic accuracy and treatment efficiency. Int J Adv Comput Sci Appl. 2023;14(7).

13.

Kadaparthi

: A deep learning based approach for analyzing the Sentiments of financial text. In: 2023 4th IEEE Global Conference for Advancement in Technology (GCAT), Bangalore, October 2023: 1-6.

14.

Huang

Tang

Chen

: A comprehensive survey on evaluating large language model applications in the medical industry. arXiv preprint arXiv:240415777 2024.

15.

Koubaa

Boulila

Ghouti

Alzahem

Latif

. Exploring ChatGPT capabilities and limitations: a critical review of the nlp game changer. Preprints 2023. https://doi.org/10.20944/preprints202303.0438.v1

16.

Gandhi

Adhvaryu

Poria

Cambria

Hussain

. Multimodal sentiment analysis: a systematic review of history, datasets, multimodal fusion methods, applications, challenges and future directions. Inf Fusion. 2023;91:424-444.

17.

Fowlkes

Yang

Dabeer

Soatto

. Guided recommendation for model fine-tuning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023;2023:3633-3642.

18.

Saheb

. Ethically contentious aspects of artificial intelligence surveillance: a social science perspective. AI and Ethics. 2023;3(2):369-379.

19.

Adıgüzel

Kaya

Cansu

. Revolutionizing education with AI: Exploring the transformative potential of ChatGPT. Contemporary Educational Technology. 2023;15(2):ep429.