Sage Journals: Discover world-class research

Abstract

Background: Timely detection of disease outbreaks is critical in public health. Artificial Intelligence (AI) can identify patterns in data that signal the onset of epidemics and pandemics. This scoping review examines the effectiveness of AI in epidemic and pandemic early warning systems (EWS). Objective: To assess the capability of AI-based systems in predicting epidemics and pandemics and to identify challenges and strategies for improvement. Methods: A systematic scoping review was conducted. The review included studies from the last 5 years, focusing on AI and machine learning applications in EWS. After screening 1087 articles, 33 were selected for thematic analysis. Results: The review found that AI-based EWS have been effectively implemented in various contexts, using a range of algorithms. Key challenges identified include data quality, model explainability, bias, data volume, velocity, variety, availability, and granularity. Strategies for mitigating AI bias and improving system adaptability were also discussed. Conclusion: AI has shown promise in enhancing the speed and accuracy of epidemic detection. However, challenges related to data quality, bias, and model transparency need to be addressed to improve the reliability and generalizability of AI-based EWS. Continuous monitoring and improvement, as well as incorporating social and environmental data, are essential for future development.

Keywords

artificial intelligence infectious diseases epidemic pandemic early warning systems public health data analysis machine learning health policy

Introduction

Timely detection of diseases, epidemics, and pandemics enables public health authorities to implement preventive measures such as quarantine, vaccination campaigns, and public awareness initiatives.¹ Hence, early warning systems for epidemics play a crucial role in preventing the rapid spread of infectious diseases and mitigating their impact on public health; they serve as a proactive defence against the escalation of epidemics, allowing for a faster and more targeted response to protect communities and save lives. These systems utilize a combination of surveillance, data analysis, and technology to detect and monitor the emergence of potential outbreaks. To speed up the detection of early signs of epidemics, it is necessary to analyze various data sources, including reports of unusual symptoms, laboratory results, and even social media trends, to identify patterns indicative of a potential epidemic.²

Integrating modern technologies like artificial intelligence (AI) and machine learning can speed up the process of data analysis and anomaly detection.³ Indeed, AI can process vast amounts of data from diverse sources, such as medical records, social media, and environmental monitoring devices, to detect patterns and anomalies that may indicate the onset of a disease, an infection outbreak, an epidemic, or a pandemic. Furthermore, AI can enhance the accuracy and efficiency of early warning systems, as they allow for detecting complex patterns in available large and small data sets.⁴ AI models can continuously learn and adapt, improving their ability to identify potential threats. Integrating AI into early warning systems enhances the speed and efficiency of epidemic detection using complex factors contributing to disease outbreaks. By utilizing the capabilities of AI, early warning systems become more proactive, enabling authorities to implement timely interventions and preventive measures, ultimately contributing to the containment and management of epidemics.

However, the use of AI includes some risks, such as technical challenges (e.g., clinical integration), bias and lack of fairness,⁵ limited generalization,⁶ data privacy and security,⁷ and ethical considerations (e.g., consent).⁸

To this date, no framework exists to develop AI for a pandemic. Also, best practices, essential features to include, or pitfalls to avoid are unavailable. Given the importance of AI in the development of epidemic and pandemic early warning systems, the evidence demonstrating the effectiveness of AI in health, the rise of research on AI for health, including during the COVID-19 pandemic, the benefits and risks inherent in the use of AI for health, it is essential to synthesize the available evidence regarding the use of AI for epidemic and pandemic early warning systems. While there is a systematic review on the use of deep learning for epidemic detection,⁹ a narrative overview on the use of AI in public health for early warning systems,¹⁰ a systematic review on the use of AI for the detection of COVID-19 pandemic,¹¹ and a systematic review on the effectiveness of EWS in detecting infectious diseases outbreaks¹²; to our knowledge, there is no review on the use of AI for epidemic and pandemic early warning systems.

We have conducted a systematic scoping review¹³ we systematically reviewed the literature on AI and diseases, epidemics, and pandemic outbreaks to address this knowledge gap. The study objectives were to examine whether AI for effective early warning systems is adequate and the lessons learned from available evidence. We reviewed the literature on using AI and diseases, epidemics, and pandemic outbreaks. The questions that guided us were as follows: (1) “What type of AI did early warning systems use?”, (2) “Was the use of AI effective in detecting outbreaks?” and (3) “What type of strategies (if any) were implemented to mitigate the risks of AI bias?”. The results will inform future AI-based early warning systems design and development. This review provides public health practitioners, AI specialists, and policymakers insights on the items to consider when building AI-based early warning systems.

Methods

Search strategy

We have followed the PRISMA Extension for Scoping Reviews (PRISMA-ScR) guideline.¹⁴ The following five online databases were used to conduct a systematic scoping review: IEEE Xplore, ACM Digital Library, Web of Science, PubMed, and PsycINFO. The literature search strategy was conducted using key search terms established with the advice of a supervisor. The terms included in the search are as follows: surveillance, forecast, foresee, pandemic, epidemic, outbreak, artificial intelligence, AI, and machine learning. Of the 1450 citations extracted, 363 were duplicated or retracted, yielding the remaining 1087 articles. The search was conducted on August 11, 2023. All the search strategies are in the Appendix.

Eligibility criteria

The inclusion criteria were predetermined and included studies published in the last 5 years, limited to journal articles and conferences focusing on how artificial intelligence or machine learning is used to develop an early warning system for various disease outbreaks. The exclusion criteria were based on removing the studies that were not written in English and were not scholarly publications. As this was a scoping review, the search terms were limited to the titles and abstracts of the publications. The most recent search was done on August 11, 2023.

Selection process

The inclusion and exclusion criteria were applied to 1087 abstracts. Two authors (second author and third author) employed Rayyan software¹⁵ for screening 1087 articles, and 64 were identified as potentially eligible. Out of the 64 one article was not retrievable, both authors reviewed the full-text version of the 63 articles retrieved to make the final selection. Only 33 articles fit the inclusion criteria and were selected for review and synthesis. The literature search and article collection were conducted by two authors (second author and third author), and the literature review and data collection from articles were conducted by both authors (second author and third author) and were reviewed by another author (first author). A discussion was held to reach an agreement in case of any ambiguities and disagreements. This review was not registered in a prospective review registry.

Results

In total, 1450 articles were identified, 363 of which were duplicated and retracted articles. Of the 1087 unique articles, 1023 were excluded based on the content of their abstracts, and 1 article was not available for retrieval. The inclusion criteria were applied to the 63 articles after reading their abstracts, and 33 were included in the analysis (Figure 1). A complete summary of the included studies is provided in Table 1.

Figure 1.

PRISMA Diagram.

Table 1.

Studies summaries.

Study	Disease	Purpose	Continent	Country	Yearly/Monthly/Weekly data	Regression or classification	Climate data (Yes/No)	Alternative sources (Yes/No)	Algorithm (best performing)	Model type	Bias
(Aleixo et al., 2022)	Dengue fever	dengue outbreak one but ideally 3 months in advance	Latin America	Brazil	Monthly	R	Yes	No	CatBoost	Ensemble learning (Gradient boosting)	No
(Althomsons et al., 2022)	Tuberculosis	Predict which clusters are likely to grow in genotyped tuberculosis clusters	North America	United States	Quarter	C	No	No	Random forest	Ensemble learning (Bagging)	No
(Bellocchio et al., 2021)	COVID-19	COVID-19 outbreak in dialysis center within a 2-weeks forecasting	Europe	Europe	Weekly	C	No		XGBoost	Ensemble learning (Gradient boosting)	No
(Benedum et al., 2020)	Dengue fever	1) Predicting weekly dengue case numbers 4 weeks in advance, and 2) predicting weekly outbreaks 12 weeks ahead	Global	Peru, Puerto Rico, and Singapore	Weekly	R	Yes	No	RF-UFA (Random forest - Univariate flagging algorithm)	Hybrid (Univariate flagging+ensemble Learning(Bagging))	No
(Bhunia & abirami, 2023)	Dengue fever	Predict the number of dengue cases weekly	Latin America	Peru and Puerto Rico	Weekly	R	Yes	No	XGBoost (XGBoost with moving average rolling features)	Ensemble learning (Gradient boosting)	No
(Campbell et al., 2020)	Cholera	Forecast environmental cholera risk in coastal India	Asia	India	Monthly	C	Yes	No	Random forest	Ensemble learning (Bagging)	No
(Ceballos-Arroyo et al., 2019)	Dengue fever	Predict dengue outbreaks up to 4 weeks in advance	Latin America	Colombia	Monthly	R	Yes	No	Multilayer perceptron (MLP) for time series	Neural network	No
(Chekol & Hagras, 2018)	Malaria	Predict malaria epidemic	Africa	Ethiopia	Yearly	R	Yes	No	A framework of two models:- support vector regression has better accuracy- adaptive neuro fuzzy inference System(ANFIS) has better transparency and better interpretability.	Hybrid (support vector machine+neuro-fuzzy system)	No
(Didi et al., 2020)	Influenza	Predict influenza cases	North America	United States	daily	R	Yes	No	LSTM (long short term memory)	Recurrent neural network	No
(Fong et al., 2020)	COVID-19	Forecasting model from small dataset	Asia	China	daily	R	No	No	Polynomial neural network with corrective feedback (PNN+cf)	Neural network	No
(Harvey et al., 2021)	Malaria	Predict epidemic based on lowest threshold for an epidemic alert (an epidemic isdefined as the point at which the instantaneous case rate rises above the 5-year mean for that period of year plus two standard deviations)	Africa	Burkina Faso	Yearly	R	No	No	Gaussian process and random forest regressor	Hybrid (ensemble learning+Bayesian)	No
(Ismail et al., 2022)	Dengue fever	Forecast a dengue outbreak 4 weeks in advance	Asia	Malaysia	Weekly	R	Yes	No	Random forest	Ensemble learning (Bagging)	No
(Kumar et al., 2022)	COVID-19	Forecasts coronavirus outbreaks	Asia	India	daily	R	No	No	KNN	Instance-based learning	No
(Malki et al., 2021)	COVID-19	Period when the virus can be stopped and the risk of a second rebound	Global	Global	Monthly	R	No	No	Seasonal auto-regressive integrated moving average (SARIMA)	Time series model	No
(Manogaran & Lopez, 2018)	Dengue fever	Predict dengue fever cases	Asia	India	Months	R	No	No	Gaussian process regression modelLISA Local Moran’s I spatial autocorrelation method is used to find the hot spot dengue outbreak region	Hybrid (Bayesian+spatial autocorrelation)	No
(Masinde, 2020)	Malaria	Malaria outbreak prediction system that is anchored on the well-established correlation between certain climatic conditions and breeding environment of the malaria carrying vector (mosquito)	Africa	Sub-saharan africa	Yearly	C	Yes	No	- Gradient boosted trees, random forest, decision tree, and deep learning are the four top-performing algorithms, respectively.- deep learning was chosen	Neural network	No
(McGough et al., 2021)	Dengue fever	Predict dengue fever epidemic years	Latin America	Brazil	Monthly	C	Yes	No	Ensemble method (SVM)	Ensemble learning (Bagging with support vector machines)	No
(Mirri et al., 2020)	COVID-19	Predict the probability of surpassing a threshold of infected people in a given province and on a certain dayThe threshold is the average of the province; when all regions in the province surpass the average a wave is deemed happening	Europe	Italy	daily	C	Yes	No	Gradient Boost (GBoost)	Ensemble learning (Gradient boosting)	No
(Mishra et al., 2019)	Dengue fever	Predicting the number of denguecases in a given week of the year	North America	United States	Weekly	R	Yes	No	Linear regression	Linear model	No
(Nguyen et al., 2022)	Dengue fever	Predict dengue fever incidence and outbreaks up to 3 months in advance	Asia	Vietnam	Weekly	R	No	No	Attention-enhanced LSTM (LSTM-ATT)	Recurrent neural network with attention mechanism	No
(Panja, Chakraborty, Kumar, et al., 2023)	Dengue fever, malaria, Hepatitis B, and influenza	Providing reliable estimates for monthly dengue cases at 26 weeks and 52 weeks	Global	Global (Australia, Japan, Philippines, Singapore, China, India, Thailand, Mexico, Colombia, Peru, Puerto Rico, Venezuela)	Weekly	R	No	No	Maximal overlap discrete wavelet transform (MODWT) based autoregressive neural network called ensemble wavelet neural network (XEWNet)	Hybrid (wavelet transform+neural network)	No
(Panja, Chakraborty, nadim, et al., 2023)	Dengue fever	Providing reliable estimates for monthly dengue cases at 26 weeks and 52 weeks	Global	Peru, Puerto Rico, and India	Weekly	R	No	No	Maximal overlap discrete wavelet transform (MODWT) based autoregressive neural network called ensemble wavelet neural network (XEWNet)	Hybrid (wavelet transform+neural network)	No
(Patil & pandya, 2021)	Dengue fever	Predicting dengue incidences and outbreaks for the next 3 years	Asia	India	daily	R	Yes	No	Random forest regression best among regression and model facebook prophet model is the best fit time series forecasting	Hybrid (ensemble learning+time series model)	No
(Prakash et al., 2022)	COVID-19	Predicts the COVID-19 daily incidence in India in the future	Asia	India	daily	R	No	No	Random forest	Ensemble learning (Bagging)	No
(Setty et al., 2021)	COVID-19	Forecast the outbreak’s conclusion in the next days	Asia	India	daily	R	Yes	No	Linear regression is best for predicting the number of cases	Linear model	No
(Shaghaghi et al., 2020)	Influenza	Predict the number of flu cases 3 weeks in advance	North America	United States	Weekly	R	No	Yes	Long short-term memory (LSTM) neural network (eVision software)	Recurrent neural network	No
(Shinde et al., 2022)	Zika virus	Predict epidemic the next outbreak based on the number of cases	Global	Global	daily	C (that uses R)	Yes	No	XGBoost (for classification) random forest regressor (for regression)	Hybrid (Gradient boosting+ensemble learning)	No
(Singh & mittal, 2023)	COVID-19	The prediction is made for the next 30 days	Europe	Italy	daily	R	No	No	Ant colony optimization-polynomial regression (PR-ACO)	Hybrid (optimization algorithm+polynomial regression)	No
(Srivastava et al., 2022)	COVID-19	Forecast the likely onset of the fourth wave of COVID-19 in India	Asia	India	daily	R	No	No	Linear regression	Linear model	No
(Su et al., 2019)	Influenza	Forecasting weekly percent per age of influenza activity	Asia	China	daily	C	Yes	Yes	Self-adaptive AI model (SAAIM): a Combination of a seasonal autoregressive integrated moving average (SARIMA) model and an XGBoost model	Hybrid (time series model+Gradient boosting)	No
(Valter et al., 2021)	Dengue fever	Identify the cumulative incidence of dengue fever cases in the subsequent months and adapt the predicted infected curve	Latin America	Brazil	Weekly	R	Yes	No	MLP (multilayer perceptron)	Neural network	No
(Xu et al., 2020)	Dengue fever	Forecasting model for dengue 1 month before an outbreak	Asia	China	Monthly	R	Yes	No	LSTM (long short term memory)	Recurrent neural network	No
(Yang et al., 2023)	Influenza	Predict influenza cases	Asia	China	daily	R	Yes	No	MultiAttention-long short-term memory (MAL) deep-learning model	Recurrent neural network with attention mechanism	No

For the data charting process, we utilized spreadsheets to systematically organize and extract relevant data from the included studies. This process was conducted independently by two reviewers to ensure accuracy and consistency. Any discrepancies between the reviewers were resolved through discussion and consensus.

The data items collected from each study included the following:

• Study Author and Year: The name(s) of the author(s) and the publication year.

• Target Disease: The specific disease the study aimed to predict.

• Main Purpose: The primary objective of the study.

• Continent and Country: The geographic location where the study was conducted.

• Period of Prediction: The temporal granularity of the data (Yearly, Monthly, Weekly).

• Type of Algorithm: Whether the study employed a regression or classification algorithm.

• Performance: The performance metrics of the best model as defined by the authors.

• Outcomes: The final results achieved by the study.

• Data Source: The origin of the data used in the study.

• Dataset Size: The size of the dataset utilized in the study.

• Features: The features or variables included in the dataset.

• Public vs Private: Whether the data source was public or private.

• Clinical vs Non-Clinical: Whether the study setting was clinical or non-clinical.

• Climate Data: Whether climate data was incorporated in the study (Yes/No).

• Alternative Sources: Whether the study used alternative data sources such as search engine data or social media (Yes/No).

• Best Performing Algorithm: The algorithm identified by the authors as the most effective based on their performance metrics.

• Model Bias Analysis: Whether the study considered and analyzed potential biases in the model (Yes/No).

Given that these items cannot fit in one table, we reported the model’s performance and outcomes in Table 2 and the data related items in Table 3.

Table 2.

Studies’ models, performance, and outcomes.

Disease	Reference	Model	Regression or classification	Performance	Outcomes
Dengue fever	(Aleixo et al., 2022)	CatBoost	R	87% precision for severe outbreaks up to 3 months in advance	Outperforms SARIMA, emphasizes the importance of incorporating intervention procedure and related data
	(Bhunia & abirami, 2023)	XGBoost	R	MSE score of 6.37, outperforms negative Binomial regression	Better than baseline but insufficient for predicting prominent peaks during the outbreak
	(Benedum et al., 2020)	RF-UFA (random forest - Univariate flagging algorithm)	R	MCC scores: 0.84 and 0.57 for 4-weeks predictions in san juan and Singapore	RF-UFA yields best performance for 12-weeks predictions in Peru, san juan, and Singapore
	(Ceballos-Arroyo et al., 2019)	Multilayer perceptron (MLP) for time series	R	Accurate prediction 4 weeks in advance, RMSE ∼0.00667	- Predict the trend for the correct prediction of dengue cases up to the first 4 weeks- Reduced prediction accuracy when applied for an additional 10 to 20 weeks
	(Ismail et al., 2022)	Random forest	R	95% accuracy, 4 weeks in advance	Weather, virus incubation period, symptoms, and public health measures
	(Manogaran & Lopez, 2018)	Gaussian process-based regression	R	Higher performance compared to multiple regression, SVM, and random forests, RMSE = 0.281, MSE = 0.078	Observational climate data and the number of dengue cases for the study area Tamil nadu, India
	(McGough et al., 2021)	Ensemble model (SVM based)	C	- Overall accuracy of 74.2%,	- The model missed half (6/12) of non-epidemics in 2014- An outbreak is defined as a year in which the number of cases exceeds 100 per 100,000 persons- Incorporates weather patterns and susceptibility cycles
	(Mishra et al., 2019)	Linear regression	R	Achieves 19.81 mean absolute error, minimal compared to traditional machine learning techniques	Improved dengue disease spread prediction
	(Nguyen et al., 2022)	Long short-term memory (LSTM)-ATT model	R	Outperforms all other techniques up to 3 months ahead, rankings of 1.60 for RMSE and 1.95 for MAE	Continues worsening of performance for forecasts beyond 6 months
	(J. Xu et al., 2020)	LSTM model	R	LSTM model reduced the average the root mean squared error (RMSE) of the predictions by 12.99% to 24.91% and reduced the average RMSE of the predictions in the outbreak period by 15.09% to 26.82% as compared with other candidate models.	LSTM model achieved superior performance in predicting dengue cases as compared with other previously published forecasting models.
	(Panja, Chakraborty, Nadim, et al., 2023)	Maximal overlap discrete wavelet transform (MODWT) based autoregressive neural network called ensemble wavelet neural network (XEWNet)	R	XEWNet model is significantly better than other forecasters (statistical models, ML models, deep learning models, hybrid models, ensemble models)For short-term prediction, the XEWNet framework is 18% more accurate than the second-best performing forecaster, EWNet.	Provides precise and concurrent forecasts for dengue outbreaks, notable enhancement related to including rainfall data as a covariateCreate a threshold around the 3-year mean of cases plus two standard deviations.
	(Patil & Pandya, 2021)	Random forest regression	R	Best-fit regression model for five out of nine cities, support vector regression for two cities, facebook prophet for six cities	Predicts high-risk geographic regions, demonstrates sensitivity and low MAE for peak and end detection
	(Valter et al., 2021)	MLP (multilayer perceptron)	R	Best sensitivity and lowest MAE for peak and end detection	Difficulty predicting correct intensity, especially several weeks in advance
COVID-19	(Bellocchio et al., 2021)	XGBoost	C	AUC score of 0.79, predicts risk 2 weeks in advance	Provides risk scores of the occurrence of a COVID-19 outbreak in each dialysis center within a 2-weeks forecasting horizonUtilizes open-source and clinical data
	(Fong et al., 2020)	Polynomial neural network with corrective feedback	R	RMSE of 136.547 for predicting case numbers 6 days ahead	GROOMS (group of optimized and multisource selection) model, predicts trends in confirmed, cured, died, suspected, and critical case numbers
	(Kumar et al., 2022)	K-Nearest neighbor	R	Accuracy 94%	Occurrence of an outbreak
	(Malki et al., 2021)	Seasonal auto-regressive integrated moving average (SARIMA)	R	RMSE and MAE	Estimate COVID-19 pandemic’s life cycle duration, potential concern of a second wave if preventive measures are completely relaxed
	(Mirri et al., 2020)	Gradient boosting machine (GBM)	C	High accuracy, almost 90%	Predicting the probability of surpassing a threshold of COVID-19-infected cases
	(Prakash et al., 2022)	Random forest	R	Lowest RMSE of 150.082 for predicting no fourth wave	Health data from daily new cases
	(Setty et al., 2021)	Linear regression	R	99% accuracy in predicting COVID-19 outbreaks	Linear relationship between confirmed cases and cured cases
	(Singh & mittal, 2023)	Ant colony optimization-polynomial regression	R	Accuracy of 92.41%RMSE	Prediction of COVID-19 pandemic outbreak for the next 30 days
	(Srivastava et al., 2022)	Linear regression	R	Outperforms other models in forecasting the fourth wave	Health data from the world health organization (WHO)
Influenza	(Didi et al., 2020)	Long short term memory (LSTM) network	R	RMSE	For 15-weeks ahead predictionAir pollution sensors, climate variable impact, proximity regions factor
	(Shaghaghi et al., 2020)	LSTM neural network embedded in epidemic Vision tool	R	Accuracy 90.15%	Use of adjacent countries or states reflecting similar climate and development levels, 7.89% and 1.98% improvement in Chile and California forecasts
	(Su et al., 2019)	Self-adaptive AI model (SAAIM): a Combination of a seasonal autoregressive integrated moving average (SARIMA) model and an XGBoost model	C	RMSE of 0.175,MAPE of 0.110,MAE of 0.117	Strong performance in approximating influenza epidemic peak values, determining onset and end of epidemic periods, real-time forecasting
	(Yang et al., 2023)	Multi-attention LSTM (MAL) model	R	R2 of 0.70, explained variance score of 0.74	Influenza-like illness (ILI) % × positive% with climate, demography, influenza term search trends, climate data, and air quality index
Malaria	(Chekol & Hagras, 2018)	- SVR and support vector regression (SVR) for regression- Adaptive neuro-fuzzy inference system (ANFIS) for regression- SVR is better performing but ANAFIS is more interpretable	R	RMSE of 0.12	Predicts malaria epidemics 3 months in advanceAmount of rainfall, relative humidity, mean temperature, elevation, and malaria cases
	(Harvey et al., 2021)	Gaussian processes and random forest regressors	R	Prediction 13 weeks in advance, relies on consultations of infants in Burkina Faso and rainfall data	Absolute number of confirmed cases and tests, as well as their standard deviations
	(Masinde, 2020)	- Gradient boosted trees, random forest, decision tree, and deep learning are the four top-performing algorithms, respectively.- Deep learning was chosen	C	- Gradient boosted trees, random forest, decision tree and deep learning were the 4 top performing algorithms, respectively.- Deep learning was chosen as advantageous.	Classification accuracy and computation time, outperforming random forest, decision tree, and deep learning
Tuberculosis	(Althomsons et al., 2022)	Random forest	C	Accuracy score: 0.556	Predict which clusters are likely to grow on a training.Cluster characteristics in predicting unexpected tuberculosis growth is more important than individual characteristics, integration into an early warning system
Cholera	(Campbell et al., 2020)	Random forest	C	AUC score of 0.984	Predicts cholera outbreaks with routine monitoring of districts and areas
Zika virus	(Shinde et al., 2022)	- XGBoost classifier for classificatiobn- Random forest regressor for regression	C (that uses R)	Classifier accuracy: 60.373%	Best regressor with mean squared error: 441754.63 and coefficient of determination (R2 score): 0.09
Dengue fever, Malaria, Hepatitis B, and influenza	(Panja, Chakraborty, Kumar, et al., 2023)	Maximal overlap discrete wavelet transform (MODWT) based autoregressive neural network called ensemble wavelet neural network (XEWNet)	R	A significant improvement in epicasting by applying the proposed EWNet framework as reported by the RMSE, MASE, MAE, and sMAPE scores.The proposed EWNet performed better compared with 22 statistical, machine learning, and deep learning models.	Forecasting short, medium, and long-term disease incidence

Table 3.

Dataset characteristics.

Study	Data source	Dataset size	Features	Public vs private	Clinical vs non-Clinical	Climate data (Yes/No)	Alternative sources (Yes/No)
(Aleixo et al., 2022)	- Health data from the national health notification information system (SINAN)- Climate data from national weather institute (INMET) - Sociodemographic data from the Brazilian institute of Geography and statistics (IBGE)- The number of health facilities national register of health facilities (CNES)	- Not mentioned.- Not mentioned.- Not mentioned.−248 health facilities	- Lab results: Case number over the last one, two, and 3 months; sum of cases in neighboring districts; dengue prevalence (normalized)- The mean temperature, precipitation, air humidity for each month-Public health data: the LIRAa (an index determined from a survey of Aedes aegypti infestation indicators), the number of zika and chikungunya cases- Demographic density- Number of health facilities	- Public- Public- Public- Unknown	- Clinical- Non-clinical- Non-clinical- Non-clinical	Yes	No
(Althomsons et al., 2022)	US national tuberculosis surveillance system for tuberculosis research	−64,942 cases reported during 2009-2017.	- Characteristics of patients: Homelessness, incarceration, injection drug use, non-injection drug use, excess alcohol use, and residence of a long-term care facility.- Patient clinical factors: Previous case of TB, sputum smear positivity, cavitary disease, human immunodeficiency virus, and multiple drug resistance.- Patient demographic and epidemiologic characteristics: Born in the United States, pediatric case (<15 years old), found via targeted testing or contact investigation, reported an epidemiologic link to another case, or attributable to recent transmission.- Cluster characteristics: Average number of cases per quarter in the baseline, number of cases in a cluster, and difference in time (months) between the first and last case.	- Private (researchers may need to email CDC headquarters at TBInfo@cdc.gov)- Private (training data can be requested at DTBESupport@cdc.gov)	- Clinical and non-clinical- Clinical	No	No
(Bellocchio et al., 2021)	The european clinical database for malaria prediction	- Not mentioned	- Open-source data- Epidemic status in the clinical country/region- Aggregated data abstracted from EuCLiD: Epidemic status in the target clinic, distance-weighted information of the adjacent clinics, other parameters related to the target clinic−74 variables are included in the final model.	- Data may be available from the author with the permission of fresenius medical care.	- Clinical	No	No
(Benedum et al., 2020)	- Health data from the ministry of health (Peru, Puerto Rico and Singapore)- Population data from the national statistical institute of Peru, the U.S. Census Bureau, ministry of trade and industry- Climate data from the national oceanic and atmospheric administration and the national environment agency (Singapore)	- Not mentioned.	- Weekly case counts, and the positive untested cases (computed by multiplying the number of untested cases rate laboratory-positive cases amongst those that were tested)- Population estimates- Temporal predictors- Daily temperature, humidity, and rainfall summaries (i.e., averages, minimums, maximums, and totals	- Public- Public- Public- Public- Public- Mostly public (private data can be purchased by emailing at sales_climo@nea.gov.sg)	- Clinical- Non-clinical- Non-clinical- Non-clinical- Non-clinical- Non-clinical	Yes	No
(Bhunia & abirami, 2023)	- Health data from Hospitals- Climate data from the DrivenData.org	- The training dataset contained 1457 rows and 24 columns with the mentioned features.- The testing dataset contained 417 rows and 24 columns.	−24 features including: city, year, week of year, week start date, reanalysis air temperature, reanalysis average temperature, reanalysis dew point temperature, reanalysis max air temperature, reanalysis min air temperature humidity, precipitation amount, relative humidity, vegetation index (in the two cities of san juan and iquitos)	- Public- Public	- Clinical- Non-clinical	Yes	No
(Campbell et al., 2020)	- Health data collected from publicly available weekly epidemiological reports published in portable document format (PDF) by the integrated disease surveillance programme of India (IDSP)- Essential climate variables (ECVs) obtained fromThe european space agency (ESA) climate change initiative (CCI) program; - Altimetry data product produced by ssalto/Duacs and distributed by AVISO (https://www.aviso.altimetry.fr/);- The ECV data of total precipitation of the european reanalysis interim (ERAI) synoptic monthly means obtained from the european centre for medium-range weather forecasts	- Not mentioned.Though, the model input dataset consisted of a total of 850 outbreak and 8504 non-outbreak data points.	- The incidences of communicable diseases for a given week- Altimetry- Sea surface temperature, sea surface salinity, sea level anomaly, Chlorophyll-A concentration, soil moisture combined product, average day Land surface temperature, synoptic means of total precipitation	- Public- Public- Researchers can gain access through an authentication process at https://www.aviso.altimetry.fr/- public	- Clinical- Non-clinical- Non-clinical- Non-clinical	Yes	No
(Ceballos-Arroyo et al., 2019)	- Health data from the system for surveillance in public health (SIVIGILA)- Climate data from early warning system for medellin and the aburr Valley (SIATA)	- A total of 28,817 probable and confirmed cases (source: SIVIGILA)	- Probable or confirmed dengue cases- Precipitation, temperature, and humidity	- Public- Public	- Clinical- Non-clinical	Yes	No
(Chekol & Hagras, 2018)	Historical climatic data, elevation and malaria data from Ethiopia	- Range of cases: 0-111 (number of positive malaria confirmed persons per month/1000 population 1 month lag)	- The amount of rainfall, relative humidity, mean temperature, elevation, and malaria cases	- Unknown	- A mix of clinical and non-clinical data	Yes	No
(Didi et al., 2020)	- Weather data from the climate data online (CDO)- Health data from the centers for disease control (CDC)- Air pollution data from the United States environmental protection agency	Not mentioned	- CDC-reported ILI flu counts- Maximum temperature, minimum temperature, precipitation, and humidity- Air pollution variables (PM2.5, PM10, NO2, CO, SO2)	- Public- Public- Public	- Non-clinical- Clinical- Non-clinical	Yes	No
(Fong et al., 2020)	Small dataset reported in the literature	- Not mentioned	- Confirmed, cured, died, suspected, critical	– Unknown	- Clinical	No	No
(Harvey et al., 2021)	Integrated e-diagnostic approach (IeDA) dataset from Burkina Faso government for dengue research	- Not mentioned	- The absolute number of cases of malaria- The rainfall data	- Private (under strict licensing)- Unknown.	- Clinical- Non-clinical	No	No
(Ismail et al., 2022)	- Health data from the ministry of health - Meteorological data from Malaysia environment department	- Not mentioned	- Epidemiological (notified dengue cases, onset cases and the number of interventions)- Entomological (the number of aedes larvae collected weekly)- Environmental (rain gauges, temperature, and humidity data, the air pollution index (API))	- Public	- Clinical- Clinical- Non-clinical	Yes	No
(Kumar et al., 2022)	Kaggle for COVID-19 dataset	−11435 records and 20 symptoms.	- The patient’s age, sex, confirmation duration (in days), systemic weakness, cough, pain, headache, fever, exhaustion, diabetes, heart illness, respiratory disease, and other factors	- Public	- Clinical	No	No
(Malki et al., 2021)	Records from the world health organization (WHO) for COVID-19 research	- dataset sizes differ per country	- Confirmed cases, daily recovery, and death rates	- Public	- Clinical	No	No
(Manogaran & Lopez, 2018)	Integrated disease surveillance project (IDSP) for data on infectious diseases	- It is not mentioned.	- Maximum temperature, minimum temperature, precipitation, wind, relative humidity and solar- The number of dengue cases	- Public- Public- Public	- Clinical- Clinical- Non-clinical	No	No
(Masinde, 2020)	- Climate data from the world Bank climate knowledge data repository- Health data from the world health organization (WHO) data repository	−272,832 rows of data (for environmental data)	- Incidences of malaria- Total monthly rainfall and monthly mean temperature- Weight³	- Public- Unknown	- Non-clinical- Clinical	Yes	No
(McGough et al., 2021)	- Health data from the Brazilian ministry of health- Climate and meteorological data (MERRA-2) from NASA Goddard space flight center	- It is not mentioned.	- Annual dengue fever cases- Daily temperature and precipitation	- Web site not referenced (but it’s ministry so should be public)- Public	- Clinical- Non-clinical	Yes	No
(Mirri et al., 2020)	- Health data from the regional environmental protection agencies (ARPA) for COVID-19 research- Climate and weather data from the Italian civil protection	- It is not mentioned.	- Daily infections- The PM2.5, PM10, and NO2 pollutants	- Public- Public	- Clinical- Non-clinical	Yes	No
(Mishra et al., 2019)	- Health data from the US government data for dengue prediction- Climate data from the climate data record agency and the national centers for environmental prediction	- It is not mentioned.	- The total number of reported dengue patients weekly- Daily rainfall estimates at a spatial resolution- Normalized difference vegetation index (NDVI)	- Public- Public	- Clinical- Non-clinical	Yes	No
(Nguyen et al., 2022)	National institute of Hygiene and epidemiology (NIHE) for dengue data in Vietnam	−1,618,767 notified cases of DF from 1997 to 2016.	- Monthly incident confirmed cases and deaths- Average monthly temperature, maximum average monthly temperature, minimum average monthly temperature, monthly absolute maximum temperature, monthly absolute minimum temperature, monthly rainfall, highest daily rainfall per month, number of rainy days per month, monthly average relative humidity, monthly minimum relative humidity, monthly evaporation, total monthly sunshine hours	- Public- Unknown	- Clinical- Non-clinical	No	No
(Panja, Chakraborty, Kumar, et al., 2023)	- Health data from the epidemic datasets are accumulated from publicly available data resources	- It is not mentioned.	- Case numbers for various diseases	- Public	- Clinical	No	No
(Panja, Chakraborty, Nadim, et al., 2023)	- Health data from the national oceanic and atmospheric administration (NOAA) repository [for Puerto Rico and and Peru]- Precipitation data from the U.S. climatology agency and the NOAA [for Puerto Rico and and Peru]- Health and precipitation data from a published study [for India]	- It is not mentioned.	- The weekly laboratory-confirmed dengue incidence cases- Precipitation	- Public	- Clinical- Non-clinical	No	No
(Patil & Pandya, 2021)	- Health data from national vector Borne disease control program (NVBDCP)- Monthly climate data from the indian meteorological department (IMD)	- It is not mentioned.	- Monthly dengue disease incidence- Monthly mean maximum temperature, monthly mean minimum temperature, total monthly rainfall, relative humidity, mean wind speed	- Public- Public	- Clinical- Non-clinical	Yes	No
(Prakash et al., 2022)	- Health data from daily new cases of COVID-19 reported in India	- It is not mentioned.	- COVID-19 incidents (susceptible, exposed, infected and recovered)	- Public	- Clinical	No	No
(Setty et al., 2021)	Kaggle for COVID-19 dataset	- It is not mentioned.	- States/UT’s, total cases, discharged, and deaths	- Public	- Clinical	Yes	No
(Shaghaghi et al., 2020)	- Health data from WHO flu season data, CDC flue season data, - Google search data from Google trend	- It is not mentioned.	- Flu season data- The key phrases selected in English for Google were ’flu’, ’sore throat’, ’cough’, and ’tamiflu’, which were then translated in Spanish to ’gripe’, ’dolor de garganta’, ’tos’, and ’tamiflu’ respectively. The only country for which Portuguese data was collected was Brazil, where the keywords were translated to ’gripe’, ’dor de garganta’, ’tosse’, and ’tamiflu’.	- Public- Public- Public	- Clinical- Non-clinical	No	Yes
(Shinde et al., 2022)	- Health data from the Zika data repository from the CDC and prevention for Zika virus research - Climate data from the european centre for medium-range weather forecasts	- It is not mentioned.	- Zika virus incidence- The population density of a region- Historical weather data (temperature, precipitation, and humidity)- Latitude an d longitude information	- Public- Unknown	- Clinical- Non-clinical	Yes	No
(Singh & Mittal, 2023)	Johns Hopkins University official repository for COVID-19	- It is not mentioned.	- Cases, death, and recovery numbers	- Public	- Clinical	No	No
(Srivastava et al., 2022)	Data from the world health organization (WHO) for COVID-19 analysis	- It is not mentioned.	- COVID-19 incidence (forecast results data from India)	- Public	- Clinical	No	No
(Su et al., 2019)	- Health data from Chinese surveillance system for influenza- Climate data from China meteorological administration- Search terms from the Baidu search engine index- Micro blogs from the sina weibo microblogging services	- It is not mentioned.	- The real-time weekly percentage of influenza-like illness (ILI%)- Weather data (average temperature, lowest temperature, highest temperature, average humidity, highest humidity, lowest humidity, atmospheric pressure)- Baidu search index- Sina weibo data of chongqing	- Unknown- Public- Public- Public	- Clinical- Non-clinical- Non-clinical- Non-clinical	Yes	Yes
(Valter et al., 2021)	- Health data from the notifiable diseases information system (SINAN) for the infected population- Climate data from Brazilian national institute of meteorology (INMET) for meteorological data- Population data from Brazilian institute of Geography and statistics (IBGE) for population characteristics	−27, 895 samples	- Infecteds SMA (simple moving average)- Weekly cumulative raining precipitation- Weekly cumulative evaporation- Daily maximum temperature SMA- Daily average temperature SMA- Daily minimum temperature SMA- Daily insolation SMA- Daily wind speed SMA- City population- City population density	- Public- Public- Public	- Clinical- Non-clinical- Non-clinical	Yes	Yes
(Xu et al., 2020)	- Health data from the individual-level records from the China national notifiable disease surveillance system (NNDSS) for dengue prediction- Climate data from meteorological variables from the national meteorological information center (NMIC) for dengue prediction in China	- It is not mentioned.	- Monthly case-level records of human dengue incidence (including gender, age, nationality, and residential address)- A total of 9 meteorological variable: Maximum pressure, average pressure, average of water pressure, minimum air temperature, maximum air temperature, average of daily highest temperature, average of daily precipitation, number of days with rainfall, average of relative humidity.	- Unknown- Unknown	- Clinical- Non-clinical	Yes	No
(Yang et al., 2023)	- Health data from The Beijing Municipal influenza monitoring system- Climate data from the meteorological data from the China meteorological data sharing service system for influenza prediction	- A different sample every year	- ILI cases¹⁵- Data of virological surveillance- Climate data (the mean temperature per week, maximum temperature per week, minimum temperature per week, mean relative humidity per week, mean air pressure per week, mean wind speed per week, mean precipitation per week, mean sunlight hours per week, air quality data,)- Demographics- Data from Baidu search engine (“influenza” as the keyword)	- Unknown- Public- Public- Public- Public- Public	- Clinical- Clinical- Non-clinical- Non-clinical- Non-clinical- Non-clinical	Yes	No

We synthesized the results by collating and summarizing the extracted data to identify patterns and trends across the included studies. This narrative synthesis was complemented by visual representations (i.e., tables) to effectively convey the range and nature of the evidence. The findings were then interpreted in the context of the existing literature, discussing their implications for practice, policy, and future research directions.

Early warning systems description: Countries, purpose, algorithms, and data characteristics

Countries

The 33 studies covered 23 countries. 13 studies (40%) were in Asia, five (15%) in Latin America, four in North America (15%), five (12%) global conducted in multiple countries, three (9%) in Africa, and three (9%) in Europe (Table 1).

Purpose

13 studies (40%) aimed at predicting outbreaks of dengue fever,^16–28 nine studies (28%) of COVID-19,^29–37 four studies (12%) of influenza,^38–41 three studies (9%) of Malaria,^42–44 one study (3%) of cholera,⁴⁵ one study (3%) of tuberculosis,⁴⁶ one study (3%) of Zika virus,⁴⁷ and most notably only one system was “generalistic” as it targeted multiple diseases.⁴⁸

Algorithms

The predictions covered either weekly or monthly case estimates in the case of regression models, and occurrence of outbreaks one or more months in advance in the case of classification. Seven studies (21%) used classification techniques to predict the occurrence of an outbreak, and 25 studies (76%) used regression techniques to predict the number of cases; one study (3%) used a classification technique to predict the occurrence of an outbreak following a regression technique that predicted the number of cases. Both types of modelling targeted weekly, monthly, or yearly predictions. Also, the type of algorithms used varied widely, topped by Random Forest and Neural Network used in six studies each, followed by Long Short-Term Memory (LSTM) in five studies, boosting-based models in five studies, linear regression in three studies, and probabilistic approaches in two. SVM, SVM-based ensemble technique, K-Nearest Neighbour (KNN), Logistic Regression, and Seasonal Auto-Regressive Integrated Moving Average (SARIMA) were all used in one study. One study was unique as it combined a SARIMA model and a Boosting (XGBoost) model. A complete summary of the studies’ best-performing models, performance measurements, and outcomes can be found in Table 2.

Dataset characteristics

The data sources for the mentioned studies can be categorized into three groups: health, meteorological/climatic, and miscellaneous (e.g., search engines and social media). Table 3 summarizes the source per study. Notably, many studies combined health data with meteorological or environmental data; two studies have resorted to search engine trends (Google and Baidu) and one has resorted to social media (i.e., Sina Weibo microblogging) to supplement their features with human interaction with the world. The data sources and their types are summarized in Table 3. Notably, 19 (49%) of the studies used climate data in addition to health data, while only three studies (9%) used input from online sources, namely Google search trends from Google Trends⁴¹ and Baidu search engine words,^38,40 and one of the latter two studies also used microblogs from Weibo.³⁸

AI techniques: Types, effectiveness and performance

The 33 studies encompass a broad range of applications of machine learning and AI techniques in predicting and managing various disease outbreaks. There were commonalities and differences among these studies in terms of types of AI techniques used, their effectiveness, and the factors influencing their performance.

Types of AI techniques

The 33 studies employed a variety of AI techniques with a diversity of approaches. There were six types of algorithms: time series, instance-based models, linear models, neural networks (including recurrent neural network), Ensemble learning, and hybrid models (see Table 1).

Time series algorithms (SARIMA) and instance-based learning (KNN) algorithms were used in one study each (3%), Linear model algorithms were used in three studies (9%), both neural networks and ensemble learning were used in 9 studies (27%), while10 studies used hybrid approaches (31%).

Deep learning models were prevalent due to their ability to detect complex relationship within the data, particularly Long Short-Term Memory (LSTM) networks (5 out of 9 neural network studies) was used for time-series forecasting due to their capacity to capture temporal dependencies.^{26,27,29,40,41,49}

Additionally, ensemble methods, which combine multiple models, particularly random forest (4 our of 9 ensemble studies),^36,46,50,51 were particularly employed due to their ability to enhance prediction accuracy by reducing variance and bias.

Effectiveness of the AI techniques

The effectiveness of these AI techniques varied across studies, but they demonstrated promising results in predicting disease outbreaks. Advanced machine learning techniques like Random Forests and Deep Learning models consistently outperformed traditional statistical methods, achieving accuracy rates ranging from 55.6%⁴⁶ to 95%.⁵¹

It worth noting that deep learning models, despite their high accuracy, were often considered “black boxes,” making their predictions difficult to interpret.^18,42,43,50 This trade-off between accuracy and interpretability highlights a well-known challenge.

Performance of the AI techniques

Several factors influenced the performance of the developed models. Data quality and quantity were paramount, with studies using extensive, high-quality datasets reporting more accurate predictions.^{18,30,32,33,35,37,41,50,51} Data preprocessing and feature engineering also played a crucial role in enhancing model performance.^17,21 However, the complexity of the models posed a risk of overfitting, especially with limited data.^16,21,31,48 Techniques like cross-validation and regularization were employed to mitigate this risk. Additionally, the computational resources required for deep learning models and ensemble methods were a limiting factor in resource-constrained settings.^20,27,31 External factors such as climate, socio-economic variables, and public health interventions also significantly influenced model performance. Integrating multi-source data, including environmental and demographic information, improved the robustness of predictions.

System challenges

The studies covered in this review underscore the complex challenges epidemic and outbreak prediction models face, ranging from data quality and availability issues to model generalizability and adaptability limitations in dynamic scenarios. System challenges were not reported in eight studies.^{28,30,35–39,43}

Model’s biases

Generalizability

Another limitation involves the model’s focus on a specific population (infants under 5 years old), reducing its generalizability to the entire population and its tendency to underestimate severe and overestimate mild outbreaks.^18,44 In the case of tuberculosis prediction models, the generalizability challenge was also encountered due to the geopolitical clustering of the data (while the disease does not know borders) and the exclusion of data from children.⁴⁶ In malaria prediction, errors are recognized due to diverse climate types and the inherent incompleteness and vagueness of health-related datasets, emphasizing the need for cautious use of prediction tools.⁴² Similarly, studies on dengue prediction note the challenge of diminishing accuracy with forward forecasting, especially for small events, and the model’s limited adaptability to various scenarios (e.g., data from cities with different populations).¹⁷ In one study assessing cholera outbreaks, overfitting was a concern as the model performed aggressively better in districts and states with superior healthcare, potentially compromising its generalizability.⁴⁵ Limitations in real-time or dynamic outbreak prediction are highlighted; additionally, the study’s focus on a specific region (Tamil Nadu, India) raises concerns about the model’s generalizability to different geographical areas.²³

One study notes that its model tends to underestimate the number of cases during severe outbreaks and overestimate them in mild outbreaks despite correctly identifying the occurrence of outbreaks.¹⁸ Another study faces a significant disadvantage as the model fails to accurately identify primary rises in case numbers, resulting in the omission of significant outbreaks.²¹

False positives

A model displays a high percentage of false negatives (i.e., low precision), raising the risk of unpreparedness for outbreaks and potential loss of lives; also, too many false positives erode confidence and can rapidly deplete limited resources. Only four studies reported on false positives in their AI models for epidemic and outbreak prediction. Benedum et al. note that their Random Forest-Unweighted Frequency Analysis (RF-UFA) model for predicting dengue outbreaks in Iquitos and San Juan experienced issues with false positives (without specifying the exact rate) leading to incorrect predictions of extended outbreak periods.¹⁶ Aleixo et al. also reported 3% of false positives in their dengue prediction model 3 months in advance.¹⁸ Harvey et al. highlighted the challenge of balancing precision and recall in their malaria epidemic prediction model, noting a high number of false positives when using higher percentiles for triggering alerts.⁵² Campbell et al. discuss how their cholera risk prediction model maintained a low number of false positives (without specifying the exact rate) across different districts in India.⁵⁰

Four studies report on false negatives. Harvey et al. noted a high number of false negatives when using lower percentiles to trigger alerts for malaria epidemics, resulting in many true epidemics being missed despite improved recall at higher percentiles.⁵² Aleixo et al. discuss a case where their model predicted 72 dengue cases against an actual value of 79, highlighting the challenge of predicting borderline cases.¹⁸ Campbell et al. identify false negatives in their cholera risk prediction model, particularly in districts with fewer outbreaks, underscoring the difficulty of accurate prediction in areas with limited data.⁵⁰ Similarly, Nguyen et al. report that their LSTM-ATT model missed true dengue outbreak months in some provinces, leading to false negatives despite generally high accuracy and specificity.²⁷ This highlights the ongoing challenge of accurately detecting all true outbreaks, especially in regions with limited data or less frequent occurrences.

The complexity of societal and political dimensions

As well-known in public health, social and political dimensions of human life influence behaviour, decisions, outbreaks, and outbreak responses. This was never part of the 33 studies; however, it was reflected upon in a few of them. Societal and political dimensions were a challenge reported during the COVID-19 prediction.³¹ Additionally, another study acknowledges that the accuracy is affected by the complex nature of transmission that is influenced by the interplay of factors, including human behaviour, healthcare infrastructure, and public health measures.³³ Finally, a third study focusing on infectious disease transmission acknowledges limitations such as incomplete datasets, the uncertainty of pandemics, and the challenges posed by the interconnected and complex nature of the world.²⁹

Data challenges

The data challenges might impact the effectiveness and reliability of AI-based early warning systems. Challenges related to data volume and variety affect accurate predictions and evaluating AI effectiveness. While data velocity issues affect real-time prediction capabilities, and ensuring data availability and granularity is fundamental for implementing and testing AI techniques. Additionally, tackling data granularity and variety helps mitigate biases, ensuring fair and accurate predictions across different populations.

Many aspects of Early Warning Systems relate to data availability, be it in the form of lack of data volume, velocity, variety, availability, and granularity.

Data volume

The dependency between data sizes and model performance is well-known.⁴⁸ In the context of dengue prediction, a study encounters challenges related to dataset size and resource constraints. Analyzing only confirmed cases reduces the dataset,²⁴ and obtaining entomological data, crucial for accurate spatiotemporal predictions, is costly and time-consuming.²⁵ Another limitation identified is the small datasets available for deep learning models and the extended training time for LSTM models, impacting their effectiveness; also, models could not be effective for some cities, possibly due to relevant potential socio-economic factors not being present in the dataset.²⁶ A study focusing on Zika virus prediction highlights the limitation of relying solely on publicly available data, emphasizing the need for additional information from government and healthcare institutions.⁴⁷ The lack of data on a geographical scale, temperature, rainfall, or other attributes impacts the model’s accuracy.⁴⁸

Data velocity

Delays in obtaining the right data present a unique challenge in one study, requiring a 3-week time series prediction to forecast events 1 week ahead of real-time due to a 2-week data lag.⁴¹

Data variety

A study on influenza prediction also indicates the need for more available and detailed influencing factors and symptom surveillance data (e.g., circulating virus strains, specimen collection rates, case selection bias, and healthcare–seeking behaviours).⁴⁰ Linked to variety, challenges were observed in addressing reporting space and time heterogeneity.¹⁹ Additionally, accuracy could be improved by incorporating various data such as geographical scale, temperature, rainfall, or other attributes impacting individual epidemics.⁴⁸

Lack of availability

Disparities between data reporting rates and methodologies across different locations were reported as a limitation.²⁷

Lack of granularity

One study’s scope was constrained to a limited number of cities, primarily due to the absence of granular data. Non-climatic factors, including population demographics, community immunity levels, socioeconomic conditions, healthcare accessibility, and environmental intervention strategies, might not be considered while they are important in modelling outbreaks. Time-variant factors like changes in mosquito density, population movements, and vector control measures were sometimes overlooked.²² Similarly, a forecasting model applied in a study focused on COVID-19 resurgence ignored crucial parameters, including the number of lockdowns, people vaccinated, social distancing measures, and self-isolation behaviours.³⁴

Discussion

We will discuss the following findings in relation to our three research questions: (1) “What type of AI did early warning systems use?”, (2) “Was the use of AI effective in detecting outbreaks?” and (3) “What type of strategies (if any) were implemented to mitigate the risks of AI bias?”.

Type of AI early warning systems

Most of the studies relied on regression (25 studies or 76%) to compute the number of cases in a week, month, or year. The classification was used less to detect the presence or absence of an epidemic (7 studies or 21%). The only study (3%) that used both techniques computed the number of cases and used a threshold to decide on the occurrence of an outbreak. Three studies gathered data from search engine search trends (Google⁴¹ and Baidu^38,40) and the Chinese Weibo microblogs.³⁸ Natural Language Processing (NLP) was not used in any case. In one of the three studies, the weekly counts of searched or blogged words related to the disease (Influenza) were included in the model.³⁸ In one study in China the weekly aggregate for several keywords related to influenza in Baidu search index was considered as a feature,³⁸ while another study in China considered solely the word “influenza” in the Baidu index.⁴⁰ The third study in the United States, Google Trends key phrases were stripped down to four words (flu, sore throat, cough, and Tamiflu) and their equivalent in Spanish and Portuguese and provided to the algorithm with the number of influenza cases sources from the World Health Organizations as features to the algorithm.

As presented in the results, the algorithms used varied across the studies, with Neural Networks Random Forest as the top-used one, followed closely by LSTM. While LSTM is a natural fit for time-series data and neural networks for complex patterns, the random forest presence as a top used may be seen as a surprise; however, this is in line with research that demonstrated that Random Forest algorithm provides not only equivalent but also enhanced predictive ability over other time series models for infectious disease outbreaks prediction such as retrospective and prospective ARIMA.⁵³ It is worth including Random Forest as part of the machine learning algorithms that could be used to build future early warning systems.

AI effectiveness in detecting outbreaks

The reviewed studies demonstrate the ability of machine learning to be used for early warning of a pandemic or epidemic. With the exception of two studies that lacked detailed reporting,^32,37 all other studies showed a very good performance for the developed models. There was deterioration noticed in the models’ performance with longer time predictions, which is expected; for instance, the model’s Mean Absolute Error (MAE) in Valter et al. study increased from 3.18 infected for 1 week to 4.07 for 5 weeks and 5.62 for 10 weeks.¹⁷ Otherwise, the models showed high performance and were satisfactory.

The effectiveness of AI models is measured using different performance metrics (e.g., AUC, precision, recall, MAE). However, a model’s performance can deteriorate with longer prediction horizons. This decline in performance over time underscores the challenge of maintaining performance in long-term forecasts, which is a critical factor in the practical application of AI models for outbreak detection.

The effectiveness of these AI models is evaluated not only through offline validation but also prospectively in real-world scenarios. Only one study implemented and tested its self-adaptive AI model (SAAIM) for real-time influenza forecasting in Chongqing, China and the model’s performance demonstrated its practical utility in guiding public health decisions and resource allocation during flu seasons.³⁸ However, there is a need for continuous monitoring and improvement of AI models to ensure their adaptability to evolving data and real-world conditions. While offline validation provides an initial assessment of model performance, real-world implementation often reveals additional challenges, such as data reporting delays and varying epidemiological patterns, that must be addressed to enhance the reliability and effectiveness of AI-based early warning system.

Strategies implemented to mitigate the risks of AI Bias

(3) “What type of strategies (if any) were implemented to mitigate the risks of AI bias?”.

Bias measurement and mitigation of AI bias are crucial for any AI use for health purposes, including public health, as they allow for the avoidance of unintended negative consequences of using AI in the health domain. A bias-free early warning system is reliable and trustworthy, increasing the chances of a timely and effective public health intervention by stakeholders. While addressing bias is a recent question in AI, it is an essential factor to consider when developing AI for health, as indicated by the COVID-19 pandemic, where racial/ethnic minority groups and those from low lows socio-economic status were more vulnerable to it.⁵⁴

While 12 of the reviewed studies touched on predictive models’ bias in terms of generalizability and complexity of the phenomenon to detect,^{16–18,21,23,29,31,33,42,44–46} none measured the model’s bias in terms of imbalanced decision-making affecting a specific population (e.g., people with disabilities, people with low socioeconomic status). This highlights the need to measure and mitigate bias in future studies related to early warning systems.

Challenges and future perspectives

Model explainability

Timely intervention and understanding the factors contributing to a model’s predictions are paramount in epidemic outbreak interventions. Hence, an explainable predictive model in an early warning system is crucial because it allows stakeholders to have an interpretable model. An interpretable model enables healthcare professionals and policymakers to grasp the rationale behind predictions, fostering trust in the evidence provided.⁵⁵

The interpretability of a predictive model of an early warning system adds much value to the associated predictions. Explainability allows for understanding the different variables (i.e., features) in play, providing targeted preventive measures and more fine-tuned policymaking. Moreover, explainability and interpretability enhance communication between stakeholders (e.g., public health officials, health professionals, and policymakers) as they facilitate communication and trust in the decisions being made.

In our review, only one study out of 33 (3%) addressed the explainability of the developed model,¹⁸ which indicates the urgent need to embed explainability as a factor in early warning systems.

SDG consideration

The United Nations Sustainable Development Goals are created to attain a fair, just, and equitable world.⁵⁶ Developing an AI-based early warning system assists in achieving SDG 3 (Good Health and Well-being), especially if it has a mechanism to detect and mitigate bias. Indeed, such a system allows us to implement a mechanism for monitoring, early detection, and response to an epidemic or pandemic, facilitating outbreak containment and reducing its impact on the population and especially disadvantaged communities. Moreover, AI-based early warning systems can allow for an optimized targeted resource allocation. Finally, if used on a global scale, early warning systems can enhance SDG three globally.

Going beyond health and climate: Use of social data

Disease outbreaks are complex phenomena affected by climate, weather, human behaviour, vector density, virus serotypes, and public health prevention and control. The need for a multitude of data sources integrating such variables is essential for the success of an AI-based early warning system. However, online data proved effective in detecting diseases⁵⁷ and can be used in public health⁵⁸; in our review, two studies ventured to use data related to human behaviour from search engines⁴¹ and microblogs.⁵⁹ By incorporating social media and online data sources, early warning systems could offer nuanced insights.

Expanding the data sources to include online sources was rarely observed in our review and was a source of data enrichment that led to accurate predictions. Expanding a dataset beyond conventional health and climate indicators to integrate online social data is a potentially beneficial route for future work in the field to allow for a holistic approach to epidemic/pandemic predictive models.

Adaptability and continuous model monitoring and improvement

While the limited data at the beginning of a pandemic is a challenge, innovative solutions such as data augmentation could be beneficial. This underscores the adaptability required in dealing with evolving data situations. Moreover, preparing for future epidemics/pandemics means continuously monitoring and improving AI models’ performance to adapt to a continually changing environment. AI-based early warning systems must be adaptive and constantly refined.

Study limitations

This is the first scoping review to address AI-based early warning systems; however, this it has several limitations. One key limitation is that the search was limited to the English language; there could be studies in other languages that were missed.

Also, scoping reviews are designed to map existing literature and identify gaps rather than provide detailed synthesis or meta-analysis. Consequently, the depth of analysis might be limited, and nuanced insights into specific factors influencing AI model performance might be underexplored.

Another significant limitation is the heterogeneity of the included studies. The diversity in study designs, AI models, evaluation metrics, and disease focus areas makes it difficult to draw uniform conclusions. This variability can obscure specific insights about the strengths and weaknesses of AI applications in particular settings or for specific diseases.

The review also faces temporal constraints due to the rapid pace of technological advancement in AI and machine learning. There may be newer models might surpass those reviewed. This limitation means that the review might not fully capture the latest best practices in AI-based outbreak detection.

Additionally, the review might not adequately address implementation challenges of AI models in real-world settings. While offline validation results are often promising, practical hurdles such as data quality issues and integration with existing public health infrastructure are not always thoroughly examined in the different studies.

Conclusion

The evidence reviewed suggests that AI can be used for pandemic preparedness and designing and developing early warning systems for diseases, epidemics, and pandemic outbreaks. However, while advanced models like deep learning and ensemble methods offer high accuracy, they also come with challenges related to interpretability and resource requirements. The choice of model should balance these factors, considering the specific context and available resources. AI-based EWS systems face data volume, velocity, variety, availability, and granularity challenges. When implementing an AI-based early warning system, there is a need to address several challenges, including data quality, model explainability, bias measurement and mitigation, sustainable development goals, use of social data, system adaptability, and continuous monitoring and improvement.

Footnotes

Acknowledgements

The authors express their sincere gratitude to colleagues within the Global South AI for Pandemic and Epidemic Preparedness & Response Network (AI4PEP), who exemplify an equitable, inclusive, innovative, and supportive environment, which has been instrumental in fostering this research. The authors are also deeply grateful for the vibrant intellectual community within York University, the Beirut Arab University, and the Lebanese Hospital-Geitaoui University Medical Center, which inspired this research and enriched our experience as scholars.

Authors’ contributions

CE, AS, YE, and ES designed the study and received the funds. CE supervised DO and YA, who performed and reported the summaries. CE verified the analysis and prepared the first draft. All authors provided critical feedback and revised it; they all approved the final version and agreed to be accountable for all aspects of the submitted paper.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research is funded by Canada’s International Development Research Centre (IDRC) (Grant No. 10998).

Ethical statement

ORCID iDs

Christo El Morr

Deniz Ozdemir

Antoine Saab

Search strategies

References

Godshall

Banach

. Pandemic preparedness. Infect Dis Clin North Am 2021; 35(4): 1077–1089.

Majumder

Santillana

Mekaru

, et al. Utilizing nontraditional data sources for near real-time estimation of transmission dynamics during the 2015-2016 Colombian Zika virus disease outbreak. JMIR Public Health Surveill 2016; 2(1): e30.

El Morr

Jammal

Ali-Hassan

, et al. Machine learning for practical decision making: a multidisciplinary perspective with applications from healthcare, engineering and business analytics [Internet]. Cham, Switzerland: Springer International Publishing, 2022, p. 250. (Price CC, editor. International Series in Operations Research & Management Science). https://link.springer.com/book/9783031169892

El Morr

Maret

Muhlenbach

, et al. A virtual community for disability advocacy: development of a searchable artificial intelligence–supported platform. JMIR Form Res 2021; 5(11): e33335. DOI: 10.2196/33335. https://formative.jmir.org/2021/11/e33335

Gurevich

El Hassan

El Morr

. Equity within AI systems: what can health leaders expect? Healthc Manag Forum 2023; 36(2): 119–124. DOI: 10.1177/08404704221125368, [cited 2023 Feb 27].

Jia

Mei

, et al. Generalizability and diagnostic performance of AI models for thyroid US. Radiology 2023; 307(5): e221157.

Elliott

Soifer

. AI technologies, privacy, and security. Front Artif Intell 2022; 5: 826737.

El Morr

(ed). AI and Society: Tensions and Opportunities. New York, NY: CRC Press Taylor & Francis, 2022. (Yampolskiy RV, editor. Chapman & Hall/CRC Artificial Intelligence and Robotics Series).

Ajagbe

Adigun

. Deep learning techniques for detection and prediction of pandemic diseases: a systematic literature review. Multimed Tool Appl 2023; 88: 5893–5927, DOI: 10.1007/s11042-023-15805-z.

10.

MacIntyre

Chen

Kunasekaran

, et al. Artificial intelligence in public health: the potential of epidemic early warning systems. J Int Med Res 2023; 51(3): 3000605231159335.

11.

Islam

Inan

Rafi

, et al. A systematic review on the use of AI and ML for fighting the COVID-19 pandemic. IEEE Trans Artif Intell 2020; 1(3): 258–270.

12.

Meckawy

Stuckler

Mehta

, et al. Effectiveness of early warning systems in the detection of infectious diseases outbreaks: a systematic review. BMC Publ Health 2022; 22(1): 2216, DOI: 10.1186/s12889-022-14625-4.

13.

Peters

MDJ

Godfrey

Khalil

, et al. Guidance for conducting systematic scoping reviews. Int J Evid Based Healthc 2015; 13(3): 141–146, https://journals.lww.com/ijebh/fulltext/2015/09000/guidance_for_conducting_systematic_scoping_reviews.5.aspx

14.

Tricco

Lillie

Zarin

, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med 2018; 169(7): 467–473. [cited 2023 Dec 8]. DOI: 10.7326/M18-0850.

15.

Ouzzani

Hammady

Fedorowicz

, et al. Rayyan—a web and mobile app for systematic reviews. Syst Rev 2016; 5(1): 210, DOI: 10.1186/s13643-016-0384-4.

16.

Benedum

Shea

Jenkins

, et al. Weekly dengue forecasts in Iquitos, Peru; San Juan, Puerto Rico; and Singapore. PLoS Negl Trop Dis 2020; 14(10): e0008710. [cited 10AD Jan 1]. Available from: ://WOS:000581741700001.

17.

Valter

Oliveira

Vitorino

, et al. Intelligent epidemiological surveillance in the Brazilian semiarid. 2020 IEEE International Conference on E-health Networking, Application & Services (HEALTHCOM); 2021. IEEE.

18.

Aleixo

Kon

Rocha

, et al. Predicting dengue outbreaks with explainable machine learning. 2022 22nd IEEE International Symposium on Cluster, Cloud and Internet Computing (CCGrid); 2022 May 16-19, Taormina, Italy. IEEE.

19.

McGough

Clemente

Kutz

, et al. A dynamic, ensemble learning approach to forecast dengue fever epidemic years in Brazil using weather and population susceptibility cycles. J R Soc Interface 2021; 18(179): 20201006. [cited 6AD Jan 1]. Available from: ://WOS:000664864200001.

20.

Panja

Chakraborty

Nadim

, et al. An ensemble neural network approach to forecast Dengue outbreak based on climatic condition. Chaos Solitons Fractals [Internet] 2023; 167: 113124. [cited 2AD Jan 1]. Available from: ://WOS:000977828500001.

21.

Bhunia

Abirami

. Correlation based feature selection and hybrid machine learning approach for forecasting disease outbreaks. 2023 4th International Conference on Electronics and Sustainable Communication Systems (ICESC); 2023 July 6-8, Coimbatore, India. IEEE.

22.

Patil

Pandya

. Forecasting dengue hotspots associated with variation in meteorological parameters using regression and time series models. Front Public Health 2021; 9: 798034.

23.

Manogaran

Lopez

. A Gaussian process based big data processing framework in cluster computing environment. Cluster Comput 2018; 21(1): 189–204. [cited 2023 Sep 7]. https://link.springer.com/10.1007/s10586-017-0982-5

24.

Ceballos-Arroyo

Maldonado-Perez

Mesa-Yepes

, et al. Towards a machine learning-based approach to forecasting Dengue virus outbreaks in Colombian cities. A case-study: medellin, Antioquia. In Proceedings of SPIE, Medellin, Colombia, Nov 6-8; 2019. vol. 11330. Available from: ://WOS:000542283100041

25.

Ismail

Fildes

Ahmad

, et al. The practicality of Malaysia dengue outbreak forecasting model as an early warning system. Infect Dis Model 2022; 7(3): 510–525, Available from: ://WOS:000847839600001.

26.

, et al. Forecast of dengue cases in 20 Chinese cities based on the deep learning method. Int J Environ Res Publ Health 2020; 17(2).

27.

Nguyen

Tuyet-Hanh

Mulhall

, et al. Deep learning models for forecasting dengue fever based on climate data in Vietnam. PLoS Negl Trop Dis 2022; 16(6): e0010509.

28.

Mishra

Tiwari

Ajaymon

, et al. Dengue disease spread prediction using twofold linear regression. In: International Conference on Advanced Computing, Tiruchirappalli, India, 2019 Dec 13-14. IEEE.

29.

Malki

Atlam

Ewis

, et al. ARIMA models for predicting the end of COVID-19 pandemic and the risk of second rebound. Neural Comput Appl 2021; 33(7): 2929–2948. [cited 4AD Jan 1]. Available from: ://WOS:000582098100004.

30.

Bellocchio

Carioni

Lonati

, et al. Enhanced sentinel surveillance system for COVID-19 outbreak prediction in a large european dialysis clinics network. Int J Environ Res Public Health [Internet] 2021; 18(18). Available from: ://WOS:000700778100001.

31.

Fong

Dey

, et al. Finding an accurate early forecasting model from small dataset: a case of 2019-nCoV novel Coronavirus outbreak. Int J Interact Multimed Artif Intell [Internet] 2020; 6(1): 132–140. Available from: ://WOS:000514841200016.

32.

Setty

Thabitha

Prasad

PPS

, et al. Prediction of Corona virus outbreak using machine learning. 2021 IEEE International Conference on Intelligent Systems, Smart and Green Technologies (ICISSGT); 2021 Nov 13-14, Visakhapatnam, India. IEEE.

33.

Mirri

Delnevo

Roccetti

. Is a COVID-19 second wave possible in Emilia-Romagna (Italy)? Forecasting a future outbreak with particulate pollution and machine learning. Computation 2020; 8(3): 74. [cited 9AD Jan 1]. Available from: ://WOS:000578131200001.

34.

Srivastava

Tripathi

Kumar

, et al. Machine learning approach for forecast analysis of novel COVID-19 scenarios in India. IEEE Access 2022; 10: 95106–95124, Available from: ://WOS:000854568000001.

35.

Singh

Mittal

. Pandemic outbreak prediction using optimization-based machine learning model. 2023 3rd International Conference on Advances in Computing, Communication, Embedded and Secure Systems (ACCESS); 2023 May 19-20, Ernakulam, India. IEEE.

36.

Prakash

Pathak

Jalal

. Predicting COVID-19 fourth wave incidence in India using machine learning algorithms and SEIR model. IEEE 9th Uttar Pradesh Section International Conference on Electrical, Electronics and Computer Engineering (UPCON); 2022 Dec 2-4, Prayagraj, India.

37.

Kumar

Gupta

. Corona virus outbreak prediction using machine learning. 2022. p. 632–635.

38.

, et al. Forecasting influenza activity using self-adaptive AI model and multi-source data in Chongqing, China. EBioMedicine 2019; 47: 284–292.

39.

Didi

Walha

Wali

. Data-driven model for influenza prediction incorporating environmental effects, 2020. pp. 15–24. Prague, CZECH REPUBLIC. Available from: ://WOS:000615960700001.

40.

Yang

, et al. Deep-learning model for influenza prediction from multisource heterogeneous data in a megacity: model development and evaluation. J Med Internet Res 2023; 25: e44238.

41.

Shaghaghi

Calle

Kouretas

, et al. Expanding eVision’s scope of influenza forecasting. In IEEE Global Humanitarian Technology Conference Proceedings, 2020 Oct 29 - Nov 1, Online: IEEE.

42.

Masinde

. Africa’s malaria epidemic predictor: application of machine learning on malaria incidence and climate data. Association for Computing Machinery 2020: 29–37. DOI: 10.1145/3388142.3388158.

43.

Chekol

Hagras

. Employing machine learning techniques for the malaria epidemic prediction in Ethiopia. 2018 10th Computer Science and Electronic Engineering (CEEC); 2018 Sep 19-21, Colchester, UK.

44.

Harvey

Valkenburg

Amara

. Predicting malaria epidemics in Burkina Faso with machine learning. PLoS One 2021; 16(6): e0253302.

45.

Campbell

Racault

Goult

, et al. Cholera risk: a machine learning approach applied to essential climate variables. Int J Environ Res Publ Health 2020; 17(24): 9378, Available from: ://WOS:000602820400001.

46.

Althomsons

Winglee

Heilig

, et al. Using machine learning techniques and national tuberculosis surveillance data to predict excess growth in genotyped tuberculosis clusters. Am J Epidemiol [Internet]. 2022;191(11):1936–1943. Available from: ://WOS:000840813700001

47.

Shinde

Yadav

Somvanshi

. Epidemic outbreak prediction using machine learning model. 5th International Conference on Advances in Science and Technology (ICAST); 2022 Dec 2-3, Mumbai, India. IEEE.

48.

Panja

Chakraborty

Kumar

, et al. Epicasting: an ensemble wavelet neural network for forecasting epidemics. Neural Netw 2023; 165: 185–212. [cited 8AD Jan 1]. Available from: ://WOS:001024500400001.

49.

Didi

Walha

Wali

. Data-driven model for influenza prediction incorporating environmental effects. 5th International Conference on Internet of Things, Big Data and Security - IoTBDS; 2020 May 7-9. Online: SciTePress.

50.

Campbell

Racault

Goult

, et al. Cholera risk: a machine learning approach applied to essential climate variables. Int J Environ Res Publ Health 2020; 17(24): 9378. [cited 12AD Jan 1]. Available from: ://WOS:000602820400001.

51.

Ismail

Fildes

Ahmad

, et al. The practicality of Malaysia dengue outbreak forecasting model as an early warning system. Infect Dis Model 2022; 7(3): 510–525. DOI: 10.1016/j.idm.2022.07.008

52.

Harvey

Valkenburg

Amara

. Predicting malaria epidemics in Burkina Faso with machine learning. PLoS One 2021; 16(6): e0253302.

53.

Kane

Price

Scotch

, et al. Comparison of ARIMA and Random Forest time series models for prediction of avian influenza H5N1 outbreaks. BMC Bioinf 2014; 15(1): 276. DOI: 10.1186/1471-2105-15-276.

54.

Khanijahani

Iezadi

Gholipour

, et al. A systematic review of racial/ethnic and socioeconomic disparities in COVID-19. Int J Equity Health 2021; 20(1): 248.

55.

Kong

Akpudo

Effoduh

, et al. Leveraging responsible, explainable, and local artificial intelligence solutions for clinical public health in the Global South. Healthc Basel Switz 2023; 11(4): 457.

56.

United Nations . Fast facts – what is sustainable development? [Internet]. New York, NY: United Nations, 2023. https://www.un.org/sustainabledevelopment/

57.

Nelson

Iannaccone

. Social media and clinical trials in pediatric disease: musings from the bench. Front Genet 2021; 12: 734691.

58.

Aiello

Renson

Zivich

. Social media- and internet-based disease surveillance for public health. Annu Rev Publ Health 2020; 41: 101–118.

59.

, et al. Forecasting influenza activity using self-adaptive AI model and multi-source data in Chongqing, China. EBioMedicine 2019; 47: 284–292, 20190830th ed.

AI-based epidemic and pandemic early warning systems: A systematic scoping review

Abstract

Keywords

Introduction

Methods

Search strategy

Eligibility criteria

Selection process

Results

Early warning systems description: Countries, purpose, algorithms, and data characteristics

Countries

Purpose

Algorithms

Dataset characteristics

AI techniques: Types, effectiveness and performance

Types of AI techniques

Effectiveness of the AI techniques

Performance of the AI techniques

System challenges

Model’s biases

Generalizability

False positives

The complexity of societal and political dimensions

Data challenges

Data volume

Data velocity

Data variety

Lack of availability

Lack of granularity

Discussion

Type of AI early warning systems

AI effectiveness in detecting outbreaks

Strategies implemented to mitigate the risks of AI Bias

Challenges and future perspectives

Model explainability

SDG consideration

Going beyond health and climate: Use of social data

Adaptability and continuous model monitoring and improvement

Study limitations

Conclusion

Footnotes

Acknowledgements

Authors’ contributions

Declaration of conflicting interests

Funding

Ethical statement

ORCID iDs

Search strategies

References