Simulation-based metamodels or surrogate models are simplified models that capture the relationship between inputs and outputs of the simulation model. The analytical expression of metamodels is defined using a sample of input/output points obtained from the simulation model. This analytical expression is then embedded in an optimization process that usually provides solutions much faster than other simulation-based optimization techniques such as metaheuristics or mathematical modeling. The goal of this paper is to describe the simulation-based metamodeling approach and to provide a thorough review of the literature on its applications to emergency healthcare systems. For this purpose, we examine the recent literature (journals and conference proceedings) published in the last 15 years (2008–2022). Finally, we identify findings and avenues of research in simulation-based metamodeling that deserve special attention from the scientific community and allow the potential of this approach to be used for better decision-making in emergency healthcare.
AringhieriRBruniMEKhodaparastiS, et al. Emergency medical services and beyond: addressing new challenges through a wide literature review. Comput Oper Res2017; 78: 349–368.
2.
CimellaroGPReinhornAMBruneauM. Performance-based metamodel for healthcare facilities. Earthq Eng Struct Dyn2011; 40: 1197–1217.
3.
AboueljinaneLSahinEJemaiZ. A review on simulation models applied to emergency medical service operations. Comput Ind Eng2013; 66: 734–750.
4.
ShoaibMRamamohanV. Simulation modelling and analysis of primary health centre operations. Simulation2022; 98: 183–208.
5.
VanbrabantLBraekersKRamaekersK, et al. Simulation of emergency department operations: a comprehensive review of KPIs and operational improvements. Comput Ind Eng2019; 131: 356–381.
6.
DefraeyeMVan NieuwenhuyseI. Staffing and scheduling under nonstationary demand for service: a literature review. Omega2015; 58: 4–25.
7.
AntunesFAmorimMPereiraFC, et al. Active learning metamodeling for policy analysis: application to an emergency medical service simulator. Simul Model Pract Theory2019; 97: 101947.
8.
ZeinaliFMahootchiMSepehriMM. Resource planning in the emergency departments: a simulation-based metamodeling approach. Simul Model Pract Theory2015; 53: 123–138.
9.
XuJHuangEChenCH, et al. Simulation optimization: a review and exploration in the new era of cloud computing and big data. Asia-Pacific J Oper Res2015; 32: 1–34.
10.
BanksJCarsonJSNelsonBL, et al. Discrete event system simulation solutions manual. London: Pearson Education, 2010, p. 639.
11.
Soares do AmaralJVMontevechiJABMirandaR, et al. Metamodel-based simulation optimization: a systematic literature review. Simul Model Pract Theory2022; 114: 102403.
12.
YousefiMYousefiMFogliattoFS. Simulation-based optimization methods applied in hospital emergency departments: a systematic review. Simulation2020; 96: 791–806.
13.
PatiDLorussoLN. How to write a systematic review of the literature. Heal Environ Res Des J2017; 11: 15–30.
14.
WangGGShanS. Review of metamodeling techniques in support of engineering design optimization. J Mech Des Trans ASME2007; 129: 370–380.
15.
BartonRR. Simulation optimization using metamodels. In: Proceeding of the winter simulation conference (WSC), Austin, TX, 13–16 December 2009, pp. 230–238. New York: IEEE.
16.
BartonRR. Tutorial: metamodeling for simulation. In: Proceedings of the winter simulation conference (WSC), Orlando, FL, 14–18 December 2020, pp. 1102–1116. New York: IEEE.
KleijnenJPCSargentRG. A methodology for fitting and validating metamodels in simulation. Eur J Oper Res2000; 120: 14–29.
19.
YuBPopplewellK. Metamodels in manufacturing: a review. Int J Prod Res1994; 32: 787–796.
20.
BartonRR. Simulation experiment design. In: Proceeding of the winter simulation conference (WSC), Baltimore, MD, 5–8 December 2010, pp. 75–86. New York: IEEE.
21.
BartonRR. Tutorial: simulation metamodeling. In: Proceedings of the winter simulation conference (WSC), Huntington Beach, CA, 6–9 December 2015, pp. 1765–1779. New York: IEEE.
22.
YousefiMYousefiM. Human resource allocation in an emergency department: a metamodel-based simulation optimization. Kybernetes2020; 49: 779–796.
23.
KangHLoboJM. Identifying the optimal configuration of an express care area in an emergency department: a des and metamodeling approach. In: Proceedings of the winter simulation conference (WSC), Washington, DC, 11–14 December 2016, pp. 1961–1969. New York: IEEE.
24.
FatmaNMohdSRamamohanV, et al. Primary healthcare delivery network simulation using stochastic metamodels. In: Proceeding of the winter simulation conference (WSC), Orlando, FL, 14–18 December 2020, pp. 818–829. New York: IEEE.
25.
CoelhoGFPintoLR. Kriging-based simulation optimization: an emergency medical system application. J Oper Res Soc2018; 69: 2006–2020.
26.
ShoaibMRamamohanV. Simulation modelling and analysis of community health centre operations. In: IIE annual conference, Anaheim, CA, 2021, pp. 447–452.
27.
SilvaPMSPintoLR. Emergency medical systems analysis by simulation and optimization. In: Proceedings of the 2010 winter simulation conference, Baltimore, MD, 5–8 December 2010, pp. 2422–2432. New York: IEEE.
28.
KleijnenJPC. Design and analysis of simulation experiments. 2nd ed.Springer International Publishing; 2018. DOI: 10.1007/978-3-319-76035-3_1.
29.
WoodsDCLewisSM. Design of experiments for screening. In: GhanemRHigdonDOwhadiH (eds) Handbook of uncertainty quantification. New York: Springer International, 2017, pp. 1143–1185.
30.
ShannonRE. Introduction to simulation. In: Proceedings of the 24th conference on Winter simulation - WSC '92 New York, New York, USA, 1 December 1992. New York: ACM Press, 1992, pp. 65–73.
31.
AlamFMMcNaughtKRRingroseTJ. A comparison of experimental designs in the development of a neural network simulation metamodel. Simul Model Pract Theory2004; 12: 559–578.
32.
VuKKD’AmbrosioCHamadiY, et al. Surrogate-based methods for black-box optimization. Int Trans Oper Res2017; 24: 393–424.
33.
SanchezSMSanchezPJWanH. Work smarter, not harder: a tutorial on designing and conducting simulation experiments. In: Proceeding of winter the simulation conference (WSC), Huntington Beach, CA, 6–9 December 2018, pp. 237–251. New York: IEEE.
34.
LiuHCaiJOngYS. Remarks on multi-output Gaussian process regression. Knowledge Based Syst2018; 144: 102–121.
35.
SongXGJungJHSonHJ, et al. Metamodel-based optimization of a control arm considering strength and durability performance. Comput Math Appl2010; 60: 976–980.
36.
ShinMSargentRGGoelAL. Gaussian radial basis functions for simulation metamodeling. In: Proceedings of the winter simulation conference, San Diego, CA, 8–11 December 2002, pp. 483–488. New York: IEEE.
37.
LiYFNgSHXieM, et al. A systematic comparison of metamodeling techniques for simulation optimization in Decision Support Systems. Appl Soft Comput J2010; 10: 1257–1273.
38.
BrownleeJ. Master machine learning algorithms: discover how they work and implement them from scratch. 5th ed.San Juan: Machine Learning Mastery, 2016, http://machinelearningmastery.com
39.
BilesWEBeersWCMVan KleijnenJPC, et al. Kriging metamodeling in constrained simulation optimization: an explorative study. In: Proceeding of the winter simulation conference (WSC), Washington, DC, 9–12 December 2007, pp. 355–362. New York: IEEE.
40.
WangL. A hybrid genetic algorithm-neural network strategy for simulation optimization. Appl Math Comput2005; 170: 1329–1343.
41.
JinRChenWSimpsonTW. Comparative studies of metamodeling techniques under multiple modeling criteria. In: 8th symposium on multidisciplinary analysis and optimization, Long Beach, CA, 6 September 2000. Reston, VA: Aeronautics and Astronautics, 2000, pp. 1–13.
42.
KimB-SLeeY-BChoiD-H. Comparison study on the accuracy of metamodeling technique for non-convex functions. J Mech Sci Technol2009; 23: 1175–1181.
43.
KianifarMRCampeanF. Performance evaluation of metamodelling methods for engineering problems: towards a practitioner guide. Struct Multidiscip Optim2020; 61: 159–186.
44.
ParnianifardAAzfanizamAAriffinM, et al. Comparative study of metamodeling and sampling design for expensive and semi-expensive simulation models under uncertainty. Simulation2020; 96: 89–110.
45.
MüllerJShoemakerCA. Influence of ensemble surrogate models and sampling strategy on the solution quality of algorithms for computationally expensive black-box global optimization problems. J Glob Optim2014; 60: 123–144.
46.
de Sousa JuniorWTMontevechiJABMirandaR, et al. Shop floor simulation optimization using machine learning to improve parallel metaheuristics. Expert Syst Appl2020; 150: 113272.
47.
MeckesheimerMBartonRRSimpsonTW, et al. Computationally inexpensive metamodel assessment strategies. In: ASME 2001 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, 9 September 2001. American Society of Mechanical Engineers, 2002, pp. 191–201.
48.
YousefiMYousefiMFerreiraRPM, et al. Chaotic genetic algorithm and Adaboost ensemble metamodeling approach for optimum resource planning in emergency departments. Artif Intell Med2018; 84: 23–33.
49.
FriedmanLWFriedmanHH. Validating the simulation metamodel: some practical approaches. Simulation1985; 45: 144–146.
50.
BartonRR. Tutorial: simulation metamodeling. In: Proceedings of the winter simulation conference (WSC), Huntington Beach, CA, 6–9 December 2015, pp. 779–790. New York: IEEE.
51.
WangXGongXGengN, et al. Metamodel-based simulation optimisation for bed allocation. Int J Prod Res2020; 58: 6315–6335.
52.
FigueiraGAlmada-LoboB. Hybrid simulation-optimization methods: a taxonomy and discussion. Simul Model Pract Theory2014; 46: 118–134.
53.
LeeS. The role of hospital selection in ambulance logistics. IIE Trans Healthc Syst Eng2014; 4: 105–117.
54.
AmorimMAntunesFFerreiraS, et al. An integrated approach for strategic and tactical decisions for the emergency medical service: exploring optimization and metamodel-based simulation for vehicle location. Comput Ind Eng2019; 137: 106057.
55.
SnyderMSmuckerBJ. Metamodel optimization of a complex, rural–urban emergency medical services system. Simul Model Pract Theory2022; 118: 102526.
56.
AntunesFAmorimMPereiraF, et al. Active learning metamodelling for survival rate analysis of simulated emergency medical systems. Transp A Transp Sci. Epub ahead of print 25March2022. DOI: 10.1080/23249935.2022.2046203.
57.
LiPChenS. A review on Gaussian process latent variable models. CAAI Trans Intell Technol2016; 1: 366–376.
58.
PichenyVGinsbourgerDRichetY, et al. Quantile-based optimization of noisy computer experiments with tunable precision. Technometrics2013; 55: 2–13.
59.
GolabianHArkatJFarughiH, et al. A simulation-optimization algorithm for return strategies in emergency medical systems. Simulation2021; 97: 565–588.
60.
AboueljinaneLSahinEJemaiZ. A discrete simulation-based optimization approach for multi-period redeployment in emergency medical services. Simulation2023; 99: 659–679.
61.
AboueljinaneLFrichiY. A simulation optimization approach to investigate resource planning and coordination mechanisms in emergency systems. Simul Model Pract Theory2022; 119: 102586.
62.
KokangulA. A combination of deterministic and stochastic approaches to optimize bed capacity in a hospital unit. Comput Methods Programs Biomed2008; 90: 56–65.
63.
YiPGeorgeSKPaulJA, et al. Hospital capacity planning for disaster emergency management. Socioecon Plann Sci2010; 44: 151–160.
64.
KaptanK. An organizational metamodel for hospital emergency departments. Disaster Med Public Health Prep2014; 8: 436–444.
65.
PiquéMCimellaroG. Organizational model of an emergency department under seismic event. In: Second European conference on earthquake engineering and seismology, Istanbul, Turkey, 25–29 August 2014, pp. 1–3.
66.
CimellaroGPMalavisiMMahinS. Using discrete event simulation models to evaluate resilience of an emergency department. J Earthq Eng2017; 21: 203–226.
67.
CimellaroGPMarascoSNooriAZ, et al. A first order evaluation of the capacity of a healthcare network under emergency. Earthq Eng Eng Vib2019; 18: 663–677.
68.
MaleyeffJCorluCGWangX. Simulation metamodeling to support hospital capacity planning. In: Proceedings of the winter simulation conference (WSC), Orlando, FL, 14–18 December 2020, pp. 1–9.
69.
AminShokraviAHeraviG. Developing the framework for evaluation of the inherent static resilience of the access to care network. J Clean Prod2020; 267: 122123.
70.
AmaralTMCostaAPC. Improving decision-making and management of hospital resources: an application of the PROMETHEE II method in an emergency department. Oper Res Heal Care2014; 3: 1–6.
71.
MüllerJPichéR. Mixture surrogate models based on Dempster-Shafer theory for global optimization problems. J Glob Optim2011; 51: 79–104.
72.
VapnikVN. Statistics for engineering and information science. 2nd ed.New York: Springer International Publishing, 2000.
73.
BilinskiA. Machine learning to improve meta-modelling and sensitivity analysis: an HIV case study. In: Society for medical decision making 40th annual meeting, Montréal, QC, Canada, 14–16 October 2018, pp. 152–153.
74.
XuLZhaoJLiC, et al. Simulation and prediction of hydrological processes based on firefly algorithm with deep learning and support vector for regression. Int J Parallel2020; 35: 288–296.
75.
HighamCFHighamDJ. Deep learning: an introduction for applied mathematicians. SIAM Rev2019; 61: 860–891.
76.
CollierNNorthM. Parallel agent-based simulation with repast for high performance computing. Simulation2013; 89: 1215–1235.
77.
SaviniecLSantosMOCostaAM. Parallel local search algorithms for high school timetabling problems. Eur J Oper Res2018; 265: 81–98.
78.
CimellaroGPReinhornABruneauM. Quantification of seismic resilience of health care facilities. Buffalo, NY: University at Buffalo.
79.
YinJNgSHNgKM. Kriging metamodel with modified nugget-effect: the heteroscedastic variance case. Comput Ind Eng2011; 61: 760–777.
80.
BlasMJGonnetS. Metamodel-based formalization of DEVS atomic models. Simulation2023; 99: 433–454.
81.
VanbrabantLMartinNRamaekersK, et al. Quality of input data in emergency department simulations: framework and assessment techniques. Simul Model Pract Theory2019; 91: 83–101.
82.
ForresterAIJKeaneAJBressloffNW. Design and analysis of “noisy” computer experiments. AIAA J2006; 44: 2331–2339.
83.
ChantaSMayorgaMEKurzME, et al. The minimum p-envy location problem: a new model for equitable distribution of emergency resources. IIE Trans Healthc Syst Eng2011; 1: 101–115.
84.
DreznerTDreznerZGuyseJ. Equitable service by a facility: minimizing the Gini coefficient. Comput Oper Res2009; 36: 3240–3246.
85.
FragnelliVGagliardoS. Open problems in cooperative location games. Int Game Theory Rev2013; 15: 1–13.
86.
BaşarAÇatayBÜnlüyurtT. A taxonomy for emergency service station location problem. Optim Lett2012; 6: 1147–1160.
87.
AfferniPMeroneMSodaP. Hospital 4.0 and its innovation in methodologies and technologies. Proc IEEE Symp Comput Med Syst2018; 2018: 333–338.
88.
JayaramanPPForkanARMMorshedA, et al. Healthcare 4.0: a review of frontiers in digital health. Wiley Interdiscip Rev Data Min Knowl Discov2020; 10: 1–23.
89.
GuptaSModgilSBhattacharyyaS, et al. Artificial intelligence for decision support systems in the field of operations research: review and future scope of research. Ann Oper Res2022; 308: 215–274.
90.
TalaveraALunaA. Machine learning: a contribution to operational research. Rev Iberoam Tecnol Aprendiz2020; 15: 70–75.
