Building simulation models that are valid and credible is an enduring challenge in the Australian Defence Organisation (ADO) context. Establishing validity and credibility can be achieved through the rigorous use of appropriate Verification, Validation, and Accreditation (VVA) processes. Such processes are well-known in modeling and simulation (M&S) practice. However, these are generally not applied within the ADO, typically due to resourcing concerns and a lack of authoritative guidance. Even if there are any, due to security concerns and commercial reasons, the application of M&S within ADO is generally not published in open-access platforms. Depending on where in the M&S life-cycle VVA is started, it may also serve a secondary aim of risk reduction, assisting in the early discovery of possible problems or mistakes. This research reviews current VVA practices from academic literature and recommends processes that are appropriate for application to combat simulation tools within the ADO context. A scoping review has been conducted to gather insight into current VVA practice in the M&S community. The results of this review are presented in the form of charting relevant characteristics from selected references. The scoping review shows that executable validation of simulation results against referent data sourced from physical experiments is the most prevalent form of VVA, with referent data from comparative models being a prevalent alternative. Furthermore, there is evident reliance on graphical comparison of data; this could be enhanced with objective data comparators, such as aggregate error measures or statistical techniques. Finally, there is an evident gap in VVA references from Australia, which could be addressed through the propagation and reporting of prevalent VVA practices within the ADO context.
BalciO. Verification, validation, and testing. In: BanksJ (ed.) Handbook of simulation: principles, methodology, advances, applications, and practice. New York: John Wiley & Sons, 1998, pp. 335–393.
2.
KangKRolandRJ. Chapter 19: military simulation. In: BanksJ (ed.) Handbook of simulation: principles, methodology, advances, applications, and practice. New York: John Wiley & Sons, 1998, pp. 645–658.
3.
PettyMD. Chapter 6: verification and validation. In: SokolowskiJABanksCM (eds) Principles of modeling and simulation: a multidisciplinary approach. Hoboken, NJ: John Wiley & Sons, 2009, pp. 121–149.
4.
PettyMD. Verification, validation, and accreditation. In: SokolowskiJABanksCM (eds) Modeling and simulation fundamentals. Hoboken, NJ: John Wiley & Sons, 2010, pp. 325–372.
5.
SargentRG. Verification and validation of simulation models. In: Proceedings of the 2010 winter simulation conference (ed JohanssonBJainSMontoya-TorresJ, et al.), Baltimore, MD, 5–8 December 2010. New York: IEEE.
6.
KleijnenJPC. Verification and validation of simulation models. Eur J Oper Res1995; 82: 145–162.
7.
Defense Modeling and Simulation Enterprise (DMSE). Modeling and simulation enterprise (MSE) verification, validation, & accreditation (VV&A) reference site. DMSE, https://vva.msco.mil/
8.
Simulation Interoperability Standards Organisation (SISO). Reference for generic methodology for verification and validation (GM-VV) to support acceptance of models, simulations and data (GM-VV volume 3: reference manual). Orlando, FL: SISO, 2013.
9.
LawAM. Simulation modeling and analysis (McGraw-Hill series in industrial engineering and management science). 5th ed.New York: McGraw-Hill, 2015.
10.
The Technical Cooperation Program (TTCP). Guide for understanding and implementing defense experimentation (GUIDEx). Ottawa, ON, Canada: Canadian Forces Experimentation Centre, 2006.
BowdenFDJCouttsAWDexterRM, et al. Impacts of unmanned ground vehicles on combined arms team performance. In: CaresJRDickmannJQ (eds) Operations research for unmanned systems. Hoboken, NJ: John Wiley & Sons, 2016, pp. 119–138.
13.
Simulation Interoperability Standards Organisation (SISO). Guide for generic methodology for verification and validation (GM-VV) to support acceptance of models, simulations, and data (GM-VV volume 1: introduction and overview). Orlando, FL: SISO, 2012.
14.
Simulation Interoperability Standards Organisation (SISO). Guide for generic methodology for verification and validation (GM-VV) to support acceptance of models, simulations, and data (GM-VV volume 2: implementation guide). Orlando, FL: SISO, 2013.
15.
MIL-STD-3022:2012. Documentation of verification, validation, and accreditation (VV&A) for models and simulations.
16.
XiaoYWatsonM. Guidance on conducting a systematic literature review. J Plan Educ Res2019; 39: 93–112.
17.
ArkseyHO’MalleyL. Scoping studies: towards a methodological framework. Int J Soc Res Method2005; 8: 19–32.
18.
DavisKDreyNGouldD. What are scoping studies? A review of the nursing literature. Int J Nurs Stud2009; 46: 1386–1400.
19.
McDonaldJH. Handbook of biological statistics. Baltimore, MD: Sparky House Publishing, 2009.
20.
McHughML. The Chi-square test of independence. Biochem Med2013; 23: 143–149.
21.
GoemanJJSolariA. Multiple hypothesis testing in genomics. Stat Med2014; 33: 1946–1978.
22.
VenkataKATrumanCEWimporyRC, et al. Numerical simulation of a three-pass TIG welding using finite element method with validation from measurements. Int J Pres Ves Pip2018; 164: 68–79.
23.
AdamsCSchenkerJWestonP, et al. Path meander of male codling moths (Cydia pomonella) foraging for sex pheromone plumes: field validation of a novel method for quantifying path meander of random movers developed using computer simulations. Insects2020; 11: 549.
24.
Al ZainJEl HajjajiOEl BardouniT, et al. Validation of DRAGON4/DONJON4 simulation methodology for a typical MNSR by calculating reactivity feedback coefficient and neutron flux. Results Phys2018; 9: 1155–1160.
25.
BarcziTKohoutMKozakovicM, et al. Discrete element method simulation and experimental validation of pattern development in a rotating drum mixer. Chem Eng Technol2018; 41: 1524–1530.
26.
BarrettTJKnezevicM. Deep drawing simulations using the finite element method embedding a multi-level crystal plasticity constitutive law: experimental verification and sensitivity analysis. Comput Method Appl M2019; 354: 245–270.
27.
BarriosGRojasJHuelszG, et al. Implementation of the equivalent-homogeneous-layers-set method in whole-building simulations: experimental validation. Appl Therm Eng2017; 125: 35–40.
28.
BasagliaTBegalliMChoiC, et al. Methods, techniques and recent results in Monte Carlo simulation validation for sensitive applications. In: Proceedings of the 2015 IEEE nuclear science symposium and medical imaging conference (NSS/MIC), San Diego, CA, 31 October–7 November 2015. New York: IEEE.
29.
BattiatoADiserensE. Tractor traction performance simulation on differently textured soils and validation: a basic study to make traction and energy requirements accessible to the practice. Soil Till Res2017; 166: 18–32.
30.
BinderTPfeifferMFasoulasS. Validation of grid current simulations using the particle-in-cell method for a miniaturized ion thruster. In: Proceedings of the 31st international symposium on rarefied gas dynamics (ed ZhangYEmersonDRLockerbyD, et al.), Glasgow, 23–27 July 2018, vol. 2132. College Park, MD: American Institute of Physics (AIP).
31.
BrembillaCÖstinROlofssonT. Predictions’ robustness of one-dimensional model of hydronic floor heating: novel validation methodology using a thermostatic booth simulator and uncertainty analysis. J Build Phys2017; 41: 418–444.
32.
CaoJLiangYMasudaY, et al. Molecular simulation of CH4 adsorption behavior in slit nanopores: verification of simulation methods and models. AIChE J2019; 65: e16733.
33.
De CarvalhoLRZem-MascarenhasSH. Construction and validation of a sepsis simulation scenario: a methodological study. Rev Esc Enferm USP2020; 54: e03638.
34.
ChairopoulosNCVartholomeosPPapadopoulosE. Modeling, simulation and experimental validation of a tendon—driven soft—arm robot configuration —a continuum mechanics method. In: Proceedings of the 2019 IEEE/RSJ international conference on intelligent robots and systems (IROS’2019), Macau, China, 3–8 November 2019, pp. 5695–5700. New York: IEEE.
35.
ChampagneOPhamX-T. Numerical simulation of moving heat source in arc welding using the element-free Galerkin method with experimental validation and numerical study. Int J Heat Mass Tran2020; 154: 119633.
36.
ChenF-JWuQ-YHuangD-D, et al. Validation of the free area method for modelling fabric air dispersion system without orifices in computational fluid dynamics simulation. Indoor Built Environ2017; 27: 969–982.
37.
ChenHWangJLiuJ, et al. Validation method for simulation model of Internet of Things-aided power system. IEEE Access2020; 8: 1185–1192.
38.
ChenJTakeyamaTO-taniH, et al. Code verification of soil dynamics simulations: a case study using the method of numerically manufactured solutions. Comput Geotech2020; 117: 103258.
ChuongHDHungNQMy LeNT, et al. Validation of gamma scanning method for optimizing NaI(Tl) detector model in Monte Carlo simulation. Appl Radiat Isotopes2019; 149: 1–8.
41.
ClarkSMainarARIruinE, et al. Designing aqueous organic electrolytes for zinc–air batteries: method, simulation, and validation. Adv Energy Mater2020; 10: 1903470.
42.
CocoMMajoranaARomanoV. Cross validation of discontinuous Galerkin method and Monte Carlo simulations of charge transport in graphene on substrate. Ric Mat2016; 66: 201–220.
43.
DangJChenH. Energy saving by using asymmetric aftbodies for merchant ships–design methodology, numerical simulation and validation. J Hydrodyn2016; 28: 1022–1028.
44.
DaniilNDruryD. Investigation and validation of methods to implement a two-quadrant battery emulator for power Hardware-in-the-Loop Simulation. In: Proceedings of the 42nd conference of the IEEE industrial electronics society (IECON’2016), Florence, 23–26 October 2016, pp. 2070–2075. New York: IEEE.
45.
DanksRGoodJ. Urban scale simulations of solar reflections in the built environment: methodology & validation. In: Proceedings of the symposium on simulation for architecture and urban design, London, 6–8 May 2016, pp. 19–26. San Diego, CA: Society for Modeling and Simulation International (SCS).
46.
DorrDJoppichTSchirmaierF, et al. A method for validation of finite element forming simulation on basis of a pointwise comparison of distance and curvature. In: Proceedings of the 19th international ESAFORM conference on material forming (ed ChinestaFCuetoEAbissetchavanneE), Nantes, 27–29 April 2016, vol. 1769. College Park, MD: American Institute of Physics (AIP).
47.
DouglasHWestonPKirkwoodD, et al. Method for validating the train motion equations used for passenger rail vehicle simulation. Proc IMechE, Part F: J Rail and Rapid Transit2016; 231: 455–469.
48.
DrawertBJacobBLiZ, et al. Validation data for a hybrid smoothed dissipative particle dynamics (SDPD) spatial stochastic simulation algorithm (sSSA) method. Data Brief2019; 22: 11–15.
49.
DuttaRXingT. Five-equation and robust three-equation methods for solution verification of large eddy simulation. J Hydrodyn2018; 30: 23–33.
50.
FangKZhaoKZhouY. Validation method for simulation models with cross iteration. J Syst Eng Electron2019; 30: 555–563.
51.
FarahaniBVBelinhaJTavaresPJ, et al. A nonlinear simulation of a bi-failure specimen through improved discretisation methods: a validation study. J Strain Anal Eng2018; 53: 616–629.
52.
FavaloroAJSommerDEDenosBR, et al. Simulation of prepreg platelet compression molding: method and orientation validation. J Rheol2018; 62: 1443–1455.
53.
FeldhusenKDeiterdingRWagnerC. Validation of a dynamically adaptive lattice Boltzmann method for 2D thermal convection simulations. In: Proceedinsg of the 2015 2nd international conference on mathematics and computers in sciences and in industry (MCSI’2015), Sliema, 17 August 2015, pp. 199–206. New York: IEEE.
54.
BlázquezJLFMaestreIRGalleroFJG, et al. Experimental test for the estimation of the evaporation rate in indoor swimming pools: validation of a new CFD-based simulation methodology. Build Environ2018; 138: 293–299.
55.
GiotisGKwonO-SSheikhSA. Weakly coupled hybrid simulation method for structural testing: theoretical framework and numerical verification. J Struct Eng: ASCE2020; 146: 040191961.
56.
GkoudesnesCDeiterdingR. Verification of the WALE large eddy simulation model for adaptive lattice Boltzmann methods implemented in the AMROC framework. In: DeiterdingRDominguesMSchneiderK (eds) Cartesian CFD methods for complex applications: thematic mini-symposium contributions from ICIAM 2019 (SEMA SIMAI Springer series), vol. 3. Berlin; Heidelberg: Springer, 2021, pp. 123–144.
57.
GloverPPearceP. Rapid assessment and review of simulation modelling. J Simul2020; 14: 145–155.
58.
GregoireL-ABlanchetteH-FBelangerJ, et al. A stability and accuracy validation method for multirate digital simulation. IEEE T Ind Inform2017; 13: 512–519.
59.
GromalaPPalczynskaAWuB, et al. Simulation driven design of novel integrated circuits—Part 4: method of validation of coupled thermal and thermo-mechanical simulation. In: Proceedings of the 17th intersociety conference on thermal and thermomechanical phenomena in electronic Systems (ITherm’2018), San Diego, CA, 29 May–1 June 2018, pp. 1366–1373. New York: IEEE.
60.
GuoHGarciaEFaureB, et al. Advanced method for neutronic simulation of control rods in sodium fast reactors: numerical and experimental validation. Ann Nucl Energy2019; 129: 90–100.
61.
GurakovNIZubrilinIAAbrashkinVY, et al. Validation of the VOF method for liquid spray process simulation from a pressure-swirl atomizer. In: Proceedings of the 2nd international conference on physics and chemistry of combustion and processes in extreme environments (ComPhysChem’20-21) and 6th international summer school on modern quantum chemistry methods in applications (ed AzyazovVNMebelAM), Samara, Russia, 26–30 July 2021, vol. 2304. College Park, MD: American Institute of Physics (AIP).
62.
GutzenRVon PapenMTrenschG, et al. Reproducible neural network simulations: statistical methods for model validation on the level of network activity data. Front Neuroinform2018; 12: 90.
63.
HammoudiKTataSLaidoudiL, et al. Numerical simulation and validity of the surface photovoltage method in amorphous silicon with a Schottky contact. Semicond Sci Tech2019; 34: 015019.
64.
HanJMaLZhouW, et al. A validation method of complex product supportability simulation results based on image similarity. In: Proceedings of the 8th IEEE prognostics and system health management conference (PHM-Harbin’2017) (ed ZhangBPengYLiaoH, et al.), Harbin, China, 9–12 July 2017. New York: IEEE.
65.
HanMOokaRKikumotoH. Validation of lattice Boltzmann method-based large-eddy simulation applied to wind flow around single 1:1:2 building model. J Wind Eng Ind Aerod2020; 206: 104277.
66.
HartleyDS. Chapter 6: verification, validation, and accreditation. In: HartleyDS (ed.) Unconventional conflict (Understanding complex systems). Berlin; Heidelberg: Springer International Publishing, 2017, pp. 147–170.
67.
HendersonJABryceRM. Verification methodology for discrete event simulation models of personnel in the Canadian Armed Force. In: Proceedings of the 2019 winter simulation conference (WSC’2019), National Harbor, MD, 8–11 December 2019. New York: IEEE, pp. 2479–2490.
68.
HsiehCCLueHTHsuTH, et al. A Monte Carlo simulation method to predict large-density NAND product memory window from small-Array test element group (TEG) verified on a 3D NAND Flash test chip. In: Proceedings of the 36th IEEE symposium on VLSI technology, Honolulu, HI, 14–16 June 2016. New York: IEEE.
69.
HuTYaoYZhuF. ACTL-based parallel discrete event simulation model verification method. In: Proceedings of the 3rd IEEE information technology, networking, electronic and automation control conference (ITNEC’2019) (ed XuB), Chengdu, China, 15–17 March 2019, pp. 2250–2255. New York: IEEE.
70.
ImaizumiYSuzukiNShiraishiF, et al. Development and validation of a simulation method, PeCHREM, for evaluating spatio-temporal concentration changes of paddy herbicides in rivers. Environ Sci: Proc Imp2018; 20: 120–132.
71.
ItenMLiuSShuklaA. Experimental validation of an air-PCM storage unit comparing the effective heat capacity and enthalpy methods through CFD simulations. Energy2018; 155: 495–503.
72.
KalogerasGKoulamasCKalogerasA, et al. Verification and validation of a simulation model for energy use in buildings. In: Proceedings of the 2018 14th IEEE international workshop on factory communication systems (WFCS), Imperia, 13–15 June 2018, pp. 1–4. New York: IEEE.
73.
KettemannJGatinIBontenC. Verification and validation of a finite volume immersed boundary method for the simulation of static and moving geometries. J Non-Newton Fluid2021; 290: 104510.
74.
KimHYeoM. Thermal bridge modeling and a dynamic analysis method using the analogy of a steady-state thermal bridge analysis and system identification process for building energy simulation: methodology and validation. Energies2020; 13: 4422.
75.
KonnigkLTornerBWurmFH. Application of verification methods on a complex flow field calculated by large-eddy simulation: blood pump flow. In: Proceedings of the 6th ECCOMAS European conference on computational mechanics: solids, structures and coupled problems (ECCM’2018) and 7th ECCOMAS European conference on computational fluid dynamics (ECFD’2018) (ed OwenRDe BorstRReeseJ, et al.), 2020, pp. 4220–4232. International Centre for Numerical Methods in Engineering (CIMNE), https://www.scopus.com/inward/record.uri?eid=2-s2.0-85065988684&partnerID=40&md5=53f657eec5d0b307248e391d7a987355
76.
KrastevVKD’AdamoABerniF, et al. Validation of a zonal hybrid URANS/LES turbulence modeling method for multi-cycle engine flow simulation. Int J Engine Res2019; 21: 632–648.
77.
KretzDCallau-MonjeSHitschlerM, et al. Discrete element method (DEM) simulation and validation of a screw feeder system. Powder Technol2016; 287: 131–138.
78.
KukaNAriaudoCVerardiR, et al. Development and validation of a methodology for closed loop simulations of the driveline in railway vehicles. Proc IMechE, Part F: J Rail and Rapid Transit2017; 232: 1625–1649.
79.
LaiQHuaZXingJ, et al. Simulation and verification for seed-filling performance of cell wheel precision seed metering device based on discrete element method. In: Proceedings of the 6th international multi-conference on engineering and technology innovation (IMETI’2017) (ed HsiehWH), Hualien, Taiwan, 27–31 October 2018, vol. 169. Les Ulis: EDP Sciences.
80.
LailiYZhangLLuoY. A pattern-based validation method for the credibility evaluation of simulation models. Simulation2019; 96: 151–167.
81.
LampeasGFotopoulosKPerogamvrosN. Development and experimental validation of explicit dynamics simulation of composite structures using a stacked thick-shell methodology. Plast Rubber Compos2016; 45: 58–67.
82.
LawAM. How to build valid and credible simulation models. In: Proceedings of the 2019 winter simulation conference, National Harbor, MD, 8–11 December 2019, pp. 1402–1414. New York: IEEE.
83.
LeiteJREPSMartinsEL.UrsiniChapter 3: a validation method for AdHoc network simulation including MANETs, VANETs and emergency scenarios. In: PalattellaMRScanzioSErgenC (eds) Ad-hoc, mobile, and wireless networks, vol. 11803 (Lecture notes in computer science). Cham: Springer International Publishing, 2019, pp. 32–47.
84.
LengLSRetnasamyVShahiminMM, et al. Validation of thermal effects of LED package by using Elmer finite element simulation method. In: KramesMRKimJKTuLW, et al. (eds) Light-emitting diodes: materials, devices, and applications for solid state lighting XXI 2017, vol. 10124. Bellingham, WA: SPIE, 2017, p. 101241M.
85.
LiSY. Method validation of FEM simulation of the sinking of Carbonate inclusion in salt. In: Proceedings of the 2016 international conference on sustainable energy, environment and information engineering, Bangkok, 20–21 March 2016, pp. 63–70. Lancaster, PA: DEStech Publications.
86.
LiventsovaNVEgorovaOVLiventsovSN, et al. Present chemical production simulator model validation method. In: Proceedings of the 22nd international congress of chemical and process engineering (CHISA’2016) and 19th conference on process integration, modelling and optimisation for energy saving and pollution reduction (PRES’2016), 2016, vol. 2, p. 1185. Czech Society of Chemical Engineering, https://www.scopus.com/inward/record.uri?eid=2-s2.0-85050686607&partnerID=40&md5=6bcec5a6d5478ca325861ab087914ac1
87.
LuZHeXZhouY. Discrete element method-based collapse simulation, validation and application to frame structures. Struct Infrastruct E2017; 14: 538–549.
88.
LuttgenASharmaSKZhouD, et al. A fast simulation methodology for touch sensor panels: formulation and experimental validation. IEEE Sens J2019; 19: 996–1007.
89.
MarieNMarrelAHerbreteauK. Statistical methodology for a quantified validation of sodium fast reactor simulation tools. J Verif Valid Uncert2019; 4: 031002.
90.
MazurD. Chapter 3: a validation method of a real network device model in the riverbed modeler simulator. In: GajPSawickiMSuchackaG, et al. (eds) Computer networks, vol. 860 (Communications in computer and information science). Berlin: Springer-Verlag, 2018, pp. 26–39.
91.
MertEPsikutaAArévaloM, et al. A validation methodology and application of 3D garment simulation software to determine the distribution of air layers in garments during walking. Measurement2018; 117: 153–164.
92.
MiyazakiSItataniKFurusawaT, et al. Validation of numerical simulation methods in aortic arch using 4D Flow MRI. Heart Vessels2017; 32: 1032–1044.
93.
MizushimaNOgumaM. Acausal modeling method and its verification of engine cycle simulation for emissions prediction. Int J Automot Techn2021; 22: 341–352.
94.
MöllerTFriederichW. Simulation of elastic wave propagation across fractures using a nodal discontinuous Galerkin method—theory, implementation and validation. Geophys J Int2019; 219: 1900–1914.
95.
MoukadiriDLeclercWKhellilK, et al. Halo approach to evaluate the stress distribution in 3D discrete element method simulation: validation and application to flax/bio based epoxy composite. Model Simul Mater Sc2019; 27: 065005.
96.
MozafariFThamburajaPMoslemiN, et al. Finite-element simulation of multi-axial fatigue loading in metals based on a novel experimentally-validated microplastic hysteresis-tracking method. Finite Elem Anal Des2021; 187: 103481.
97.
NagelerPZahrerGHeimrathR, et al. Novel validated method for GIS based automated dynamic urban building energy simulations. Energy2017; 139: 142–154.
98.
Najafi-SaniMAMansourpourZ. Application of a new method for experimental validation of polydispersed DEM simulation of silo discharge. Adv Powder Technol2020; 31: 4457–4469.
99.
NguyenHTCossonBLacrampeM-F, et al. Numerical simulation of reactive polymer flow during rotational molding using smoothed particle hydrodynamics method and experimental verification. Int J Mater Form2018; 11: 583–592.
100.
NonakaTYamadaSNodaT. Verification of a macro—element method in the numerical simulation of the pore water pressure dissipation method—a case study on a liquefaction countermeasure with vertical drains under an embankment. Soils Found2017; 57: 472–487.
101.
OakleyDOnggoBSWorthingtonD. Symbiotic simulation for the operational management of inpatient beds: model development and validation using Delta-method. Health Care Manag Sci2020; 23: 153–169.
102.
OkadaNHayashiYOnukiY, et al. Mechanical stress simulation of scored tablets based on the finite element method and experimental verification. Chem Pharm Bull2016; 64: 1142–1148.
103.
OlsenMRaunakM. Increasing validity of simulation models through metamorphic testing. IEEE T Reliab2019; 68: 91–108.
104.
OtkovićIIDeluka-TibljašAŠurdonjaS. Validation of the calibration methodology of the micro-simulation traffic model. In: Proceedings of the 2019 transport infrastructure and systems in a changing world. Towards a more sustainable, reliable and smarter mobility (TIS Roma 2019) (ed IgnaccoloMTiboniM), Rome, 23–24 September 2019, vol. 45, pp. 684–691. Amsterdam: Elsevier B.V.
105.
OuJWeiCCockcroftS, et al. Advanced process simulation of low pressure die cast A356 aluminum automotive wheels—Part II modeling methodology and validation. Metals2020; 10: 1418.
106.
PanM-HTangW-CXingY, et al. Numerical simulation analysis for deformation deviation and experimental verification for an antenna thin-wall parts considering riveting assembly with finite element method. J Cent South Univ2018; 25: 60–77.
107.
PelikanLKrbalMOrsagovaJ. Simulation of impulse generator followed by practical verification. In: Proceedings of the 21st international scientific conference on electric power engineering (EPE’2020) (ed MullerZMullerM), Prague, 19–21 October 2020. New York: IEEE.
108.
PieprzycaJWarzechaPMerderT, et al. Experimental methods of validation for numerical simulation results on steel flow through tundish. Arch Metall Mater2016; 61: 2057–2060.
109.
PintoDFPintoMAPintoPR, et al. Validation of a simplified computer simulation method for plastic forming of metals by conventional tensile tests. Int Eng J2017; 70: 437–443.
110.
PojtingerSKapschRPDohmOS, et al. A finite element method for the determination of the relative response of ionization chambers in MR-linacs: simulation and experimental validation up to 1.5 T. Phys Med Biol2019; 64: 135011.
111.
PopovicsGPfeifferAMonostoriL. Generic data structure and validation methodology for simulation of manufacturing systems. Int J Comput Integ M2016; 29: 1272–1286.
112.
RahmawatiTPutriGLMarthantyDR, et al. Methods for verification of groundwater flow simulators (case study: Jakarta groundwater basin in the urban villages of Jatiluhur and Jatirangga, Bekasi City). In: Proceedings of the 2020 3rd international conference on green energy and environment engineering (CGEEE’2020), Jeju Island, South Korea, 22–24 July 2020, vol. 633. 1st ed.Bristol: IOP Publishing Ltd.
113.
Raja ShekharSSSrinivasCVRakeshPT, et al. Online Nuclear Emergency Response System (ONERS) for consequence assessment and decision support in the early phase of nuclear accidents—simulations for postulated events and methodology validation. Prog Nucl Energ2020; 119: 103177.
114.
RebouçasGFdSIFSantosJJ.ThomsenValidation of vibro-impact force models by numerical simulation, perturbation methods and experiments. J Sound Vib2018; 413: 291–307.
115.
RiedmaierSDanquahBSchickB, et al. Unified framework and survey for model verification, validation and uncertainty quantification. Arch Comput Method E2020; 28: 2655–2688.
116.
RiedmaierSSchneiderJDanquahB, et al. Non-deterministic model validation methodology for simulation-based safety assessment of automated vehicles. Simul Model Pract Th2021; 109: 102274.
117.
RivaFBeadleCFRicciP. A methodology for the rigorous verification of Particle-in-Cell simulations. Phys Plasmas2017; 24: 055703.
118.
RozsaJSongYWebbD, et al. A verified simulation method for image charge detection with non-cylindrical electrodes. In: Proceedings of the 1st intermountain engineering, technology and computing (IETC’2020), Orem, UT, 2–3 October 2020. New York: IEEE.
119.
SanoTSasakiDKoyamaM, et al. Proposal and verification of novel fatigue crack propagation simulation method by finite element method. In: Proceedings of the 22nd European conference on fracture (ECF’2018) (ed SedmakARakinMRadakovicZ), Belgrade, 26–31 August 2018, vol. 13, pp. 1154–1158. Amsterdam: Elsevier B.V.
120.
SchmidtAAPlánkaJSchmidtT, et al. Validation of a dry sliding wear simulation method for wastegate bearings in automotive turbochargers. Tribol Int2021; 155: 106711.
121.
ShiTYangHZhangN, et al. Research on verification method of electromechanical transient simulation model of battery energy storage system. In: Proceedings of the 2020 Asia energy and electrical engineering symposium (AEEES’2020), Chengdu, China, 29–31 May 2020, pp. 767–770. New York: IEEE.
122.
SiewertMNeugebauerFEppJ, et al. Validation of Mechanical Layer Equivalent Method for simulation of residual stresses in additive manufactured components. Comput Math Appl2019; 78: 2407–2416.
123.
SinaieSNgoTDNguyenVP, et al. Validation of the material point method for the simulation of thin-walled tubes under lateral compression. Thin Wall Struct2018; 130: 32–46.
124.
SiscaLQuacchiaPTLMessanaA, et al. Validation of a simulation methodology for thermoplastic and thermosetting composite materials considering the effect of forming process on the structural performance. Polymers2020; 12: 2801.
125.
SoleimaniMWriggersPRathH, et al. Numerical simulation and experimental validation of biofilm in a multi-physics framework using an SPH based method. Comput Mech2016; 58: 619–633.
126.
StotskyJAHammondJFPavlovskyL, et al. Variable viscosity and density biofilm simulations using an immersed boundary method, Part II: experimental validation and the heterogeneous rheology-IBM. J Comput Phys2016; 317: 204–222.
127.
SunX-WYangX-LLiuW. Validation method of aero-optical effect simulation for supersonic turbulent boundary layer. AIAA J2021; 59: 410–416.
128.
TateishiAWatanabeTHimenoT. Development and validation of a compressible large-eddy simulation code with overset mesh method. In: Proceedings of the ASME turbo expo 2017: turbomachinery technical conference and exposition (GT’2017), Charlotte, NC, 26–30 June 2017, vol. 2B. New York: American Society of Mechanical Engineers (ASME).
129.
TotiAVierendeelsJBelloniF. Improved numerical algorithm and experimental validation of a system thermal-hydraulic/CFD coupling method for multi-scale transient simulations of pool-type reactors. Ann Nucl Energy2017; 103: 36–48.
130.
TotiAVierendeelsJBelloniF. Development and preliminary validation of a STH-CFD coupling method for multiscale thermal-hydraulic simulations of the Myrrha reactor. In: Proceedings of the 24th international conference on nuclear engineering, Charlotte, NC, 26–30 June 2016, vol. 4. New York: American Society of Mechanical Engineers (ASME).
131.
TuttleJFWhiteNMohammadiK, et al. A novel dynamic simulation methodology for high temperature packed-bed thermal energy storage with experimental validation. Sustain Energy Technol Assess2020; 42: 100888.
132.
UrracaRPernía-EspinozaADíazI, et al. Practical methodology for validating constitutive models for the simulation of rubber compounds in extrusion processes. Int J Adv Manuf Tech2016; 90: 2377–2387.
133.
UtiugovGOChirkovVADobrovolskiiIA. The experimental verification of electrodeformation and electrocoalescence numerical simulation based on the arbitrary Lagrangian-Eulerian method. In: Proceedings of the 3rd IEEE international conference on Dielectrics (ICD’2020), Valencia, 5–31 July 2020, pp. 529–532. New York: IEEE.
134.
WangPCMcGuirkJJ. Validation of a large eddy simulation methodology for accelerated nozzle flows. Aeronaut J2020; 124: 1070–1098.
135.
WangZShibataKKoshizukaS. Verification and validation of explicit moving particle simulation method for application to internal flooding analysis in nuclear reactor building. J Nucl Sci Technol2018; 55: 461–477.
136.
WegerhoffMJacobsGDrichelP. Noise, vibration and harshness validation methodology for complex elastic multibody simulation models: with application to an electrified drive train. J Vib Control2019; 25: 243–254.
137.
WollherrSGabrielA-AUphoffC. Off-fault plasticity in three-dimensional dynamic rupture simulations using a modal Discontinuous Galerkin method on unstructured meshes: implementation, verification and application. Geophys J Int2018; 214: 1556–1584.
138.
WooMWörnerMTischerS, et al. Validation of a numerical method for interface-resolving simulation of multicomponent gas-liquid mass transfer and evaluation of multicomponent diffusion models. Heat Mass Transfer2017; 54: 697–713.
139.
WuCTBuiTQWuY, et al. Numerical and experimental validation of a particle Galerkin method for metal grinding simulation. Comput Mech2017; 61: 365–383.
140.
WuYQZhouHJBaoXF, et al. The application of the NIPC method and the FSV method on the validation of em simulation under HIRF environment. In: Proceedings of the 2017 IEEE international symposium on electromagnetic compatibility, signal and power integrity (EMCSI’2017), Washington, DC, 7–11 August 2017, pp. 561–566. New York: IEEE.
141.
XuW-JDongX-Y. Simulation and verification of landslide tsunamis using a 3D SPH-DEM coupling method. Comput Geotech2021; 129: 103803.
142.
YoshidaSFujitaniHMukaiY, et al. Real-time hybrid simulation of semi-active control using shaking table: proposal and verification of a testing method for mid-story isolated buildings. Jpn Archit Rev2018; 1: 221–234.
143.
ŽabotaBKobalM. A new methodology for mapping past rockfall events: from mobile crowdsourcing to rockfall simulation validation. ISPRS Int Geo-Inf2020; 9: 514.
144.
ZastawnyM. Numerical simulation of wing vortex generators—methodologies and validation. Aeronaut J2016; 120: 627–650.
145.
ZhangCWangFLiZ, et al. Dynamic simulation and experimental verification of foam transport in porous media based on level set method. Energy Sci Eng2019; 7: 1795–1807.
146.
ZhangJSunMHouD, et al. External sulfate attack to reinforced concrete under drying-wetting cycles and loading condition: numerical simulation and experimental validation by ultrasonic array method. Constr Build Mater2017; 139: 365–373.
147.
ZhangZTanQCuiY, et al. Experimental validation of residual stress thermomechanical simulation in as-quenched superalloy discs by using diffraction and incremental hole-drilling methods. Mater Today2021; 27: 102229.
148.
ZhengHShioyaR. Verifications for large-scale parallel simulations of 3D fluid–structure interaction using moving particle simulation (MPS) and finite element method (FEM). Int J Comput Methods2019; 16: 1840014.
149.
ZhongWZhangTTamuraT. CFD simulation of convective heat transfer on vernacular sustainable architecture: validation and application of methodology. Sustainability2019; 11: 4231.
150.
ZhouBXuYFanJ-Q, et al. Numerical simulation and experimental validation for the filtration performance of fibrous air filter media with LB method. Aerosol Air Qual Res2017; 17: 2645–2658.
151.
ZhouLZhangLShiW, et al. Transient computational fluid dynamics/discrete element method simulation of gas-solid flow in a spouted bed and its validation by high-speed imaging experiment. J Energ Resour: ASME2018; 140: 012206.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
