We present a framework for verification and validation of simulation models of System of Systems that is based on an existing framework for modeling and simulation. The framework addresses problems arising especially in recently emerging Systems of Systems such as cyber-physical autonomous cooperative systems. The design of such systems presents challenges to the currently employed independent use of simplified models for formal verification or brute-force simulations which are severely limited in the range of conditions they can test. The proposed framework is applied to integration of formal analytic and simulation verification methods where there is a need to have confidence that the properties proved for idealized abstract models also hold in more realistic models which gave rise to the abstractions. Taking both logical and probabilistic perspectives clarifies the situation and suggests where more research is needed.
AggarwalS. Ergodic Machines—Probabilistic and Approximate Homomorphic Simplifications. PhD Thesis, The University of Michigan, Ann Arbor, Michigan, USA, 1975.
2.
ZeiglerBP. Theory of modeling and simulation. 1st ed. New York: John Wiley and Sons, 1976.
3.
CaughlinD. Verification, validation, and accreditation (VV&A) of models and simulations through reduced order metamodels. In: Proceedings of the 27th conference on winter simulation. IEEE Computer Society, 1995.
4.
LeeKFishwickPA. Dynamic model abstraction. In: Proceedings of 1996 winter simulation conference, San Diego, CA, December1996, pp. 764–771.
5.
VangheluweH. Foundations of modelling and simulation of complex systems. In: ECEASST10, 2008.
6.
BaierCJoost-PieterK. Principles of model checking. Cambridge, MA: MIT Press, 2008.
7.
JamshidiM. Systems of systems—innovations for the 21st century. New York: John Wiley and Sons, 2008.
8.
ZeiglerBPPraehoferHKimTG. Theory of modeling and simulation. 2nd ed. Boston, MA: Academic Press, 2000.
9.
ShafferAR. The value of modeling and simulation for the department of defense. M&S Journal2012; (Fall): 2–3.
10.
MittalSRisco-MartínJL. Model-driven systems engineering for netcentric system of systems with DEVS unified process. In: PasupathyRKimS-HTolkAHillRME KuhlME (eds) Proceedings of the 2013 winter simulation conference, 2013.
11.
NutaroJJ. Building software for simulation: Theory and algorithms with applications in C++. New York: John Wiley and Sons, 2011.
12.
ZeiglerBPSongHKimTPraehoferH. DEVS framework for modeling, simulation, analysis, and design of hybrid systems. In: AntsaklisPKohnWNerodeASastryS (eds) Proceedings of hybrid systems II (Lecture Notes in Computer Science, vol. 999). Berlin: Springer-Verlag, 1995, pp. 529–551.
13.
ZeiglerBPSarjoughianHS. Guide to modeling and simulation of systems of systems. New York: Springer, 2012.
14.
DenilJ. Design, verification and deployment of software intensive systems: A multi-paradigm modelling approach. PhD Dissertation, University of Antwerp, 2013.
15.
PiersallCHGrangeFE. The necessity of intended use specification for successful modeling and simulation of a systems-of-systems (Crosstalk Online). http://m.crosstalkonline.org/issues/15/130/
16.
ZeiglerBP. Multifaceted modelling and discrete event simulation. New York: Academic Press, 1984.
17.
ZeiglerBP. Structuring the organization of partial models. Int J Gen Syst1978: 4: 81–88.
18.
ÖrenTIZeiglerBP. System theoretic foundations of modeling and simulation: A historic perspective and the legacy of A. Wayne Wymore. Simulation2012; 88: 1033–1046.
19.
WymoreAW. A mathematical theory of systems engineering: The elements. New York: John Wiley and Sons, 1967.
20.
WymoreAW. Model-based systems engineering: An introduction to the mathematical theory of discrete systems and to the tricotyledon theory of system design. Boca Raton, FL: CRC Press, 1993.
21.
MittalSRisco-MartínJL. DEVS net-centric system of systems engineering with DEVS unified process (CRC–Taylor & Francis Series on System of Systems Engineering). New York: Taylor & Francis, 2012.
TraoréMKMuzyA. Capturing the dual relationship between simulation models and their context. Sim Model Practice Theory2006; 14: 126–142.
24.
BalciO. Golden rules of verification, validation, testing, and certification of modeling and simulation applications. M&S Mag2010; n4(Oct): 1–7.
25.
RozaZC. Simulation fidelity theory and practice: a unified approach to defining, specifying and measuring realism of simulations. Delft, The Netherlands: Delft University Press Science, 2004.
26.
FouresDAlbertVNkestaA. Formal compatibility of experimental frame concept and FD-DEVS model. In: Proceedings of 9th international conference of modeling, optimization and simulation—MOSIM’12, 2012.
27.
ZeiglerBPNutaroJJ. Combining DEVS and model-Checking: Using systems morphisms for integrating simulation and analysis in model engineering. In: EMSS2014, Bordeaux, France, 2014.
28.
SaadawiHWainerG. Principles of discrete event system specification model verification. Simulation2013; 89(1): 41–67.
29.
SaadawiHWainerGMoallemiM. Principles of models verification for real-time embedded applications. In: PopoviciKMostermanP (eds) Real-time simulation technologies: principles, methodologies, and applications. Boca Raton, FL: CRC Press, 2012.
30.
HumphreyLR. Model checking for verification in UAV cooperative control applications recent advances in research on unmanned aerial vehicles. Lecture Notes in Control and Information Sciences2013; 444: 69–117.
31.
FooN. Stability preservation under homomorphisms. IEEE Trans SMC1977; 7: 750–754.
32.
SierockiI. 1986. A note on structural inference in systems theory. Int J Gen Syst1986, 13: 17–22. DOI: 10.1080/03081078608934951.
33.
RozaMVoogdJSebaljD. The generic methodology for verification and validation to support acceptance of models, simulations and data. JDMS2013; 10: 347–365.
34.
BalciO. Verification, validation, and accreditation. In: Proceedings of the 30th conference on winter simulation. IEEE Computer Society Press, 1998.
35.
BalciO. A life cycle for modeling and simulation. Simulation2012; 88: 870–883.
36.
BalciOOrmsbyW. Well-defined intended uses: an explicit requirement for accreditation of modeling and simulation applications. In: Proceedings of the 2000 winter simulation conference. IEEE Computer Society Press, 1998
37.
BalciOSargentRG. A methodology for cost–risk analysis in the statistical validation of simulation models. Commun ACM1981; 24: 190–197.
38.
KleijnenJPC. Verification and validation of simulation models. Eur J Op Res1995, 82: 145–162.
39.
TolkA. Verification and validation. Engineering principles of combat modeling and distributed simulation. Hoboken, NJ: John Wiley & Sons, 2012, pp. 263–294.
40.
PaceDK. 2004. Modeling and simulation verification and validation challenges. John Hopkins APL Tech Digest2004; 25(2): 163–172.
41.
PaceDK. 2012 Comprehensive consideration of uncertainty in simulation use. JDMS2013; 10: 367–380. DOI: 10.1177/1548512912455471.
42.
ObaidatMSBoudrigaN. Fundamentals of performance evaluation of computer and telecommunications systems. New York: John Wiley & Sons, 2010.
43.
WeiselEW. Towards a foundational theory for validation of models and simulations. In: Proceedings of the spring 2011 simulation interoperability workshop, 11S–SIW–074. Orlando, FL: Simulation Interoperability Standards Organization (SISO), 2011. http://www.sisostds.org/.
44.
WeiselEW. Decision theoretic approach to defining use for computer simulation. In: Proceedings of the 2012 autumn simulation conference. San Diego, CA: Society for Modeling and Simulation International (SCS), 2012.
45.
WeiselEWPettyMPMielkeRR. Validity of models and classes of models in semantic composability. In: Proceedings of the fall 2003 simulation interoperability workshop, 03F–SIW–073. Orlando, FL: Simulation Interoperability Standards Organization (SISO), 2003. http://www.sisostds.org/.
46.
ReynoldsRAIskenderianHOuztsSO. Using multiple representations and resolutions to compose simulated METOC environments. In: Proceedings of 2002 spring simulation interoperability workshop, 2002.
47.
BaohongL. A formal description specification for multi-resolution modeling based on DEVS formalism and its applications. JDMS2007; 4: 229–251.
48.
CetinkayaD. Model driven development of simulation models: defining and transforming conceptual models into simulation models by using metamodels and model transformation. PhD Thesis, Technical University Delft, The Netherlands, 2013.
49.
CetinkayaDVerbraeckASeckMD. MDD4MS: A model driven development framework for modeling and simulation. In: Proceedings of the summer computer simulation conference. SCS Press, 2011, pp. 113–121
50.
CetinkayaDMittalSVerbraeckASeckMD. Model driven engineering and its application in modeling and simulation. In MittalSMartinJLR (eds) Netcentric system of systems engineering with DEVS unified process. Boca Raton, FL: CRC Press, 2013, pp. 221–248.
51.
ZeiglerBPMoonY. DEVS representation and aggregation of spatially distributed systems: speed versus error tradeoffs. Trans SCS, 1996; 13: 179–190.
PettyMD. Verification and validation. In: SokolowskiJABanksCM (eds) Principles of modeling and simulation: a multidisciplinary approach. Hoboken, NJ: John Wiley & Sons, 2009.
54.
ÖrenTI. Artificial intelligence in quality assurance of simulation studies. In ElzasMSÖrenTIZeiglerBP (eds) Modelling and simulation methodology in the artificial intelligence era. Amsterdam: North-Holland, 1986, pp. 267–278
55.
ZhangLShenYWZhangXSSongXTaoFLiuY. The model engineering for complex system simulation. In: Proceedings of the 26th European modeling and simulation symposium (simulation in industry), Bordeaux, France, 10–12 September 2014.
56.
CastroRKofmanEWainerG (2010) A formal framework for stochastic discrete event system specification modeling and simulation, Simulation86 (10), 587–611
