A Simulation Validation Method Based on Bootstrapping Applied to an Agent-based Simulation of the Bay of Biscay Historical Scenario

Abstract

Combat, unlike many real-world processes, tends to be singular in nature. This makes statistical analysis of the combat data problematic. Building stochastic simulation models of combat scenarios provides a means of studying in some detail the particulars of a combat scenario provided that the model accurately captures the scenario. The scenario of interest in this paper is the WWII Bay of Biscay U-boat campaign, and the model is an agent-based simulation. The challenge is how to validate, or ensure the adequacy of, the simulation representation of the combat scenario with the actual scenario on the basis of comparing simulation output to typically small-sample-size actual combat data. In this paper we give the details of a new statistical methodology for use in validating a mission-level, agent-based model of a historical combat scenario. We use the Bay of Biscay agent-based simulation and develop a bootstrapping technique applied to the small-sample-size actual data from the Bay of Biscay campaign. Results from this bootstrapping statistical technique are compared with commonly used statistical techniques and are shown to improve the comparison of agent-based simulation output to actual real-world data.