Traffic simulation models support detailed analysis of the dynamics of traffic phenomena and are important tools for analysis of transportation systems. In order to evaluate correctly the impact of different traffic management schemes, simulation models must be able to replicate reality adequately. Model validation (i.e., the process of checking to what extent the model replicates reality) is discussed. The role of validation is defined within the scope of model development and calibration, and the framework for performing the validation is discussed. A hierarchy of statistical methods to validate different types of simulation outputs against observed data is examined. Also, a validation method is proposed on the basis of statistical tests on metamodels fitted to the observed and simulated data. A case study illustrates the applicability of the various methods.
Get full access to this article
View all access options for this article.
References
1.
ToledoT., KoutsopoulosH. N., DavolA., Ben-AkivaM. E., BurghoutW., AndréassonI., JohanssonT., and LundinC.. Calibration and Validation of Microscopic Traffic Simulation Tools: Stockholm Case Study. In Transportation Research Record: Journal of the Transportation Research Board, No. 1831, TRB, National Research Council, Washington, D.C., 2003, pp. 65–75.
2.
RaoL., and OwenL.. Validation of High-Fidelity Traffic Simulation Models. In Transportation Research Record: Journal of the Transportation Research Board, No. 1710, TRB, National Research Council, Washington, D.C., 2000, pp. 69–78.
3.
TuringA. M.Computing Machinery and Intelligence. Mind, Vol. 59, 1950, pp. 433–460.
4.
SchrubenL.Confidence Interval Estimation using Standardized Time Series. Operations Research, Vol. 31, 1980, pp. 1090–1108.
5.
AbdulhaiB., SheuJ. B., and ReckerW.. Simulation of ITS on the Irvine FOT Area Using "Paramics 1.5" Scalable Microscopic Traffic Simulator: Phase I: Model Calibration and Validation. California PATH Research Report UCB-ITS-PRR-99-12. University of California, Berkeley, 1999.
6.
RakhaH., HellingaB., Van AerdeM., and PerezW. A.. Systematic Verification, Validation, and Calibration of Traffic Simulation Models. Presented at 75th Annual Meeting of the Transportation Research Board, Washington, D.C., 1996.
7.
BloombergL., and DaleJ.. Comparison of VISSIM and CORSIM Traffic Simulation Models on a Congested Network. In Transportation Research Record: Journal of the Transportation Research Board, No. 1727, TRB, National Research Council, Washington, D.C., 2000, pp. 52–60.
8.
HallF. L., BloombergL., RouphailN. M., EadsB., and MayA. D.. Validation Results for Four Models of Oversaturated Freeway Facilities. In Transportation Research Record: Journal of the Transportation Research Board, No. 1710, TRB, National Research Council, Washington, D.C., 2000, pp. 161–170.
9.
RilettL. R., KimK.-O., and RaneyB.. Comparison of Low-Fidelity TRANSIMS and High-Fidelity CORSIM Highway Simulation Models with Intelligent Transportation System Data. In Transportation Research Record: Journal of the Transportation Research Board, No. 1739, TRB, National Research Council, Washington, D.C., 2000, pp. 1–8.
10.
FellendorfM., and VortischP.. Validation of Microscopic Traffic Flow Model VISSIM in Different Real-World Situations. Presented at 80th Annual Meeting of the Transportation Research Board, Washington, D.C., 2001.
11.
LeeD.-H., YangX., and ChandrasekarP.. Parameter Calibration for PARAMICS Using Genetic Algorithm. Presented at 80th Annual Meeting of the Transportation Research Board, Washington, D.C., 2001.
12.
JayakrishnanR., OhJ.-S., and SahraouiA.-E.-K.. Calibration and Path Dynamics Issues in Microscopic Simulation for Advanced Traffic Management and Information Systems. In Transportation Research Record: Journal of the Transportation Research Board, No. 1771, TRB, National Research Council, Washington, D.C., 2001, pp. 9–17.
13.
BenekohalR. F.Procedure for Validation of Microscopic Traffic Flow Simulation Models. In Transportation Research Record 1320, TRB, National Research Council, Washington, D.C., 1991, pp. 190–202.
14.
RaoL., GoldsmanD., and OwenL.. Development and Application of a Validation Framework for Traffic Simulation Models. Proc., 1998 Winter Simulation Conference (MedeirosD. J., WatsonE. F., CarsonJ. S., and ManivannanM. S., eds.), Washington, D.C., 1998.
15.
HazeltonM. L.Inference for Origin-Destination Matrices: Estimation, Prediction and Reconstruction. Transportation Research B, Vol. 35, 2001, pp. 667–676.
16.
DoanD. L., ZiliaskopoulosA., and MahmassaniH.. On-Line Monitoring System for Real-Time Traffic Management Applications. In Transportation Research Record: Journal of the Transportation Research Board, No. 1678, TRB, National Research Council, Washington, D.C., 1999, pp. 142–149.
17.
AlexopoulosC., and SeilaA.. Output Data Analysis. In Handbook of Simulation: Principles, Methodology, Advances, Applications, and Practice (BanksJ., ed.), John Wiley & Sons, New York, 1998.
18.
FishmanG. S.Principles of Discrete Event Simulation. John Wiley & Sons, New York, 1978.
19.
TheilH.Economic Forecasts and Policy. North-Holland, Amsterdam, Netherlands, 1961.
20.
LawA. M., and KeltonW. D.. Simulation Modeling and Analysis, 3rd ed.McGraw-Hill, New York, 2000.
21.
ScheffeH.Practical Solutions of the Behrens-Fisher Problem. Journal of the American Statistical Association, Vol. 65, 1970, pp. 1501–1508.
22.
KleijnenJ. P. C.Verification and Validation of Simulation Models. European Journal of Operational Research, Vol. 82, 1995, pp. 145–162.
23.
HolmS.A Simple Sequentially Rejective Multiple Test Procedure. Scandinavian Journal of Statistics, Vol. 6, 1979, pp. 65–70.
24.
ShafferJ. P.Multiple Hypothesis Testing. Annual Review of Psychology, Vol. 46, 1995, pp. 561–584.
25.
FasanoG., and FranceschiniA.. A Multidimensional Version of the Kolmogorov-Smirnov Test. Monthly Notices of the Royal Astronomical Society, Vol. 225, 1987, pp. 155–170.
26.
PressW. H., TeukolskyS. A., VetterlingW. T., and FlanneryB. P.. Numerical Recipes in C, 2nd ed. Cambridge University Press, Cambridge, United Kingdom, 1992.
27.
KleijnenJ. P. C.Validation of Models: Statistical Techniques and Data Availability. Presented at 1999 Winter Simulation Conference, Phoenix, Ariz., 1999.
28.
PipesL. A.Car-Following Models and the Fundamental Diagram of Road Traffic. Transportation Research, Vol. 1, 1967, pp. 21–29.
29.
GallantA. R., and GoebelJ. J.. Nonlinear Regression with Autocorrelated Errors. Journal of the American Statistical Association, Vol. 71, 1976, pp. 961–967.
