Overfitting Prevention in Accident Prediction Models: Bayesian Regularization of Artificial Neural Networks

Abstract

In the present paper, we implemented the Bayesian regularization (BR) backpropagation algorithm for calibrating an artificial neural network (ANN) as an accident prediction model (APM) to be used on Italian four-lane divided roads. We chose the BR-ANN since it efficiently allows for dealing with small sample size and avoiding overfitting issues by adding a regularization term in the objective function to be minimized during training. Moreover, BR-ANNs are sparsely employed in road safety analyses, and their peculiarities deserve to be emphasized. In our work, the BR-ANN aims to predict the number of fatal and injury (FI) crashes across 236 road elements, for a total length of 78 km. The input features are road element length, horizontal and vertical alignment, cross-section geometry, operating speed, traffic flow, sight distance, and road area type (i.e., a categorical predictor accounting for the potential influence of merge and diverge influence areas). Training and test phases of the BR-ANN have been evaluated by determination coefficient (R²), root mean square error (RMSE), overfitting ratio (OR), scatterplots, residuals analysis, and by the same ANN architecture trained with the gradient descent (GD) with momentum and adaptive learning rate backpropagation algorithm (GD-ANN). Results demonstrate that the BR-ANN markedly outperforms the GD-ANN, which suffers severe overfitting issues. Furthermore, BR-ANN does not overfit data (OR close to the unity), reports a satisfactory R² (0.726), and shows a Gaussian residual distribution with zero mean. Therefore, road authorities could consider regularized ANNs for performing appropriate safety analyses, especially when dealing with small road sample sizes.

Keywords

accident prediction models crash analysis artificial neural networks Bayesian regularization backpropagation algorithm overfitting Issue road safety four-lane divided roads

Get full access to this article

View all access options for this article.

References

World Health Organization. Global Status Report on Road Safety: Summary 2018. World Health Organization, Geneva, 2018.

Fiorentini

Losa

Handling Imbalanced Data in Road Crash Severity Prediction by Machine Learning Algorithms. Infrastructures, Vol. 5, No. 7, 2020, P. 61. https://doi.org/10.3390/infrastructures5070061.

Iranitalab

Khattak

Comparison of Four Statistical and Machine Learning Methods for Crash Severity Prediction. Accident Analysis & Prevention, Vol. 108, 2017, pp. 27–36. https://doi.org/10.1016/J.AAP.2017.08.008.

Rahim

M. A.

Hassan

H. M.

A Deep Learning Based Traffic Crash Severity Prediction Framework. Accident Analysis Prevention, Vol. 154, 2021, P. 106090. https://doi.org/10.1016/j.aap.2021.106090.

Theofilatos

Chen

Antoniou

Comparing Machine Learning and Deep Learning Methods for Real-Time Crash Prediction. Transportation Research Record: Journal of the Transportation Research Board, 2019. 2673: 169–178.

Hossain

Muromachi

A Bayesian Network Based Framework for Real-Time Crash Prediction on the Basic Freeway Segments of Urban Expressways. Accident Analysis & Prevention, Vol. 45, 2012, pp. 373–381. https://doi.org/10.1016/j.aap.2011.08.004.

Fiorentini

Losa

Long-Term-Based Road Blackspot Screening Procedures by Machine Learning Algorithms. Sustainability, Vol. 12, No. 15, 2020, P. 5972. https://doi.org/10.3390/su12155972.

Washington

Haque

M. M.

Lee

Applying Quantile Regression for Modeling Equivalent Property Damage Only Crashes to Identify Accident Blackspots. Accident Analysis & Prevention, Vol. 66, 2014, pp. 136–146. https://doi.org/10.1016/j.aap.2014.01.007.

Fiorentini

Leandri

Losa

Defining Machine Learning Algorithms as Accident Prediction Models for Italian Two-Lane Rural, Suburban, and Urban Roads. International Journal of Injury Control and Safety Promotion, 2022, pp. 1–13. https://doi.org/10.1080/17457300.2022.2075397

10.

Hall

Tarko

A. P.

Adequacy of Negative Binomial Models for Managing Safety on Rural Local Roads. Accident Analysis & Prevention, Vol. 128, 2019, pp. 148–158. https://doi.org/10.1016/j.aap.2019.03.001.

11.

Tang

Donnell

E. T.

Application of a Model-Based Recursive Partitioning Algorithm to Predict Crash Frequency. Accident Analysis & Prevention, Vol. 132, 2019, P. 105274. https://doi.org/10.1016/j.aap.2019.105274.

12.

Mannering

F. L.

Bhat

C. R.

Analytic Methods in Accident Research: Methodological Frontier and Future Directions. Analytic Methods in Accident Research, Vol. 1, 2014, pp. 1–22. https://doi.org/10.1016/j.amar.2013.09.001.

13.

Mannering

Bhat

C. R.

Shankar

Abdel-Aty

Big Data, Traditional Data and the Tradeoffs between Prediction and Causality in Highway-Safety Analysis. Analytic Methods in Accident Research, Vol. 25, 2020, P. 100113. https://doi.org/10.1016/j.amar.2020.100113.

14.

Yannis

Dragomanovits

Laiou

La Torre

Domenichini

Richter

Ruhl

Graham

Karathodorou

Road Traffic Accident Prediction Modelling: A Literature Review. Proceedings of the Institution of Civil Engineers: Transport, Vol. 170, No. 5, 2017, pp. 245–254. https://doi.org/10.1680/jtran.16.00067.

15.

Wang

Zhao

Jackson

Functional Forms of the Negative Binomial Models in Safety Performance Functions for Rural Two-Lane Intersections. Accident Analysis & Prevention, Vol. 124, 2019, pp. 193–201. https://doi.org/10.1016/j.aap.2019.01.015.

16.

Washington

S. P.

Karlaftis

M. G.

Mannering

F. L.

Anastasopoulos

. . Statistical and Econometric Methods for Transportation Data Analysis, 3rd ed. Chapman & HALL/CRC, New York, NY, 2020.

17.

Zhang

Comparing Prediction Performance for Crash Injury Severity among Various Machine Learning and Statistical Methods. IEEE Access, Vol. 6, 2018, pp. 60079–60087. https://doi.org/10.1109/ACCESS.2018.2874979.

18.

Alkheder

Taamneh

Severity Prediction of Traffic Accident Using an Artificial Neural Network. Journal of Forecasting, Vol. 36, No. 1, 2017, pp. 100–108. https://doi.org/10.1002/for.2425.

19.

Singh

Pal

Yadav

Singla

Deep Neural Network-Based Predictive Modeling of Road Accidents. Neural Computing and Applications, Vol. 32, No. 16, 2020, pp. 12417–12426. https://doi.org/10.1007/s00521-019-04695-8.

20.

Abdel-Aty

Haleem

Analyzing Angle Crashes at Unsignalized Intersections Using Machine Learning Techniques. Accident Analysis & Prevention, Vol. 43, No. 1, 2011, pp. 461–470. https://doi.org/10.1016/J.AAP.2010.10.002.

21.

Dong

Shao

Xiong

An Improved Deep Learning Model for Traffic Crash Prediction. Journal of Advanced Transportation, Vol. 2018, 2018, pp. 1–13. https://doi.org/10.1155/2018/3869106.

22.

Cawley

G. C.

Talbot

N. L. C.

Preventing Over-Fitting during Model Selection via Bayesian Regularisation of the Hyper-Parameters. Journal of Machine Learning Research, Vol. 8, 2007, pp. 841–861.

23.

MacKay

D. J. C.

Bayesian Interpolation. Neural Computation, Vol. 4, No. 3, 1992, pp. 415–447. https://doi.org/10.1162/neco.1992.4.3.415.

24.

Schlögl

Stütz

Laaha

Melcher

A Comparison of Statistical Learning Methods for Deriving Determining Factors of Accident Occurrence from an Imbalanced High Resolution Dataset. Accident Analysis & Prevention, Vol. 127, 2019, pp. 134–149. https://doi.org/10.1016/J.AAP.2019.02.008.

25.

Chakraborty

Mukherjee

Mitra

Development of Pedestrian Crash Prediction Model for a Developing Country Using Artificial Neural Network. International Journal of Injury Control and Safety Promotion, Vol. 26, No. 3, 2019, pp. 283–293. https://doi.org/10.1080/17457300.2019.1627463.

26.

Xie

Lord

Zhang

Predicting Motor Vehicle Collisions Using Bayesian Neural Network Models: An Empirical Analysis. Accident Analysis & Prevention, Vol. 39, No. 5, 2007, pp. 922–933. https://doi.org/10.1016/j.aap.2006.12.014.

27.

Riviere

Lauret

Ramsamy

J. F. F. M.

Page

A Bayesian Neural Network Approach to Estimating the Energy Equivalent Speed. Accident Analysis & Prevention, Vol. 38, No. 2, 2006, pp. 248–259. https://doi.org/10.1016/j.aap.2005.08.008.

28.

Mcculloch

W. S.

Pitts

A Logical Calculus of the Ideas Immanent in Nervous Activity. Bulletin of Mathematical Biophysics, Vol. 5, 1943, pp. 115–133. https://doi.org/10.1007/978-3-030-01370-7_61.

29.

Rumelhart

D. E.

Hinton

G. E.

Williams

R. J.

Learning Representations by Back-Propagating Errors. Nature, Vol. 323, No. 6088, 1986, pp. 533–536. https://doi.org/10.1038/323533a0.

30.

Lau

K. T.

Guo

Kiernan

Slater

Diamond

Non-Linear Carbon Dioxide Determination Using Infrared Gas Sensors and Neural Networks with Bayesian Regularization. Sensors and Actuators, B: Chemical, Vol. 136, No. 1, 2009, pp. 242–247. https://doi.org/10.1016/j.snb.2008.11.030.

31.

Yue

Songzheng

Tianshi

Bayesian Regularization BP Neural Network Model for Predicting Oil-Gas Drilling Cost. Proc., International Conference on Business Management and Electronic Information, IEEE, New York, 2011, pp. 483–487.

32.

Kayri

. Predictive Abilities of Bayesian Regularization and Levenberg-Marquardt Algorithms in Artificial Neural Networks: A Comparative Empirical Study on Social Data. Mathematical and Computational Applications, Vol. 21, No. 2, 2016, P. 20. https://doi.org/10.3390/mca21020020.

33.

Liang

An Effective Bayesian Neural Network Classifier with a Comparison Study to Support Vector Machine. Neural Computation, Vol. 15, No. 8, 2003, pp. 1959–1989. https://doi.org/10.1162/08997660360675107.

34.

Liang

Bayesian Neural Networks for Nonlinear Time Series Forecasting. Statistics and Computing, Vol. 15, No. 1, 2005, pp. 13–29. https://doi.org/10.1007/s11222-005-4786-8.

35.

Jaynes

E. T.

Prior Probabilities. IEEE Transactions on Systems Science and Cybernetics, Vol. 4, No. 3, 1968, pp. 227–241. https://doi.org/10.1109/TSSC.1968.300117.

36.

Foresee

F. D.

Hagan

M. T.

Gauss-Newton Approximation to Bayesian Learning. Proc., International Conference on Neural Networks (ICNN’97), Vol. 3, Houston, TX, IEEE, New York, 1997, pp. 1930–1935.

37.

Levenberg

Arsenal

A Method for the Solution of Certain Non-Linear Problems in Least Squares. Quarterly of Applied Mathematics, Vol. 1, No. 278, 1943, pp. 536–538.

38.

Marquardt

D. W.

An Algorithm for Least-Squares Estimation of Nonlinear. Journal of the Society for Industrial and Applied Mathematics, Vol. 11, No. 2, 1963, pp. 431–441.

39.

Fiorentini

Leandri

Losa

Predicting International Roughness Index by Deep Neural Networks with Levenberg-Marquardt Backpropagation Learning Algorithm. Proc., SPIE 11863, Earth Resources and Environmental Remote Sensing/GIS Applications XII, Vol. 11863, 118630P, September12, 2021, pp. 194–207. https://doi.org/10.1117/12.2598005.

40.

Géron

Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems. O’Reilly Media, Sebastopol, CA, 2019.

41.

Kohavi

A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection. Proc., 14th International Joint Conference on Artificial Intelligence, American Association for Artificial Intelligence, Montreal, Quebec, Canada, August 20–25, 1995, pp. 1137–1143.

42.

Breiman

Random Forest. Machine Learning, Vol. 1, 2001, P. 33. https://doi.org/10.1023/A:1010933404324.

43.

Breiman

Friedman

J. H.

Olshen

R. A.

Stone

C. J.

Classification and Regression Trees. Chapman & Hall/CRC, New York, 1984.

44.

Louppe

Wehenkel

Sutera

Geurts

Understanding Variable Importances in Forests of Randomized Trees. Advances in Neural Information Processing Systems, Vol. 26, 2013, pp. 1–9.

45.

Ministry of Infrastructure and Transport. Decreto Ministeriale 5 Novembre2001, n. 6792 (S.O. n.5 Alla G.U. n.3. Del 4.1.02) Norme Funzionali e Geometriche per La Costruzione Delle Strade.

46.

ESRI. ArcGIS Desktop: Release 10.5. Environmental Systems Research Institute, Redlands, CA, 2011.

47.

Misaghi

Hassan

Modeling Operating Speed and Speed Differential on Two-Lane Rural Roads. Journal of Transportation Engineering, Vol. 131, No. 6, 2005, pp. 408–418. https://doi.org/10.1061/(ASCE)0733-947X(2005)131:6(408).

48.

National Research Council of the Transportation Research Board. Highway Capacity Manual 6th Edition: A Guide for Multimodal Mobility Analysis. Transportation Research Board, Washington, DC, 2016.

49.

Ministry of Infrastructure and Transport. Decreto Ministeriale Numero 137 Del 02/05/2012, Linee Guida per La Gestione Della Sicurezza Delle Infrastrutture Stradali Ai Sensi Dell’art. 8 Del Decreto Legislativo 15 Marzo 2011, n. 35.