Comparing Machine Learning and Deep Learning Methods for Real-Time Crash Prediction

Abstract

Although there are numerous studies examining the impact of real-time traffic and weather parameters on crash occurrence on freeways, to the best of the authors’ knowledge there are no studies which have compared the prediction performances of machine learning (ML) and deep learning (DL) models. The present study adds to current knowledge by comparing and validating ML and DL methods to predict real-time crash occurrence. To achieve this, real-time traffic and weather data from Attica Tollway in Greece were linked with historical crash data. The total data set was split into training/estimation (75%) and validation (25%) subsets, which were then standardized. First, the ML and DL prediction models were trained/estimated using the training data set. Afterwards, the models were compared on the basis of their performance metrics (accuracy, sensitivity, specificity, and area under curve, or AUC) on the test set. The models considered were k-nearest neighbor, Naïve Bayes, decision tree, random forest, support vector machine, shallow neural network, and, lastly, deep neural network. Overall, the DL model seems to be more appropriate, because it outperformed all other candidate models. More specifically, the DL model managed to achieve a balanced performance among all metrics compared with other models (total accuracy = 68.95%, sensitivity = 0.521, specificity = 0.77, AUC = 0.641). It is surprising though that the Naïve Bayes model achieved a good performance despite being far less complex than other models. The study findings are particularly useful, because they provide a first insight into performance of ML and DL models.