Road safety analysis is typically performed by domain experts on the basis of the information contained in accident reports. The main challenges are the difficulty of considering a large number of reports in textual form and the subjectivity of the expert judgments contained in reports. This work develops a framework based on the combination of Natural Language Processing (NLP) and Machine Learning (ML) for the automatic classification of accidents with the final aim of assisting experts in performing road safety analyses. Two different models for the representation of the textual reports (Hierarchical Dirichlet Processes (HDPs) and Doc2vec) and three ML-based classifiers (Artificial Neural Networks (ANNs), Decision Trees (DTs) and Random Forests (RFs)) are compared. The framework is applied to a repository of road accident reports provided by the US National Highway Traffic Safety Administration. The best trade-off between accuracy of the classification and explainability of the obtained results is achieved by combining HDP topic modeling and RF classification.
World Health Organization. Global status report on road safety. Geneva: World Health Organization, 2018.
2.
ImprialouMQuddusM.Crash data quality for road safety research: Current state and future directions.. Accid Anal Prev2019; 130: 84–90.
3.
RolisonJJ.Identifying the causes of road traffic collisions: Using police officers’ expertise to improve the reporting of contributory factors data. Accid Anal Prev2020; 135: 105390. DOI: 10.1016/j.aap.2019.105390
4.
KrauseSBuschF.New insights into road accident analysis through the use of text mining methods. In: 6th international conference on models and technologies for intelligent transportation systems (MT-ITS), 2019. New York: IEEE.
5.
KwonOHRheeWYoonY.Application of classification algorithms for analysis of road safety risk factor dependencies. Accid Anal Prev2015; 75: 1–15.
6.
TehYWJordanMIBealMJ, et al. Hierarchical Dirichlet processes. J Am Stat Assoc2006; 101: 1566–1581.
7.
LeQMikolovT.Distributed Representations of Sentences and Documents. In: Proceedings of the 31 st international conference on machine learning, 2014.
8.
BleiDCarinLDunsonD.Probabilistic topic models. In: IEEE signal processing magazine, 2010, Vol. 27, pp.55–65. New York: IEEE.
9.
LandauerTKFoltzPWLahamD.An introduction to latent semantic analysis. Discourse Process1998; 25: 259–284.
10.
BleiDMNgAYJordanMT.Latent dirichlet allocation. Adv Neural Inf Process Syst2002; 3: 993–1022.
11.
Martinčić-IpšićSMiličićTTodorovskiA.The influence of feature representation of text on the performance of Document Classification. Appl Sci2019; 9: 743.
12.
VaswaniAShazeerNParmarN, et al. Attention Is All You Need. In: Proceeding of the 31st conference on neural information processing systems (NIPS 2017), 2017.
13.
FangWLuoHXuS, et al. Automated text classification of near-misses from safety reports: an improved deep learning approach. Adv Eng Inform2020; 44: 44.
14.
MauniHZHossainTRabR.Classification of underrepresented text data in an imbalanced dataset using deep neural network. In: IEEE region 10 symposium (TENSYMP), 2020, pp.997–1000. New York: IEEE.
15.
MishuSZRafiuddinS.Performance analysis of supervised machine learning algorithms for text classification. In: 19th international conference on computer and information technology (ICCIT), 2016, pp.409–413. New York: IEEE.
16.
ZaghloulWLeeSMTrimiS.Text classification: neural networks vs support vector machines. Ind Manag Data Syst2009; 109: 708–717.
17.
SunYLiYZengQ, et al. Application research of text classification based on random forest algorithm. In: 3rd International conference on advanced electronic materials, computers and software engineering (AEMCSE), 2020.
18.
VriesVDClassification of aviation safety reports using machine learning. In: International conference on artificial intelligence and data analytics for air transportation (AIDA-AT), 2020, pp.1–6. New York: IEEE.
19.
ValcamonicoDBaraldiPAmigoniF, et al. Text mining for the automatic classification of road accident reports. In: Proceedings of the 30th European safety and reliability conference and the 15th probabilistic safety assessment and management conference, 2020.
GasparettoAMarcuzzoMZangariA, et al. A survey on text classification algorithms: from text to predictions. Information2022; 13: 83–39.
22.
YangZBaraldiPZioE.A novel method for maintenance record clustering and its application to a case study of maintenance optimization. Reliab Eng Syst Saf2020; 203: 107103.
23.
GuimarãesMSGomes de AraújoHHLucasTC, et al. An NLP and text mining – based approach to categorize occupational accidents. In: Proceedings of the 30th European safety and reliability conference and the 15th probabilistic safety assessment and management conference, Venice, Italy, 2020.
24.
BezerraCde SantanaJMMMouraMdasC, et al. Automated classification of injury leave based on accident description and natural language processing. In: Proceedings of the 30th European safety and reliability conference and the 15th probabilistic safety assessment and management conference, Venice, Italy, 2020. DOI: 10.3850/981-973-0000-00-0.
25.
ZhangFFleyehHWangX, et al. Construction site accident analysis using text mining and natural language processing techniques. Autom Constr2019; 99: 238–248.
26.
HeidarysafaMKowsariKBarnesLE, et al. Analysis of railway accidents’ narratives using deep learning. In: 17th international conference on machine learning and applications, 2018. DOI: 10.1109/ICMLA.2018.00235.
27.
ZhangF.A hybrid structured deep neural network with Word2Vec for construction accident causes classification. Int J Constr Manag2022; 22: 1120–1140.
28.
RaneAKumarA. Sentiment classification system of twitter data for US airline service analysis. In: Proceedings of the 42nd IEEE international conference computer software and applications, 2018, pp.769–773. New York: IEEE.
29.
SarkarSVinaySMaitiJ. Text mining based safety risk assessment and prediction of occupational accidents in a steel plant. In: 2016 international conference on computational techniques in information and communication technologies (ICCTICT), 2016, pp.439–444. New York: IEEE.
30.
WilliamsTBetakJ.A comparison of LSA and LDA for the analysis of railroad accident text. Procedia Comput Sci2018; 130: 98–102.
31.
KwayuKMKwigizileVLeeK, et al. Discovering latent themes in traffic fatal crash narratives using text mining analytics and network topology. Accid Anal Prev2021; 150: 105899.
32.
LimsetthoNHataHMatsumotoKI. Comparing hierarchical dirichlet process with latent dirichlet allocation in bug report multiclass classification. In: 2014 IEEE/ACIS 15th international conference on software engineering, artificial intelligence, networking and parallel/distributed computing, SNPD 2014, 2014. New York: IEEE.
33.
TahviliSHatvaniLFeldererM, et al. Automated functional dependency detection between test cases using Doc2Vec and clustering. In: IEEE international conference on artificial intelligence testing (AITest), 2019, pp.19–26. New York: IEEE.
34.
BragattoPAnsaldiSAgnelloP, et al. Ageing management and monitoring of critical equipment at Seveso sites: An ontological approach. J Loss Prev Process Ind2020; 66: 104204
35.
MacêdoJBdas Chagas MouraMAicheleD, et al. Identification of risk features using text mining and BERT-based models_ application to an oil refinery. Process Saf Environ Prot2022; 158: 382–399.
36.
BinCBaigenCWeiS. Text mining in fault analysis for on-board equipment of high-speed train control system. In: Chinese automation congress (CAC), Jinan, China2017; pp.6907–6911. New York: IEEE.
37.
WeissSMIndurkhyaNZhangT, et al. Text mining. Predicitive methods for analyzing unstructured information. Springer, Sydney, Australia, 2005.
38.
PereiraJSaraivaF. A comparative analysis of unbalanced data handling techniques for machine learning algorithms to electricity theft detection. In: 2020 IEEE Congress on evolutionary computation (CEC), 2020, pp.1–8. New York: IEEE.
39.
SojkaPŘehůřekR.Software framework for topic modelling with large corpora. In: Proceeding Lr 2010 Work new challenges NLP Fram, 2010.
40.
MikolovTSutskeverIChenK, et al. Distributed representations of words and phrases and their compositionality. In: Proceedings of the 26th international conference on neural information processing systems, 2013, Vol. 2, pp.3111–3119. Red Hook, NY: Curran Associates Inc.
41.
AmatiGVan RijsbergenCJ.Probabilistic models of information retrieval based on measuring the divergence from randomness. ACM Trans Inf Syst2002; 20: 357–389.
42.
GriffithsTLSteyversMTenenbaumJB.Topics in semantic representation. Psychol Rev2007; 114: 211–244.
43.
AlghamdiRAlfalqiK.A survey of topic modeling in Text Mining. Int J Adv Comput Sci Appl2015; 6: 147–153.
44.
BleiDMLaffertyJD.Correlated topic models. In: Proceeding of Advances in Neural Information Processing Systems 18 conference (NIPS 2005), Vancouver, Canada, 2005, pp.147–154.
45.
BleiDMLaffertyJD.Topic models. In: SrivastavaANSahamiM (eds.), Text mining. Chapman and Hall/CRC, New York, 2009, pp.101–124.
46.
WangCPaisleyJBleiDM.Online variational inference for the hierarchical Dirichlet process. J Mach Learn Res2011; 15: 752–760.
47.
AlmeidaFXexéoG.Word Embeddings: A Survey. Computing research repository (CoRR), 2019.
48.
SchakelAMJWilsonBJ.Measuring word significance using distributed representations of words. Computing research repository (CoRR), 2015.
49.
BengioYDucharmeRVincentP, et al. A neural probabilistic language model. J Mach Learn Res2003; 3: 1137–1155.
50.
NwankpaCEIjomahWGachaganA, et al. Activation Functions : Comparison of trends in practice and research for Deep Learning. arXiv:181103378v12018; 1–20.
51.
ZhangKSuHDouY, et al. Evaluation of the influences of hyper-parameters and L2 norm regularization on ANN model for MNIST recognition. In: International conference on intelligent computing, automation and systems (ICICAS), 2019, pp. 379–386. New York: IEEE.
52.
ConroyCTominagaGTErwinS, et al. The influence of vehicle damage on injury severity of drivers in head-on motor vehicle crashes. Accid Anal Prev2008; 40: 1589–1594.
53.
AugensteinJPerdeckEStrattonJ, et al. Characteristics of the crashes that increase the risk of serious injuries. Assoc Advacement Autom Med2003; 47: 561–576.
54.
ShannonDMurphyFMullinsM, et al. Applying crash data to injury claims – an investigation of determinant factors in severe motor vehicle accidents. Accid Anal Prev2018; 113: 244–256.
55.
LiJChiuBShangS, et al. Neural text segmentation and its application to sentiment analysis. IEEE Trans Knowl Data Eng2022; 34: 828–842.
56.
LeeJSKimYHYunJS, et al. Characteristics of patients injured in road traffic accidents according to the New Injury Severity Score. Ann Rehabil Med2016; 40: 288–293.
57.
ChenMJiXShenD.Short text classification improved by learning multi-granularity topics. In: Proceedings of the 22nd international joint conference of artificial intelligence. AAAI Press, Barcelona, Spain, 2011, pp. 1776–1781.
58.
WeiJZouK. EDA : Easy data augmentation techniques for boosting performance on text classification tasks. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing, Association for Computational Linguistics, Hong Kong, China, 2019, pp.6382–6388.
59.
DogruHBTilkiSJamilA, et al. Deep learning-based classification of news texts using Doc2Vec model. In: 2021 1st international conference on artificial intelligence and data analytics, CAIDA 2021, 2021, pp.91–96. New York: IEEE.
60.
LiuXWuJZhouZ. Exploratory undersampling for class-imbalance learning. In: IEEE transactions on systems, man, and cybernetics, 2009, Vol. 39, pp.539–550. New York: IEEE.
61.
PalczewskaAPalczewskiJRobinsonRM, et al. Interpreting random forest classification models using a feature contribution method. In: Integration of Reusable Systems. Springer, Cham, 2014, pp.193–218.
62.
LoecherM.From unbiased MDI feature importance to explainable AI for Trees. arXiv Prepr (statistic, Machine Learning), 2021.
63.
MarjaiPLehotay-KéryPKissA.Document similarity for error prediction. J Inf Telecommun2021; 5: 407–420.
64.
BaraldiPCompareMZioE, et al. Identification of contradictory patterns in experimental datasets for the development of models for electrical cables diagnostics. Int J Performability Eng2011; 7: 43–60.
65.
VigJBelinkovY. Analyzing the structure of attention in a transformer language model. In: Proceedings of the 2019 ACL workshop blackboxNLP: Analyzing and interpreting neural networks for NLP, Florence, Italy, 2019, pp.63–76.
