Comparison of Machine Learning Approaches for Classifying and Evaluating Freeway Guide Signs

Abstract

Well-designed guide signs improve traffic safety by reducing reaction times and aiding decision-making. Current studies on guide sign assessments typically rely on simulated environments or expert subjective evaluations, which lack real data from actual driving conditions. Such data are crucial for accurately evaluating the practical effectiveness of guide signs from the driver’s perspective in complex and dynamic traffic environments. To systematically assess drivers’ physiological and psychological responses and identify guide sign deficiencies, this paper proposes an innovative approach by integrating emotional indicators, such as eye tracking, electroencephalography, and facial expressions, into guide sign effectiveness evaluation. These indicators help determine whether a guide sign is effective or ineffective. A comparison of several machine learning classification algorithms was undertaken to classify and evaluate freeway guide signs, using real-world road-testing data collected from Shijiazhuang, China. These indicators are used in the analysis, including accuracy, precision, recall, F1 score, and precision-recall curves. The extreme gradient boosting algorithm demonstrated the best performance, achieving 0.83 accuracy, 0.66 precision, 0.83 recall, 0.73 F1 score, and 0.71 area under the precision-recall curve for the minority class, while also excelling in the majority class. Percentage of eyelid closure, frowning, anger, and pupil area have particularly significant effects on the effectiveness of guide signs. Drivers preferred signs with place names, while inconsistent reliability and complex ramps led to lower ratings. This study provides empirical guidance for optimizing the effectiveness of freeway guide signs, emphasizing the critical role of physiological and psychological indicators in the evaluation process. It underscores that improving the clarity and reliability of guide signs is essential for enhancing road safety and promoting traffic efficiency.