Accelerated safety evaluation of automated vehicles using modified Bayesian optimization

Abstract

A comprehensive safety evaluation of automated vehicles, along with well-defined safety boundaries and operational design domains, is essential for ensuring the safe deployment of vehicles. However, the rare failure modes, the neural networks, and continuous test scenarios make the efficient safety evaluation of automated vehicles particularly challenging. To address these challenges, we propose a new Modified Bayesian Optimization (MBO) framework to accelerate the safety evaluation of automated vehicles. We utilize the Gaussian Processes (GP) with logit and logistic functions to iteratively fit a surrogate model for predicting the failure modes of automated vehicles in untested test scenarios. Three acquisition functions are introduced to comprehensively explore the test scenario space, accurately identify the safety boundaries, and sample in predicted unsafe regions. Additionally, likelihood weighting is employed to efficiently estimate the failure probability of the automated vehicle by using the fitted surrogate model. To validate the effectiveness of the proposed framework, we evaluate the safety of Reinforcement Learning–based T-BONE Collision Damage Mitigation (RL-TCDM) system in a 2-dimensional T-junction scenario, and evaluate the safety of an Adaptive Cruise Control (ACC) system in a 4-dimensional cut-in scenario. The experimental results show that, in both the T-junction scenario and the cut-in scenario, when the number of test scenarios is 990, the traditional algorithms are unable to obtain unbiased estimates of the failure probability. In contrast, the MBO achieves the lowest relative error in failure probability estimation, with values of 0.005 and 0.03, respectively.