Railroad Crossing Surveillance and Foreground Extraction Network: Weakly Supervised Artificial-Intelligence Approach

Abstract

According to the report to Congress National Strategy to Prevent Trespassing on Railroad Property issued by the Federal Railroad Administration, trespassing is currently the number one cause of all railroad-related deaths. The number of fatalities resulting from trespassing, including both illegally entering and remaining in the railroad right-of-way, is even higher than the number of deaths caused by collisions between vehicles and trains. Accurately and effectively detecting objects at rail crossings is critical for improving railway safety. The convolutional neural network (CNN) is one of the most potent tools in image processing and video surveillance. However, the current CNNs under supervised learning rely on a substantial amount of manually labeled images, which is labor-intensive and time-consuming. This study proposes a weakly supervised learning technique for training foreground-segmentation networks and develops a tailored CNN network, RC-SAFE (railroad crossing surveillance and foreground extraction), to accurately and effectively detect and track both still and moving objects in the railroad crossing area. The weakly supervised learning significantly reduces the load for image labeling. The experimental results on the public CDnet 2014 dataset show that the weakly supervised network outperforms the supervised network and some other state-of-the-art foreground-segmentation methods. Railroad crossing monitoring videos are collected at different railroad crossings to validate the proposed RC-SAFE network. The results confirm the robustness of the proposed RC-SAFE network, and that RC-SAFE can detect any object that does not belong to the railroad crossing, which outperforms object-detection-based models, such as Mask RCNN (a region-based CNN).