Data-Cleaning Technique for Reliable Real-Life Travel Time Estimation: Use of Dedicated Short-Range Communications Probes on Rural Highways

Abstract

With increasing market penetration, dedicated short-range communications (DSRC) probes are attracting more interest in South Korea for application in advanced traveler information systems as a way to efficiently alleviate traffic congestion. Generally, DSRC probes, thanks to their ability to directly collect point-to-point travel times, are considered to be superior to conventional point detectors. However, outlying observations caused by vehicles entering and exiting between corridors, parking, driving on the shoulder lane during congestion, and so on are inevitable in DSRC probe data and can erroneously indicate abnormally long or short travel times. Moreover, since DSRC scanners cannot discern directional maneuvers, outlying observations, especially for the interrupted facilities this study is based on, seem to be more significant than in conventional automatic license plate recognition. In this paper, a novel algorithm is proposed to filter out outliers in DSRC probe data for rural highways. The suggested algorithm is divided into two parts. In the case of a small sample, the algorithm uses a previous interval value to determine a valid range for the current interval values; otherwise, the algorithm uses a modified median filter that uses the modified z-score of the current interval observations to determine the valid range. To address the problem of a collection interval in which only outliers exist or in which the number of outliers is greater than the number of valid observations, the logics of the valid median range and the maximum coefficient of variation are applied to enhance the performance of the developed algorithm. The algorithm has been thoroughly verified through the use of various types of DSRC probe data for rural highways near Seoul, South Korea. The algorithm has been proved to be sufficient in overcoming the deficiencies of previous techniques as well as in generating reliable real-life travel time information with errors of less than 5%.