An anisotropic heat diffusion model for enhancing the extraction of underground rail track fasteners under extremely low and uneven illumination conditions

Abstract

During the past two decades, subway systems have become one of the most dominant infrastructural developments in China at an unprecedented pace and scale. More than 60 metro lines in 25 cities have been completed, transporting more than 70 million passengers daily. Operating the subway systems safely and efficiently is a continuously pressing demand from both the management companies and the public. Although many automated or semi-automated methods for extracting critical components of the rail track systems, e.g. rail, fastener, sleeper, etc., have significantly improved the productivity of routine inspection, the unique challenges posed by the subway systems have hindered these existing methods from successful implementation because of the extremely low illumination in the underground environment, whereas additional artificial lighting often poses extremely uneven illumination. In this study, a generalized local illumination adaptation model using an anisotropic heat equation is proposed to dynamically adjust the acquired rail track images with extremely low and uneven illumination conditions. An integration flow is then proposed to seamlessly incorporate the proposed model into the state-of-the-art automated fastener detection algorithms. The results show that the proposed local illumination adaptation model can significantly improve the performance of the tested state-of-the-art fastener detection algorithms when they are applied to the images collected in the environment with extremely low and uneven illumination conditions, e.g. subway systems.

Keywords

Anisotropic heat equation fastener extraction local illumination adaptation model rail track systems visual inspection

Get full access to this article

View all access options for this article.

References

Han

et al.

Urban rail transit in China: Progress report and analysis (2008–2015). Urban Rail Transit 2016; 2: 93–105.

Singh M, Singh S, Jaiswal J, et al. Autonomous rail track inspection using vision-based system. In: IEEE international conference on computational intelligence for homeland security and personal safety, 2006, pp.56–59. New York, NY: IEEE.

Trinh H, Haas N, Li Y, et al. Enhanced rail component detection and consolidation for rail track inspection. In: IEEE workshop on the applications of computer vision (WACV), 2012, pp.289–295. New York, NY: IEEE.

Barke

Chiu

. Structural health monitoring in the railway industry: A review. Struct Health Monitor 2005; 4: 81–93.

Mazzeo

Nitti

Stella

et al.

Visual recognition of fastening bolts for railroad maintenance. Pattern Recognit Lett 2004; 25: 669–677.

Aytekin

Rezaeitabar

Dogru

et al.

Railway fastener inspection by real-time machine vision. IEEE Trans Syst Man Cybernet: Syst 2015; 45: 1101–1107.

Gibert X, Patel VM and Chellappa R. Robust fastener detection for autonomous visual railway track inspection. In: IEEE winter conference on applications of computer vision, 2015, pp.694–701. New York, NY: IEEE.

Gibert

Patel

Chellappa

. Deep multitask learning for railway track inspection. IEEE Trans Intell Transp Syst 2017; 18: 153–164.

Ruvo PD, Distante A, Stella E, et al. A GPU-based vision system for real-time detection of fastening elements in railway inspection. In: 16th IEEE international conference on image processing (ICIP), 2009, pp.2333–2336. New York, NY: IEEE.

10.

Xia Y, Xie F and Jiang Z. Broken railway fastener detection based on AdaBoost Algorithm. In: International conference on optoelectronics and image processing, 2010, pp.313–316.

11.

Feng

Jiang

Xie

et al.

Automatic fastener classification and defect detection in vision-based railway inspection systems. IEEE Trans Instrum Meas 2014; 63: 877–888.

12.

Gibert X, Patel VM and Chellappa R. Material classification and semantic segmentation of railway track images with deep convolutional neural networks. In: IEEE international conference on image processing (ICIP), 2015, pp.621–625. New York, NY: IEEE.

13.

Killing

Surgenor

Mechefske

. A machine vision system for the detection of missing fasteners on steel stampings. Int J Adv Manuf Technol 2009; 41: 808–808.

14.

Trinh

Haas

et al.

Rail component detection, optimization, and assessment for automatic rail track inspection. IEEE Trans Intell Transp Syst 2014; 15: 760–770.

15.

Lorente

Llorca

Velasco

et al.

Detection of range-based rail gage and missing rail fasteners. J Transp Res Board 2014; 2448: 125–132.

16.

Gibert-Serra X, Berry A, Diaz C, et al. A machine vision system for automated joint bar inspection from a moving rail vehicle. In: ASME/IEEE joint rail conference & internal combustion engine spring technical conference, At Pueblo, CO, USA, 2007, pp.289–296.

17.

Khan RA, Islam S and Biswas R. Automatic detection of defective rail anchors. In: 17th international IEEE conference on intelligent transportation systems (ITSC), 2014, pp.1583–1588. New York, NY: IEEE.

18.

Stella E, Mazzeo P, Nitti M, et al. Visual recognition of missing fastening elements for railroad maintenance. In: Proceedings of the IEEE 5th international conference on intelligent transportation systems, 2002, pp.94–99. New York, NY: IEEE.

19.

Resendiz

Hart

Ahuja

. Automated visual inspection of railroad tracks. IEEE Trans Intell Transp Syst 2013; 14: 751–760.

20.

Hsieh H-Y, Chen N and Liao C-L. Visual recognition system of elastic rail clips for mass rapid transit systems. In: ASME/IEEE joint rail conference & internal combustion engine spring technical conference, 2007, pp.319–325. New York, NY: IEEE.

21.

Babenko P. Visual inspection of railroad tracks. Master Thesis, University of Central Florida, USA, 2009.

22.

Yang

Tao

Liu

et al.

An efficient direction field-based method for the detection of fasteners on high-speed railways. Sensors 2011; 11: 7364–7381.

23.

Clark R. Rail flaw detection: Overview and needs for future developments. NDT & E Int 2004; 37: 111–118.

24.

Marino

Distante

Mazzeo

et al.

A real-time visual inspection system for railway maintenance: Automatic hexagonal-headed bolts detection. IEEE Trans Syst Man Cybernet C 2007; 37: 418–428.

25.

Manay

Yezzi

. Anti-geometric diffusion for adaptive thresholding and fast segmentation. IEEE Trans Image Process 2003; 12: 1310–1323.

26.

Witkin A. Scale-space filtering: A new approach to multi-scale description. In: IEEE international conference on acoustics, speech, and signal processing, ICASSP ‘84, 1984, pp.150–153.

27.

Eriksson

Estep

Johnson

. Applied mathematics: Body and soul: Calculus in several dimensions, Berlin, Heidelberg: Springer, 2004.

28.

Kimia

Siddiqi

. Geometric heat equation and nonlinear diffusion of shapes and images. Comput Vision Image Understand 1996; 64: 305–322.

29.

Kelley

Keyes

. Convergence analysis of pseudo-transient continuation. SIAM J Numer Anal 1998; 35: 508–523.

30.

Felzenszwalb P, McAllester D and Ramanan D. A discriminatively trained, multiscale, deformable part model. In: IEEE conference on computer vision and pattern recognition, 2008, 1–8. New York, NY: IEEE.

31.

Viola

Jones

. Robust real-time face detection. Int J Comput Vision 2004; 57: 137–154.

32.

Guo

Zhang

. A completed modeling of local binary pattern operator for texture classification. IEEE Trans Image Process 2010; 19: 1657–1663.

33.

Zhang

Yang

. Fault detection based on multi-scale local binary patterns operator and improved teaching-learning-based optimization algorithm. Symmetry 2015; 7: 1734–1750.

34.

Huang C, Al H, Wu B, et al. Boosting nested cascade detector for multi-view face detection. In: Proceedings of the 17th international conference on pattern recognition, Cambridge, UK, 26 August 2004, pp.415–418.

35.

Hastie T, Tibshirani R and Friedman J. The elements of statistical learning: Data mining, inference, and prediction. 2nd ed. Springer Series in Statistics. New York, NY: Springer, 2009.