Sage Journals: Discover world-class research

Abstract

Road obstacle detection is an important component of the advanced driver assistance system, and to improve the speed and accuracy of road obstacle detection method is a vital task. In this article, fast image region-matching method based on the maximally stable extremal regions method is proposed to improve the speed of image matching. The theoretical feasibility of detection method combining monocular camera with inertial measurement unit (IMU) is clarified. The fast road obstacle detection method based on maximally stable extremal regions combining fast image region-matching method based on maximally stable extremal regions and the vision-IMU-based obstacle detection method is proposed to bypass obstacle classification and to reduce time and space complexity for road environment perception. The AdaBoost cascade detector, the speeded-up robust features-based obstacle detection method, and the proposed method are used to detect obstacles in outdoor contrast tests. Test results show that the proposed method has higher accuracy, and the reason of high accuracy is analyzed. The processing time of AdaBoost cascade detector, speeded-up robust features-based obstacle detection method, and proposed method are compared, and the results show that the proposed method has faster processing speed, and the reason of faster processing speed is analyzed.

Keywords

Road obstacle detection MSER pinhole camera model IMU

Introduction

Road obstacle detection is an important component of the advanced driver assistance system and has attracted an extensive amount of interest from both academia and automobile industry. Although LiDAR and millimeter wave radar have higher robustness and accuracy, the cost has restricted their application in road obstacle detection. Machine vision has been paid more attention to and has been studied deeply in the aspect of road obstacle detection because of meeting the human visual cognitive habit and low cost. With the development of artificial intelligence, machine learning algorithm has been gradually introduced into the obstacle detection method to improve the accuracy. Pomerleau applied artificial neural network to the identification of traffic environment.¹ Xiao et al. used random forest method to obtain better results in structured road detection.² Sivaraman et al. proposed an active learning framework based on Haar features and adaptive boosting algorithm (AdaBoost)³ to detect vehicles in high way environment. Song et al. improved the robustness and real time of vehicle detection through the integration of AdaBoost and convolutional neural network (CNN).⁴ The continuous development of generative adversarial networks, SegNet, Faster-CNN, and other new concept of machine learning increases the potential for improving the accuracy of monocular obstacle detection.^5
–7 Obstacle detection methods above are based on accurate classification of obstacles, and accurate classification requires large amount of samples and high computational cost. In sparse optical flow field-based obstacle detection methods^8

–11 and motion compensation-based obstacle detection methods,^12
–14 lots of feature points must be extracted by Harris corner detector,¹⁵ scale-invariant feature transform (SIFT),^, ^16,17 speeded-up robust features (SURF),^18
–20 features from accelerated segment test,^21,22 or other traditional feature point detectors. In traditional feature point detectors, feature points are determined by detecting local feature of every pixel in the images, thus leading to a large number of feature points. The processing and analysis of large numbers of feature points increases the computational cost of road obstacle detection, and road obstacle detection speed is influenced.

Road obstacles in two-dimensional (2D) images can be described by regions, the number of regions is less than feature points, and the regions can be easily tracked if they are stable. The maximally stable extremal regions (MSER) method denotes a set of distinguished regions that are detected in a grayscale image. All of these regions are defined by an extremal property of the intensity function in the region and on its outer boundary. MSERs have properties that form their superior performance as stable local detector.^23,24 Thus, we propose a fast road obstacle detection method based on MSER to speed up obstacle detection using improved MSER region-matching method and pinhole camera model.

Feature points are determined by detecting local feature of every pixel in the images for traditional feature point detectors, thus leading to a large number of feature points, and the huge number of feature points increases the computational cost of road obstacle detection and road obstacle detection speed is influenced. Small amount of feature points can be extracted by MSER-based image region-matching method. However, clustering method and pattern recognition method cannot be used if there are few feature points. Therefore, the fast road obstacle detection method based on MSER combining fast image region-matching method based on MSER and the vision-inertial measurement unit (IMU)-based obstacle detection method is proposed to bypass obstacle classification and to reduce time and space complexity for road environment perception.

In this article, a fast region-matching method based on MSER is proposed to speed up MSERs’ matching, and this method is presented in the “Fast image region-matching method based on MSER” section. Monocular camera- and IMU based obstacle detection method (also called the vision-IMU-based obstacle detection method) is presented in the “Vision-IMU-based obstacle detection method” section. The fast road obstacle detection method based on MSER combining the fast region-matching method and the vision-IMU-based obstacle detection method is presented in the “Fast road obstacle detection method based on MSER” section. Effect analysis of fast road obstacle detection method based on MSER and conclusions of this article are presented in the “Effect analysis of fast road obstacle detection method based on MSER” and “Conclusion” sections, respectively.

Fast image region-matching method based on MSER

The MSER algorithm is widely used in image registration and region matching.^25

–31 In most MSER-based image region matching, first, MSERs are extracted by MSER extraction algorithm firstly, and all MSERs in the reference image and the target image are fitted into elliptical regions to provide more useful information secondly, and feature point detection methods, such as SIFT and affine-SIFT (ASIFT), are used to improve matching precision finally. In the process of road obstacle detection, if images are collected at a short time interval, the position and area shape of MSERs in the two images will not change greatly, and affine change will not be obvious. Therefore, we propose a fast image region-matching method using the stability of MSERs and ignoring MSERs’ position and shape difference between two images, to simplify matching process and to improve matching speed. Process of fast image region-matching method based on MSER is as follows and is shown in Figure 1:

MSER extraction. Extract maximally stable extremal regions using traditional MSER method.³²

Area calculation. Let Ab ( $A b = {A b_{1}, A b_{2}, \dots, A b_{m}}$ ) be area set of MSERs in previous image, let Aa( $A a = {A a_{1}, A a_{2}, \dots, A a_{n}}$ ) be area set of MSERs in latter image, and let A_i ( $A_{i} = | A b_{i} - A a |$ ) be absolute value set of area difference between the i th MSER in the previous image and unmatched MSERs in the latter image.

Distance calculation. Let Cb( $C b = {C b_{1}, C b_{2}, \dots, C b_{m}}$ , $C b_{i} = {x b_{i}, y b_{i}}$ ) be centroid set of MSERs in previous image, let Ca( $C a = {C a_{1}, C a_{2}, \dots, C a_{n}}$ , $C a_{i} = {x a_{i}, y a_{i}}$ , $X a = {x a_{1}, x a_{2}, \dots, x a_{n}}$ , $Y a = {y a_{1}, y a_{2}, \dots, y a_{n}}$ ) be centroid set of MSERs in latter image, and let D_i( $D_{i} = \sqrt{(x b_{i} - X a) + (y b_{i} - Y a)}$ ) be distance set between the i th MSER in the previous image and unmatched MSERs in the latter.

Image region matching. Let M_i be match value set of the i th MSER, and the MSER corresponding to the minimum M_i is considered as the matching region.

M_{i} = A N_{i} + D N_{i}

A N_{i} = (A_{i} - min (A_{i})) / (max (A_{i}) - min (A_{i}))

D N_{i} = (D_{i} - min (D_{i})) / (max (D_{i}) - min (D_{i}))

Figure 1.

Schematic diagram of fast image region-matching method based on MSER. MSER: maximally stable extremal regions.

Vision-IMU-based obstacle detection method

Small amount of feature points can be extracted by MSER-based image region-matching method. Clustering method and pattern recognition method cannot be used to detect obstacles. Therefore, the vision-IMU-based method is used to detect obstacle directly and accurately.

Static obstacle detection

Image acquisition is the process of mapping objects in 3D space to 2D image plane, and this process can be simplified as a pinhole camera model (see Figure 2). The effective focal length of the camera is f, the installation height of the camera is h, and the pitch angle of the camera is ∂. The coordinate origin of the plane coordinate system $(x_{0}, y_{0})$ , that is, the intersection between the image plane and the camera optical axis, is usually set to (0, 0). The intersection of the front obstacle and the road plane is P, and the coordinates of point P in the image plane coordinate system is (x, y). The horizontal distance (d) between the point P and the camera can be obtained

d = \frac{h}{tan (\partial + arctan [(y_{0} - y) / f])}

Figure 2.

Schematic diagram of pinhole camera model.

The first imaging point of obstacle is A (see Figure 3), y axis is moved from y₁ to y₂ in the image plane because of the camera’s movement, and the imaging point of obstacle’s top is B. Assuming that A is the imaging point of A′ on the road plane and B is the imaging point of B′ on the road plane, then the horizontal distance from the camera to A′ is d₁ and the horizontal distance from the camera to B′ is d₂. d₁ and d₂ can be calculated by equation (1), and the relationship is $d_{1} = d_{2} + Δ d$ . But the real relationship is $d_{1} = d_{2} + Δ d + Δ l$ . Therefore, the target point is not on the road if $d_{1} \neq d_{2} + Δ d$ , and thus, static obstacles can be recognized by Δl if Δd can be acquired using IMU.

Figure 3.

Schematic diagram of static obstacle imaging.

Moving obstacle detection

When the front obstacle moves along the horizontal direction (see Figure 4), the distance from camera to obstacle’s top point at the previous moment is s₁, the distance from camera to obstacle’s top point at the following moment is s₂, and the relationship between d₁, d₂, s₁, and s₂ is

{\begin{matrix} d_{2} = d_{1} + Δ l - Δ d \\ s_{2} = s_{1} + s - Δ d \end{matrix}

Figure 4.

Schematic diagram of moving obstacle imaging.

The relationship between h_v, h, d₁, d₂, s₁, and s₂ according to the characteristic of right triangle is

{\begin{matrix} \frac{h_{v}}{h} = \frac{d_{1} - s_{1}}{d_{1}} \\ \frac{h_{v}}{h} = \frac{d_{2} - s_{2}}{d_{2}} \end{matrix}

Δl can be calculated according to equations (2) and (3) as $Δ l = \frac{h \times s - h_{v} \times Δ d}{h_{v} - h}$ . Obstacles can thus be recognized under any circumstances that do not meet $h \times s = h_{v} \times Δ d$ . Thus, it can be seen that the method of distinguishing obstacles from road surface using monocular camera and IMU is feasible, and the process of differentiation only needs to track and calculate the position of feature points; therefore, the consumption of time and space can be reduced.

Fast road obstacle detection method based on MSER

Fast road obstacle detection method based on MSER is proposed by combining fast image region-matching method based on MSER and vision-IMU-based obstacle detection method. The fast image region-matching method based on MSER is used to simplify matching process and to improve matching speed, and the vision-IMU-based obstacle detection method is used to detect obstacles using less feature points.

Process of fast road obstacle detection method based on MSER

Process of the fast road obstacle detection method based on MSER is as follows and is shown in Figure 5.

Figure 5.

Fast road obstacle detection method based on MSER. MSER: maximally stable extremal regions.

Camera parameter updating based on IMU data

Calibration of camera initial parameters. Calibrate the monocular camera mounted on the vehicle and get the camera focal length f, the mounting height h, the pitch angle ∂, and the pixel size of the photosensitive chip p.

Continuous inertial data acquisition. At the beginning of t = 0, continuously acquire inertial data by IMU rigidly connected with monocular camera with frequency F.

Camera parameters updating. Calculate Δd in period Δt according to inertial data.

Image region matching and obstacle detection

Image region matching based on MSER. Acquire road image at t and t + Δt and match image regions on the basis of MSER.

Feature points position. Find centroids of matched regions as feature points.

Horizontal distance calculation and obstacle detection. Assuming feature points are at the horizontal plane, calculate horizontal distance d₁ from feature points to the camera at t; assuming feature points are at the horizontal plane, calculate horizontal distance d₂ from feature points to the camera at t + Δt; compare Δl ( $Δ l = | d_{1} - Δ d - d_{2} |$ ) and k (k is a set threshold, k > 0); the feature point is at horizontal plane and the region does not belong to the obstacle if Δl ≤ k; the feature point is not at horizontal plane and the region belongs to the obstacle if Δl > k.

Obstacle detection experiment

OV5640 camera unit (OmniVision Technologies, Inc.) and JY61p IMU (Wit Motion Intelligent Technology Co., Ltd.) are mounted on a movable platform (see Figure 6 (a)). The obstacle is simulated by vehicle scaling model (see Figure 6 (b)). Traffic mark, road repair patches, and other visually significant non-obstacles are simulated by pieces of paper attached to the plane (see Figure 6 (c)). One of the indoor obstacle experiments is processed as follows.

Figure 6.

Indoor experiment equipment. (a) Movable platform, camera unit, and IMU. (b) Vehicle scaling model (obstacle). (c) Traffic mark and road patch. IMU: inertial measurement unit.

Effective focal length of the camera f = 6.779 mm, installation height of the camera h = 6.572 cm, pitch angle of the camera ∂ = 0.132 rad, and pixel size of the photosensitive chip p = 1.4 µm.

The angular acceleration and acceleration data are acquired by the IMU with F = 100 Hz. The camera pose is solved using the quaternion method, and the pitch angle of the camera ∂ is updated. The horizontal distance Δd = 2.00 cm in period Δt = 2 s is calculated using acceleration data.

As mentioned in the third step of image region matching and obstacle detection in the “Process of fast road obstacle detection method based on MSER” subsection, t represents the moment of acquiring image data. The images at t = 0 and t = 2 are processed by fast image region-matching method based on MSER. Fourteen centroids of matched regions are found as feature points. The extraction of MSERs is shown in Figure 7, and the feature points are shown in Figure 8.

Figure 7.

MSER extraction. (a) Image at t = 0. (b) Image at t = 2. (c) Matched images. Red regions and o are MSERs and centroids of MSERs in the image at t = 0, cyan regions, and + are MSERs and centroids of MSERs in the image at t = 2. MSER: maximally stable extremal regions.

Figure 8.

Feature points. (a) Feature points in the image at t = 0. (b) Feature points in the image at t = 2.

Assuming feature points are at the horizontal plane, calculate horizontal distance d₁ from feature points to the camera at t = 0; assuming feature points are at the horizontal plane, calculate horizontal distance d₂ from feature points to the camera at t = 2; Δl and k (k = 2 cm) are compared to confirm obstacles. Calculation results of d ₁, d ₂, and Δl are shown in Table 1.

Table 1.

Calculation results of d ₁, d ₂, and Δl.

Feature point	d ₁, cm	d ₂, cm	Δd, cm	Δl, cm
1	34.07	32.23	2.00	0.16
2	25.47	24.24	2.00	0.77
3	25.86	23.78	2.00	0.08
4	51.08	48.91	2.00	0.17
5	51.07	49.44	2.00	0.37
6	99.03	94.81	2.00	2.22
7	99.02	94.80	2.00	2.22
8	80.96	76.80	2.00	2.16
9	68.04	63.74	2.00	2.30
10	52.67	50.92	2.00	0.25
11	52.67	50.92	2.00	0.25
12	52.67	50.92	2.00	0.25
13	52.06	49.81	2.00	0.25
14	52.99	50.63	2.00	0.36

Calculation results show that feature points 6, 7, 8, and 9 are not at horizontal plane, and corresponding MSERs are considered as regions belonging to obstacles.

MSERs belonging to obstacles are labeled as obstacle region. The bottom of the obstacle region is considered as the intersection of the obstacle and the road, and the distance from obstacles to camera is calculated by pinhole camera model (see Figure 9).

Figure 9.

Obstacle region. Yellow box is the detected obstacle region, and cyan box shows the distance from obstacle to camera.

Effect analysis of fast road obstacle detection method based on MSER

AdaBoost cascade detection method is a typical machine learning method, and it is widely used in obstacle because of its high accuracy and speed in target recognition.^33

–36 In contrast tests, a 20-level AdaBoost cascade detector using the Histogram of Oriented Gradient (HOG) feature is built, and the maximum false detection rate for each level of the cascade detector is 0.2. The car image data set of Stanford University Krause and pedestrian image data set of Center for Biological & Computational Learning at Massachusetts Institute of Technology (MIT CBCL) are artificially labeled, and the labeled regions are taken as the positive samples. The Pasadena_Houses_2000 image data set of the computer vision research group of California Institute of Technology is selected to provide negative samples. AdaBoost cascade detector (also called AdaBoost method) is trained using the positive and negative samples.

The SURF-based detector is several times faster than SIFT and is more robust against different image transformations than SIFT.³⁷ In contrast tests, the SURF is used to detect feature points in motion compensation-based road obstacle detection method. In SURF-based detection method (hereafter, SURF method) and fast road obstacle detection method based on MSER (hereafter, MSER method), if feature points or regions are detected as obstacle region and the points or regions close to each other, they will be classified as one obstacle.

The traffic environment on campus road is recorded by OV2710 camera unit, camera pose data is recorded by HEC295 IMU, and the image data and camera pose data are processed using the AdaBoost method, SURF-based detection method, and MSER method. The results are compared to analyze the accuracy and detection speed of the above three methods.

Analysis of detection accuracy

Producer’s accuracy (PA), user’s accuracy (UA), overall accuracy (OA), and κ are widely used in the field of remote sensing and pattern recognition because of high universality.^38
–40 PA, UA, OA, and κ are thus used as the evaluation indexes.

The number of pixels that are detected as obstacles by detection method but actually are not obstacles is a_i. The number of pixels that are detected as obstacles and actually are obstacles is b_i. The number of pixels that are not detected as obstacles by detection method but actually are obstacles is c_i. The number of pixels that are not detected as obstacles by detection method and actually are not obstacles is d_i. Confusion matrix of detection results is shown in Table 2.

a = \sum_{i = 1}^{n} a_{i}, b = \sum_{i = 1}^{n} b_{i}, c = \sum_{i = 1}^{n} c_{i}, d = \sum_{i = 1}^{n} d_{i}

PA can be calculated as

P A = \frac{b}{b + c}

UA can be calculated as

U A = \frac{b}{a + b}

OA can be calculated as

O A = \frac{b + d}{a + b + c + d}

κ can be calculated as

κ = \frac{(a + b) \times (b + d) + (c + d) \times (b - a)}{(a + b) \times (a + d) + (b + c) \times (c + d)}

Table 2.

Confusion matrix of detection result.

		Actual
		Positive	Negative
Detected	Positive	b_i	a_i
Detected	Negative	c_i	d_i

Confusion matrix of AdaBoost method-detected results is shown in Table 3. Confusion matrix of SURF method-detected results is shown in Table 4. Confusion matrix of MSER method-detected results is shown in Table 5. Comparison of detection accuracy is shown in Table 6 and Figure 10. The obstacle detection results are shown in Figure 11.

Table 3.

Confusion matrix of AdaBoost method detected results.

		Actual
		Positive	Positive
Detected	Positive	3.611 × 10⁷	1.659 × 10⁷
Detected	Negative	1.176 × 10⁷	2.342 × 10⁸

Table 4.

Confusion matrix of SURF method detected results.

		Actual
		Positive	Negative
Detected	Positive	3.926 × 10⁷	8.266 × 10⁶
Detected	Negative	3.100 × 10⁶	2.480 × 10⁸

SURF: speeded-up robust features.

Table 5.

Confusion matrix of MSER method detected results.

		Actual
		Positive	Positive
Detected	Positive	3.886 × 10⁷	2.045 × 10⁶
Detected	Negative	1.023 × 10⁷	2.475 × 10⁸

MSER: maximally stable extremal regions.

Table 6.

Comparison of detection accuracy.

	PA	UA	OA	κ
AdaBoost method	0.755	0.685	0.905	0.763
SURF method	0.927	0.826	0.962	0.939
MSER method	0.792	0.950	0.959	0.928

PA: producer’s accuracy; UA: user’s accuracy; OA: overall accuracy; SURF: speeded-up robust features; MSER: maximally stable extremal regions.

Figure 10.

Histogram of detection accuracy.

Figure 11.

Obstacle detection results. (a) Result of AdaBoost method. (b) Result of MSER method. The AdaBoost method did not detect the temporary traffic lights. MSER: maximally stable extremal regions.

It can be seen in Table 6 and Figure 10 that the PA of MSER method is lower than that of SURF method, and the UA of MSER method is higher than that of SURF method. That is because in complicated traffic environment, some obstacles are far from the camera or moving rapidly, and these obstacle regions may not be detected as stable regions by MSER.

It is also shown that the accuracy indexes of MSER method and SURF methods are higher than those of AdaBoost method in vehicle obstacle detection. The essence of AdaBoost method is bias classification, its accuracy is affected by the quality and quantity of training samples, and the accuracy of AdaBoost method is thus reduced. In addition, an AdaBoost method can only detect the trained targets, and if obstacles in test are not vehicles or pedestrians, AdaBoost method will not detect obstacles effectively.

Analysis of detection speed

Data of images and camera poses before and after moving should be processed in the SURF method and the MSER method, and all the time needed for detecting obstacles in an image is counted to compare detection speeds of the three methods. The average detection time of three detection methods is shown in Table 7.

Table 7.

Average detection time of three methods.

	AdaBoost method	SURF method	MSER method
Detection time, s	0.587	0.808	0.732

SURF: speeded-up robust features; MSER: maximally stable extremal regions.

It can be seen in Table 7 that the speed of AdaBoost method is faster than that of the SURF and MSER methods. That is because the AdaBoost method is trained using large amount of positive and negative samples and if the time of detector training is counted, the detection speed will be slower.

The speed of MSER method is faster than the SURF method because in MSER method, the processes of elliptical region fitting and feature point detected by SIFT or ASIFT are omitted. What’s more, MSER method detect region feature instead of local feature of every pixel in the image; thus, less number of stable feature points can be detected. Less points cause less calculation and faster obstacle detection speed.

Conclusion

In this article, fast image region-matching method based on MSER is proposed. In the fast image region-matching method, the processes of elliptical region fitting and feature point detecting by SIFT or ASIFT are omitted to improve the speed of image matching. The theoretical feasibility of detection method combining monocular camera with IMU is clarified through deriving horizontal distances from camera to static obstacle and moving obstacle. The fast road obstacle detection method based on MSER combining fast image region-matching method based on MSER and the vision-IMU-based obstacle detection method is proposed to bypass obstacle classification and to reduce time and space complexity for road environment perception.

Obstacle detection steps and indoor experiments are shown to expound the detection process of the fast road obstacle detection method based on MSER. The AdaBoost cascade detector, SURF-based obstacle detection method, and the proposed method are used to detect obstacles in outdoor contrast tests, and the PA, UA, OA, and κ are used as evaluating indexes to compare test results. The results show that the proposed method has higher accuracy, and the reason of high accuracy is analyzed. The processing time of AdaBoost cascade detector, SURF-based obstacle detection method, and proposed method are compared; the results show that the proposed method has faster processing speed, and the reason of faster processing speed is analyzed.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research is supported by the National Key Research and Development Program of China (2016YFD0701101), the Scientific Research Initial Foundation of Shandong University of Technology (4041416053), the National Natural Science Foundation of China (51508315), and the Natural Science Foundation of Shandong Province (ZR2016EL19).

References

Pomerleau

. Knowledge-based training of artificial neural networks for autonomous robot driving. Robot Learn 1993; 233: 19–43. DOI: 10.1007/978-1-4615-3184-5_2.

Xiao

Dai

Liu

. Monocular road detection using structured random forest. Int J Adv Robot Syst 2016; 13: 101. DOI: 10.5772/63561.

Sivaraman

Mohan

. A general active-learning framework for on-road vehicle recognition and tracking. IEEE Trans Int Trans Syst 2010; 11(2): 267–276. DOI: 10.1109/TITS.2010.2040177.

Song

Rui

Zha

. The AdaBoost algorithm for vehicle detection based on CNN features. In: Proceedings of the 7th international conference on Internet multimedia computing and service, 2015, pp. 1–5. ACM. DOI: 10.1145/2808492.2808497.

Goodfellow

Pouget-Abadie

Mirza

. Generative adversarial nets. In: Advances in Neural Information Processing Systems 2014, pp. 2672–2680.

Badrinarayanan

Kendall

Cipolla

. SegNet: a deep convolutional encoder-decoder architecture for scene segmentation. IEEE Trans Patt Anal Mach Int 2017; 39(12): 2481–2495. DOI: 10.1109/TPAMI.2016.2644615.

Ren

Girshick

. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Patt Anal Mach Int 2017; 39: 1137–1149. DOI: 10.1109/TPAMI.2016.2577031.

Pan

Bingham

Chen

. Breaking camouflage and detecting targets require optic flow and image structure information. Appl Opt 2017; 56: 6410–6418. DOI: 10.1364/AO.56.006410.

Braillon

Pradalier

Crowley

. Real-time moving obstacle detection using optical flow models. In: Intelligent vehicles symposium, 2006, IEEE, Tokyo, Japan, 13–15 June 2006, pp. 466–471. IEEE. DOI: 10.1109/IVS.2006.1689672.

10.

Mahmoudi

Kierzynka

Manneback

. Real-time motion tracking using optical flow on multiple GPUs. Bull Polis Acad Sci Tech Sci 2014; 62: 139–150. DOI: 10.2478/bpasts-2014-0016.

11.

Lefaix

Marchand

Bouthemy

. Motion-based obstacle detection and tracking for car driving assistance. In: 16th international conference on pattern recognition, Quebec City, Canada, 11–15 August 2002. IEEE.

12.

Zhu

Sun

Guo

. Surf points based moving target detection and long-term tracking in aerial videos. KSII Trans Int Inform Syst 2016; 10: 5624–5638. DOI: 10.3837/tiis.2016.11.023.

13.

Pan

Chen

Peng

. A new moving objects detection method based on improved SURF algorithm. In: 2013 25th Chinese on Control and decision conference (CCDC), Guiyang, China, 25–27 May 2013, pp. 901–906. IEEE. DOI: 10.1109/CCDC.2013.6561051.

14.

Wei

YC,

Tang

ACW

. Visual tracking using compensated motion model for mobile cameras. In: 2011 18th IEEE international conference on image processing (ICIP), 2011, pp. 489–492. DOI: 10.1109/ICIP.2011.6116558.

15.

Harris

Stephens

. A combined corner and edge detector. Alvey Vision Conf 1988; 15: 147–152. DOI: 10.5244/C.2.23.

16.

Song

George

. Remote sensing image registration approach based on a retrofitted SIFT algorithm and Lissajous-curve trajectories. Opt Exp 2010; 18: 513–522. DOI: 10.1364/OE.18.000513.

17.

Kobayashi

Okamoto

Onishi

. Generation of obstacle avoidance based on image features and embodiment. Int J Robot Autom 2012; 27: 364–376. DOI: 10.2316/Journal.206.2012.4.206-3631.

18.

Kang

Choi

Park

. Local environment recognition system using modified SURF-based 3D panoramic environment map for obstacle avoidance of a humanoid robot. Int J Adv Robot Syst 2013; 10(6): 275. DOI: 10.5772/56552.

19.

Aguilar

Casaliglla

VP,

Pólit

. Obstacle avoidance based-visual navigation for micro aerial vehicles. Electronics 2017; 6(1): 10. DOI: 10.3390/electronics6010010.

20.

Aguilar

Casaliglla

VP,

Pólit

. Obstacle avoidance for low-cost UAVs. In: 2017 IEEE 11th international conference on semantic computing (ICSC), San Diego, CA, USA, 30 January–1 February 2017, pp. 503–508. IEEE. DOI: 10.1109/ICSC.2017.96.

21.

Rosten

Drummond

. Machine learning for high-speed corner detection. In: European conference on computer vision (eds Leonardis

Bischof

Pinz

.), 2006, pp. 430–443. Berlin: Springer. DOI: 10.1007/11744023_34.

22.

Rosten

Porter

Drummond

. Faster and better: a machine learning approach to corner detection. IEEE Trans Patt Anal Mach Int 2010; 32: 105–119. DOI: 10.1109/TPAMI.2008.275.

23.

Zhou

Shi

Gao

. Wildfire smoke detection based on local extremal region segmentation and surveillance. Fire Safet J 2016; 85: 50–58. DOI: 10.1016/j.firesaf.2016.08.004.

24.

Donoser

Bischof

. Efficient maximally stable extremal region (MSER) tracking. In: 2006 IEEE computer society conference on computer vision and pattern recognition, New York, NY, USA, 17–22 June 2006, pp. 553–560. IEEE. DOI: 10.1109/CVPR.2006.107.

25.

Henderson

Izquierdo

. Robust feature matching in long-running poor-quality videos. IEEE Trans Circ Syst Video Technol 2016; 26: 1161–1174. DOI: 10.1109/TCSVT.2015.2441411.

26.

Mammeri

Boukerche

. Lane detection and tracking system based on the MSER algorithm, Hough transform and Kalman filter. In: Proceedings of the 17th ACM international conference on modeling, analysis and simulation of wireless and mobile systems, Montreal, QC, Canada, 21–26 September 2014, pp. 259–266. New York, NY, USA: ACM. DOI: 10.1145/2641798.2641807.

27.

Forssén

Lowe

. Shape descriptors for maximally stable extremal regions. In: 2007 ICCV 2007 IEEE 11th international conference on computer vision, Rio de Janeiro, Brazil, 14–21 October 2007, pp. 1–8. IEEE. DOI: 10.1109/ICCV.2007.4409025.

28.

Zhang

Guo

. Robust feature matching and selection methods for multisensor image registration. In: 2009 IEEE international, IGARSS 2009 geoscience and remote sensing symposium, Cape Town, South Africa, 12–17 July 2009, pp. 255–258. IEEE. DOI: 10.1109/IGARSS.2009.5417786.

29.

Mikolajczyk

Tuytelaars

Schmid

. A comparison of affine region detectors. Int J Comput Vision 2005; 65: 43–72. DOI: 10.1007/s11263-005-3848-x.

30.

Liu

Tuo

. Multi-sensor image registration using edge-enhanced maximally stable extremal region. In: 2012 5th international congress on image and signal processing (CISP), Chongqing, China, 16–18 October 2012, pp. 901–905. IEEE. DOI: 10.1109/CISP.2012.6469944.

31.

Zhang

Wang

. Registration of images with affine geometric distortion based on maximally stable extremal regions and phase congruency. Image Vision Comput 2015; 36: 23–39. DOI: 10.1016/j.imavis.2015.01.008.

32.

Matas

Chum

Urban

. Robust wide-baseline stereo from maximally stable extremal regions. Image Vision Comput 2004; 22: 761–767. DOI: 10.1016/j.imavis.2004.02.006.

33.

Viola

Jones

. Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition, Kauai, HI, USA, 8–14 December 2001, pp. 511–518, 10.1109/CVPR.2001.990517.

34.

Schapire

. The strength of weak learn ability. Mach Learn 1990; 5: 28–33.

35.

Schapire

. A brief introduction to boosting. In: Sixteenth international joint conference on artificial intelligence, Vol. 14, 1999, pp. 1401–1406. Morgan Kaufmann Publishers Inc. ISBN: 1-55860-613-0.

36.

Freund

Schapire

. A decision-theoretic generalization of on-line learning and an application to boosting. J Comput Syst Sci 1999; 55: 119–139. ISBN: 3-540-59119-2.

37.

Bay

Ess

Tuytelaars

. Speeded-up robust features. Comput Vision Image Understand 2008; 110: 404–417. DOI: 10.1016/j.cviu.2007.09.014.

38.

Patel

Chatterjee

Gorai

. Development of machine vision-based ore classification model using support vector machine (SVM) algorithm. Arab J Geosci 2017; 10: 107. DOI: 10.1007/s12517-017-2909-0.

39.

Perea-Moreno

Aguilera-Ureña

Meroño-De Larriva

. Assessment of the potential of UAV video image analysis for planning irrigation needs of golf courses. Water 2016; 8(12): 584. DOI: 10.3390/w8120584.

40.

Abdat

Amouroux

Guermeur

. Hybrid feature selection and SVM-based classification for mouse skin precancerous stages diagnosis from bimodal spectroscopy. Optics Express 2012; 20: 228–244. DOI: 10.1364/OE.20.000228.

Fast road obstacle detection method based on maximally stable extremal regions

Abstract

Keywords

Introduction

Fast image region-matching method based on MSER

Vision-IMU-based obstacle detection method

Static obstacle detection

Moving obstacle detection

Fast road obstacle detection method based on MSER

Process of fast road obstacle detection method based on MSER

Camera parameter updating based on IMU data

Image region matching and obstacle detection

Obstacle detection experiment

Effect analysis of fast road obstacle detection method based on MSER

Analysis of detection accuracy

Analysis of detection speed

Conclusion

Footnotes

Declaration of conflicting interests

Funding

References