Sage Journals: Discover world-class research

Abstract

The details of an onboard visual-based navigation system that allows an unmanned aerial vehicle to perform power line following are presented in this article. A histogram of oriented segments method was developed to establish the main angle orientation of power lines. The histogram of oriented segment is also used to discard segments that are not power lines in the images. The components of the unmanned aerial vehicle flight system which include the platform configuration and the hardware used are also presented. A set of tests were designed to validate the methods and the overall system. The system was tested with a sequence of images taken from different perspectives in real inspection tasks. Tests were performed with images taken using an unmanned aerial vehicle with a camera, pointing vertically downward, and also using different kinds of cameras. The techniques developed for unmanned aerial vehicle navigation were employed over power lines using a multirotor. The results showed that a vision system based on the developed techniques is able to give important information to an unmanned aerial vehicle in order to perform autonomous flight missions.

Keywords

Onboard visual system power line detection visual-based navigation UAV

Introduction

Due to the importance of emergent methods that improve the inspection and maintenance of electrical infrastructure, current trends of research show advances in technologies for power line inspection using computer vision techniques. Unmanned aerial vehicles (UAVs) have received increasing attention and have proliferated due to their low cost, advances in control theory, mechanical design, and suitable power supply. All this has propitiated the development of more stable systems in comparison with the traditional helicopters that consist of more than two propellers, some of them are called quadcopters, hexacopters, and octocopters.

The use of UAVs for inspection tasks is very attractive as they help to prevent risks and minimize costs for capturing images and videos^1,2 in comparison with the use of helicopters. Nowadays, the UAVs are an alternative for aerial electrical infrastructure inspection.³

Most of the work related with visual inspection of electrical infrastructure is based on image processing techniques. Since power lines are fundamental to maintain the electrical supply and distribution, different methods for line detection have been used.^4

–8

Other line detection methods, such as the straight line detection described in the work of Burns et al.,⁹ the line segment detector (LSD)¹⁰ and edge drawing lines (EDLines),¹¹ present good results in terms of accuracy and have been considered an alternative for comparing power line detection methods⁵ Recently, a method for line detection based on integer operations which uses a circle-based search (CBS) is presented in Ceron et al.¹²

The line detection process requires a segmentation stage using different filters such as Prewitt, Sobel, Scharr,¹³ and edge drawing¹⁴ (which is used in the EDLines line detection method). Another option is the use of steerable filters¹⁵ as presented in the work by Liu et al.⁵ A method for background removal with a pulse-coupled neural network (PCNN) filter has been presented in Li et al.¹⁶ This method was improved in Zhang et al.,¹⁷ where the Hough transform and the Kalman filter were used. In the same work, the Otsu filter is employed instead of PCNN with good results. A method for line detection based on region growing, ridge filters, and chain codes was developed in Yao et al.⁷

After the application of a line detection method a post-processing stage can be used. This stage is in charge of connecting segments by using geometrical properties as shown in Liu et al.⁵ Also, this stage can consist of a clustering of related lines as presented in Zhang et al.¹⁷

Additionally, different sensors have been used, such as infrared⁷ and Laser Imaging, Detection and Ranging (LiDAR). Multisource feature fusion information such as point clouds, color, and texture have been used for power line corridor inspection processes that include power line and vegetation¹⁸ Although the use of a combination of different sensors such as LiDAR or RGB-D is highly useful for object detection and recognition, the payload of a UAV is limited. For this reason, this work seeks to explore 2-D image techniques for object detection with common cameras.

Related work

Autonomous visual-based navigation for UAVs is a complex process that requires short computing times and accurate measurements in order to provide suitable and safe control commands to the device. It is clear that UAV navigation requires real-time measurements to produce a response within a specified time (at least 100 ms), otherwise severe consequences including failure may affect the device. The evaluation of the performance of a visual-based navigation system for power line following is presented in simulation for a fixed wing UAV in some works for example in the study by Mills et al.¹⁹ In a previous work, the simulation of a visual-based navigation process for power line following in a 3-D environment using a closed-loop control is presented in Ceron et al.²⁰

The relevance of visual-based navigation methods has been proved viable as shown in works for example in the study by Sanfourche et al.²¹ Also, in the work of Sa et al.,²² high-speed vision for pole inspection was used. This is a case where the GPS information is not enough and unsuitable for close fly navigation. Additionally, in the work of Zarandy et al.,²³ a real-time multi-camera vision system based on Field-programmable gate array implementation is presented. These examples show the advantage of the vision systems for UAV autonomous navigation.

The analysis of the line orientation from images taken with a camera pointing vertically downward is valuable (see Figure 1) for the detection of power lines and for correcting the UAV orientation when it is navigating autonomously.

Figure 1.

Images of power lines taken from a top-down view.

Problem and main contributions

In this work, a robot operating system (ROS)-based environment was used. It is key to highlight, the evaluation with a real multirotor UAV of a power line following system represents an important concern in the area of robotics.

The main contribution presented in this article is the development of a set of techniques to detect line features and this includes an adapted version of the CBS algorithm, a method to obtain the power line orientation defined as a histogram of oriented segments (HOS), and a process to detect power lines. Another contribution is the design and development of a system that performs onboard visual-based navigation for power line following using the developed techniques.

The remainder of this article is organized as follows: Line detection methods are introduced in the “Power line detection using CBS” section. HOS is described in the “Histogram of oriented segments” section. The autonomous navigation system architecture is illustrated in the “Autonomous navigation system” section. Experiments and results are presented in the “Experiments and results” section and finally, the conclusions and future work are presented in the “Conclusions” section.

Power line detection using CBS

In Ceron et al.,¹² a line detection method called CBS was presented. This process shows a favorable performance compared to Hough, LSD, and EDLines. In this work, we extend the process to power line orientation and UAV navigation. The process used a discrete circle algorithm to generate positions for line searching.

The CBS method is presented in Algorithm 1. In this algorithm, the method ValidPoint is in charge of searching in the range of possible line orientations [0, 2π] with the values of Dx and Dy, where there are more points that belong to a line. This is done using a drawing circle algorithm to generate the positions of points by symmetry. The method NextCircle is in charge of finding the next valid points in the same direction detected with the values of Dx and Dy in order to move to the end of the line (see Figure 2). Also, r is the actual radius that is increased by incr in order to repeat the searching with different radii. It is good to note that the points that belong to a detected line are deleted. The output of the algorithm is a list of segments. Each segment has an initial point (x_i,y_i ) and final point (x_f,y_f ).

Algorithm 1.

LocateLine.

Figure 2.

CBS process. CBS: circle-based search.

The main difference with the original version presented in Ceron et al.¹² is that the linking stage is avoided in order to reduce the computational cost. Instead of a linking stage an HOS is used.

The results of different state-of-the-art line detection algorithms using the navigation images of this work have been added in Table 1. In this table, we can observe that the CBS method is competitive with respect to the other methods.

Table 1.

Computation times of line detection methods in miliseconds.

Method	Hough	LSD	EDLines	CBS
Mean	38.6	136.4	20.8	20.3
Standard deviation	4.5	10.1	2.7	2.4

LSD: line segment detector; EDLines: edge drawing lines; CBS: circle-based search.

In Table 2, a comparison between different methods for line detection using 14 images is presented. In the used images, three power lines are presented and the number of false positives (FP), true positives (TP), false positives (FP), and false negatives (FN) are obtained by counting each segment detected. In this table, it is remarkable that the number of FP is greater in the LSD method, since it detects a lot of small segments. The CBS presents a good balance in short time compared with the number of FP compared with the other methods.

Table 2.

Quantitative results of line segment detected for 14 images.

	Hough			LSD			EDLines			CBS
Image	TP	FP	FN	TP	FP	FN	TP	FP	FN	TP	FP	FN
1	19	5	0	25	136	0	15	90	0	36	8	0
2	17	8	0	154	30	0	30	86	0	12	22	0
3	10	6	0	38	102	0	33	138	0	11	0	0
4	14	1	0	84	31	0	10	35	0	17	0	0
5	5	0	0	83	36	0	7	39	0	45	7	0
6	21	30	0	54	158	0	23	133	0	20	58	0
7	17	8	0	31	115	0	10	114	0	24	4	1
8	18	6	0	17	218	0	13	123	0	27	8	0
9	11	1	1	61	110	0	31	16	0	16	3	0
10	19	1	0	82	41	0	10	24	0	32	1	0
11	13	0	0	50	47	0	7	51	0	24	4	0
12	17	0	0	47	27	0	10	36	0	13	0	0
13	13	9	0	51	147	0	49	94	0	10	6	0
14	16	3	0	33	61	0	12	98	0	30	3	0
Total	210	78	1	810	1259	0	260	1093	0	317	119	1

LSD: line segment detector; EDLines: edge drawing lines; CBS: circle-based search.

Precision Recall: Using the total of TP, FP, and FN for each line detection method. The precision and recall values were obtained in testing by using the follow expressions

Precision = \frac{TP}{TP + FP}

Recall = \frac{TP}{TP + FN}

In Table 3, the average of precision and recall values is shown. The CBS method shows a good precision which is similar or superior to the other methods.

Table 3.

Precision and recall for line detection methods.

	Hough	LSD	EDLines	CBS
Precision	0.72	0.39	0.19	0.72
Recall	0.99	1	1	0.99

LSD: line segment detector; EDLines: edge drawing lines; CBS: circle-based search.

Nevertheless, the improvement of this performance requires the use of more advanced methods related with computational intelligence that are proposed as a future work.

Histogram of oriented segments

According to Ceron et al.,¹² in a power line detection process using a camera pointing vertically downward, different characteristics can be highlighted aiming at improving the detection such as length of the line (the longer segments should have more importance than smaller ones), direction with higher number of long lines (a histogram of orientations is considered), and structure of lines (power lines present a pattern of parallel lines in a top-down view).

Following the abovementioned characteristics, in this work, a measure based on the linear content of the image is proposed. The measure is obtained from an HOS that has three purposes: First, obtaining the orientation angle of the power lines for UAV navigation. Second, selecting the power lines in the scene. Third, defining a way to differentiate between a scene with and without power lines.

As a prerequisite, the image has to be taken, when possible, from a top-down view, and the radial distortions must be corrected. It is assumed that the main linear components in the scene are power lines. This histogram is defined in the interval from 0° to 180°. The main idea is to obtain a histogram of line segment angles that are pondered with the length of each line.

Let (x_i,y_i ) be the coordinates of the initial point and (x_f,y_f ) the coordinates of the end point of a k segment.

The following are the steps to obtain the HOS:

Obtain lines in the overall image. The segments detected are collected in a list of M elements.

Create an array H of 180 elements which corresponds to angles in the interval [0°, 180°).

Initialize the array with zeros.

For each line segment line_k

dx = x_f − x_i .

dy = y_f − y_i .

length = norm(line_k).

θ = ComputeAngle(dy, dx).

Correct θ to interval [0°, 180°).

Use this angle as an index in the histogram and accumulate the length of line.

H[θ]+ = length.

The ComputeAngle(dy, dx) function is obtained by using the atan2(dx, dy) function and is corrected in order to have angles within the interval [0°, 180°) in the histogram

angle = {\begin{array}{l} θ - 180 & if θ \geq 180 \\ θ + 180 & if θ < 0 \end{array}

Main angle

Due to the fact that the HOS contains the most representative angles that are present in the scene, it is possible to get the maximum values. These values are an estimation of the line orientation. After the histogram is obtained, the main angle θ _main is extracted as the index which corresponds to the maximum of the histogram as shown in the following

arg max_{θ} H [θ] = {θ | H [θ] = max_{θ^{'}} H [θ^{'}]}

After this, the angle range is transformed to the range [−90°, 90°] in order to use this as a control variable in the visual navigation system as shown in Figure 3. The correction of range is given by θ = −(90 − θ _main).

Figure 3.

Main angle of power lines from a top-down view.

In Table 4, the main angles for the set of images in Figure 1 are presented.

Table 4.

Main angle detection in images of Figure 1.

Image	1	2	3	4	5	6	7	8
Angle (°)	−1	8	6	−41	−3	−5	22	28

Once the main angle θ is obtained, the image can be segmented by selecting lines with similar angles. This way, only lines with angles close to the main angle are selected.

The algorithms were tested in different rural scenarios where it is possible to find different man-made objects such as water channels, roads, and fences. For instance, in Figure 1(g), the power lines intersect a set of fences. In this case, the angle of the power lines is correctly detected as can be seen in the eighth column of Table 4. The developed method is able to detect power lines from a top-down view.

HOS as a global descriptor

The information obtained with the HOS is useful not only for obtaining the main angle of the power lines but also to discriminate whether power lines exist in the scene. This is done by comparing the maximum value of the histogram with a threshold. Angle estimations are selected only when the maximum of the histogram is greater than a threshold which is obtained from the data set.

Autonomous navigation system

UAV system description

In this work, a low-cost hexacopter platform-type DJI 550 that was adapted to carry the equipment is used (see Figure 4). In order to enable the navigation tasks that include vision processing and position control, an onboard companion computer was employed. We used an ODROID U3 computer with an ARM processor and a 32 bits Pixhawk flight controller of controlled by the companion computer. The system runs under linux Ubuntu 14.04. OpenCV is used for implementing computer vision algorithms and ROS Indigo to integrate the robotic platform.

Figure 4.

UAV platform and flight tests. (a) Hexacopter, (b) camera pointing vertically downwards, and (c and d) test flight. UAV: unmanned aerial vehicles.

For the vision system, two USB cameras were tested. An action camera Mobius (equipped with wide-angle lens) and an ELP industrial camera USBFHD01M (see Figure 4(b)).[Please provide manufacturer’s details for “ELP industrial camera USBFHD01M,” if applicable.] Although it is possible to obtain the calibration parameters of the action cameras it is a challenge to reduce the radial distortion. This makes it difficult to detect rectilinear objects since they generate curves instead of lines. The detected lines appear as curves, specially at the extremes of the image. Nevertheless, with the main angle obtained from our line detection method, it is possible to get an estimation of the line orientation.

In order to get better results in terms of the rectification of images an ELP camera was used. This device provided images that are suitable for the line detection process plus a frame rate that permits closed-loop control. The UAV system consisted of a set of related components that are shown in Figure 5. The main components of the system are the UAV flight platform, the flight controller, the sensors that include GPS, IMU (Inertial Measurement Unit, that includes three axes magnetometers, gyroscopes, accelerometers, and compass), camera, and the onboard computer to run the developed software. In this system, the flight controller receives setpoints from the onboard computer and sends sensored information to it. The vision sensor (camera) sends frames of images to the onboard computer.

Figure 5.

UAV system. UAV: unmanned aerial vehicles.

System architecture

In this work, tasks involved in a mission of power line navigation are coordinated by a task manager (see Figure 5). The task manager receives data through sensors that include an accelerometer and a GPS. Also, it uses high-level information obtained from the image provided by the camera. The information extracted by the developed vision module using the proposed algorithms is an input for the control in the task manager.

A position control interface for the Pixhawk flight controller is used; this interface receives the desired commands from the task manager during the UAV mission.

For the implementation, a set of ROS-based modules have been developed. The most representative procedures are explained in Algorithms 2, 3, and 4. The main modules are the following:

Vision module: This module performs the video capture, image smoothing, camera calibration, edge segmentation, line detection, histogram of oriented lines, and computing of the main angle.

Navigation task manager: This module determines the set of autonomous actions to be performed by the UAV during a flight mission. The main tasks are Take off, Ascend (vertical flight), Rotate, Move_forward, Move_until (see Algorithm 2), Align (see Algorithm 3), Follow lines (see Algorithm 4), Return, Descend, and Land.

Algorithm 2.

Move until.

Algorithm 3

Align.

Algorithm 4.

Follow_lines.

In these algorithms, the initial yaw orientation of the UAV with respect to the north is θ ₀. The angle variable is the orientation in UAV coordinates. In the alignment algorithm, the control loop aims to reduce the angle error. This loop continues until the absolute value of error is less than a threshold ε. In the designed high-level architecture, a proportional control is implemented based on the main angle of the detected lines in order to align and follow lines. The scheme of the visual servoing system is presented in Figure 6. An image-based visual servoing was implemented. This control uses the angle obtained in the vision module. In a low-level part, a position control receives the reference for power line following to keep a safe flight.

Figure 6.

Control system.

This kind of control was used because the addressed problem considers only power line detection from a top-down view; this is to simplify the flight strategy and avoid collisions. In Figure 6, the computer vision techniques are located in the block (CV) that receives the image from the camera.

The increment used in each iteration is defined with the r variable; an incremental error is considered. The function Set(x, y, z, angle) mentioned in the algorithms is used to send a position setpoint and a yaw angle to the UAV. The position (x, y, z) is updated in the algorithms: Move_until and Follow_lines.

Experiments and results

Several experiments have been conducted in order to validate the proposed approach for power line following that includes the evaluation of CBS line detection method, the HOS algorithm, and the UAV autonomous system.

Three types of experiments were performed. First, using images from a real power line inspection in an off-line process. Second, using a video taken with an action camera. Third, using an embedded real-time system to accomplish the autonomous visual-based navigation strategy of power line following with the presented algorithms.

Data set

Three data sets of power lines captured using aerial vehicles were employed. First, a subset of 800 images from a video sequence of a real inspection task taken from a helicopter. This information was used to compute the main angle with the HOS. Second, a set of 1097 images of a video sequence are taken with a UAV with a camera pointing vertically downward in continuous flight during the line following process.

First test: Data collection of power lines inspection

A set of images taken from different perspectives of a power line inspection process in a continuous flight are used. In Figure 7, some images are shown.

Figure 7.

Some images from a power line inspection sequence.

The results of the main angle calculation obtained in a sequence of frames from an electrical inspection are shown in Figure 8. In this figure, the vertical axis is the detected angle (in degrees) and the horizontal axis is the number of the frame. Also, four images related to the approximate angle orientation are shown.

Figure 8.

Angle obtained in a sequence of images from a perspective view.

It is relevant to note that the presented process showed a good estimation of the power line orientation since the images in this sequence were taken from different perspectives and backgrounds. The variation in the results among consecutive frames oscillates around 5°.

Taking into consideration that the process shown in Figure 8 has the purpose of angle detection in a frame sequence that contains power lines taken from a perspective view, each frame contains a different view of the objects in the scene. The changes in camera pose due to the movement modify the linear content at the scene. This makes that some edges can be more distinctive at different times then some changes are produced in the detected lines. As a result, the detected orientation angle oscillates from frame to frame as it is shown in the mentioned figure.

For this kind of behavior, it is possible to use a moving average filter in order to reduce the variations in the control signal as it is shown in Figure 9. The average filter is shown in the following expression

y [n] = \frac{1}{5} \sum_{k = - 4}^{k = 0} x [n + k]

Figure 9.

Filtered signal of the angle obtained.

Due to the fact that obtaining the main angle from a perspective view is a particular case with many oscillations, the next tests use sequences taken from a top-down view.

Second test: Line detection using a video taken with an action camera

In this experiment, an action camera pointing downward was attached to the UAV in a manual flight. For this, the CBS line detection algorithm was validated in a video sequence of 2 min. The frames had to be rectified using a camera calibration toolbox. The result of the HOS method applied to some frames of an action camera video of the main angle detection is shown in Figure 10, where the main angle is represented with a blue line in the center of the image. The rest of segments are colored with different colors in order to differentiate from each other. In spite that the corrected image shows some curvature distortion, the algorithm was able to detect the power lines in the scene and reject the others as can be seen in the following video:

Main angle detection with an action camera [https://youtu.be/RsKAst6o-0o].

Figure 10.

Results of HOS in some frames taken with an action camera. (a) frame 1, (b) frame 2, (c) frame 3, and (d) frame 4. HOS: histogram of oriented segments.

As it can be seen in the video, the proposed method is able to detect the main direction of the power lines, since the accumulator used gives important information about the principal segments with the same direction in the scene. It is good to note that although the frames of the video present many different objects such as parts of a water channel, a pole, and trees that can be detected as line segments, the system is able to detect the main angle of the power lines accurately.

Third test: Lines following using an onboard vision system

The proposed approach was tested with the companion computer equipped with the autonomous navigation system. The system included the CBS algorithm and the HOS process execution. With the onboard vision system, the UAV performed a navigation over power lines. The UAV had to make a horizontal flight to detect the lines with the camera pointing vertically downward. The camera shots frames with a resolution of 640 × 480 pixels. As it is mentioned in Table 1, the CBS algorithm takes 20 ms aproximately with this image size. The HOS process takes less than 5 ms to obtain the main angle. An experiment of a power line following mission was prepared. The mission consisted of the following tasks: take off, ascend, move_until, align, follow_lines, rotate (180°), follow_lines, rotate (−90°), move_forward (the same distance required by UAV to reach the power lines from the origin), descend, and land as shown in Figure 11.

Figure 11.

Autonomous mission process stages.

A data logger was used to record the information that included the position and captured images in each flight. Four flight missions were performed in different days. The HOS is used to detect if power lines are present in the scene. The UAV flight forward continues until the power lines are detected n times. The number n is used to score the number of detections. When n is higher than a threshold (e.g. four detections), the alignment process is initiated. The alignment process continues until the alignment error in the image is less than 5°. After that, the following lines process is started.

In this test, the results of different autonomous navigation missions are presented. In Figure 12, some images obtained and recorded with the data logger during the process of an autonomous navigation mission are shown. The blue lines represent the detected orientation in each image.

Figure 12.

Complete autonomous power line following mission. (a) Take off, (b) ascend, (c) move_until, (d) detect, (e–h) align, (i, j) follow, (k) return, and (l) descend and land.

The flight information recorded with the data logger is presented in Figure 13. Here, four trajectories of power line following are shown (trajectory in blue and power lines in red). In first place, we tested a mission composed of the following stages: take off, ascend, move_until, align, follow_lines, and return (the return stage was performed manually) as it is shown in Figure 13(a) and (b), from a perspective view.

Figure 13.

Trajectories obtained in two different missions of four flights. The positions in these figures have been measured in meters. (a) Test 1, (b) test 2, (c) test 3, (d) test 4, (e) upside view test 3, and (f) upside view test 4.

In second place, we tested a completely autonomous mission of power line following consisting of the stages presented in Figure 11. One of the main purposes of the stages prepared was that the UAV returns to its initial take off location. The results of two autonomous missions are shown in Figure 13(c) and (d), from a perspective view, and in Figure 13(e) and (f), from an upside view. As it can be seen the UAV satisfactorily performed the complete process using the onboard companion computer with no human intervention during the flight. Such achievement was done using our line detection and main angle detection methods.

This process can be seen in the following videos:

Test flight [https://youtu.be/c60UfQ7FKyI].

Data logger [https://youtu.be/pa4fqoGJmDU].

In the last video, the lines appear in the image during the autonomous process of power line following.

Finally, a linear regression using singular value decomposition was computed with the 3-D positions of the autonomous trajectory of power line following of the visual-based control. The results of the linear regression are shown in Figure 14.

Figure 14.

Linear regression of a power line following path (trajectory in blue and regression in red). (a) Following 1, (b) following 2, (c) following 3, and (d) following 4.

As a result a 3-D line is obtained in the parametric form (see equation (6))

\vec{p} = {\vec{p}}_{0} + \vec{v} t

where $\vec{p} = [x, y, z]$ , $\vec{v} = [v_{x}, v_{y}, v_{z}]$ , and t ∈ [0, 1]. The distance from each point of the obtained trajectory to the parametric line is calculated as it is shown in equation (7), in this case v is a normalized vector with the line direction

{dist}_{i} = | {\vec{p}}_{0} - {\vec{p}}_{i} - (({\vec{p}}_{0} - {\vec{p}}_{i}) \cdot \vec{v}) \vec{v} |

where i is an index for the points in the trajectory.

The results of the average error calculation (see equation (8)), expressed in meters, for each trajectory of N points (3-D positions) and for each one of the four missions, are shown in Table 5. The obtained error gives a measurement of the variations in the desired trajectory. This error depends on different aspects such as wind presence, camera resolution, and the delay in response of the control system. This error can be reduced by using a camera with better resolution and an onboard computer with better configuration for performance

E_{avg} = \frac{1}{N} \sum_{1}^{N} {dist}_{i}

Table 5.

Average error of each mission in meters.

Mission	1	2	3	4
Average error	0.1937	0.0437	0.0880	0.2191

On the other hand, a measurement of the location of lines in the images was obtained in order to quantify the offset of the detected line to the center of the image. This distance is considered as an approximation of the error in each frame.

The error in pixels of the line following process in an autonomous mission was less than the 18% of the width of the image. In Figure 15, an example of the measured Euclidean distance in pixels is shown.

Figure 15.

Distance between power lines and the center of the image.

Conclusions

The monitoring process of electrical infrastructure requires the development of automated non-intensive methods of inspection that allow the evaluation of the actual state of lines without expensive or dangerous interventions over the installation surroundings.

The HOS permits computing the direction of linear content in images. The main peak of this histogram is an estimation of the orientation of power lines in countryside environments. Since many of these lines are presented in the environment surrounded by trees and other types of vegetation, histograms can be used not only to obtain an orientation but also for segmenting the lines that are related with the main orientation angle. Additionally, the HOS can give an estimation of the presence of power lines in the image.

The results shown in Figure 8 indicate that the line orientation help the UAV to remain in a trajectory following power lines. An additional advantage of the proposed methods is that the computing time of the detection algorithms (see CBS method and HOS) for different types of images (including from UAV or aerial inspection) is short. The use of this information is very valuable for the UAV navigation system.

The proposed navigation system enables the UAV to perform an autonomous flight without any manual intervention of an operator (or pilot) using the proposed techniques that include the CBS line detection method and the HOS method with a low-cost UAV.

From the experimental work, we conclude that power lines are very difficult to detect with the same accuracy in all frames of a video. This is due to different reasons: There are thin elements which are affected by the abrupt movements of the UAV, the difference of backgrounds found while flying, and illumination changes.

As future work, a technique of power line detection based on a machine learning approach could be useful to deal with complex scenes in which edge detectors are not effective. The development of a system with a fixed wing platform could be useful for long-distance inspections. Finally, the study of the effects of oscillation of the detected main angle can be considered in order to improve the control strategy using methods such as filtering.

Footnotes

Acknowledgements

The authors appreciate the collaboration of Professor Luis Mejias of Queensland University for sharing a data set of power line images. The authors appreciate the collaboration of Professor Pascual Campoy of the Universidad Politecnica de Madrid for his valuable recommendations and his Computer Vision Group.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Vicerrectoria de Investigaciones - Universidad Militar Nueva Granada through the INV ING 2120 project.

ORCID iD

Alexander Cerón

References

Walker

Hayward

. Advances in vegetation management for power line corridor monitoring using aerial remote sensing techniques. In: Proceedings of the first international conference on applied robotics for the power industry (CARPI), Montreal, QC, Canada, 5–7 October 2010, IEEE. ISBN: 9781424466351, pp. 1–6. DOI: 10.1109/CARPI.2010.5624431.

Sheng

Liu

. Inspection Based On Unmanned Aerial Vehicle. In: Power and energy engineering conference (APPEEC), Shanghai, China, 27–29 March 2012, pp. 1–5. ISBN: 9781457705472.

Wang

Chen

Wang

. Power line inspection with a flying robot. In: 2010 first international conference on applied robotics for the power industry (CARPI 2010), Montreal, QC, Canada, 5–7 October 2010, ISBN 978-1-4244-6633-7, pp. 1–6. DOI: 10.1109/CARPI.2010.5624430.

Hough

. Method and means for recognizing complex patterns. US Patent: 3,069,654, 1962.

Liu

Mejias

. Fast power line detection and localization using steerable filter for active UAV guidance. In: 12th international society for photogrammetry & remote sensing (ISPRS2012), Melbourne Convention and Exhibition Centre, Melbourne, 25 August–1 September 2012, vol. 39, pp. 491–496.

Liu

Mejias

. Real-time power line extraction from unmanned aerial system video images. In: Second international conference on applied robotics for the power industry (CARPI), Zurich, Switzerland, 11–13 September 2012, pp. 52–57.

Yao

Guo

Zhao

. Power line detection based on region growing and ridge-based line detector. In: Sun

Deng

(eds) Chinese intelligent automation conference, Lecture notes in electrical engineering, vol. 255. Berlin, Heidelberg: Springer, pp. 431–437. DOI: 10.1007/978-3-642-38460-8. ISBN: 978-3-642-38459-2.

Song

. Power line detection from optical images. Neurocomputing 2014; 129: 350–361. DOI: 10.1016/j.neucom.2013.09.023.

Burns

Hanson

AR,

Riseman

. Extracting straight lines. IEEE Trans Pattern Anal Mach Intell 1986; PAMI-8(4): 425–455. DOI: 10.1109/TPAMI.1986.4767808.

10.

Grompone von Gioi

Jakubowicz

Morel

. LSD: a fast line segment detector with a false detection control. IEEE Trans Pattern Anal Mach Intell 2010; 32(4): 722–32. DOI: 10.1109/TPAMI.2008.300. URL http://www.ncbi.nlm.nih.gov/pubmed/20224126.

11.

Akinlar

Topal

. EDLines: a real-time line segment detector with a false detection control. Pattern Recognit Lett 2011; 32(13): 1633–1642. DOI: 10.1016/j.patrec.2011.06.001.

12.

Ceron

Prieto

Mondragon

. Power line detection using a circle based search with UAV images. In: 2014 international conference on unmanned aircraft systems (ICUAS), Orlando, FL, USA, 27–30 May 2014, ISBN: 9781479923762. IEEE.

13.

Canny

. A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell 1986; PAMI-8(6): 679–698.

14.

Topal

Akinlar

. Edge drawing: a combined real-time edge and segment detector. J Vis Commun Image Represent 2012; 23(6): 862–872. DOI: 10.1016/j.jvcir.2012.05.004.

15.

Freeman

Adelson

. The design and use of steerable filters. IEEE Trans Pattern Anal Mach Intell 1991; 13(9): 891–906.

16.

Liu

Hayward

. Knowledge-based power line detection for UAV surveillance and inspection systems. In: 2008 23rd international conference image and vision computing New Zealand, Christchurch, New Zealand, 26–28 November 2008, p. 1–6. DOI: 10.1109/IVCNZ.2008.4762118.

17.

Zhang

Liu

Wang

. High speed automatic power line detection and tracking for a UAV-based inspection. In: International conference on industrial control and electronics engineering, Xi’an, China, 23–25 August 2012, ISBN: 9780769547923. DOI: 10.1109/ICICEE.2012.77.

18.

Bruggemann

Ford

. Toward automated power line corridor monitoring using advanced aircraft control and multisource feature fusion. J Field Robot 2012; 29(1): 4–24. DOI: 10.1002/rob.

19.

Mills

Ford

Mejias

. Vision based control for fixed wing UAVs inspecting locally linear infrastructure using skid-to-turn maneuvers. Journal of Intelligent & Robotic Systems 2011; 61(1–4): 29–42.

20.

Ceron

Modragon

IF,

Prieto

. Towards visual based navigation with power line detection. In: Proceedings of ISVC 2014, Lecture Notes in Computer Science, vol. 8887, 2014, pp. 827–836. Springer.

21.

Sanfourche

Delaune

Le Besnerais

. Perception for UAV: vision-based navigation and environment modeling. Aerosp Lab 2012; (4): 1–19.

22.

Hrabar

Corke

. Outdoor flight testing of a pole inspection UAV incorporating high-speed vision. Cham: Springer International Publishing, pp. 107–121. ISBN 978-3-319-07488-7, 2015. DOI: 10.1007/978-3-319-07488-7_8. URL

23.

Zarandy

Nemeth

Nagy

. A real-time multi-camera vision system for UAV collision warning and navigation. J Real-Time Image Process 2016; 12(4): 709–724. DOI: 10.1007/s11554-014-0449-3.

Onboard visual-based navigation system for power line following with UAV

Abstract

Keywords

Introduction

Related work

Problem and main contributions

Power line detection using CBS

Histogram of oriented segments

Main angle

HOS as a global descriptor

Autonomous navigation system

UAV system description

System architecture

Experiments and results

Data set

First test: Data collection of power lines inspection

Second test: Line detection using a video taken with an action camera

Third test: Lines following using an onboard vision system

Conclusions

Footnotes

Acknowledgements

Declaration of conflicting interests

Funding

ORCID iD

References