Image recognition in online monitoring of power equipment

Abstract

The application of remote digital video surveillance and image recognition technology in online monitoring of power equipment is conducive to timely equipment maintenance and troubleshooting. In order to solve the problem of slow speed and large amount of computation of traditional template matching algorithm for power image recognition, a second template matching algorithm for fast recognition of target image is proposed in this article. Firstly, a quarter of the template data is taken and matched within a quarter of the source image, and a reasonable error threshold is given in the matching process. Then, the neighborhood of the minimum error point in rough matching is matched to get the final result. Finally, the algorithm is applied to identify the power equipment and detect the abnormal state of the power equipment. The experimental results show that the matching algorithm can not only accurately locate and identify power equipment and detect equipment faults, but also greatly improve the matching speed compared with other commonly used template matching algorithms.

Introduction

The rapid growth of the national economy is the rapid growth of industrial and agricultural electricity demand, the shortage of power supply, and the frequent occurrence of power shortage, which has brought unprecedented challenges to the power system. As an important part of the power grid, the normal operation of high-voltage electrical equipment directly affects the safe and stable operation of the power grid. ^1,2 Therefore, we need to monitor the high-voltage electrical equipment to ensure its safe and reliable operation. At present, ^3,4 many power plants and substations have installed a remote video surveillance system. Through the remote video surveillance system, field equipment monitoring, remote camera motion control, digital video recording, and other functions can be realized. However, 24 h of real-time video surveillance is difficult to meet real-time requirements by manual inspection. The human impact of manual inspection will greatly affect the monitoring effect. At the same time, because the monitoring image is too large, and when the surface of power equipment is defective, it is difficult for the human eye to distinguish the subtle changes of the image. Therefore, it is impossible for inspectors to quickly identify the abnormal situation of the equipment, so the situation of false alarm and missed report of the equipment abnormality often occurs. ⁵ How to solve this problem has always been a topic that researchers are working hard to study.

The emergence of image recognition technology fundamentally solves some problems existing in the online monitoring of power equipment. ⁶ Image recognition ^7,8 is an important field of artificial intelligence. It is a technology of object recognition for images to recognize targets and objects of different modes. This technology has been applied to many fields and has achieved certain results. ⁹ In the field of traffic, image recognition technology is used to provide navigation for vehicles; image recognition is used to deal with traffic accidents, to judge whether the traffic is illegal, as well as vehicle statistics on highways, speed measurement, and so on. In the medical field, doctors judge diseases by observing the photographs taken by patients, and then carry out targeted treatment to provide a scientific basis for the treatment of diseases. In the field of agriculture, image recognition technology is used to observe the growth of plants and determine the influencing factors of plants. It can also use image recognition technology to scientifically identify the quality of agricultural products, and then provide scientific guidance for promoting agricultural development. In the field of security, the combination of monitoring system and image recognition technology to realize monitoring automation not only reduces the workload of staff but also improves work efficiency. It also provides a basis for accident liability identification. In addition, with the development of image recognition technology, many researchers have done a lot of research on it. For example, Jia et al. ¹⁰ used image recognition technology to obtain morphological and physical indicators of a single wheat seed, which provided a reference for the research of image recognition technology in crop seed processing technology and accurate selection of parameters. Xia ¹¹ takes Alipay brush face as an example to study the application of image recognition technology in the field of Internet finance. In addition, a great deal of research has been done on the intellectualization of image recognition technology. For example, Sun ¹² has made a detailed exposition of some theoretical breakthroughs in intelligent image recognition technology. Yongmei et al. ¹³ have done a deep research on the computer intelligent image recognition algorithm. Therefore, so far, image recognition technology has been quite mature.

At present, digital image processing technology is also widely used in the power system ¹⁴ ; for example, the application of Luo et al.’s ¹⁵ remote image monitoring in the power system. Liu and Lei ¹⁶ studied infrared image enhancement locomotive power system fault diagnosis and location research. Therefore, the application of image processing technology in the power system is also very extensive. However, the research on combining remote video surveillance and image recognition technology is still in its infancy. For example, Luo et al. ¹⁷ discussed the application of image processing in recognition technology in the remote video monitoring system. At present, many power enterprises have built centralized remote image monitoring system. It can real-time video surveillance for various operating equipment, transmission lines, and so on and then real-time image of each monitoring point is transmitted to the monitoring center or dispatching center through the communication network, so as to real-time monitor the operation status of each power equipment. When the power equipment is in abnormal condition, it can give an alarm in time, and the staff can deal with the abnormal condition of each equipment according to the alarm, so as to ensure the normal operation of the whole system. However, the monitoring system still relies on the staff on duty to observe and analyze the collected images and judge the operation status of power equipment, which lacks the function of automatic identification and analysis of power equipment. In order to improve the ability of automatic identification and analysis of power equipment, Liang ¹⁸ designed a remote video monitoring system with a network hard disk video recorder as the core. The system integrates image technology, pattern recognition technology, and video monitoring technology to realize unmanned monitoring. Sun et al. ¹⁹ put forward the application of remote video monitoring and image recognition technology in the power system for equipment maintenance and troubleshooting. However, in the process of image recognition, most researchers use the traditional template matching algorithm, the traditional template matching uses a template matching, so the speed is slow, the calculation is large, and the matching accuracy is not very high.

Many researchers have also improved the traditional matching algorithm. ²⁰ For example, some researchers have proposed mismatch weighted matching recognition method, central moment invariant template matching recognition method, and template matching algorithm based on vectorization feature extraction. Although they improve the matching accuracy or speed to a certain extent, each image recognition algorithm has its limitations. For example, the mismatch weighting algorithm has a better recognition effect for deformed and damaged images, but it has a bad recognition effect in zoomed and rotated images. The algorithm of central moment invariance has better recognition effect in the case of object scale and rotation. Although the mismatch weighting algorithm has good performance, it is too complex, and the central moment invariants are relatively simple, but the recognition speed is fast. Therefore, all kinds of matching algorithms have their own advantages and disadvantages. ²¹ Different occasions need to choose different recognition algorithms according to their own situation. In view of the shortcomings of traditional template matching, considering the performance and time, this article proposes the algorithm of secondary template matching. For the first time, the improved algorithm performs rough matching in part range. Then after the rough matching, the second accurate matching is carried out. In rough matching, the neighborhood of the point with the smallest error is taken and then matched to get the final result.

Application of image recognition technology in online monitoring of power equipment

In the digital video surveillance system, an image recognition system is added. Firstly, the real-time image is intercepted and saved, then the image preprocessing, image feature extraction, and image recognition are carried out, as shown in Figure 1 and Figure 2.

Figure 1.

Recognition process of video images.

Figure 2.

Preprocessing of image.

Image preprocessing

In the acquisition process, due to the hardware equipment and the external environment, the image often contains some random noise and distortion. Therefore, image preprocessing is needed before image recognition is processed. Image preprocessing ²² includes histogram enhancement, image denoising, image sharpening, image edge detection, image segmentation, and so on.

• (1) Histogram enhancement

The main purpose of enhancing the histogram is to make the image clear. It can balance the histogram of the image to make the gray histogram of the image tend to be uniform in a large dynamic range, so as to increase the image. Assuming that the image is f(x, y), the source image histogram P_r (r) is transformed into a uniformly distributed histogram P_r (s) by gray function s = T[r]. The function s = T[r] is

$s=T\left[r\right]={\displaystyle {\int }_0^r{P}_r(\omega )\text{d}\omega }$

1

where r represents the grayscale of the original image and s represents the grayscale of the histogram after modification, $0\le r,s\le 1$ .

• (2) Image denoising

In general, people use image smoothing technology to remove noise. Common smoothing techniques include median filtering, mask smoothing, spatial low-pass filtering, and so on. Smoothing filtering and median filtering are commonly used.

Smooth filtering mainly uses smooth template filtering. The idea of smoothing template ²³ is to remove the abruptly changed points by averaging the points to be processed and the nine adjacent points around it, so as to filter out some noise. The mathematical expression of smooth template (box template) is

$\frac{1}{9}\left[\begin{array}{ccc}1& 1& 1\\ 1& 1& 1\\ 1& 1& 1\end{array}\right]$

2

For image denoising, we only need to convolution operation between the box template and the original image. ²⁴ Median filtering is a nonlinear processing method for suppressing noise. The basic principle of median filtering is to replace the value of a point in a digital image or sequence with the median value of each point in a neighborhood of the point, so that the surrounding pixel values are close to each other, thus eliminating isolated noise points. The output of two-dimensional median filter is

$g(x,y)=\text{med}\{f(x-k,y-l),(k,l\in W)\}$

3

where f(x, y) is the source image and g(x, y) is the processed image. W is a two-dimensional template, usually in 3 × 3 and 5 × 5 regions. It can also have different shapes, such as linear, circular, cross, circular, and so on. The K and l belong to template W. Figure 3 shows the effect of median filtering on a transformer image.

Figure 3.

The effect of median filtering on the image. (a) Original graph. (b) Image after median filter.

As can be seen from Figure 3, the effect of median filtering is obvious, eliminating a large number of noise scanning lines and outliers in the original image.

• (3) Image sharpening

Commonly used image sharpening methods ²⁵ have differential operator algorithm and Laplasse operator algorithm. One of the commonly used differential methods in image processing is gradient. The gradient is a vector. The gradient of a function at a point is the direction in which the derivative of the function at that point reaches its maximum value. Its modulus is equal to the maximum value of directional derivative. Gradient is often applied to image edge detection and edge enhancement. For a continuous function f(x, y), its gradient at point (x, y) is a vector, defined as

$g(x,y)={[\partial f(x,y)/\partial x,\partial f(x,y)/\partial y]}^T$

4

The magnitude of the point (x, y) gradient is the modulus of the gradient vector

$G(x,y)=|g(x,y)|=\sqrt{{(\partial f(x,y)/\partial x)}^2+{(\partial f(x,y)/\partial y)}^2}$

5

Because of the discreteness of digital image, the differential calculus is approximately replaced by the differential operation. At its pixels (i, j), the first-order difference in the X direction and the Y direction is defined as

$\begin{array}{l}{\text{Δ}}_{x}f(i,j)=f(i,j)-f(i+1,j)\\ {\text{Δ}}_{y}f(i,j)=f(i,j)-f(i,j+1)\end{array}$

6

Therefore, the modulus G(x, y) of the gradient vector can be approximately represented as

$G(x,y)=\sqrt{{[f(i,j)-f(i+1,j)]}^2+{[f(i,j)-f(i,j+1)]}^2}$

7

The gradient method mentioned above is called the horizontal vertical difference method. There is also a gradient method for Robert gradient. Its mathematical expression is

$G(x,y)=\sqrt{{[f(i,j)-f(i+1,j+1)]}^2+{[f(i+1,j)-f(i,j+1)]}^2}$

8

Through gradient calculation, it is found that the gradient value is larger in the edge area where the gray level changes greatly, smaller in the area where the gray level changes gently, and zero in the area where the gray level changes uniformly. The change of the gray value at the edge is very large, so the gradient is the largest.

• (4) Image edge detection

The purpose of edge detection is to identify the points with obvious brightness changes in digital images. In addition, image edge detection can greatly reduce the amount of data, eliminate irrelevant information, and retain the important structural attributes of the image. ^26,27 The classical edge detection method is usually based on the variation of the first or second derivatives of the adjacent edges. The steps of edge detection generally consist of four steps: filtering, enhancement, test, and position. An image of power transformer is detected by edge detection. The effect of edge detection is shown in Figure 4.

Figure 4.

Effect of transformer edge detected. (a) Original graph. (b) Edge detection of transformer.

Here, the Sobel operator is used to detect the edges of transformers directly. As can be seen from Figure 4, the detection effect is good.

• (5) Image segmentation

Image segmentation is the key step from image processing to image analysis. It is the technology and process of dividing the image into several specific and unique regions and proposing interested objects. At present, the existing image segmentation methods in literature ^28,29 are mainly divided into the following categories: threshold-based segmentation method, region-based segmentation method, edge-based segmentation method, and specific theory-based segmentation method. As shown in Figure 5, an image is segmented into several different regions by thresholding.

Figure 5.

Image segmentation effect.

Image feature extraction

At present, the main features of power equipment image ^30,31 are the color, texture, and shape contour of the equipment.

• (1) Color feature analysis

Color feature is a global feature. It is the most abundant information in images. Because different power equipment has different colors, such as red transformer, gray transformer, and so on. For those electric power equipments with obvious color features, we can use color features as the main basis for identifying equipments. For power equipment with significant color features, we can select its color as the main feature vector for recognition and analysis of power equipment.

• (2) Texture feature analysis

Texture feature is also a global feature. Unlike color features, texture features need to be statistically computed in regions containing multiple pixels. This regional feature has great advantages in pattern matching. It cannot be matched successfully because of local deviation. As a statistical feature, texture features have rotation invariance and strong resistance to noise.

• (3) Shape feature analysis

After denoising, enhancement, and edge detection, the image edges and regions of power equipment are obtained, and then the shape of equipment image is obtained. Usually, there are two kinds of representation methods for shape feature, one is the contour feature and the other is the region feature. In this article, we mainly use the global method based on the region. The shape is regarded as a whole based on the regional global method, and the feature quantities describing the shape are extracted, mainly including geometric invariant moments and Zernike moments. The shape representation method based on geometric moment invariants and similarity measures also achieve good results in the process of identifying substation power equipment.

Image recognition

Image recognition ⁷ refers to the technology of using computers to process, analyze, and understand images in order to identify targets and objects of different modes. In order to complete the image, image preprocessing is needed. The feature data of the image is extracted according to the image features, and then the model base is obtained through the training process. Finally, according to the model features in the model base, the image to be detected is searched to find the target to be recognized.

Among the identification methods of power equipment, template matching based on a gray level is one of the most commonly used identification methods of power equipment. It takes the known small target image as the template. When looking for the target image in the source image, it is necessary to compare the template image with the source image to determine whether the source image contains the same (or similar) area as the template, if so, determine its location and extract the area. The basic principle of template matching is to find its coordinate position by calculating the correlation function. As shown in Figure 6, in the source image S, move downward from the upper left corner to search for the full image to see if there is an image of the same size as the template T, and if there is one, take out the image area.

Figure 6.

Standard template matching process.

Let the template $T\left(n\times m\hspace{0.17em}\text{Pixel}\hspace{0.17em}\text{point}\right)$ be stacked on the search graph S and translated. The search graph covered by the template is called subgraph S^i,j , ij is the coordinates of the upper left corner image points of the subgraph Sⁱ , $1\le i,j\le n-m+1$ . Compare the contents of T and S^ij . If the two are in agreement, the difference between T and S^i,j is zero. The formula of measurement is

$\begin{array}{c}D(i,j)={\displaystyle \sum _{m=1}^M{\displaystyle \sum _{n=1}^N[S^{{}_{i,j}}}(m,n)-T(m,n){]}^2}\\ ={\displaystyle \sum _{m=1}^M{\displaystyle \sum _{n=1}^N[S^{i,j}}}(m,n){]}^2-2{\displaystyle \sum _{m=1}^M{\displaystyle \sum _{n=1}^N{S}^{i,j}(m,n)}}\times T(m,n)\\ +{\displaystyle \sum _{m=1}^M{\displaystyle \sum _{n=1}^N[T}}(m,n){]}^2\end{array}$

9

$D(i,j)$ represents the difference between the contents of T and S^ij . The first item is subgraph energy, the third item is template energy, and the second item is the correlation between template and subgraph, which changes with (i, j). When the template matches the subgraph, the item has the maximum value. The correlation coefficient of template matching is obtained by normalizing it

$R(i,j)=\frac{{\displaystyle \sum _{m=1}^M{\displaystyle \sum _{n=1}^N{S}^{i,j}(m,n)}}\times T(m,n)}{\sqrt{{\displaystyle \sum _{m=1}^M{\displaystyle \sum _{n=1}^N[S^{i,j}}}(m,n){]}^2}\sqrt{{\displaystyle \sum _{m=1}^M{\displaystyle \sum _{n=1}^N[T}}(m,n){]}^2}}$

10

If the subgraph and template are exactly the same, the correlation coefficient R (I, J) = 1. After completing all searches, find the maximum value of R, and the corresponding subgraph is the matching target. The algorithm can effectively eliminate the influence of illumination change on template recognition results. But the computation is large and the speed is slow. In order to reduce the computational complexity of template matching and improve the speed and accuracy of template matching algorithm, a second template matching method is proposed in this article. For the first time, a rough match was made in some areas. Firstly, a quarter of the data of the template is taken and matched within a quarter of the source image. In the matching process, a reasonable error threshold D ₀ is given

$D_0=d_0\times \frac{m+1}{2}\times \frac{n+1}{2}$

11

where m represents the length of the template and n represents the width of the template. d ₀ represents the maximum error of each point. After rough matching, second exact matches are made. In the rough matching, the neighborhood with the minimum error is matched to get the final result.

This matching method is based on the absolute error algorithm. In order to measure the error between S^i,j (x, y) and T (x, y), absolute error is used as the error standard and the error formula is

$D(i,j)={\displaystyle \sum _{m=1}^M{\displaystyle \sum _{n=1}^N|}}{S}^{i,j}(m,n)-T(m,n)|$

12

D(i, j) is the $S^{i,j}(x,y)-T(x,y)$ absolute error. In the matching process, when D(i, j) >D ₀, the matching of that point is over. The matching target is D(i, j) as the minimum.

In order to test the performance of the secondary template matching algorithm proposed in this article, the absolute error method, the correlation coefficient R method, and the secondary matching method proposed in this article are respectively used for template matching on the two images, and then the results are compared. The comparison results are given in Table 1.

Table 1.

Speed comparison of three template matching algorithms.

Image size	Template size	Absolute error method (s)	Correlation coefficient R method (s)	Quadratic matching method
640 × 480	12 × 16	1.523	2.393	0.932
640 × 480	24 × 31	2.108	7.361	0.981
640 × 480	36 × 45	4.217	16.602	1.013
640 × 480	48 × 62	5.392	21.433	3.087
200 × 200	12 × 16	1.187	1.974	0.613
200 × 200	24 × 31	1.808	3.752	0.724
200 × 200	36 × 45	3.131	10.724	0.801
200 × 200	48 × 62	5.018	16.831	0.942

It can be seen from Table 1 that among the three algorithms, the matching speed of correlation coefficient R method is the slowest, while that of quadratic matching method is the fastest. The matching speed of absolute error method is also relatively fast, and it is also very fast under certain threshold conditions, but the selection of threshold has certain randomness, so the application of absolute error method is less.

Applied experiment

Experimental data

The remote video monitoring image used in this article is the transformer image of the substation, and the size of the image is 790 × 600. The transformer in the image is sf110-31500/110 produced by Changzhou transformer in Jiangsu Province, which was delivered on New Year’s day in 2000 and put into operation on April 4, 2002. The main structure of the transformer includes oil tank, iron core, coil, conservator, bushing, tap changer, and so on.

Identification experiment of power equipment in substation

Because the background of power equipment image in the substation is complex and the illumination often changes, we can find out the part of the equipment which has some special characteristics as the template feature, and then find the location of the equipment in the image according to the template feature in the image to be recognized. The purpose of this experiment is to give a picture of an insulator cable, and then find out whether the cable image exists in the corresponding transformer image. The recognition process is shown in Figure. 7. According to the template in Figure 7(b), the images in Figure 7(a) are searched sequentially. Starting from the upper left corner, the image in Figure 7(a) is searched to find out whether there is the same target image as the template in Figure 7(b), if so, it is identified. The search results are shown in Figure 7(c). The correlation coefficient R method, absolute error method and quadratic template matching algorithm are used to determine whether there is a target object in the image in the article, where the template size is set to 42 × 156.

Figure 7.

Identification of power transformers. (a) Source picture. (b) Identifying target images.

Anomaly detection experiment for power equipment

In the future, substations are unattended. It is not very meaningful to simply collect and transmit the image signals of substations. The purpose of this experiment is to classify the real-time images collected in the substation, and then decide whether to transmit the monitoring image to the dispatcher according to whether the equipment is abnormal or not. If the result of monitoring image analysis is that the equipment is normal, only the result of analysis is transmitted to the dispatcher; if the substation equipment is abnormal, the abnormal part of the image and the alarm information need to be transmitted to the dispatcher. According to the computer alarm, the dispatcher observes and processes the abnormal parts of the image. The power equipment is identified by means of equipment color characteristics, texture features and template matching. In this article, subtraction is used to detect whether the image of the device and the normal state of the database are changed. The subtraction formula is

$\text{Δ}{P}_i(x,y)=P_i(x,y)-P(x,y)$

13

Among them, the image that needs to be discriminated is the normal state image in the database. If the equipment is normal, $\text{Δ}{P}_i(x,y)$ is 0. If there is damage to the surface of the equipment or oil leakage of the equipment and other faults, there will be protrusions or burrs in the image. If the result of $\text{Δ}{P}_i(x,y)$ calculation is not 0, means transformer leakage detection, as shown in Figure 8.

Figure 8.

Identification of transformer abnormal state. (a) The equipment is in normal condition. (b) The equipment is in the condition of oil leakage.

Analysis of experimental results

Analysis of identification test results of power equipment in the substation

As shown in Figure 7, three cables with the same contents as those in Figure 7(b) are found in Figure 7(a) by the algorithm. The figure shows that the three matching algorithms can identify the corresponding cable image in the transformer image, as shown in Figure 7(c). That piece of cable was identified. But the matching speed of the three algorithms is not the same. The recognition time of correlation coefficient R method, absolute error method, and quadratic template matching algorithm is 8.212, 2.783, and 1.357 s, respectively. The time of quadratic template matching method is about 1/7 of that of R algorithm and about 1/2 of that of absolute error method. The matching speed of quadratic template matching method is faster than that of R algorithm.

Analysis of test results of power equipment anomaly detection

According to the color features and texture features of the equipment, the template matching method is used to detect the power equipment, and then the subtraction formula is used to process the image of the detected area, which is compared with the normal operation image of the benchmark in the database. There is a circular hole in the lower right corner of the third transformer tube in Figure 6(b), which is different from Figure 6(a). So through software calculation, the $\text{Δ}{P}_i(x,y)$ value is not 0, and the change is very large. From this, it can be judged that the oil leakage fault occurred at the transformer oil leakage detection site, and alarm signals are sent out, which requires staff to go to the site for maintenance.

In the detection process, the secondary template matching method is used to detect the power equipment. The experimental results also show that the secondary template matching proposed in this article can accurately detect the corresponding power equipment, which provides sufficient guarantee for the equipment anomaly detection.

Conclusion

Remote digital video surveillance has the function of remote monitoring equipment, but it not only wastes human resources, but also increases the probability of false alarm and false alarm. However, the image recognition system can establish the corresponding decision rules through pattern recognition, catch the abnormal phenomena of the equipment, and send the alarm signal to the supervisor in time. Therefore, the effective combination of remote digital video surveillance and image recognition system can not only achieve real-time monitoring and fault alarm of power transformer operation status but also play an anti-theft and early warning role for power transformer, so as to ensure the safe and stable operation of transformer.

There are three main works in the article:

The whole process of equipment image recognition and anomaly detection by combining remote digital video surveillance and image recognition system is described.

A second template matching algorithm for fast recognition of target image is proposed. Firstly, a quarter of the template data is taken and matched within a quarter of the source image, and a reasonable error threshold is given in the matching process. Then, the neighborhood of the minimum error point in rough matching is matched to get the final result.

The simulation results show that this method can not only accurately identify the location of the target but also reduce the time of the proposed matching algorithm compared with other commonly used template matching algorithms.