Detection of smoke from infrared image frames in the aircraft cargoes

Abstract

The existing equipment of civil aircraft cargo fire detection mainly uses photoelectric smoke detectors, which has a high false alarm rate. According to Federal Aviation Agency’s statistics, the false alarm rate is as high as 99%. Since, in the cargo of civil aircraft, visible image processing technology cannot be used to detect smoke in the event of a fire due to the closed dark environment, a novel smoke detection method using infrared image processing technology is presented. Experiments were conducted under different environment pressures in the full-size cargo of civil aircraft. The results show that the proposed method can effectively detect smoke at the early stage of fire which is applicable for fire detection in civil aircraft cargoes.

Introduction

Cargo fire detection is an important guarantee for the flight safety of civil aircraft. In recent years, flight safety accidents caused by false alarms in aircraft cargo fires have emerged one after another. ¹ According to the Federal Aviation Agency (FAA) Technical Center, only one of every 200 fires is a real fire. ² The existing equipment of civil aircraft cargo fire detection mainly uses photoelectric smoke detectors ³ which are effective for the early detection ⁴ of the fire. Since the smoke particle size of the cargo fire is related to the combustion material, the photoelectric smoke detector cannot accurately distinguish the smoke particles and the dust in the air as well as the goods with floating characteristics such as feathers. ⁵ Visual images for smoke detection cannot be achieved due to the dark cargo environment. Hence, infrared image frames considering the visual smoke detection ⁶ algorithms are used to obtain smoke features. Due to complexity and cost, there is no infrared camera for smoke detection in the cargo hold of aircraft in the world. Therefore, this article studies the infrared detection methods from the perspective of reducing false positives and makes preparatory tests for infrared detection and refitting aircraft. Because the high-altitude variable pressure environment will directly lead to changes in oxygen concentration, it will have a greater impact on the combustion rate and smoke characteristics of different materials.^7,8 We also take into account the pressure variance during the experiments, and the variable pressure environment is established in the full-size simulated cargo hold.

Infrared image frames captured from the aircraft cargoes are divided into blocks using the conventional vision-based smoke detection techniques. The outstanding features are extracted and employed to classify the blocks into smoke and non-smoke. ⁹ A multiple dictionaries–based method with respect to the atmospheric pressure is proposed for features extracting. First, based on the atmospheric scattering model, ¹⁰ an infrared smoke image formation model is established. The model believes that infrared images can be segmented into linear combinations of smoke and non-smoke areas. Over-complete dictionaries ¹¹ are used to achieve the sparse representations of smoke component and non-smoke component in one block of the frame which leads to a convex optimization problem. To solve this problem, dictionaries learning process for infrared smoke image frames is conducted in an enclosed air cargo under different pressure conditions and trained with real samples to accommodate different image content. Thus, the sparse representations of two components with relation to the atmospheric pressures can be achieved and imported to the classifier. The innovation of this article lies in two points, one is the infrared image method in smoke detection and the other is the experiments under different pressure parameters which influence the oxygen concentration during the burning process.

Smoke detection method

In visible light imaging systems, smoke acts like the scattering medium such as fog and haze. Much differently, in infrared imaging systems, smoke acts like a heat sink and occlusion medium. Inspired by the physics-based visible light image formation model, ¹² an infrared smoke image formation model is proposed. After eliminating the non-uniform distortions, the infrared image is divided into blocks and processed separately within one block. The observed intensity of the block at pixel (x, y) can be expressed as

$$F(x,y)=S(x,y)+\psi (\tau ,p,x,y)B(x,y)$$

(1)

In equation (1), the image brightness at location (x, y) is expressed as the sum of the two variables S and ψB. The parameter τ indicates the density of the smoke and p is the atmospheric pressure inside the aircraft cargo. S(x, y) represents the brightness of smoke which is considered to be unobstructed. If the background component B(x, y) is not covered by smoke, equation (1) reduces to F(x, y) = S(x, y) + B(x, y), which means the pixel intensity is the direct sum of the thermal radiation energy of two components. If B(x, y) is blocked by the front smoke, part of the thermal radiation energy may be blocked while the residues may be absorbed by the front smoke. This situation is reflected by the blocking coefficient ψ(τ, p, x, y) here which is supposed to be the function of the smoke density and the atmospheric pressure which affects the average particle size of smoke. It is assumed that in each block parameters, τ and p are constants so that the coefficient ψ is constant within one block. A multiple dictionaries method based on different atmospheric pressures using sparse representation of the smoke is proposed. Let f be a given infrared image with N pixels, s and b be the corresponding smoke and background components satisfying {f, s,b∈ RN}. Then, the infrared image formation model described in equation (1) can be written as

$$\mathbf{f}=\mathbf{s}+\phi \mathbf{b}+\mathbf{n}$$

(2)

where $n\in R^N$ represents the modeling noise plus by the thermal noise and φ is the blocking coefficient within one block f. Thus, the modeling process can be formulated as the minimization of the residual noise

$$min\Vert \mathbf{f}-\mathbf{s}-\phi \mathbf{b}{\Vert }_2^2\phi \in [0,1]$$

(3)

Given certain constrains, equation (3) can be solved to obtain a good estimation of b, s, and φ. The sparse representation of the pure smoke component is

$$\mathbf{s}={\mathbf{D}}_{1p}{\mathbf{x}}_{1p}\Vert {\mathbf{x}}_{1p}{\Vert }_0\le L_1$$

(4)

where x_1p is the sparse coefficients and D_1p is the dictionary for pure smoke component under the atmospheric pressure p. The background component can be sparse represented by

$$\mathbf{b}={\mathbf{D}}_{2p}{\mathbf{x}}_{2p}\Vert {\mathbf{x}}_{2p}{\Vert }_0\le L_2$$

(5)

Similar to equation (4), the dictionary D_2p and the corresponding sparse coefficient x_2p are multiplied to form the background component b. Here, L₁ and L₂ are the upper bounds for numbers of the non-zero term with respect of x_1p and x_2p. Then, equation (3) can be rewritten as

$$y=min\Vert \mathbf{f}-{\mathbf{D}}_{1p}{\mathbf{x}}_{1p}-\phi {\mathbf{D}}_{2p}{\mathbf{x}}_{2p}{\Vert }_2^2+a_1\Vert {\mathbf{x}}_{1p}{\Vert }_1+a_2\Vert {\mathbf{x}}_{2p}{\Vert }_1$$

(6)

where a₁ and a₂ are regularization parameters of the convex approximation of x_1p and x_2p with the first-order norm. The objection function y in equation (6) can be solved by sparse coding algorithms ¹³ for x_1p and x_2p. The optimal results can be obtained while the difference of the two consecutive iterations is less than a predefined threshold. Since both sparse coefficient vectors contain the information of whether the block f has smoke or not, the extracted feature^12,14 from them is bundled together and input into the support vector machine (SVM) classifier. The pseudo code of the entire detection algorithm is given in Table 1.

Table 1.

Algorithm of the block detection method.

Algorithm processes.
Input Data: parameters f, p, D1p, D2p, a1, a2, T, dInitialize x1p, x2p, y(0), y(1)Choose the dictionary series D1p and D2pby the measure of pWhile $|y(1)-y(0)|\ge T$ do 1. Calculate x1p, x2p by solving equation (6) with one fixed;2. Update y(0), y(1); 3. end While 4. Input x1p and x2p to an SVM classifier. Output: classification label of f

Experimental results

To illustrate the reliability of the proposed algorithm, infrared image smoke detection experiments based on a simulated full-size aircraft cargo under different atmospheric pressure were conducted, as shown in Figure 1. The dimension of inner chamber of the full-scale aircraft simulation cargo is 8.11 × 4.16 × 1.67 m with the cavity volume of 56.6 m³. The performance parameter of the infrared camera used in the article is 640 × 480 pixels with 17 μm pixel spacing, and the performance parameter of the video camera is 12 megapixels with motorized zoom lens.

Figure 1.

Full-size aircraft cargo environment.

Two data sets were constructed for the dictionary learning with visual assessment adopted to determine whether the block contains the smoke component. The data sets consist of 500 × 3 pure smoke and non-smoke infrared images under 3 different pressure intervals with the size of 10 × 10 pixels and were used to learn D_1p and D_2p. Since medium-sized and above civil aviation aircrafts fly at high altitude, where the high altitude refers to the altitude of 7000–12,000 m, the pressure distribution outside the cabin along the airline is in the range of 19.3–41 kPa. Considering the large fluctuation of the atmospheric pressure outside the cabin during the aircraft take-off and landing process, the 3 intervals of the atmospheric pressures are chosen as 30, 60, and 90 kPa. Both dictionary series were trained by K-SVD method. Part of the dictionaries with different pressure intervals is presented in Figure 2. Given 3000 infrared smoke image blocks and 2000 non-smoke image blocks in the aircraft cargo environment, the discriminate power of the proposed smoke detection method was studied.

Figure 2.

Samples of dictionaries under different atmospheric pressures: (a) Samples under the atmospheric pressure of 30 kPa. (b) Samples under the atmospheric pressure of 60 kPa. (c) Samples under the atmospheric pressure of 90 kPa.

The sparse coefficients were estimated by following the steps in Table 1. Since there are rare smoke detection methods for infrared images, the proposed method was compared with conventional video-based smoke detection methods. The texture feature LBP (Local Binary Pattern) was chosen as offered in the video-based methods. This feature was also extracted from the original infrared image block and fed into the SVM classifier. The classification accuracy is reported in Table 2. Hereafter, LBPI indicates the LBP extracted from the infrared image and LBPV means the one from the visual image. The dictionaries were chosen based on the changes in atmospheric pressure parameter.

Table 2.

Accuracy comparison of three methods.

Method	LBPI	LBPV	Proposed
Accuracy	56.3%	66.7%	80.2%

LBPI: LBP extracted from the infrared image; LBPV: LBP extracted from the visual image; LBP: Local Binary Pattern.

As shown in Table 2, the proposed method achieves the highest accuracy in the binary classification of smoke and non-smoke compared with LBPI and LBPV methods. The receiver operating characteristic (ROC) curves of the methods mentioned above are presented in Figure 3 along with area under the curve (AUC). It is evident that the proposed method outperforms the others.

Figure 3.

ROC curves for the classification of smoke and non-smoke.

The false alarm rate of the block in 5 video series with different cargo environment pressures is computed for comparison between the proposed and LBP method. The smoke is generated by the burning process of 20 FAA standard cartons in the full-size cargo. The pressure in the cargo is set from 30 to 100 kPa. The comparison of detection result of the 5 videos and the traditional photoelectric detector is shown in Figure 4.

Figure 4.

Block FAR with different pressures.

The simulation results indicate that the proposed method shows better false alarm rate performance than the photoelectric detector, and the detection performance is vulnerable to the significant change in air pressure. When the air pressure is much lower, the false alarm rate increases a lot in both the proposed and LBP methods. The proposed method has advantages over the traditional LBP method while the pressure condition changes. On the basis of the result of smoke block identification, the smoke content in the frame is judged according to the improved neighboring block rule as depicted in Figure 5. We assumed that if there are three image blocks of nine in one square that have smoke content synchronously, the frame is considered to be contaminated by the smoke.

Figure 5.

Improved neighboring block rule.

However, textures of the smoke such as the jittering of the camera may cause the change of texture which is similar to the motion of the smoke. Meanwhile, the oxygen concentration may vary with the air pressure fluctuation caused by the flight altitude change. These situations which commonly occur in flight may decrease the performance of both the LBP and proposed methods. A solution to this problem is to add more atoms of the dictionary at several different pressure intervals for smoke detection of image blocks. It also needs to be clear that the samples used in the smoke identification of blocks are artificial marked. Due to visual fatigue and other reasons, the mark error will be larger, and the recognition result has a great correlation with the sample mark, so the difference between different videos is large. The schematic diagram of detection and the performance with variable pressure ratios are shown in Figures 6 and 7. The prop1 is the proposed method whose dictionaries are obtained under different pressures, and the prop2 takes the same algorithm as prop1 but using the dictionaries only obtained under the normal atmospheric pressure. The results show that the proposed method achieves the best false alarm rate performance at low atmospheric pressures which is the evidence for using variable pressure environment of aircraft cargoes.

Figure 6.

Smoke detection in the aircraft cargo.

Figure 7.

Smoke detection performance with variable pressure ratios.

Conclusion

The study in this article provides a new perspective on smoke detection. A method for smoke detection from infrared frames is proposed through the construction of the dual over-complete dictionaries under different atmospheric pressure environment. The features of the smoke and the background components were extracted from the infrared blocks by the sparse representations of the dictionaries. Experiments on smoke detection in the full-size aircraft cargo were conducted and the results indicate that the atmospheric pressure has effects on smoke detection with the proposed method and the texture methods. The method with multiple atmospheric pressure parameters is effective and outperforms the LBP methods and dictionary method with the single atmospheric pressure parameter. All the results are achieved under the environment of the enclosed aircraft cargo and show the prospect of smoke and early fire detection during the flight. More experiments should be conducted to demonstrate whether it can reduce the false alarm ratio in the airline transportation. Further improvement may be achieved through comparing different classifiers and modifying the supervised learning schemes. Also, other categories of methods are expected to be researched based on the infrared image for smoke detection.