Sage Journals: Discover world-class research

Abstract

Since the impact of haze on outdoor imaging varies sharply with the scene depth, hazy image enhancement proves to be a challenging task. In this article, we propose an adaptive image enhancement method for haze removal. In the proposed method, a quantitative measurement has been presented to evaluate the haze density, in which the characteristics of hazy images on brightness, texture, and color are taken into account. On the basis of the estimated haze density distribution, the intensity and color saturation have been enhanced. To achieve the increased visibility, the image texture component is extracted through a total variation model and sharpened depending on the haze density function. In addition, the corresponding correction on color saturation is also conducted to gain better color fidelity. Experimental results demonstrate that our approach is effective to produce the output images with compelling visual quality in terms of visibility and color fidelity.

Keywords

Haze removal haze dense function texture enhancement saturation correction total variation smoothing

Introduction

In foggy weather, outdoor images usually degrade in color, texture, and contrast due to the light scattering from the atmosphere suspended particles. Most automatic vision systems, such as traffic monitoring and auto navigation, fail to work normally with destructive features from input hazy images. Therefore, effective automatic haze removal techniques will benefit many computer vision applications and the research on image dehazing is of practical significance.

Image haze removal aims at restoring the scene radiance from the input hazy image. In the literature, researchers have developed many methods used in hazy image restoration. Among them, some methods dehaze images by image enhancement techniques. For instance, Huang et al.¹ used adaptive gamma correction with weighting distribution to enhance hazy images. Ancuti and Ancuti² derived two images from an original hazy image through white balance and contrast enhancing and generated the outcome by fusing them in the multi-scale manner with three weight maps measured in terms of luminance, saturation, and saliency. In Zhou and Zhou,³ the Retinex method was applied in image haze removal, exploiting the lightness-color constancy of human vision system for contrast enhancement. Choi et al.⁴ proposed the image defogging method based on the referenceless prediction model called fog aware density evaluator (FADE), which proved effective to evaluate the perceptual fog density. In spite of their capability of improving images’ visibility, these methods of image enhancement are inadequate in image haze removal. The major reason relies on the fact that the haze effect changes exponentially in the distance of scenes from observers. If the spatial distribution of haze is not taken into account in image processing, the restoration results of hazy images are far from satisfactory in most cases.

In recent years, it has caught great attention to remove haze on the basis of the atmospheric scattering model. Narasimhan and Nayar⁵ described the formation of a hazy image as follows

I (x, y) = L_{\infty} \cdot ρ (x, y) \cdot e^{- β \cdot d (x, y)} + L_{\infty} \cdot (1 - e^{- β \cdot d (x, y)})

(1)

where I is the observed image, $ρ$ is scene reflectance, $L_{\infty}$ is the global atmospheric light, $β$ denotes the medium transmission coefficient, and d refers to the scene depth. For the sake of simplicity, $L_{\infty}$ and $β$ are usually regarded as constant when dealing with haze removal.

In order to restore the original scene radiance, it is critical to estimate the scene depth from the input image. Kopf et al.⁶ constructed the scene depth map from the corresponding known three-dimensional (3D) model. In Schechner et al.,⁷ multiple images taken with different degrees of polarization were used to derive the scene depth. Narasimhan and Nayar⁸ estimated scene depth through two or more images captured under different weather conditions. These depth-based methods are restrictive since the requirements are usually hard to be fulfilled in most practical applications.

The estimation of scene depth remains to be more challenging if only one single hazy image is provided as input. Hence, most existing endeavors are devoted to finding valuable prior knowledge or constraints, which are helpful in hazy image restoration.

Tan⁹ calculated the best transmission combination through maximizing the contrast in neighboring regions under the assumption that the local contrast in haze-free images is much higher than hazy ones. Although Tan’s results are impressive, they tend to be over-saturated.

Fattal¹⁰ made the assumption of local irrelevance between transmission and surface shading in neighbor regions and utilized the independent component analysis (ICA) to remove haze. His algorithm is time consuming and restrictive in color images.

From the statistics of haze-free outdoor images, He et al.¹¹ discovered the dark channel prior (DCP) that most local patches contain some pixels with very low intensity in at least one color channel and applied DCP in haze removal. In spite of the compelling dehazing results, his method is computationally intensive and may lose quality in restoration when the original image contains sky regions.

To accelerate the haze removal process, the joint guided bilateral filtering,^12,13 the guided filter,¹⁴ and median of median filter¹⁵ are used to replace the time-consuming soft matting.¹¹ Chen et al.¹⁶ refined the transmission map by the proposed gain intervention refinement (GIR) filter, resulting in much less computational cost as well as the suppression of halo artifacts.

To further improve the restoration visual effect, Nishino et al.¹⁷ modeled the image with a factorial Markov Random Field (MRF) to estimate the scene radiance more accurately. Nishino’s approach often fails to restore the sky regions, resulting in serous artifacts. Tang et al.¹⁸ combined four types of haze relevant features with random forests when calculating the transmission. Chen and Huang¹⁹ exploited the fact that the entropy of gradient magnitude in each hazy image is much lower than its haze-free version and introduced a self-adjusting parameter to dynamically modulate the transmission map, leading to the increased visibility as well as reduced computational burden. Huang et al.²⁰ combined the median filter and the adaptive gamma correction operation in the transmission map refinement, not only avoiding halo effects in images with complex structures, but also solving the insufficient estimation of transmission caused by the different distribution in RGB color histograms. Huang’s method proved to have strong abilities to adapt to varying weather conditions. Meng et al.²¹ exploited the inherent boundary constraint on the transmission function and incorporated this constraint and a weighted L1-norm based contextual regularization into an optimization model to refine the transmission map. Meng’s method behaves well in removing halo artifacts in the dehazing results. However, it fails to handle color distortion when large sky regions are contained in the hazy image. Zhu et al.²² constructed a linear model to derive the scene depth via the color attenuation prior, that the scene depth is positively correlated with the difference between brightness and saturation. To overcome the problem of underestimating and overestimating the transmission map of the background and foreground regions in real-world scene with deep depth of field, Chen et al.²³ combined gamma correction and histogram equalization to produce a bi-histogram modification-based visibility restoration approach. Chen’s method is capable of increasing the visibility of heavy haze regions while maintaining the structure of the foreground objects.

Some methods are developed to restore visibility for special real-world hazy scenes. Li et al.²⁴ proposed a framework to dehaze nighttime images, in which the glow was eliminated and a spatially varying atmospheric light map was introduced to deal with the problem of multiple light colors. The images restored by Li’s method show good quality in terms of visibility and color. Nevertheless, the boosting of noise and compression artifacts sometimes can be found in the dehazing results. For images captured during sandstorm conditions, Huang et al.²⁵ built the image dehazing method on the Laplacian strategy and combined gamma correction and the white patch-Retinex techniques to overcome the color cast problem as well as insufficient estimation of haze thickness. Huang’s method proved effective in removing the color cast effects and recovering the visibility of the degraded images. For road scenes captured by intelligent transportation system, Huang et al.²⁶ utilized the hybrid DCP to prevent the localized light from being misjudged as the atmospheric light and produced the color spectrum adjustment parameter to avoid the color shifts in the restored images. Also, in Chen and Huang,²⁷ Fisher’s linear discriminant method and the dual DCP scheme were combined to estimate the global atmospheric, avoiding the misjudging caused by the presence of localized lights. Moreover, based on the gray world assumption and the Retinex theory, the chromatic adaptable parameters were calculated and used to cope with the problem of color shift when dehazing images captured in the sandstorm weather conditions.

Unfortunately, most prior rules or assumptions that have been discovered or made originate from statistical data and may not be valid all the time. The physics-based methods based on prior knowledge inevitably have their own limitations and may produce less than satisfactory results sometimes.

In this article, we present a hazy image enhancement method, which does not rely on the physical scattering model. Motivated by the fact that haze concentration is different from place to place, our proposed method makes full use of the inherent relevance between the image degradation and haze distribution to perform an adaptive enhancement on the input images. In our method, the characteristics of hazy images in terms of brightness, texture, and color saturation are combined to roughly estimate the haze density in the input image. And on the basis of the estimated haze distribution, both the intensity and the color saturation are adjusted respectively. Experimental results show that the enhanced images by our method are visually compelling in both visibility and color fidelity.

Image enhancement for haze removal

In this section, we conduct hazy image enhancement on intensity and color saturation of pixels. The overall framework of our algorithm is illustrated in Figure 1.

Figure 1.

The framework of our method.

Our method first applies the Retinex algorithm to perform contrast stretching on pixels’ intensity. In the Retinex model, the input image can be represented as the product of the source illumination (incident light) and the scene reflectance. Under the assumption that the incident light varies slowly, the source illumination can be approximately estimated through Gaussian smoothing filtering on the input image.³ And the Retinex output can be expressed as

V_{i} (x, y) = \log I_{i} (x, y) - \log (I_{i} (x, y) * G (x, y))

(2)

where I_i is the input image in the ith color spectral band and G(x,y) is the Gaussian surround function. “*” denotes the convolution operation.

Figure 2(c) shows the image brightness by Retinex enhancement. As can be seen, the image intensity after Retinex processing becomes brighter and the visibility is increased to some extent.

Figure 2.

(a) The hazy image, (b) haze density map of (a), (c) the intensity by Retinex processing, (d) the enhanced intensity with sharpened texture, (e) the corrected saturation, (f) the restoration result by our algorithm removing Retinex processing, and (g) the restoration result by our algorithm.

Then the operation of sharpening on image texture follows. To extract the surface texture, a guided total variation (TV) model is constructed and a fast iterative method is applied to find the solution of the model. The subsequent enhancement on texture depends on the haze density distribution or the scene depth. The detailed steps in texture abstraction and sharpening are described in section “The adaptive enhancement on image texture.”

Except for the brightness, the enhancement on color is also required to achieve the pleasing visual effect. Considering the inherent relevance between color degradation and scene depth, the haze density-dependent correction on color saturation is proposed to restore color fidelity and described in section “Saturation correction.”

Haze distribution estimation

Since the enhancement on both intensity and color saturation relies on the haze density, an effective measurement is required to evaluate the haze distribution.

For the convenience of description, we define D_H(x) to indicate the haze density. Intuitively, the concentration of the haze increases along with the change of the scene depth. In other words, haze density is positively correlated with the scene depth. In order to estimate D_H(x) from the single input image, we have to take full advantage of some valuable priors hidden in hazy images. Through observation, we find that in haze-free regions, the brightness is moderate, the color saturation is pretty high, and the texture is clear. In the presence of haze, these three features vary sharply with haze distribution.

As depicted in equation (1), the impact of fog or haze on pixels’ intensity is both multiplicative and additive. The multiplicative influence comes from the decay of scene radiance in the medium, while the additive influence results from the incident light scattered by atmosphere particles. Both these two portions are dependent on the scene depth. It indicates those objects more distant from the camera are bound to be affected significantly, and the corresponding brightness tends to be larger. Except for brightness, hazy images suffer from poor color fidelity as well, and the degree that color saturation declines is also strongly associated with the distance of scene object from the observers. Clearly, in the hazy weather condition, images captured from distant scene are more likely to fade. Likewise, haze imaging may blur the texture details of scene objects, leading to poor visibility. And the blurred degree depends on the scene depth as well.

Based on the above analysis, we define D_H(x) as

\begin{array}{l} D_{H} (x) = \exp (\frac{- {(\nabla V (x))}^{2}}{C_{1}}) \cdot \\ \exp (\frac{- {(\bar{V} (x) - 1)}^{2}}{C_{2}}) \cdot \exp (\frac{- {(S (x))}^{2}}{C_{3}}) \end{array}

(3)

where x denotes the index or position of pixels and $\bar{V}$ is computed from the local smooth operation on pixels’ intensity V. ∇ represents the gradient operator. S denotes color saturation.

For each pixel, the corresponding D_H(x) is computed from the product of three terms in the exponential form. The first term originates from the texture prior in hazy region. Small gradient indicates low contrast and high similarity in brightness among neighbor pixels. For thick haze, the first term tends to return large value. The second term describes the contribution from the brightness. Apparently, it is close to 1 when the pixel brightness approaches 1. The brighter pixel is more likely to be considered as hazy one. The third term takes the impact of color saturation into account, and it approaches 1 when the difference among three color channels is trivial. For pixels in hazy regions, the corresponding saturation S is low and the resultant value of the third term is large accordingly.

C ₁, C₂, C₃ are scaling factors for balancing these three multiplicative terms. If C₁ is much larger than the other two, it means the change of intensity plays more important role in estimating haze density. Accordingly, the ultimate enhancement will be focused on those regions with blurred texture or low contrast, which sometimes may result in texture over-enhancement. In our experiment, we treat these three terms as equally important and therefore set C₁ = C₂ = C₃.

In order to preserve the local smoothness, we perform the erosion operation on D_H(x) as

D_{H}^{'} = D_{H} Θ Λ

(4)

where $Θ$ denotes the erosion operator and $Λ$ is the template used in erosion. To limit $D'_{H} (x)$ in the range [0,1], we carry out the normalization on $D'_{H} (x)$ as

D_{HN} (x) = \frac{D_{H}^{'} (x)}{Max (D_{H}^{'})}

(5)

Figure 3 shows the maps of haze density or scene depth estimated by our algorithm, Zhu et al.,²² and Chen et al.¹¹. Among them, Zhu et al.²² took full advantage of the color attenuation prior to estimate the depth map, and He et al.¹¹ considered the dark channel of hazy images can be used as the rough estimate of haze density. To make a fair comparison, the depth map or haze density estimated by Zhu et al.²² and He et al.¹¹ has been conducted using the same smoothing process as our algorithm. As seen in Figure 3, the estimated results by our algorithm are more consistent with the hazy images, preventing the haze density of the close range objects from being underestimated.

Figure 3.

(a) The hazy images, (b) haze density estimated by our proposed algorithm, (c) haze density estimated by Zhu et al.,²² and (d) haze density estimated by He et al.¹¹

The adaptive enhancement on image texture

In order to increase the visibility, we propose an algorithm dedicated to sharpening image texture. The algorithm is composed of two steps: texture extraction and texture enhancement. In order to extract the texture component, a TV model is first applied to smooth the hazy image and then the texture details are obtained by subtracting the smooth version from the input. Due to the dependency of blurring effect on the haze density, we intend to enhance texture details adaptively based on haze distribution. Our enhancement method on image texture can be described as follows:

Step 1. Compute the smooth part from the input image. In common sense, edges in images can be classified into two types. One is the outline of scene objects near regions with the depth jump, and the other is surface texture with consistent depth. For ease of distinction, we use the term “edge” to represent the former and “texture” to denote the latter. Normally, the blurring impact of haze on texture is more prominent. So our major goal is to make the blurred texture clearer. In order to separate the texture details from the Retinex output V, we are meant to obtain the smooth component with major edges first. The traditional low-pass filters, such as the mean filter or the Gaussian filter, are not capable of preserving edges. Here, we propose to perform smooth operation on the input image through a TV model

\underset{X}{\arg} \min E = \frac{A}{2} ‖ X - V ‖^{2} + \frac{B}{2} \cdot W \cdot ‖ \nabla X ‖^{2}

(6)

In equation (6), A, B are two factors, deciding the contribution of the corresponding term to the optimization. The TV model utilizes the first term to ensure the fidelity of X to V. The greater the parameter A, the closer the solution to the original input. The second term is to impose the local dependency constraint on X. Setting larger value of B results in smoother solution. In our experiment, we set A = 3, B = 1. W is a weighting function defined as

W (x) = e^{- \frac{‖ \nabla V (x) ‖_{2}^{2}}{2 σ}}

(7)

where x denotes pixels’ position, ∇ V denotes the gradient of V, and $σ$ is the scaling factor ( $σ = 0.1$ in our experiment). Clearly, the weight W varies inversely with the brightness gradient. It makes sense that pixels within smooth regions are likely to be assigned large weights, whereas those near or on the edges receive small values.

To speed up the solution process, we adopt an iterative procedure with gradient approximation and compute $\nabla X$ through averaging all the first-order difference between the current pixel and its neighbors

\nabla X (y) = \frac{1}{r^{2} - 1} \sum_{i = 1}^{r^{2} - 1} (X (y) - X_{i} (y))

(8)

where r denotes the neighborhood width (r = 15 in our experiment), and X_i(y) is the ith neighbor pixel in the patch centered at y. For simplicity, in the jth iteration, we replace $X_{i}^{(j)} (y)$ with $X_{i}^{(j - 1)} (y)$ . So the jth iterative objective function can be expressed as

\begin{matrix} E^{(j)} & = \frac{A}{2} ‖ X^{(j)} - V ‖^{2} + \frac{B}{2} \cdot W \cdot ‖ \nabla X^{(j)} ‖^{2} \\ = \frac{A}{2} ‖ X^{(j)} - V ‖^{2} + \frac{B}{2} \cdot W \cdot ‖ \frac{1}{r^{2} - 1} \sum_{i = 1}^{r^{2} - 1} (X^{(j)} - {\hat{X}}_{i}^{(j - 1)}) ‖^{2} \end{matrix}

(9)

where ${\hat{X}}^{(j - 1)}$ is the outcome of the previous iteration.

To find the optimal result in equation (9), we calculate the partial derivative of E^(j) with respect to X^(j) and make it equal to 0

\frac{\partial E^{(j)}}{\partial X^{(j)}} = 0

(10)

{\hat{X}}^{(j)} = \frac{AV + \frac{W}{r^{2} - 1} \sum_{i = 1}^{r^{2} - 1} B {\hat{X}}_{i}^{(j - 1)}}{A + BW}

(11)

We treat the outcome of the last iteration as the smooth component V_s. To facilitate the description below, we define the above TV smooth operator as the function F(X). Here, X is the input image, F(X) returns the image by the TV smoothing processing.

Step 2. Extract the texture map V_d from the input as $V_{d} = V - V_{s}$ and make a further smooth operation on V_d to get $V_{df} = F (V_{d})$ to prevent from over-enhancement on texture details.

Step 3. Sharpen the detail component and produce the output through

V' = min {V + K \lg^{(1 + D_{HN} / α)} V_{df}, V_{0}}

(12)

Here, we use D_HN to control the magnitude that the texture details are exaggerated. In equation (12), K and α are positive constants. K decides the strength of enhancement, and the scaling factor α controls the rate that the logarithmic function varies with the haze density. Obviously, the amplitude of texture enhancement grows with the increase in K or the decrease in α. To achieve better visual quality, K and α should be set differently when removing haze from different types of hazy images. For example, in dense haze condition, the texture of scene objects tends to be blurred seriously and K can be set to a larger value. Actually, it is not easy to adaptively set the appropriate value for K and α, which we will conduct further investigation of. For the sake of simplicity, we set K = 1, α = 0.01 in our experiment. V₀ is the upper bound.

Figure 2(d) shows the intensity map after texture sharpening. Clearly, the visibility has been improved significantly and the texture details have been enhanced. Obviously, compared with the original image, the boundaries between adjacent floors of the distant buildings look clearer.

Saturation correction

For most outdoor hazy images, the overall color tends to be faint gray. Especially in heavy haze, the color may suffer from severe distortion. Similar to pixels’ intensity, the color saturation varies sharply along with the change in the haze concentration. Thus, it is necessary to correct the color saturation and restore the color fidelity. To this end, we adjust the saturation component by equation (13), where D_HN is also adopted to control the magnitude of enhancement

S' (x) = min {(1 + δ D_{HN} (x)) S (x), S_{0}}

(13)

In equation (13), S₀ is the upper bound, and the constant $δ$ controls the strength of enhancement. The large value of $δ$ may lead to over-enhancement in color in the restored images. To prevent an excessive increase in saturation level, we set S₀ = 1 and $δ = 0.5$ . Figure 2(e) and (g) displays the saturation map enhanced by equation (13) and the final restored image, respectively. Clearly, the dehazing result is appealing in visual quality as well as color fidelity.

To illustrate the role of Retinex treatment, we removed the Retinex operator from our algorithm and obtained the restored image as shown in Figure 2(f). Obviously, compared with Figure 2(g), the contrast of the distant buildings is declined and the overall color fidelity is decreased to some extent.

Experimental comparison

To demonstrate the effectiveness of our method, we compare our approach with some state-of-the-art works in both qualitative and quantitative manners. Those algorithms we select for comparison include those of He et al.,¹¹ Zhu et al.,²² Tarel and Hautière,¹⁵ and Choi et al.⁴ All the experiments were conducted in the MATLAB R2012a environment on a PC with a 2.5 GHz Intel Core(TM) i5-3210M CPU. For the sake of fairness, we run those algorithms^4,11,15,22 with the parameters recommended by the authors.

Figure 4 shows the restored results by our proposed method. There are nine test images in Figure 4. The images on the first and third row are input hazy ones, and those on the second and fourth row are corresponding restored results. As can be seen in Figure 4, our approach works well in removing haze from these test images. For images with distant scenery, such as Figure 4(a), (b), (c), (e), and (h), both the visibility and color fidelity are enhanced significantly. More than that, the clear texture details are restored in Figure 4(g), and the vivid color information is recovered in Figure 4(d) and (f).

Figure 4.

The dehazed results by our algorithm: (a)-(i) display the nine hazy images on the top and the corresponding restored images on the bottom.

Figures 5 –9 show the comparison results. In most cases, He’s¹¹ method behaves well in haze removal. However, as can be seen from Figure 5(b), the severe color distortion can be found in the sky region, and there are halo artifacts around the boundary of distant buildings. The inherent reason is that the image contains white buildings of large size, which may lead to the invalidity of dark channel as well as the mistake in estimating the global atmospheric light. In addition, a small amount of mist remains near the scene depth jump in Figure 7(b), because of the erosion operation in estimating medium transmission.

Figure 5.

Test images (2) from (a) to (f): hazy image, the dehazed results from He et al.,¹¹ Choi et al.,⁴ Tarel and Hautière,¹⁵ Zhu et al.,²² and ours.

Figure 6.

Test images (3) from (a) to (f): hazy image, the dehazed results from He et al.,¹¹ Choi et al.,⁴ Tarel and Hautière,¹⁵ Zhu et al.,²² and ours.

Figure 7.

Test images (4) from (a) to (f): hazy image, the dehazed results from He et al.,¹¹ Choi et al.,⁴ Tarel and Hautière,¹⁵ Zhu et al.,²² and ours.

Figure 8.

Test images (5) from (a) to (f): hazy image, the dehazed results from He et al.,¹¹ Choi et al.,⁴ Tarel and Hautière,¹⁵ Zhu et al.,²² and ours.

Figure 9.

Test images (6) from (a) to (f): hazy image, the dehazed results from He et al.,¹¹ Choi et al.,⁴ Tarel and Hautière,¹⁵ Zhu et al.,²² and ours.

It can be observed from Tarel’s results that most texture details are well restored after haze removal. However, due to the median filter used in estimating the dissipation function, Tarel and Hautière¹⁵ show poor performance in removing haze from regions with depth discontinuities. For instance, in Figures 5(d) and 7(d), halo effects can be found near the scene depth discontinuities. Moreover, Figures 8(d) and 9(d) look dark overall, and the color fidelity is less than satisfactory.

Zhu et al.²² is effective in preventing from excessive haze removal and the results are not over-enhanced (see Figures 5(e) and 6(e)). Yet, the fixed setting of scattering coefficient usually results in incomplete haze removal (see Figures 7(e) and 8(e)).

Rather than the atmospheric scattering model, Choi et al.⁴ built the reference-less perceptual haze density prediction model in haze removal. The restored images are visually dark and the color fidelity is not very good.

Comparatively, our algorithm is able to produce compelling results with clear texture and excellent color fidelity. And the restored scene looks more natural, as seen in Figures 5(f) and 7(f).

To quantitatively assess the visual quality after haze removal, we employ three variables (g, r, d). As defined in equation (14), g denotes the average gradient magnitude, which can be used to evaluate the visibility of texture in the restoration results to some extent. r is recommended in the literature²⁸ and used to evaluate the change in visibility between images before and after haze removal. Generally, a higher value of r indicates better performance of haze removal. d is defined in Choi et al.⁴ to measure the amount of haze remained in the image, and the lower d is expected in the restoration results

g = \frac{\sum_{1 \leq i < h, 1 \leq j < w} \sqrt{\nabla_{x}^{2} (i, j) + \nabla_{y}^{2} (i, j)}}{(h - 1) \times (w - 1)}

\begin{matrix} \nabla_{x} (i, j) = I (i + 1, j) - I (i, j) \\ \nabla_{y} (i, j) = I (i, j + 1) - I (i, j) \end{matrix}

(14)

We calculate these three measurements (g, r, d) from the images in Figures 5 –8 and show the quantitative comparison results in Figure 10. Comparatively, our results possess the highest value of g, which results from our enhancement on image texture. Meanwhile, our results also rank first in terms of r, which conforms to the appealing visual effect of our dehazed outcome. As shown in Figure 10(c), all values of d from Zhu’s results exceed other algorithms, which is consistent with the subjective observations in Figures 5 –8.

Figure 10.

The quantitative comparison results from (a) to (c): the three measurements g, r and d calculated from the images in Figure 5 –8.

The test images used in our experiments are mainly collected from online Baidu images. The experimental results demonstrate that our proposed algorithm works well for about 70% of images not covered with dense fog. Since the enhancement on intensity and color depends on the haze density estimation, errors in estimating haze distribution may decrease the capability of haze removal, resulting in intensity over-enhancement or color distortion. Especially, when the original input images contain scene objects of large size with similar features with haze, or the distribution of atmospheric particles is uneven, the restored results are far from satisfactory, and the visual effect is even worse than the original hazy images, as shown in Figure 11.

Figure 11.

Test images (7): the left and the right row of (a) and (b) display input hazy images and the restored results.

Table 1 lists the time consumed in processing hazy images in Figures 5 –9 by He et al.,¹¹ Zhu et al.,²² Tarel and Hautière,¹⁵ Choi et al.,⁴ and ours. Evidently, our method has distinct speed advantages over He et al.¹¹ and Choi et al.⁴ and consumes a little bit more time than Tarel and Hautière.¹⁵

Table 1.

Processing speed comparison.

Hazy image	Time consumed by different algorithms (s)
	He et al.¹¹	Zhu et al.²²	Choi et al.⁴	Tarel and Hautière¹⁵	Ours
Figure 5(a)	3.03	0.35	2.89	0.11	0.12
Figure 6(a)	3.12	0.37	2.26	0.08	0.15
Figure 7(a)	4.05	0.35	2.80	0.11	0.22
Figure 8(a)	3.12	0.38	2.30	0.08	0.16
Figure 9(a)	3.15	0.38	2.83	0.08	0.18

Conclusion

In this article, we propose an enhancement method to restore a single hazy image. The method mainly focuses on sharpening the texture details and correcting color saturation on the basis of haze distribution, which is estimated via the characteristics of brightness, texture, and color fidelity. In order to extract the weak texture blurred by the haze, we construct a TV model to smooth pixels’ intensity and utilize an iterative process with gradient approximation to find the optimal solution. In addition, in order to restore the color fidelity, pixels’ saturation is corrected accordingly based on the estimated haze density as well.

However, as mentioned in section “Experimental comparison,” our proposed algorithm is still inadequate in dealing with images under the circumstance of uneven haze or those containing bright objects of large size. Therefore, our future work would focus on (1) investigating flexible methods to achieve a more precise estimate of haze distribution, preventing the interference from various scenes of real world, and (2) improving the atmospheric scattering model to solve the problem of haze removal in case of the inhomogeneous atmosphere and uneven incident light.

Footnotes

Handling editor: Alexandar Djordjevich

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported in part by the National Natural Science Foundations of China under grant 61571241, Industry University Research Prospective Joint Project of Jiangsu Province under grant BY2014014, Major Projects of Jiangsu Province University Natural Science Research under grant 15KJA510002, Jiangsu Province Graduate Research Innovation Project under grant CXZZ13_0476, and Science Research Fund of Nanjing University of Posts and Telecommunications under grant NY215169.

References

Huang

Cheng

Chiu

. Efficient contrast enhancement using adaptive gamma correction with weighting distribution. IEEE T Image Process 2013; 22: 1032–1041.

Ancuti

. Single image dehazing by multi-scale fusion. IEEE T Image Process 2013; 22: 3271–3282.

Zhou

. Single image dehazing motivated by Retinex theory. In: Proceedings of the 2nd international symposium on instrumentation and measurement, sensor network and automation, Toronto, ON, 23–24 December 2013, pp.243–247. New York: IEEE.

Choi

You

Bovik

. Referenceless perceptual image defogging. In: Proceedings of the southwest symposium on image analysis and interpretation, San Diego, CA, 6–8 April 2014, pp.165–168. New York: IEEE.

Narasimhan

Nayar

. Vision and the atmosphere. Int J Comput Vision 2002; 48: 233–254.

Kopf

Neubert

Chen

et al . Deep photo: model-based photograph enhancement and viewing. ACM T Graphic 2010; 27: 32–39.

Schechner

Narasimhan

Nayar

. Instant dehazing of images using polarization. In: Proceedings of the computer society conference on computer vision and pattern recognition, Kauai, HI, 8–14 December 2001, vol. 1, pp.325–332. New York: IEEE.

Narasimhan

Nayar

. Contrast restoration of weather degraded images. IEEE T Pattern Anal 2003; 25: 713–724.

Tan

. Visibility in bad weather from a single image. In: Proceedings of the conference on computer vision and pattern recognition, Anchorage, AK, 23–28 June 2008, pp.1–8. New York: IEEE.

10.

Fattal

. Single image dehazing. ACM T Graphic 2008; 27: 1–9.

11.

Sun

Tang

. Single image haze removal using dark channel prior. IEEE T Pattern Anal 2011; 33: 2341–2353.

12.

Sun

Wang

Zhou

et al . Real time image haze removal using bilateral filter. Trans Beijing Inst Technol 2011; 31: 810–814.

13.

Xiao

. Physics-based fast single image fog removal. In: Proceedings of the10th international conference on signal processing, Beijing, China, 24–28 October 2010, pp.1048–1052. New York: IEEE.

14.

Sun

Tang

. Guided image filtering. IEEE T Pattern Anal 2013; 35: 1397–1409.

15.

Tarel

Hautière

. Fast visibility restoration from a single color or gray level image. In: Proceedings of the12th international conference on computer vision, Kyoto, Japan, 29 September–2 October 2009, pp.2201–2208. New York: IEEE.

16.

Chen

Huang

Cheng

. A high-efficiency and high-speed gain intervention refinement filter for haze removal. J Disp Technol 2016; 12: 753–759.

17.

Nishino

Kratz

Lombardi

. Bayesian defogging. Int J Comput Vision 2012; 98: 263–278.

18.

Tang

Yang

Wang

. Investigating haze-relevant features in a learning framework for image dehazing. In: Proceedings of the conference on computer vision and pattern recognition, Columbus, OH, 23–28 June 2014, pp.2995–3002. New York: IEEE.

19.

Chen

Huang

. Edge collapse-based dehazing algorithm for visibility restoration in real scenes. J Disp Technol 2016; 12: 964–970.

20.

Huang

Chen

Wang

. Visibility restoration of single hazy images captured in real-world weather conditions. IEEE T Circ Syst Vid 2014; 24: 1814–1824.

21.

Meng

Wang

Duan

et al . Efficient image dehazing with boundary constraint and contextual regularization. In: Proceedings of the international conference on computer vision, Sydney, NSW, 1–8 December 2013, pp.617–624. New York: IEEE.

22.

Zhu

Mai

Shao

. A fast single image haze removal algorithm using color attenuation prior. IEEE T Image Process 2015; 24: 3522–3533.

23.

Chen

Huang

. Hazy image restoration by Bi-Histogram modification. ACM T Intell Syst Tech 2015; 6: 1–17.

24.

Tan

Brown

. Nighttime haze removal with glow and multiple light colors. In: Proceedings of the international conference on computer vision, Santiago, Chile, 7–13 December 2015, pp.226–234. New York: IEEE.

25.

Huang

Chen

. An advanced single-image visibility restoration algorithm for real-world hazy scenes. IEEE T Ind Electron 2015; 62: 2962–2972.

26.

Huang

Chen

. An efficient visibility enhancement algorithm for road scenes captured by intelligent transportation systems. IEEE T Intell Transp 2014; 15: 2321–2332.

27.

Chen

Huang

. An advanced visibility restoration algorithm for single hazy images. ACM T Multim Comput 2015; 11: 53.

28.

Hautière

Tarel

Aubert

. Blind contrast enhancement assessment by gradient rationing at visible edge. Image Anal Stereol 2008; 27: 87–95.

Automatic hazy image enhancement via haze distribution estimation

Abstract

Keywords

Introduction

Image enhancement for haze removal

Haze distribution estimation

The adaptive enhancement on image texture

Saturation correction

Experimental comparison

Conclusion

Footnotes

Declaration of conflicting interests

Funding

References