Sage Journals: Discover world-class research

Abstract

Illumination variation makes automatic face recognition a challenging task, especially in low light environments. A very simple and efficient novel low-light image denoising of low frequency noise (DeLFN) is proposed. The noise frequency distribution of low-light images is presented based on massive experimental results. The low and very low frequency noise are dominant in low light conditions. DeLFN is a three-level image denoising method. The first level denoises mixed noises by histogram equalization (HE) to improve overall contrast. The second level denoises low frequency noise by logarithmic transformation (LOG) to enhance the image detail. The third level denoises residual very low frequency noise by high-pass filtering to recover more features of the true images. The PCA (Principal Component Analysis) recognition method is applied to test recognition rate of the preprocessed face images with DeLFN. DeLFN are compared with several representative illumination preprocessing methods on the Yale Face Database B, the Extended Yale face database B, and the CMU PIE face database, respectively. DeLFN not only outperformed other algorithms in improving visual quality and face recognition rate, but also is simpler and computationally efficient for real time applications.

1. Introduction

Human-robot interaction is an important topic in robot research. Robots need to actively obtain information about the people around them through variety of perceptual systems and respond in real time. In these applications, robot vision system is a critical component [1]. For example, home service robots should be able to recognize the members of a family to be served and then provide service for them. However, the recognition accuracy of the person being served is affected greatly by the external environmental condition in application, such as lack of light and uneven illumination. Low-light noise is a significant problem in robotic automatic face recognition. In real applications, the deep shadow happens under both dim light and strong light in some angles. In low-light environments, the performance of face recognition degrades seriously [2]. Current existing approaches have difficulties to fulfill face recognition successfully in dim environments due to the lack of the study in the characteristics of the noises in low-light images.

The illumination-reflectance model [3] described the greyscale value f(x, y) of each pixel in an image as the product of the reflectance information and illumination information as follows:

f (x, y) = i (x, y) \cdot r (x, y),

(1)

where i(x, y) is the illumination component and r(x, y) is the reflectance component at pixel (x, y). The reflectance information denotes only the surface features of the object without any view of information. Moreover, the grayscale value reduces when light is insufficient while it increases under the strong illumination.

According to illumination-reflectance model, r(x, y) is regarded as the true image and the image noise was generated by the illumination. Based on this theory, algorithms are developed to extract reflectance information which are illumination invariant or illumination insensitive and use the information for face recognition. The representative approaches include retinex model [4] and quotient image (QI) [5]. However, it is quite difficult to separate r(x, y) from i(x, y) in practice. These approaches proposed conceptual and mathematical models, but empirical studies showed that none of these representations were sufficient to overcome image variations due to changes of illumination.

On the other hand, Buades et al. described the observed pixel grayscale value as the decomposition of the true image and the noise perturbation [6] as follows:

f (x, y) = r (x, y) + n (x, y),

(2)

where r(x, y) is the “true” value and n(x, y) is the noise perturbation.

Hence, we can extract the true image from the noisy image by removing noise. Most classical image preprocessing methods are denoising algorithms, including spatial filtering methods (histogram equalization, logarithmic transformation) and frequency domain filtering methods (low-/high-pass filters). These methods are simple and widely used in image preprocessing. However, each individual method can only remove some limited noise and cannot fulfill the illumination compensation. Buades et al. [6] proposed a novel denoising algorithm that is called nonlocal means, which attracts researchers’ interests to denoising methods again. However, its denoising effect is not good for the face images under dim light environment.

All above approaches do not distinguish the difference between dim and strong illumination. The denoising in low-light images is discussed by Chatterjee et al. [7]. However, it only focuses on the spot noise in low-light images which does not have deep shadow. Few researchers tested and analyzed the noise characteristics in low-light images. In general, researchers consider any low-frequency component to be signal, and noise mainly locates in high-frequency spectrum. But low-frequency noise becomes dominant in low-light images, and the noise value is distributed in a large-frequency range.

We focus on low-light face image denoising in this paper. A simple and efficient novel method based on denoising of low-frequency noise (DeLFN) is proposed. The main contribution of this paper includes the following. (1) The noise characteristics are analyzed based on experimental results. The frequency distribution of low-light image is given and discussed in-depth, and we proved there is very low-frequency noise dominant in low-light face image by performing a large range of systematical experiments. (2) A three-level denoising method based on (1) is presented to suppress the noise in different range in low-light face images. The DeLFN method and other methods are then compared on the Yale Face Database B [8], the Extended Yale Face Database B [9], and the CMU PIE Face Database [10]. The experimental results showed that DeLFN can significantly improve the performance of the face recognition systems in various low-light conditions and it is very fast for real applications.

The rest of this paper is structured as follows. In Section 2, the related work is outlined. In Section 3, the noise characteristics and distribution in low-light face images are discussed. In Section 4, the three-level image denoising method DeLFN is described in detail. In Section 5, experimental results are given and analyzed. Finally, the conclusion is given in Section 6.

2. Related Work

There are two categories of methods to cope with illumination variation. The first approach attempts to extract illumination invariant features or illumination insensitive measure, which is then used for face recognition. Retinex model [4] is proposed to find the pure reflectance information without illumination interference. In order to reduce halo artifacts, Jobson et al. [11] proposed the multiscale retinex (MSR) method which cancels much of the low-frequency information through dividing the image by a smoothed version of itself. Another famous concept quotient image (QI) [5] is regarded as an illumination invariant, and the self-quotient image (SQI) model [12] is presented to extend the QI theory. Chen et al. proposed a method in [13] which utilizes the logarithmic total variation (LTV) model to factorize an image and then obtain illumination invariant. Many other approaches are working out in this division [14 –18]. These approaches proposed conceptual and mathematical models, but empirical studies showed that none of these representations were sufficient to overcome image variations due to changes of illumination.

The second approach solving varying lighting is to restrain the disturbance or enhance the information of true image which is more useful for recognition. This class of methods cannot create new information of true image. But it can extract the true image from the raw image by using some image processing techniques. Face image preprocessing, which is the first step of automatic face identification system, mainly uses image enhancement technique to identify images. Image enhancement algorithms can be basically divided into two categories: space domain methods and frequency domain methods.

Algorithms based on spatial domain are basically divided into two types: gray scale of pixels computing algorithm and neighborhood enhancement algorithm. The former is to enhance the image contrast by transforming the grayscale of pixels, including grey level transformation, histogram equalization (HE) [19], and logarithmic transformation (LOG) [20]. The latter approach includes removal of noise and sharpening of image. This class of method includes Gaussian filter [21], median filter [22], and bilateral filter [23]. Some of these methods become basic and common in image processing.

On the other hand, image processing in frequency domain is an indirect denoising algorithm. We are familiar with Gaussian low-/high-pass filter, Butterworth low-/high-pass filter, and so on, whose objective is denoising or sharpening. Many other similar methods [24 –26] have been continuously proposed. An approach being sought after in recent years is homomorphic filtering [27, 28]. Delac et al. proposed a simple illumination normalization method [29] based on standard homomorphic filtering (HOMOD) technique. Some comparisons to other illumination compensation methods can be found in [30] but with insufficient theoretical and implementation details. However, homomorphic filter removes low frequency information which might not be noise. To solve this problem, Fan and Zhang proposed a homomorphic filtering-based illumination normalization method (HF + HE) in [31], which adds the low-frequency information while enhancing high-frequency component by high-pass filtering. These denoising methods in frequency domain always remove the information by a threshold or cutoff frequency which is too simple and crude for the wide range mixture of noise and true signal in real images.

Removing or reducing image noise is always a very hot research area in image processing [32 –38]. Buades et al. [6] developed a nonlocal image denoising algorithm which takes full advantage of image redundancy. After that, other researches are proposed to analyze and improve nonlocal image denoising method [39 –41]. The main advantage of these methods is not only simple in calculation but also very useful. However, their ability to handle extreme uneven illumination variations remains limited. The noise becomes significant especially in low-light conditions where the number of incoming photons is limited [42]. Chatterjee et al. [7] address the effects of noise in low-light images and proposed an algorithm for combined denoising and demosaicing of low-light images, specifically targeted at reducing the splotches and noise effects in such images.

3. Extra-Low-Frequency Noise in Low-Light Face Images

Generally, researchers consider the low-frequency information in an image as signal while high-frequency information as noise. Actually, the pixels are impacted through different noise which distributed in a large noise range. Low-frequency noise is dominant in low-light images.

In low-light images, it is the low frequency noise to dominantly disturb the true image and seriously degrade the picture quality. We can design a high-pass filter to detect critical frequency for low-frequency noises from superposition of true signal and noise. Firstly, the frequency spectrum F(u, v) of an image f(x, y) with size of M × N can be derived by discrete Fourier transformation (DFT) on the grayscale of pixels. Then Butterworth high-pass filter H(u, v) is applied for detecting the critical frequency. The function is given by

H (u, v) = \frac{1}{1 + {[D_{0} / D (u, v)]}^{2 n}},

(3)

where

D (u, v) = {[{(u - \frac{M}{2})}^{2} + {(v - \frac{N}{2})}^{2}]}^{1 / 2},

(4)

where F(u, v) is the frequency spectrum of the image after removing low-frequency noise. D(u, v) is the distance from the origin of the centered Fourier transform. n is the order of Butterworth filter, and n is set as 2 in this paper. D₀ is cutoff distance measured from the origin. D(u, v) = D₀ when (u₀, v₀) = (0, 0).

The critical D₀ can be obtained by changing cutoff distance and test the face recognition rate by PCA. The data is obtained by massive calculation. Table 1 shows the face recognition rate versus various cutoff frequencies. The subset 1 is used as training set in PCA. We performed experiments on all images of subsets 2–5 in the Yale Face Database B and the Extended Yale Face Database B. The critical D₀ equals 0.6 in Table 1. The value is relatively small. The best recognition rate at D₀ = 0.6 is not satisfactory because the signal and noise mix seriously in the frequency range when D₀ > 0.6 due to the low-frequency noise. Therefore, the face recognition rate decreases with increasing D₀ when D₀ > 0.6.

Table 1:

Recognition rates versus parameter D₀.

D ₀	0	0.2	0.4	0.6	0.8	1	2	3

Subset 2	91.01	93.20	96.27	99.34	97.15	94.96	89.69	86.18
Subset 3	85.03	87.72	90.35	91.67	90.79	87.28	82.46	80.26
Subset 4	18.66	19.92	21.62	22.74	21.24	19.36	14.66	10.53
Subset 5	12.05	16.20	20.64	22.30	18.01	15.51	7.06	1.80

However, the recognition rate improves in all low-light cases by removing the information whose frequency is lower than the frequency corresponding to D₀ = 0.6. It shows that there is some very low-frequency noise in low-light face images. Actually, 1/f noise (sometimes also called flicker noise or pink noise) is a kind of noise which is always generated by cameras when the picture is taken. It becomes dominant under dim lighting conditions and becomes significant at very low frequency [43]. The lower the light level is, the noisier the image is. And the lower the light level is, the more dominant the very low-frequency noise is.

The noise value is distributed in a large noise range from high frequency to very low frequency as shown in Figure 1.

Figure 1:

Noise distribution in low-light face images.

However, there are useful signals in the noise frequency range. Denoising methods in frequency domain always simply remove all information whose frequency is less than the cutoff frequency. To keep the useful signals, we need to remove the noise in different frequency but not to cut it off at certain frequency. On the other hand, there is also noise in the frequency range where the frequency is higher than the cutoff frequency. So multilevel denoising is necessary for comprehensive denoising and keeping useful signals.

4. Proposed Method

To get more comprehensive improvement on low-light face images, we should suppress the noises in whole noise frequency range as shown in Figure 1. Some denoising methods in spatial domain are very effective when there are plenty of noises distributing in a large noise range, although they do not look precise like those in frequency domain. Moreover, the denoising methods in frequency domain can completely remove noise in certain frequency range.

Hence, we propose a multilevel denoising method to remove noises in diverse frequency range by combining denoising methods in spatial domain and frequency domain. The proposed algorithm includes three-level denoising methods. The first level is preliminary denoising to reduce high-frequency noise, moderate-frequency noise, and low-frequency noise from the hybrid information of noises and signals by histogram equalization [19, 44] in spatial domain, which will promote the overall contrast of an image. The second level is roughly denoising to suppress low-frequency noise by modified logarithmic transformation in spatial domain, which can adapt to the skin gray. It will enhance the sharpness of the image. The third level is fine denoising to remove the residual significant very low-frequency noise in frequency domain by high-pass filtering. It will recover more information of true images. The multilevel denoising method can comprehensively recover most significant true image information step by step to greatly improve face recognition rate for low-light images.

The detailed flow diagram of DeLFN is shown in Figure 2. The original image is preliminarily denoised to reduce massive mixed noises with histogram equalization based on gray normalization. Then the image is roughly denoised to suppress low-frequency noise with modified Log transformation. After denoising in spatial domain, we transform the image into frequency domain by DFT. Then the image is finely denoised to remove the residual significant very low-frequency noise with high-pass filtering. Finally, the image is transformed back to spatial domain by inverse discrete Fourier transformation (IDFT), and the gray range of pixels is stretched to the whole gray range with grayscale stretching. Each step of this cascade denoising algorithm has the different extent of effects to the quality of images. Therefore, the sequence of the cascade is extremely important. The experiment is taken in Section 4 to prove the right cascade order which can get the best recognition result. In short, images are processed from coarse to fine, and the change of the image quality is from large to small.

Figure 2:

The flow of proposed method DeLFN.

4.1. Preliminary Denoising of Miscellaneous Noises

Original image in low-light level is very noisy. The lower the light level is, the lower frequency the noise gathers to. But there is still some high frequency noise as shown in Figure 3. Figure 3 shows face images in subsets 1–5 under different lighting conditions and their Fourier spectrums. There is more high-frequency information in the darker face image.

Figure 3:

Original face images in subsets 1–5 and their Fourier spectrums.

Histogram equalization is a nonlinear method based on cumulative function. It is effective for significant noises in whole noise frequency range because it redistributes the concentrated gray levels of an image to approximately uniform distribution in the whole greyscale range 0–255. The new gray value S_k is calculated by

S_{k} = T (r_{k}) = \sum_{j = 0}^{k} P_{r} (r_{j}) = \sum_{j = 0}^{k} \frac{n_{j}}{N},

(5)

where r_k is the kth gray level, n_k is the number of pixels in the image with that gray level, N is the total number of pixels in the image, and k = 0, 1, 2,…, L − 1. L is the greyscale of the original histogram and L′ is the greyscale of the modified histogram. The new gray value S_k is established from original gray value r_k (see Figure 4).

Figure 4:

The process of greyscale mapping with histogram equalization.

Greyscale distribution becomes more uniform after histogram equalization. In Figure 5, pictures in (a) are face images from subsets 1, 3, and 5 of Yale Face Database B, and figures in (b) are their gray histograms, and (c) and (d) show the results of histogram equalization. As shown in Figure 5, high intensity greyscales are stretched and low intensity greyscales are compressed. Obviously, the contrasts of the images were enhanced and the visual graininess of face images was improved considerably. The first face image becomes clearer whose facial features are nearly invisible before histogram equalization.

Figure 5:

Enhancement of the image contrast by gray histogram equalization.

4.2. Rough Denoising of Low-Frequency Noise

As we know, histogram equalization is a very basic image processing method. It can improve the general contrast of a noisy image. But the face image is still dark after the first level denoising and the improvement of face recognition rate is limited (which is proved by experiments in Section 5). That implies there are still some low-frequency noises dominant in the image.

To suppress the low-frequency noise further, logarithmic transformation is applied. Logarithmic transform can expand values of dark pixels and compress values of bright pixels. Therefore, the details of an image with low-level gray scale values will be enhanced. To adapt to the skin color, we modify the logarithmic transformation a little bit.

Most area of a face is covered with skin. The distribution of a face's grayscale under a uniform natural lighting condition trends to a normal distribution as shown in Figure 6. Through a lot of experiments, we get the skin greyscale values of the faces in subset 1 of Yale Face Database B. The grayscale values distribute in [110, 190] in general. If the face image is brighter, the upper limit value of the skin grayscale will be bigger.

Figure 6:

A face image in subset 1 of Yale Face Database B and its gray histogram.

Actually, the skin color is quite different for different races. It also has some difference for people in the same race. So we hope to adjust the skin gray value to the true skin value while image denoising. Suppose f_HE(x, y) is input image and f_HE+LOG(x, y) is output image; the logarithmic mapping function is modified as

f_{HE + LOG} (x, y) = α + β \ln [f_{HE} (x, y) + 1],

(6)

where α and β are parameters, which can modify the position and curvature of the log transformation and map the peak of gray histogram to the skin gray value.

To suppress the noise of high-light face images, exponent transformation can be applied. In this paper, we focus on the low-light face image denoising.

4.3. Fine Denoising of Very Low-Frequency Noise

Denoising in spatial domain suppresses massive miscellaneous noise and most low-frequency noise. There is still some dominant low-frequency noise after previous coarse denoising. Assume 1/f noise is an important negative factor for low-light face image recognition. We do not want to degrade useful low-frequency signal which represents the general view of an image [3]. Assuming 1/f noise is an important negative factor for low-light face image recognition, it is necessary to do fine denoising in frequency domain to remove residual very low-frequency noise.

Butterworth high-pass filter is applied in DeLFN to remove the residual very low-frequency noise. The frequency response of Butterworth high-pass filter is maximally flat in the pass-band because it has no ripples in the pass-band and slowly rolls off towards zero in the stop-band. Butterworth high-pass filter has a monotonically changing magnitude function with omega, unlike other filter types that have nonmonotonic ripple in the pass-band and/or the stop-band. It has a more linear phase response in the pass-band than the others.

The key issue is the selection of parameters to remove the dominant very low-frequency noise and keep useful low-frequency signal as much as possible.

4.4. Selection and Analysis of the Parameter D₀

The value of D₀ determines what frequency will be filtered. We did a large number of experiments to look for the critical value of D₀ which might correspond to the dominant low-frequency noise. If there is a critical value of D₀ at which sudden change of the recognition results occurs, there should be a dominant low-frequency noise.

Table 2 illustrates the recognition rates with PCA based on the images denoised by DeLFN versus parameter D₀. Subset 1 is used as training set. The inverse change occurred at D₀ = 2 for images in subsets 2–4. It is very low. The experimental results proved our hypothesis. There is indeed very low-frequency noise residual in low-light face images even after the images have been denoised in two levels.

Table 2:

The face recognition rate versus various D₀ values.

D ₀	0	0.05	0.5	1	1.5	1.6	1.8	1.9	2	2.1	2.2	2.4	2.5	3	4	5

Subset 2	95.17	96.71	97.36	99.34	99.56	99.56	99.78	100	100	100	100	99.56	99.12	98.68	97.15	92.11
Subset 3	92.11	93.20	94.07	97.15	98.46	99.12	99.56	100	100	100	100	99.56	99.34	98.9	94.53	90.13
Subset 4	56.02	57.33	58.46	59.77	61.09	62.03	63.16	64.10	64.10	63.91	63.91	63.16	62.78	61.84	55.45	53.95
Subset 5	73.82	75.76	78.25	80.88	87.12	88.78	91.69	92.11	92.11	92.11	91.97	90.86	88.09	86.84	79.78	71.05

4.5. Stretching Greyscale to the Whole Gray Range

After above denoising, the greyscale value of the image f(x, y) distributes in the range of a-b, 0 < a < b. Greyscale stretching is applied to convert the greyscale range from a-b to 0–255. The final face image f_DeLFN(x, y) is obtained by

f_{DeLFN} (x, y) = {\begin{cases} 0 & f (x, y) = a \\ \frac{255}{b - a} f (x, y) - \frac{255 a}{(b - a)} & f (x, y) \in (a, b) \\ 255 & f (x, y) = b . \end{cases}

(7)

Based on above discussion, the main procedures of DeLFN algorithm are given as in Algorithm 1.

Algorithm 1:

The DeLFN algorithm.

5. Experiments

In this section, we present and analyze the experimental results. We evaluate our DeLFN algorithm on the Yale Face Database B, the Extended Yale Face Database B, and the CMU PIE Face Database, respectively. Under the same experimental conditions, DeLFN is compared with different illumination processing methods including no preprocessing (None), histogram equalization (HE), homomorphic filtering (HF), the multiscale retinex (MSR) method, the self-quotient image (SQI) model, the logarithmic total variation (LTV) model, and the modified homomorphic method proposed in [31] (HF + HE).

The algorithms are evaluated based on the Yale Face Database B, the Extended Yale Face Database B and, the CMU PIE Face Database in terms of three performance metrics: visual quality, recognition rate, and time complexity, respectively. To comprehensively evaluate the recognition rate of the face image preprocessed by our algorithm, we select different training sets to test the algorithm in various illumination conditions. The training sets and test indices are shown in Table 3.

Table 3:

Recognition rate evaluation in various conditions.

Face database	Illumination condition	Training set

Yale Face Database B and the Extended Yale Face Database B	Relative ideal (angle < 12° from optical axis)	266 images of subset 1
	Bad (60° < angle < 77°)	532 images of subset 4
	Random	10 images per person randomly chosen from all 5 subsets

CMU PIE Face Database	Random	The same amount of images randomly selected for each person

The test of time complexity for different methods is performed on a computer with Intel Core i3-2330 M 2.20 GHz 2.19 GHz CPU and 6.00 GB memory. To reduce the computation cost, some optimizing methods [45, 46] are adopted in the implementation.

We set D₀ = 2 in DeLFN for all experiments. PCA recognition method is used to evaluate the performance of different preprocessing methods because PCA is easily affected by the variance of lighting. Therefore, recognition rates obtained by PCA method can directly show the capability of handling illumination by different algorithms. Besides, the minimum distance classifier is chosen for its simplicity, and the Euclidean metric L1 is used as distance measure.

5.1. Experiments on Yale Face Database B and the Extended Yale Face Database B

The Yale Face Database B and the Extended Yale Face Database B are both used to test the proposed method. The Yale Face Database B contains 5760 single light-source images of 10 persons. Each person has 9 poses and each pose has 64 different illumination conditions. The size of each image is 640(width) × 480(height). The Extended Yale Face Database B contains 16128 single light-source images of 28 persons; similarly, each person has 9 poses and each pose has 64 different illumination conditions. The form of the face database is the same as that of the Yale Face Database B. All testing images used in the experiments are manually aligned, cropped, and then resized to 168 × 192 pixels. The images of 38 persons in the face database are shown in Figure 7.

Figure 7:

The images of 38 persons in the face databases.

Since we focused on the illumination problem in this paper, only the 64 frontal pose images obtained under 64 different illumination conditions for each of the 38 persons were selected. The images were classified into five subsets based on the light-source directions (azimuth and elevation): subset 1 (angle < 12 degrees from optical axis), subset 2 (20 < angle < 25 degrees), subset 3 (35 < angle < 50 degrees), subset 4 (60 < angle < 77 degrees), and subset 5 (others). As a result, there are total 2432 images: 266, 456, 456, 532, and722 images in subsets 1–5, respectively.

5.1.1. Visual Quality of Images

Figure 8 shows the comparison of proposed method DeLFN to other methods. As can be seen, the visual effect of images preprocessed by our method is more clear and natural.

Figure 8:

Comparison between our DeLFN, histogram equalization (HE), homomorphic filtering (HF), the multiscale retinex (MSR) method, the self-quotient image (SQI) model, the logarithmic total variation (LTV) model, and the modified homomorphic method proposed in [31].

Figure 9 shows five images of each subset and the corresponding processed images obtained by DeLFN algorithm. It is shown that the proposed method made the visual quality of face images become stable and clear no matter how dark they were originally.

Figure 9:

(a) The same person in five subsets; (b) the corresponding processed images obtained by DeLFN.

5.1.2. Recognition Rate Comparison to Existing Methods

Under the same experimental conditions, we compared the recognition rates of our method DeLFN where D₀ = 2 with the following illumination compensation methods: no preprocessing (None), histogram equalization (HE), homomorphic filtering (HF), the multiscale retinex (MSR) method, the self-quotient image (SQI) model, the logarithmic total variation (LTV) model, and the homomorphic filter based method (HF + HE).

In the first experiment, subset 1 taken under good light conditions is used as the training set and all the images in the subsets from 2 to 5 are selected for testing, each of which is matched to the training set to get a best match result. The recognition rates are given in Table 4. The results show that DeLFN outperforms the other methods including HE, HF, MSR, SQI, LTV, and HF + HE methods in every subset. Especially, the recognition rate is far greater than that of the other methods for subsets 4 and 5. The recognition rates of subsets 2 and 3 are not less than 85% because images in the two subsets are captured under the better illumination condition than in subsets 4 and 5. By contrast, images in subset 5 are obtained in the low-light environment while there are serious uneven illumination issues with images of subset 4. As a result, recognition rates of the latter two subsets are less than those of the former two, but compared with other methods, our algorithm has more obvious effect in serious bad illumination condition. Thus, it can be seen that our method has obvious effect on illumination compensation for face recognition.

Table 4:

The recognition rate (%) using images of subset 1 as training set.

Method	Subset 2	Subset 3	Subset 4	Subset 5

None	91.01	85.03	18.66	12.05
HE	98.03	86.84	30.83	36.98
HF	96.05	85.96	28.95	34.90
MSR	100.00	89.75	44.92	88.38
SQI	100.00	91.67	45.11	89.20
LTV	100.00	100.00	60.39	90.86
HF + HE	92.98	86.40	48.68	38.09
DeLFN	100.00	100.00	64.10	92.11

In the second experiment, subset 4 taken under hostile light conditions is chosen as the training set, and other images in subsets 1, 2, 3, and 5 are used as testing images. Table 5 presents the recognition results. The proposed method is robust to training samples under varying lighting. The worst recognition rate is more than 98% and greater than that of other methods. The performance of DeLFN is satisfactory on subsets 1 and 2. Hence, our method is robust to different illumination conditions.

Table 5:

The recognition rate (%) using images of subset 4 as training set.

Method	Subset 1	Subset 2	Subset 3	Subset 5

None	17.29	24.12	43.42	28.25
HE	58.64	59.87	66.01	61.22
HF	42.11	38.38	47.37	47.65
MSR	88.72	84.87	85.75	89.61
SQI	96.99	97.59	97.59	97.78
LTV	99.42	99.13	97.67	98.31
HF + HE	91.35	93.86	97.15	89.47
DeLFN	100.00	100.00	98.25	99.03

In the third experiment, ten images per person are randomly chosen from all 5 subsets to make up the training set and the remaining images for testing. The recognition results are given in Table 6. DeLFN achieved a 97.27% recognition rate on average. DeLFN method outperformed HE, HF, HF + HE, MSR, SQI, and LTV methods.

Table 6:

The recognition rate (%) using 10 images of different illumination as training set.

Method	Average

None	38.11
HE	63.26
HF	56.43
MSR	91.28
SQI	93.16
LTV	96.59
HF + HE	82.65
DeLFN	97.27

The three experiments are implemented under different training samples (relative ideal images, bad illumination images, and different illumination images selected randomly), respectively. As we can see, our method consistently achieved high recognition rate.

5.1.3. Analysis of the Process of DeLFN

In order to prove that the preprocessing flow is reasonable for low-light face images, we did the following experiments. We tested the recognition rate of images processed by individual methods on their combinations used in each step of our algorithm. We also tested the recognition rate by changing the preprocessing flow and the combination. The training set includes 10 images randomly chosen for each person from 5 subsets, and other images are used as testing images. The experimental results are given in Figure 10.

Figure 10:

Recognition rate comparison of the proposed methods and their combinations.

From the experimental results, the sequence of gray scale transformation in spatial domain has obvious effect on the result. When logarithmic transformation was used before histogram equalization, the recognition rate decreased greatly. This validated our noise theory in low-light face images. There are massive mixed noises in original low-light images. Histogram equalization can denoise the complexly mixed noises which include both low and high frequency noises. Because it is based on cumulative function, it does not cut off information by a threshold. The feature of the logarithmic transformation indicated that it is biased to the information in some range.

As it can be seen in Figure 10, the method DeLFN proposed in this paper is proved to be very efficient. Denoising in each level influences each other. And each step has made improvement on the quality of images and the recognition rate.

5.1.4. Time Cost

The face images are randomly selected from the Yale Face Database B and the Extended Yale Face Database B. The size of the images is 168 × 192 pixels. Figure 11 shows the time comparison in preprocessing the images with different methods. Compared to other methods, our implementation can approach an excellent recognition rate while consuming less computational time. As we can see, DeLFN takes only about 11 ms. Hence, it can be used in a real time application.

Figure 11:

The time cost comparison in preprocessing face images randomly selected from the Yale Face Database B and the Extended Yale Face Database B with different methods.

5.2. Experiments on the CMU PIE Face Database

The CMU PIE Face Database [10] contains 68 subjects with different poses, illuminations, and expressions. Our work focuses on the illumination variations, so we only choose the illumination subset (C27) for testing. There are 21 distinct sources of lights used to illuminate the face. In addition, these 21 sources are sampled both with the background room lights on and the background room lights off. We used 68 subjects with 1428 face images in CMU PIE Face Database for this experiment, and each with 21 different illuminations. All test images used in our experiment are cropped in the same way as the Yale Face Database B and resized to 64 × 64.

5.2.1. Visual Quality of Images

Figure 12 shows the 21 different lighting images for a single subject on the CMU PIE Face Database and corresponding illumination normalized images obtained by the proposed method (DeLFN).

Figure 12:

Original images and their illumination compensation images obtained by DeLFN method on the CMU PIE Face Database.

5.2.2. Recognition Rate Comparison between DeLFN and Other Existing Methods

In this experiment, we randomly choose 3, 4, and 5 images from each subject to form the training set and use the rest of the images for testing. We compute the average value of the results over 30 random splits. The recognition results are shown in Table 7. As can be seen, the proposed method outperforms the other methods, including None, HE, HF, MSR, SQI, and HF + HE methods, and is better than or close to the LTV method. By using three images per subject as the training set, the DeLFN algorithm reached 99.84 percent recognition rate on average. By using four and five images per subject as the training set, DeLFN reached a 100 percent recognition rate on average.

Table 7:

Recognition rate comparisons of different methods when using the randomly selected 3, 4, and 5 images as training set, respectively.

Training set	Recognition rate (%)
Training set	3	4	5

None	40.85	43.55	44.87
HE	60.29	65.31	65.99
HF	56.45	57.18	59.19
MSR	80.46	85.62	89.21
SQI	92.05	94.63	95.79
LTV	98.93	99.28	99.86
HF + HE	90.42	91.69	94.52
DeLFN	99.84	100.00	100.00

5.2.3. Time Cost

The face images used in the experiments are randomly selected from CMU PIE Face Database. The size of the images is 64 × 64 pixels. Figure 13 shows the time cost comparison in preprocessing face images with different methods. As we can see, our algorithm consumes less computational time than MSR, SQI, LTV, and HF + HE. DeLFN takes only about 2.7 ms.

Figure 13:

The time cost comparison in preprocessing the face images randomly selected from the CMU PIE Face Database with different methods.

6. Conclusion

In this paper we focused on illumination compensation in low-light face images by denoising. A very simple and efficient three-level image denoising method DeLFN was proposed for face recognition under low-light level environments. DeLFN is implemented with hypothesis of noise frequency distribution in low-light images. The experiments prove that the low- and very low-frequency noise are dominant in low-light images. We used spatial denoising to remove the mixed noise and most of the low-frequency noise. Then we applied high-pass filter in frequency domain to remove residual very low-frequency noise. Compared to other methods, the output image of DeLFN contains more essential information for face recognition and greatly reduces the influence of illumination changes. Experimental results showed that the DeLFN could make a significant progress in image quality for visual effects and recognition under various illumination conditions and it is very stable to violent illumination variation. Finally, the test of time cost shows that the method can be used in a real time application.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

References

Kang

Xie

, and Hu

, “A scale adaptive mean-shift tracking algorithm for robot vision,” Advances in Mechanical Engineering, vol. 2013, Article ID 601612, 11 pages, 2013.

Wang

, and Ao

, “Illumination processing in face recognition,” in Face Recognition, Oravec

, Ed., InTech, 2010.

Land

E. H.

, “An alternative technique for the computation of the designator in the retinex theory of color vision,” Proceedings of the National Academy of Sciences of the United States of America, vol. 83, no. 10, pp. 3078–3080, 1986.

Land

E. H.

and McCann

J. J.

, “Lightness and retinex theory,” Journal of the Optical Society of America, vol. 61, no. 1, pp. 1–11, 1971.

Shashua

and Riklin-Raviv

, “The quotient image: class-based re-rendering and recognition with varying illuminations,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 2, pp. 129–139, 2001.

Buades

Coll

, and Morel

J.-M.

, “A non-local algorithm for image denoising,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '05) vol. 2, pp. 60–65, June 2005.

Chatterjee

Joshi

Kang

S. B.

, and Matsushita

, “Noise suppression in low-light images through joint denoising and demosaicing,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR '11), pp. 321–328, June 2011.

Georghiades

A. S.

Belhumeur

P. N.

, and Kriegman

D. J.

, “From few to many: illumination cone models for face recognition under variable lighting and pose,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 6, pp. 643–660, 2001.

Lee

K.-C.

, and Kriegman

D. J.

, “Acquiring linear subspaces for face recognition under variable lighting,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 5, pp. 684–698, 2005.

10.

Sim

Baker

, and Bsat

, “The CMU pose, illumination, and expression (PIE) database,” in Proceedings of the 5th IEEE International Conference on Automatic Face and Gesture Recognition, pp. 46–51, Washington, DC, USA, 2002.

11.

Jobson

D. J.

Rahman

Z.-U.

, and Woodell

G. A.

, “A multiscale retinex for bridging the gap between color images and the human observation of scenes,” IEEE Transactions on Image Processing, vol. 6, no. 7, pp. 965–976, 1997.

12.

Wang

S. Z.

, and Wang

, “Face recognition under varying lighting conditions using self quotient image,” in Proceedings of the 6th IEEE International Conference on Automatic Face and Gesture Recognition (FGR '04), pp. 819–824, May 2004.

13.

Chen

Yin

Zhou

X. S.

Comaniciu

, and Huang

T. S.

, “Total variation models for variable lighting face recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 9, pp. 1519–1524, 2006.

14.

Zhang

Fang

Yuan

, “Multiscale facial structure representation for face recognition under varying illumination,” Pattern Recognition, vol. 42, no. 2, pp. 251–258, 2009.

15.

Sao

A. K.

and Yegnanarayana

, “On the use of phase of the Fourier transform for face recognition under variations in illumination,” Signal, Image and Video Processing, vol. 4, no. 3, pp. 353–358, 2010.

16.

Zhang

Tang

Y. Y.

Fang

Shang

, and Liu

, “Face recognition under varying illumination using gradientfaces,” IEEE Transactions on Image Processing, vol. 18, no. 11, pp. 2599–2606, 2009.

17.

Cheng

Hou

Zhao

, and Wang

, “Robust face recognition based on illumination invariant in nonsubsampled contourlet transform domain,” Neurocomputing, vol. 73, no. 10–12, pp. 2217–2224, 2010.

18.

Xie

Lai

, and Zheng

W.-S.

, “Extraction of illumination invariant facial features from a single image using nonsubsampled contourlet transform,” Pattern Recognition, vol. 43, no. 12, pp. 4177–4189, 2010.

19.

Pizer

S. M.

Amburn

E. P.

Austin

J. D.

, “Adaptive histogram equalization and its variations,” Computer Vision, Graphics, and Image Processing, vol. 39, no. 3, pp. 355–368, 1987.

20.

Savvides

and Kumar

, “Illumination normalization using logarithm transforms for face authentication,” in Proceedings of the 4th IAPR AVBPA International Conference, pp. 549–556, 2003.

21.

Lindenbaum

Fischer

, and Bruckstein

, “On Gabor's contribution to image enhancement,” Pattern Recognition, vol. 27, no. 1, pp. 1–8, 1994.

22.

Yin

Yang

Gabbouj

, and Neuvo

, “Weighted median filters: a tutorial,” IEEE Transactions on Circuits and Systems, vol. 43, no. 3, pp. 157–192, 1996.

23.

Tomasi

and Manduchi

, “Bilateral filtering for gray and color images,” in Proceedings of the 6th International Conference on Computer Vision, pp. 839–846, Bombay, India, January 1998.

24.

Alvarez

Lions

P.-L.

, and Morel

J.-M.

, “Image selective smoothing and edge detection by nonlinear diffusion. II,” SIAM Journal on Numerical Analysis, vol. 29, no. 3, pp. 845–866, 1992.

25.

Romberg

J. K.

Choi

, and Baraniuk

R. G.

, “Bayesian tree-structured image modeling using wavelet-domain hidden Markov models,” IEEE Transactions on Image Processing, vol. 10, no. 7, pp. 1056–1068, 2001.

26.

Portilla

Strela

Wainwright

M. J.

, and Simoncelli

E. P.

, “Image denoising using scale mixtures of Gaussians in the wavelet domain,” IEEE Transactions on Image Processing, vol. 12, no. 11, pp. 1338–1351, 2003.

27.

Gonzalez

R. C.

and Woods

R. E.

, Digital Image Processing, Pearson/Prentice Hall, Upper Saddle River, NJ, USA, 3rd edition, 2008.

28.

Umbaugh

S. E.

, Computer Imaging: Digital Image Analysis and Processing, CRC Press, Boca Raton, Fla, USA, 2005.

29.

Delac

Grgic

, and Kos

, “Sub-image homomorphic filtering technique for improving facial identification under difficult illumination conditions,” in Proceedings of the International Conference on Systems, Signals and Image Processing (IWSSIP '06), pp. 95–98, 2006.

30.

Short

Kittler

, and Messer

, “A comparison of photometric normalisation algorithms for face verification,” in Proceedings of the 6th IEEE International Conference on Automatic Face and Gesture Recognition (FGR '04), pp. 254–259, May 2004.

31.

Fan

C.-N.

and Zhang

F.-Y.

, “Homomorphic filtering based illumination normalization method for face recognition,” Pattern Recognition Letters, vol. 32, no. 10, pp. 1468–1479, 2011.

32.

Buades

Coll

, and Morel

J. M.

, “A review of image denoising algorithms, with a new one,” SIAM Journal of Multiscale Modeling and Simulation, vol. 4, no. 2, pp. 490–530, 2005.

33.

Hirakawa

and Parks

T. W.

, “Joint demosaicing and denoising,” IEEE Transactions on Image Processing, vol. 15, no. 8, pp. 2146–2157, 2006.

34.

Chatterjee

and Milanfar

, “Is denoising dead?” IEEE Transactions on Image Processing, vol. 19, no. 4, pp. 895–911, 2010.

35.

Sahu

Runger

, and Apley

, “Image denoising with a multi-phase kernel principal component approach and an ensemble version,” in Proceedings of the IEEE Applied Imagery Pattern Recognition Workshop (AIPR '11), pp. 1–7, 2011.

36.

Jiang

and Feng

, “Robust low-rank subspace recovery and face image denoising for face recognition,” in Proceedings of the 18th IEEE International Conference on Image Processing (ICIP '11), pp. 3033–3036, September 2011.

37.

Bharadwaj

Bhatt

Vatsa

Singh

, and Noore

, “Quality assessment based denoising to improve face recognition performance,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW '11), pp. 140–145, June 2011.

38.

Wang

X.-Y.

Yang

H.-Y.

Zhang

, and Fu

Z.-K.

, “Image denoising using SVM classification in nonsubsampled contourlet transform domain,” Information Sciences, vol. 246, pp. 155–176, 2013.

39.

Liu

Y.-L.

Wang

Chen

Guo

Y.-W.

, and Peng

Q.-S.

, “A robust and fast non-local means algorithm for image denoising,” Journal of Computer Science and Technology, vol. 23, no. 2, pp. 270–279, 2008.

40.

Fernando

, “Image Denoising with the Non-local Means Algorithm,” 2010, http://ittc.ku.edu.

41.

Goossens

Luong

Pižurica

, and Philips

, “An improved non-local denoising algorithm,” in Proceedings of the International Workshop on Local and Non-Local Approximation in Image Processing (LNLA '08), pp. 143–156, 2008.

42.

Alter

Matsushita

, and Tang

, “An intensity similarity measure in low-light conditions,” in Computer Vision—ECCV, pp. 267–280, Springer, Berlin, Germany, 2006.

43.

Hoshino

Nielsen

, and Nishimura

, “Noise reduction in CMOS image sensors for high quality imaging: the autocorrelation function filter on burst image sequences,” ICGST-GVIP Journal, vol. 7, no. 3, pp. 17–24, 2007.

44.

Shan

Gao

Cao

, and Zhao

, “Illumination normalization for robust face recognition against varying lighting conditions,” in Proceedings of the IEEE Workshop on AMFG, pp. 157–164, 2003.

45.

Turk

and Pentland

, “Eigenfaces for recognition,” Journal of Cognitive Neuroscience, vol. 3, no. 1, pp. 71–86, 1991.

46.

Liu

Wang

Shao

Guo

, and Xue

, “Optimally maximizing iteration-level loop parallelism,” IEEE Transactions on Parallel and Distributed Systems, vol. 23, no. 3, pp. 564–572, 2012.

A Novel Approach of Low-Light Image Denoising for Face Recognition

Abstract

1. Introduction

2. Related Work

3. Extra-Low-Frequency Noise in Low-Light Face Images

4. Proposed Method

4.1. Preliminary Denoising of Miscellaneous Noises

4.2. Rough Denoising of Low-Frequency Noise

4.3. Fine Denoising of Very Low-Frequency Noise

4.4. Selection and Analysis of the Parameter D0

4.5. Stretching Greyscale to the Whole Gray Range

5. Experiments

5.1. Experiments on Yale Face Database B and the Extended Yale Face Database B

5.1.1. Visual Quality of Images

5.1.2. Recognition Rate Comparison to Existing Methods

5.1.3. Analysis of the Process of DeLFN

5.1.4. Time Cost

5.2. Experiments on the CMU PIE Face Database

5.2.1. Visual Quality of Images

5.2.2. Recognition Rate Comparison between DeLFN and Other Existing Methods

5.2.3. Time Cost

6. Conclusion

Conflict of Interests

References

4.4. Selection and Analysis of the Parameter D₀