Sage Journals: Discover world-class research

Abstract

Internal patch prior (e.g. self-similarity) has achieved a great success in image denoising. However, it is a challenging task to utilize clean external natural patches for denoising. Natural image patch comes from very complex distributions which are hard to learn without supervision. In this paper, we use an autoencoder to discover and utilize these underlying distributions to learn a compact representation that is more robust to realistic noises. By exploiting learned external prior and internal self-similarity jointly, we develop an efficient patch sparse coding scheme for real-world image denoising. Numerical experiments demonstrate that the proposed method outperforms many state-of-the-art denoising methods, especially on removing realistic noise.

Keywords

Image denoising patch prior autoencoder self-similarity sparse coding real-world image

Introduction

Image denoising is a classical ill-posed problem in low-level vision. It aims to recover the original image signal x from its observed noisy version $y = x + n$ , where n represents the additive noise. Without utilizing any image prior, denoising is often performed poorly. Various methods have been proposed to solve such ill-posed problem, including partial differential equations,^1,2 spatial domain filters,^3,4 transform domain filters,^5,6 nonlocal techniques,^7,8 sparse representation,^9,10 etc.

The seminal work of nonlocal means⁷ is based on the assumption that a local patch often has many nonlocal similar patches to it across the image. The use of such image internal self-similarity has significantly enhanced the denoising performance and has led to many good denoising algorithms, such as block-matching three-dimensional filtering (BM3D).⁸ Based on sparse representation, another popular approach is to encode an image patch as a linear combination of a few atoms selected from a dictionary. Due to the seminal work of KSVD,¹¹ learning dictionaries from natural image patches has attracted much attention.^10,12,13 Given a noisy patch matrix Y and the dictionary D of coding atoms, the sparse coding model can be formulates as

\hat{Z} = \arg \min_{_{Z}} | | Y - DZ | |_{F}^{2} + τ | | Z | |_{1}

(1)where τ is the regularization parameter. Once the sparse coefficients matrix

\hat{Z}

is computed, the latent clean patch matrix

\hat{X}

can be estimated as

\hat{X} = D \hat{Z}

. Sparse coding approaches^14,15 toward modeling nonlocal self-similarity have demonstrated promising denoising performance.

Internal self-similarity-based patch methods have been successful in denoising. However, learning good prior from natural patches is a great challenge. The plain multilayer perceptron (MLP) method¹⁵ uses a neural network to learn a denoising procedure from training examples consisting of pairs of noisy and noise-free image patches. By viewing image patches as samples of a multivariate variable vector and considering that natural images are non-Gaussian, Zoran and Weiss¹⁶ learned clean natural image patches using Gaussian mixture models (GMM) with the means and full covariance matrices, and mixing weights over all pixels. Recently, GMM is developed to patch group learning¹⁷ and patch clustering^18,19 for high-performance denoising.

Since image patch space is very complex, there is no guarantee that GMM is a good choice for patch prior learning. Figure 1 shows two patches having very similar values of average intensity and covariance matrix. The GMM classifier¹⁶ could not distinguish them directly. As a consequence, their collaborative filtering may not be effective to restore image intensity. In contrast to GMM which mainly learns the covariance matrices of the clean natural patches, in this paper we take a different approach, inspired by recent advances in unsupervised learning.^20–22 By using clean natural patches, we train an autoencoder to learn patch features that are more robust against realistic noises, because we do not assume the corrupted noise to be additive white Gaussian noise (AWGN). In the denoising stage, the learned external prior will guide internal noisy patch clustering, and followed by a sparse coding scheme to estimate the latent patch group for image recovery.

Figure 1.

Two patches have very similar values of average intensity and covariance matrix.

Image patch learning by autoencoder

An autoencoder neural network is an unsupervised learning algorithm.^20,22 It attempts to map inputs to their hidden representations. Suppose we have a total of N training patches from clean natural images. For the ith patch, let $x_{i}$ denote the input patch and $y_{i}$ be the output version of $x_{i}$ . W is the encoding weight and b is the corresponding bias vector. Similarly, $W'$ and $b'$ are the decoding weight and the decoding bias vectors, respectively. An autoencoder neural network is learned by solving the following optimization problem

\min_{_{W, W', b, b'}} \sum_{i = 1}^{N} | | x_{i} - y_{i} | |_{2}^{2} + γ (| | W | |_{F}^{2} + | | W' | |_{F}^{2})

(2)where

y_{i} = f (W' h_{i} + b')

and

h_{i} = f (W x_{i} + b)

. The first term is an average sum of squares error term. The second term is a regularization term. Here

γ > 0

is a parameter that controls the relative importance of the two terms. Let

f (\cdot)

is Sigmoid function and

| | \cdot | |_{F}

denotes the Frobenius norm. By setting the output

y_{i}

to be equal to the input

x_{i}

, the autoencoder will attempt to replicate its input at its output. When the number of hidden units is less than the size of the input, the autoencoder learns a compressed representation of the input. Encoding image patches to their hidden units gives us compact features, which we could use to build patch prior for clustering.

The hidden features $h_{i}$ learned from the autoencoder network is unsupervised, so it could not be expected to be discriminative. Therefore, we further employ the K-means for clustering. Assume that the obtained N hidden representations are separated into K groups. We introduce class label $C = (c_{1}, c_{2}, \dots, c_{m})$ and let $c_{i} \in {1, \dots, K}$ denote which patch $x_{i}$ is from. After patch clustering by using K-means, we could obtain a set of cluster centers $U = {u_{1}, u_{2}, \dots, u_{K}}$ , and it gives rise to a partition of all training patches into K classes.

Modeling natural patches is challenging because image patches are continuous and high-dimensional. Figure 2(a) visualizes eigenvectors of one Gaussian component of the learned GMM from the Berkeley segmentation dataset.²³ It can be seen that the different eigenvectors only encode a kind of patch structure. Figure 2(b) visualizes the encoding weights of the learned autoencoder. Each input pixels in the encoder has a vector of weights associated with it which will be trained to respond to a particular visual feature of image patches. The encoding weights could extract more compact features from image patches. Therefore, the learned parameters $Θ = {W, b, U}$ of autoencoder could be used to distinguish noisy patches.

Figure 2.

Eigenvectors of one Gaussian component learned by the GMM and encoding weights by the learned autoencoder. (a) Eigenvectors of one Gaussian component learned by the GMM and (b) Encoding weights by the learned autoencoder.

Image denoising by using external patch prior

Nonlocal self-similarity has been widely adopted in patch-based image denoising. However, how to learn the patch prior from clean natural images and apply it to image restoration is still an open problem. Based on the idea that good patch prior should be robust to noises, we include autoencoder-based external patch prior into the denoising framework.

Including external patch prior into the framework

Given a noisy image y, we want to recover the latent image x and extract all overlapped patches into a set, denoted by $X ≐ [x_{1}, x_{2}, \dots, x_{M}]$ , where $x_{i}$ is the ith patch extracted from x. We map each patch to its hidden representation and minimize the within-cluster sum of squares by using learned parameters of autoencoder.

c_{i} = \arg \min_{_{k}} | | f (W x_{i} + b) - u_{k} | |_{2}^{2}

(3)

Then, all overlapped noisy patches could be partitioned into K classes. Let $X_{k}$ denote the matrix formed by the set of vectorized patches from the kth cluster. For the dictionary $D_{k}$ , we learn it adaptively by applying the singular value decomposition (SVD) to the given data matrix $X_{k}$ as $X_{k} = D_{k} S_{k} V_{k}^{⊤}$ . Since X_k exhibits perfect mutual similarity, efficient collaborative filtering can be performed by weighted sparse coding model¹⁹ as follows

{\hat{Z}}_{k} = \arg \min_{_{Z_{k}}} | | X_{k} - D_{k} Z_{k} | |_{F}^{2} + | | T_{k} Z_{k} | |_{1}

(4)

Note that the parameter τ has been implicitly incorporated into the weight matrix $T_{k}$ . Including external patch prior into the denoising framework, the proposed model with weighted sparse coding regularization is the following

\begin{matrix} (\hat{x}, \hat{C}, {{\hat{Z}}_{k}}) = \underset{x, C, {Z_{k}}}{\arg \min} \frac{λ}{σ^{2}} | | y - x | |_{2}^{2} \\ - \sum_{i = 1}^{M} | | f (W x_{i} + b) - u_{c_{i}} | |_{2}^{2} \\ + \sum_{k = 1}^{K} (| | X_{k} - D_{k} Z_{k} | |_{F}^{2} + | | T_{k} Z_{k} | |_{1}) \end{matrix}

(5)where σ is the standard deviation of AWGN. The first term is a fidelity term which ensures that the denoised image x be close to the given image y and is weighted by the positive parameter λ. The second term is to minimize the mean squared distance from each image patch to its nearest cluster center. The proposed model can effectively incorporate external patch priors and internal nonlocal self-similarity for image denoising. Our approach differs from the PCLR¹⁸ in three ways.

Firstly, GMM-based image patch modeling could not effectively capture the natural patch distribution. For example, Figure 3(a) shows a noisy image Monarch with σ = 30 and indicates a noisy patch; Figure 3(b) shows the most similar patches to it by using GMM-based patch clustering; Figure 3(c) shows the most similar patches to it by using the proposed autoencoder-based patch clustering. It can be seen that GMM-based patch prior could not distinguish these patches which have the similar average intensity and covariance matrix. By visual comparison, the proposed method can find better similar patches to the noisy one, and their collaborative filtering could preserve effectively patch structure features.

Figure 3.

GMM model vs. autoencoder. (a) A noisy patch from image Monarch, (b) The most similar patches to it by using GMM-based patch prior and (c) The most similar patches to it by using autoencoder-based patch prior.

Secondly, GMM-based patch clustering generally depends on noise type. Since denoising procedure needs several iterations for better denoising results, the noise distribution is no longer Gaussian after the first iteration. Unlike existing methods^14,17,18 which require estimating the standard deviation of noise to search similar patches in each iteration, auto-encoder-based patch learning could obtain more compact representation, and it could be considered as a non-linear mapping model which is more robust to noises.

Thirdly, the realistic noise in real-world noisy images is much more complex than white Gaussian noise. The low-rank approximation used in PCLR becomes much less effective when applied to real-world noisy images captured by CCD or CMOS cameras. However, the weighted sparse coding scheme can characterize the statistics of realistic noise in patch group.

Algorithm 1

Image denoising by using external patch prior

Input: Noisy image y and learned patch prior $Θ = {W, b, U}$

Output:The recovered image $x^{T}$

Initialization: $x^{0} = y, y^{0} = y$

for $t = 1 : T$ do

Iterative Regularization: $y^{t} = x^{t - 1} + δ (y - y^{t - 1})$ ;

Patch clustering by computing the class label by (3);

For each class do weighted sparse coding model by (6);

Reconstruct the image by aggregation

end for

Denoising algorithm

Owe to the rapid development of non-convex optimization techniques,¹⁹ it is shown that equation (4) has a closed-form solution. By a soft-thresholding operation on the sparse coding coefficient matrix Z_k, the solution can be obtained by

{\hat{Z}}_{k} = sign (D_{k}^{⊤} X_{k}) ⊙ max (| D_{k}^{⊤} X_{k} | - T_{k}, 0)

(6)where

sign (\cdot)

is the sign function and

⊙

means element-wise multiplication. Here,

X_{k} = D_{k} S_{k} V_{k}^{⊤}

is the SVD of X_k. Let

S_{k} (i)

denote the ith singular value of diagonal singular value matrix S_k, and we have the matrix of regularization parameter

T_{k} = diag (τ)

, where

τ_{i} = 1 / (\sqrt{S_{k} (i)} + ϵ)

Due to the non-convexity of the object function, equation (5) is difficult to solve, and the alternating minimization algorithm is commonly employed. Given the prior $Θ = {W, b, U}$ , we start with initial guess $x^{0} = y$ . In the tth iteration, $t \in ℕ$ , we extract all overlapped patches from $x^{t - 1}$ and map each patch to its hidden representation, and then solve for the class label by equation (3). For each cluster, we do weighted sparse coding model for collaborative filtering by equation (6). Next, we return the patches into their place in the image, and reconstruct the image by aggregation. Besides, iterative regularization is adopted to update the observed noisy image in every iteration. Such technique has appeared in existing works,^17,24 so detailed discussions are omitted here. The proposed denoising algorithm is summarized in Algorithm 1.

Experimental results and discussion

To validate the effectiveness of our proposed method, we apply it to both synthetic AWGN corrupted images and real-world noisy images captured by CCD or CMOS cameras. The proposed method contains two stages, the prior learning stage and the denoising stage. In the autoencoder-based learning stage, we use autoencoder with default parameter settings to learn the patch prior from a set of $2 \times 10^{6}$ patches, uniformly sampled from the 200 training images from the Berkeley segmentation dataset.²³ In the denoising stage, we test it on 12 popularly used test images and 15 cropped real-world noisy images, which are shown in Figures 4 and 5, respectively. We empirically set the parameter $δ = 0.18$ and K = 350. Denoted by h_s and $p_{s} \times p_{s}$ the number of hidden units and the patch size, respectively, and we set $h_{s} = 15, p_{s} = 6, T = 3$ for $0 < σ \leq 15$ ; $h_{s} = 20, p_{s} = 7, T = 4$ for $15 < σ \leq 30$ ; $h_{s} = 25, p_{s} = 8, T = 4$ for $30 < σ \leq 50$ ; $h_{s} = 30, p_{s} = 9, T = 5$ for $50 < σ \leq 75$ ; $h_{s} = 35, p_{s} = 10, T = 5$ for $75 < σ \leq 100$ . All parameters are fixed in our experiments.

Figure 4.

The 12 popularly used test images.

Figure 5.

The 15 cropped real-world noisy images used in the dataset.²⁵

Results on AWGN noise removal

To better demonstrate the role of the external patch prior in our model, we compare it with several state-of-the-art denoising methods, including BM3D,⁸ EPLL,¹⁶ DnCNN,²⁶ SAIST,¹⁴ PGPD,¹⁷ and PCLR.¹⁸ In the denoising stage, we test it on 12 popularly used test images. Gaussian white noise with standard deviations $σ = 30, 50, 75$ are added to those test images.

As shown in Table 1, the best two PSNR results of each image are highlighted in bold, and the proposed algorithm outperforms the other methods in most cases in terms of PSNR. When standard deviation of noise fluctuates from 10 to 90, the average PSNR of the proposed method is about 0.4 dB higher than BM3D. The visual comparison of the denoising methods on noise level (σ = 75) is shown in Figure 6. It can be seen that BM3D tends to over-smooth the image, while EPLL, SAIST, and PGPD are likely to generate artifacts when noise level is high. The DnCNN can achieve higher PSNR results by using large-scale training dataset and deep learning; however, it do not work well on images with strong self-similarity (e.g. house, Barbara). Owing to the learned external prior, the proposed method is more robust against artifacts, and it preserves more textures and fine details on images with similar structure than the competing methods. In our MATLAB implementation, it takes approximately 3 min to denoise a 256 × 256 image with standard deviations σ = 30 on a laptop with Intel Xeon E3 CPU(3.40 GHz).

Figure 6.

Denoising results on image Lena by different methods (σ = 75). (a) Clean image, (b) Noisy image, (c) BM3D (PSNR: 27.26 dB), (d) EPLL (PSNR: 26.57 dB), (e) DnCNN (PSNR: 27.57 dB), (f) SAIST (PSNR: 27.23 dB), (g) PGPD (PSNR: 27.40 dB), (h) PCLR (PSNR: 27.41 dB) and (i) Ours (PSNR: 27.54 dB).

Table 1.

Denoising PSNR (dB) results by different denoising methods.

	BM3D	EPLL	DnCNN	SAIS	PGPD	PCLR	Ours
Images
σ = 30
C.man	28.64	28.36	29.27	28.36	28.53	28.82	28.77
Monarch	28.36	28.35	29.25	28.65	28.49	28.83	28.90
House	32.09	31.23	32.29	32.30	32.24	32.17	32.44
Parrot	28.12	28.07	28.61	28.12	28.07	28.34	28.25
Peppers	29.28	29.16	29.93	29.24	29.35	29.56	29.44
Montage	31.38	30.17	31.89	31.06	30.88	31.35	31.60
Lena	31.26	30.79	31.59	31.27	31.27	31.36	31.41
Barbara	29.81	27.57	28.91	30.14	29.38	29.70	30.14
Boat	29.12	28.89	29.38	28.98	29.05	29.21	29.21
Couple	28.87	28.62	29.23	28.72	28.84	28.89	28.88
Hill	29.16	28.90	29.28	29.06	29.09	29.15	29.19
Man	28.86	28.83	29.31	28.81	28.86	29.00	28.98
σ = 50
C.man	26.12	26.02	27.00	26.15	26.46	26.55	26.44
Monarch	25.82	25.78	26.76	26.10	26.00	26.25	26.40
House	29.69	28.76	30.01	30.17	29.93	29.78	30.46
Parrot	25.90	25.84	26.48	25.95	25.93	26.14	26.14
Peppers	26.68	26.63	27.29	26.73	26.80	27.02	27.03
Montage	27.90	27.17	28.95	28.00	28.06	28.20	28.51
Lena	29.05	28.42	29.36	29.01	29.11	29.12	29.23
Barbara	27.23	24.82	26.23	27.51	26.81	27.11	27.52
Boat	26.78	26.65	27.19	26.63	26.85	26.99	26.98
Couple	26.46	26.24	26.89	26.30	26.50	26.56	26.57
Hill	27.19	26.96	27.44	27.04	27.22	27.24	27.27
Man	26.81	26.72	27.24	26.68	26.86	26.94	26.96
σ = 75
C.man	24.33	24.20	25.09	24.30	24.64	24.73	24.64
Monarch	23.91	23.72	24.70	23.98	24.00	24.27	24.45
House	27.51	26.68	27.83	28.08	27.81	27.57	28.39
Parrot	24.19	24.04	24.70	24.17	24.26	24.44	24.46
Peppers	24.73	24.56	25.15	24.66	24.85	25.07	25.11
Montage	25.52	24.90	26.30	25.46	25.70	25.70	26.02
Lena	27.26	26.57	27.57	27.23	27.40	27.41	27.54
Barbara	25.12	22.94	23.62	25.54	24.84	25.06	25.38
Boat	25.12	24.89	25.47	24.99	25.19	25.29	25.34
Couple	24.70	24.45	24.99	24.54	24.70	24.81	24.85
Hill	25.68	25.46	25.95	25.56	25.73	25.82	25.85
Man	25.32	25.14	25.64	25.14	25.36	25.46	25.47

Results on realistic noise removal

This subsection evaluates the proposed method on the publicly available real-world noisy image dataset.²⁵ Since the dataset is very large, Xu et al.²⁷ cropped 15 smaller images of size 512 × 512 to perform experiments. These noisy images are captured by Canon 5D Mark 3, Nikon D600, and Nikon D800 cameras, which are shown in Figure 5. Each scene was captured 500 shots, and the mean image of these 500 shots can be used as a kind of ground-truth to compute the PSNR and SSIM.²⁸ We compare the proposed method with CBM3D,²⁹ DnCNN,²⁶ the commercial software Neat Image (NI),³⁰ the Noise Clinic algorithm (NC),³¹ and GID.¹⁹ For CBM3D, PCLR, and the proposed method, we use statistical method³² to estimate the standard deviation of noise. For blind mode DnCNN, we use its color version provided by the authors and there is no need to estimate the standard deviation of noise. In order to handle RGB images, we extend PCLR and the proposed method by stacking three channels of color images.

The results on PSNR (dB) and average computational time of different methods are listed in Table 2. The best PSNR results of each image are highlighted in bold. One can see that on 11 out of the 15 images, our method achieves the best PSNR values. This is because real world noise is much more complex than Gaussian noise. The visual comparisons of the denoising methods are shown in Figure 7. In addition, all experiments are run on a laptop with Intel Xeon E3 CPU (3.40 GHz). The fastest result is highlighted in bold. One can see that CBM3D and DnCNN generate some noise-caused color artifacts across the whole image, while PCLR tends to over-smooth the image a little. These results show that the methods designed for AWGN are not effective for realistic noise removal. The methods NI and NC do not work well on removing the realistic noises effectively. Since autoencoder-based patch learning is more robust to realistic noise, the proposed method works much better in removing the noise while maintaining the details (see the zoom-in window) than the competing method GID.

Figure 7.

Denoised images of a region cropped from the real-world noisy image by different methods. The images are better to be zoomed-in on screen. (a) Noisy image (PSNR 29.63 dB), (b) CBM3D (PSNR 31.13 dB), (c) DnCNN (PSNR 29.83 dB), (d) NI (PSNR 31.28 dB), (e) NC (PSNR 33.49 dB), (f) PCLR (PSNR 35.04 dB), (g) GID (PSNR 33.28 dB), (h) Ours (PSNR 35.15 dB) and (i) Mean image.

Table 2.

PSNR (dB) results and averaged computational time (s) on 15 cropped real-world noisy images used in the dataset.²⁵

Real-world images	CBM3D	DnCNN	NI	NC	PCLR	GID	Ours
Canon 5D ISO3200 1	39.76	37.26	37.68	38.76	40.25	40.50	40.02
Canon 5D ISO3200 2	36.40	34.13	34.87	35.69	36.09	37.05	36.21
Canon 5D ISO3200 3	36.37	34.09	34.77	35.54	35.31	36.11	35.51
Nikon D600 ISO3200 1	34.18	33.62	34.12	35.57	35.40	34.88	35.46
Nikon D600 ISO3200 2	35.07	34.48	35.36	36.70	36.96	36.31	36.88
Nikon D600 ISO3200 3	37.13	35.41	38.68	39.28	40.61	39.23	41.02
Nikon D800 ISO1600 1	36.81	35.79	37.34	38.01	39.29	38.40	39.49
Nikon D800 ISO1600 2	37.76	36.08	38.57	39.05	42.12	40.92	42.20
Nikon D800 ISO1600 3	37.51	35.48	37.87	38.20	38.45	38.97	38.49
Nikon D800 ISO3200 1	35.05	34.08	36.95	38.07	39.51	38.66	40.03
Nikon D800 ISO3200 2	34.07	33.70	35.09	35.72	37.17	37.07	37.14
Nikon D800 ISO3200 3	34.42	33.31	36.91	36.76	40.29	38.52	41.67
Nikon D800 ISO6400 1	31.13	29.83	31.28	33.49	35.04	33.76	35.15
Nikon D800 ISO6400 2	31.22	30.55	31.38	32.79	33.94	33.43	34.13
Nikon D800 ISO6400 3	30.97	30.09	31.40	32.86	34.02	33.58	34.05
Average	35.19	33.86	35.49	36.43	37.63	37.15	37.83
Time (s)	8.11	180.72	1.53	18.92	251.64	204.73	235.52

Conclusion

This paper extends patch clustering-based image denoising by including external patch prior, since the inherent complexity of patch space, autoencoder, is used to learn the patch feature in order to build a good low-dimensional representation. In contrast to GMM, the proposed denoising algorithm can find better similar patches and make it more efficient to preserve edge and texture areas. Experimental results showed the proposed algorithm can achieve very competitive denoising performance. In particular, it can preserve better texture structures under realistic noise environment than the other state-of-the-art denoising algorithms.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Natural Science Foundation of China (61771141), the Natural Science Foundation of Fujian Province (2017J01751, 2017J01502), and the Scientific Research Foundation of Fuzhou University (XRC-17015).

References

Perona

Malik

Scale-space and edge detection using anisotropic diffusion. IEEE Trans Pattern Anal Machine Intell 1990; 12: 629–639.

Rudin

Osher

Fatemi

Nonlinear total variation based noise removal algorithms. Physica D 1992; 60: 259–268.

Chen

Zeng

Wang

Image denoising via local and nonlocal circulant similarity. J Vis Commun Image R 2015; 30:117–124.

Tomasi

Manduchi

Bilateral filtering for gray and color images. In: Proceedings of the ICCV, 7 January. 1998.

Portilla

Strela

Wainwright

, et al. Image denoising using scale mixtures of Gaussians in the wavelet domain. IEEE Trans Image Process 2003; 12: 1338–1351.

Tian

Jiao

Guo

An affinity-based algorithm in nonsubsampled contourlet transform domain: application to synthetic aperture radar image denoising. J Sign Process Syst 2016; 83: 373–388.

Buades

Coll

, and Morel

JM.

A non-local algorithm for image denoising. In: Proceedings of the CVPR, 20-25 June 2005.

Dabov

Foi

Katkovnik

, et al. Image denoising by sparse 3-d transform-domain collaborative filtering. IEEE Trans Image Process 2007; 16: 2080–2095.

Dong

Zhang

Shi

Centralized sparse representation for image restoration. In: Proceedings of the ICCV, 6-13 November 2011.

10.

Elad

Aharon

Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans Image Process 2006; 15: 3736–3745.

11.

Aharon

Elad

Bruckstein

K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans Signal Process 2006; 54: 4311–4322.

12.

Chatterjee

Milanfar

Clustering-based denoising with locally learned dictionaries. IEEE Trans Image Process 2009; 18: 1438–1451.

13.

Rubinstein

Bruckstein

Elad

Dictionaries for sparse representation modeling. Proc IEEE 2010; 98: 1045–1057.

14.

Dong

Shi

Nonlocal image restoration with bilateral variance estimation: a low-rank approach. IEEE Trans Image Process 2013; 22: 700–711.

15.

Burger

Schuler

Harmeling

Image denoising: can plain neural networks compete with bm3d? In: Proceedings of the CVPR, 16-21 June 2012.

16.

Zoran

Weiss

From learning models of natural image patches to whole image restoration. In: ICCV, 6-13 November 2011.

17.

Zhang

Zuo

, et al. Patch group based nonlocal self-similarity prior learning for image denoising. In: ICCV, 7-13 December 2015.

18.

Chen

Zhang

External patch prior guided internal clustering for image denoising. In: ICCV, 7-13 December 2015.

19.

Zhang

External prior guided internal prior learning for real-world noisy image denoising. IEEE Trans Image Process 2018；27：2996 - 3010.

20.

Coates

HLA

An analysis of single-layer networks in unsupervised feature learning. In: NIPS, 10 December 2010.

21.

Hinton

GE.

A practical guide to training restricted Boltzmann machines. In: Neural networks: tricks of the trade. 2nd ed. Berlin: Springer, 2012, pp.599–619.

22.

Vincent

Larochelle

Bengio

, et al. Extracting and composing robust features with denoising autoencoders. In: ICML, 5-9 July 2008.

23.

Martin

Fowlkes

Tal

, et al. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings of the ICCV, 7-14 July 2001.

24.

Zhang

Zuo

, et al. Weighted nuclear norm minimization with application to image denoising. In: Proceedings of the CVPR, 24-27 June 2014.

25.

Nam

Hwang

Matsushita

, et al. A holistic approach to cross-channel image noise modeling and its application to image denoising. In: CVPR, 27-30 June 2016.

26.

Zhang

Zuo

Chen

, et al. Beyond a Gaussian denoiser: residual learning of deep CNN for image denoising. IEEE Trans Image Process 2017；26：3142 - 3155.

27.

Zhang

A trilateral weighted sparse coding scheme for real-world image denoising. In: ECCV, 8-14 September 2018.

28.

Wang

Bovik

Sheikh

, et al. Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 2004; 13: 600–612.

29.

Dabov

Foi

Katkovnik

, et al. Color image denoising via sparse 3d collaborative filtering with grouping constraint in luminance-chrominance space. In: ICIP, 16-19 September 2007.

30.

ABSoft

Neat image, https://ni.neatvideo.com/home (accessed 1 October 2019).

31.

Lebrun

Colom

Morel

Multiscale image blind denoising. IEEE Trans Image Process 2015; 24: 3149–3161.

32.

Chen

Zhu

Pheng

An efficient statistical method for image noise level estimation. In: ICCV, 7-13 December 2015.

Autoencoder-based patch learning for real-world image denoising

Abstract

Keywords

Introduction

Image patch learning by autoencoder

Image denoising by using external patch prior

Including external patch prior into the framework

Denoising algorithm

Experimental results and discussion

Results on AWGN noise removal

Results on realistic noise removal

Conclusion

Footnotes

Declaration of Conflicting Interests

Funding

References