Abstract
Considering the objects by different granularity reflects the recognition common law of people, granular computing embodies the transformation between different granularity spaces. We present the image denoising algorithm by using the dictionary trained by granular computing with L∞-norm, which realizes three transformations, (1) the transformation from image space to patch granule space, (2) the transformation between granule spaces with different granularities, and (3) the transformation from patch granule space to image space. We demonstrate that the granular computing with L∞-norm achieved the comparable peak signal to noise ratio (PSNR) measure compared with BM3D and patch group prior based denoising for eight natural images.
Introduction
With the popularization of various digital instruments and digital products, image and video have become the main way to obtain the original information of the outside world and the most common information carriers in human activities. However, in the process of image acquisition, transmission, and storage, image quality is often affected by noise, and image preprocessing algorithm is directly related to the subsequent image processing, such as image segmentation, object recognition, edge extraction, and so on, it is necessary to obtain high-quality digital images.1–3 Therefore, image denoising has been a hot spot in the research of image processing and computer vision.
As a classical problem in low level vision, image denoising is one of the well-studied problems in the field of image processing, yet it is still an active topic for that it provides an ideal test bed for image-modeling techniques.
The search for efficient image denoising methods is still a valid challenge at the crossing of functional analysis and statistics. In spite of the sophistication of the recently proposed methods, most algorithms have not yet attained a desirable level of applicability. 4
Spatial filtering is a direct data operation on the original image, the gray value of the pixel is processed. The common spatial domain image denoising algorithm has the low pass filter, the neighborhood average method, the median filter, etc.5,6
Transform domain image denoising method is a transform of the image. First, the image is transformed from spatial domain to transform domain. Second, the coefficients of transform are processed in the transform domain. Third, the inverse transform of the image from the transform domain into spatial domain to remove image noise. There are many transform domain method, such as, fast fourier transform/discrete fourier transform (FFT/DFT), wavelet transform, which are the common methods for image denoising.7,8
A patch-based generalization of the bilateral filter 9 was proposed in Buades et al. 4 and Awate and Whitaker, 10 where the concept of locality was extended to the entire image. Although the results were encouraging, the true potential for the nonlocal means (NLMs) method was only realized in Kervrann and Boulanger 11 and Kervrann and Boulanger. 12 Another patch redundancy-based framework, i.e., BM3D, 13 adopts a hybrid approach of grouping similar patches and performing collaborative filtering in some transform domain. It ranks among the best performing methods that define the current state of the art. A significantly different approach to denoising was introduced in K-singular value decomposition (SVD). 14 Building on the notion of image patches being sparse representation, 15 Elad et al., proposed a greedy approach for dictionary learning tuned for denoising. In Chatterjee and Milanfar, 16 P. Chatterjee and P. Milanfar proposed a hybrid approach K-locally learned dictionary (K-LLD) that bridged such dictionary-based approaches with the regression-based frameworks.4,9,11 The motivation there was that similar patches shared similar subdictionaries, and such subdictionaries could be used for better image modeling. A similar observation was exploited in the form of a nonlocal sparse model (NLSM) 17 to improve performance of the K-SVD 15 framework. L. Zhang proposed the image denoising algorithm of patch group prior–based denoising (PGPD), in which patch groups are extracted from training images by putting nonlocal similar patches into groups, and a PG-based Gaussian Mixture Model (PG-GMM) learning algorithm is developed to learn the nonlocal self-similarity (NSS) prior. 18
In this paper, we present the image denoising algorithm by using the dictionary trained by granular computing (GrC) with L∞-norm, which realizes three transformations, (1) the transformation from image space to granule space, (2) the transformation between granule spaces with different granularities, and (3) the transformation from granule space-to-image space. We demonstrate that the GrC with L∞-norm achieved the comparable PSNR measure compared with BM3D and PGPD for eight natural images.
The training process by GrC with L∞-norm
For a given clear training image I, we extract N patches with size p × p, and form the training set including N vectors, which are induced by transformation from patch into vector. There is redundancy in the training set S, the purpose of training process is to reduce redundancy by GrC, and then obtain the centers of granules, covariance matrices, mixed Gaussian components by likelihood estimation, such as GMM proposed in ZoranWeiss. 19
In this section, we mainly discuss the GrC with L∞-norm, which includes the representation of granule, the join operation between two granules, and the GrC algorithms with L∞-norm.
The partition of image
The image I is partitioned into some patches with the patch size p × p and the overlapped parameter step by the feature values of pixel points, such as the gray value, RGB (stands for red, green, and blue), HSV (stands for hue, saturation, and value), HSL (stands for hue, saturation, and luminance), and gradient feature (first-order gradient, second gradient). Suppose p = 4 and step = 2, the gray image with 7 × 6 can be partitioned the four patches with 4 × 4 shown in Figure 1.

The partition of image with patch size p = 4 and step = 2.
Granular computing with L∞-norm
As mentioned above, for the given image I, we can partition it into some image patches and form patch set. Each patch is represented as a vector, such as xi = (xi1, xi2, …, xin) and |xi| = p × p for the image patch with size p × p, namely each patch corresponds to a vector.
Representation of granules. In the paper, for the patch set S, the patch granule is composed of the data x whose distance from the center C is less than the granularity. The distance between center vector C = (c1, c2 …, cN) and vector x = (x1, x2 …, xN) can be defined as follows
So we can see the patch granule is determined by two factors, such as the center and the granularity. The patch granule is defined as follows
In general, the patch granule includes some patches when R > 0, especially, the patch granule is a point and indivisible and regarded as atomic granule when R = 0.
For the patch set S = {xi|i = 1,2 …, n} in N-dimensional space, GrC is to train the dictionary including some patch granules by the following steps. First, the granule representation is proposed for the patch. Second, join between two granules are designed, and the threshold of granularity controls the process of operations. Third, the dictionary including patch granules is obtained.
Figure 2 shows the process of image partition, granulation of patches from Figure 1. Each patch is represented as an atomic granule with the granularity 0. As mentioned in Figure 1, each patch is composed by 16(=4 × 4) feature values, which are sorted by indices, so each patch is transferred a vector and represented as a granule named as patch granule.

The representation of patches by granules.
Operations between two granules. In reality, for the same image, the resolution changes with the distance between the image and the human eyes. If we recognize the image patch in detail, we must close to it. In GrC, the granularity is generally the size of granule, and reflects the degree that people recognize object. The less granularity means the more detail recognized by human, on the contrary, people understand objects only as a whole. The operations between two granules realize the transformation between granule space with different granularities. Operation ∨ unites the granules with small granularities to the granules with the large granularities. Inversely, operation ∧ divides the granules with large granularities into the granules with small granularities.
In the paper, we only discuss the operation ∨ which is used to train the dictionary composed by the image patches. A point is regarded as atomic granule with the granularity 0, which is indivisible, the join process is the key to obtain the larger granules compared with atomic granules.
For two patch granules G1 = (C1, R1) and G2 = (C2, R2), the join patch granule is
G = G1∨G2 = (C, R)
The center C and the granularity R of G are computed as follows.
First, the vector from C1 to C2 and vector from C2 to C1 are computed. If C1 = C2, then C12 = 0 and C21 = 0. If C1 ≠ C2, then C12 = (C2 − C1)/d∞(C1,C2) and C21 = (C1 − C2)/d∞(C2,C1), where d∞(.,.) is induced by the formula (1), C12 is the vector with the lenth 1 from C1 to C2, C21 is the vector with the length 1 from C2 to C1.
Second, the crosspoints of G and G1 are P1 = C1 − C12R1 and P2 = C1 + C12R1. The crosspoints of G and G2 are Q1 = C2 − R2C21 and Q2 = C2 + R2C21.
Third, the center C and granularity R of the join patch granule G is computed by algorithm 1.
The framework of GrC. For the patch set S, the GrC includes the following steps for image denoising. First, all the patches induced by the training image sequence are represented as the atomic granules. Second, the threshold of granularity is introduced to conditionally operate ∨ the atomic granules by the aforementioned join operation. Third, the patch granules are obtained and used to form image denoising algorithms. The GrC is described as follows.
For algorithm 2, the patch set is composed of the image patch from an image or image sequence, each PG consists of some patches with the similarity. We explain the algorithm 2 by the selected image named t53 from the image sequence. 20 The image has the size 277 × 285, if the patch size is set to 20 × 20, namely p = 20, and the overlap size is set to 0, namely step = 0, the image is partitioned into 182 image patches which form the patch set. For the image patch marked the red lines shown in Figure 3, there are six image patches (Figure 3(a)) marked the green lines around it if we set ρ = 0.2, there are 24 image patches (Figure 3(b)) marked the green lines around it if we set ρ = 0.4. The situation coincides with the reality. In general, the PG with the larger granularity contains the more patches compared with the PG with the smaller granularity.

The patch granules with the different granularities for the same image.
GrC–GMM learning
A GMM is a parametric probability density function represented as a weighted sum of Gaussian component densities. GMMs are commonly used as a parametric model of the probability distribution of continuous measurements or features. GMM parameters are estimated from training data using the iterative expectation–maximization (EM) algorithm or maximum A posteriori (MAP) estimation from a well-trained prior model. We estimate the parameters of GMM by GrC (GrC–GMM)
We take the learning problem in two-dimensional space as an example to illustrate the GrC–GMM learning process.
The mean and covariance of cluster 1 are [1 4] and
First, we mix all the training data into the data set X and perform GrC to obtain the three clusters by adjusting the parameter ρ, the three clusters are obtained by GrC when the parameter ρ is 4.9.
Second, for each cluster, we estimate the mean and covariance. For the training data, the learned mean and covariance of cluster 1, by GrC with L∞-norm, are m1 = [0.8917 4.1745] and
Third, by SVD, covariance matrix Σ can be factorized as
For the input noisy datum x, the clean x is estimated by the following formula
When
Image denoising via GrC with L∞-norm
For a noisy image y, we partition y into some patches with the same patch size as the patch in learned patch granule dictionary obtained by the aforementioned GrC–GMM, for each patch we search its similar patch granule in the W × W window, denoted by Y = {y1, y2 …, yM}. Then the group mean of Y, denoted by uy, is calculated and subtracted from each patch, leading to the mean subtracted PG
In practice, the clean patch
EM is often combined with the GMM for data clustering in machine learning and computer vision. For algorithm 3, the GrC clustering algorithm is used to perform GMM for image denoising in XuZhang et al., 18 instead of EM. Since the scanning data set is completed one time, the computational complexity of GrC is O(n), where n is the number of data set, for the image denoising, n is the number of patches from the training image sequence.
Experiments
Implementation details
The five training images selected from Yang et al. 20 are used to perform the training process (see Figure 4). If the size of patch is set to 7, there are 576,732 atomic granules induced by the patches. The GrC is performed to reduce the redundancy by the suitable parameter, and the centers of patch granules are used to perform the denoising process.

The training image.
Parameter setting
The patch group–based GrC with L∞-norm for image denoising contains two stages, the prior learning stage by GrC and the denoising stage. In learning stage, there are p and ρ. We set the patch size (p) as p = 7 for σ = 10, σ = 20, σ = 30, σ = 40 in order to compared the proposed image denoising algorithm with state-of-the-art algorithms. 18 In the denoising stage, there are three parameters: c, δ, and η. We set the same c, δ, and η as PGPD in our implementation, namely (c, δ, η) are set to (0:32; 0:08; 0:77), (0:27; 0:10; 0:71), (0:17; 0:07; 0:86), (0:13; 0:05; 0:96) when σ = 10; 20; 30; 40, respectively. In addition, on all noise levels, we stop Algorithm 1 in four iterations.
Comparison methods
We compare the proposed patch group–based GrC for image denoising algorithm with BM3D, 13 EPLL, 19 and PGPD, 18 which represent the state-of-the-arts of modern image denoising techniques and all of them exploit image nonlocal self-similarity (NSS). The source codes of all competing algorithms are downloaded from the authors’ websites and we use the default parameter settings.
Result analysis
We evaluate the competing methods from PSNR shown in Table 1, σ is the noise level, PSNR is the peak signal-to-noise ratio between the denoising image and the original image, the denoising image with the higher PSNR is closer to the original image than the denoising image with lower PSNR. We can analyze the data from the table vertically and horizontally, from a vertical point of view, the PSNR value decreases as the σ value increases, that is why we choose lower noise levels for the four selected denoising algorithms, such as σ = 10, 20, 30, 40, the performances of denoising algorithms are compared from a horizontal point of view.
PSNR (dB) results of different denoising algorithms on eight natural images.
*Bold values indicate the best results.
In Table 1, we present the PSNR results on four noise levels σ = 10, 20, 30, 40, the best PSNRs are highlighted by boldface. From Table 1, we have several observations. First, denoised image by GrC is much better than by EPLL because there are 31 denoised images by GrC, which are better than the denoised images by EPLL for 32 images with different noise level. Second, PGPD achieves much better PSNR results than EPLL, BM3D, and GrC, this demonstrates that the dictionary learned by GrC needs to further optimization. Third, GrC achieved the best results for 7 images of 32 images compared with the other three state-or-the-art image denoising techniques, this shows that the denoising performance can be improved by the more training images. Considering that human subjects are the ultimate judge of the image quality, the visual quality of denoised images is also critical to evaluate a denoising algorithm. The denoised images with the best PSNR compared with the other three methods are shown from Figures 5 to 9.

Denoised images of bloodcell by different methods (σ = 20). (a) Original image. (b) Noisy image. (c) BM3D. (d) EPLL. (e) PGPD. (f) GrC.

Denoised images of cameraman by different methods (σ = 40). (a) Original image. (b) Noisy image. (c) BM3D. (d) EPLL. (e) PGPD. (f) GrC.

Denoised images of house by different methods (σ = 20). (a) Original image. (b) Noisy image. (c) BM3D. (d) EPLL. (e) PGPD. (f) GrC.

Denoised images of lena by different methods (σ = 10). (a) Original image. (b) Noisy image. (c) BM3D. (d) EPLL. (e) PGPD. (f) GrC.

Denoised images of monarch by different methods (σ = 40). (a) Original image. (b) Noisy image. (c) BM3D. (d) EPLL. (e) PGPD. (f) GrC.
Conclusion
NSS prior for image restoration is a class of popular image denoising method and an open problem, and we made a good attempt using the dictionary trained by GrC with L∞-norm on the basis of patch group. After group mean subtraction, a granule induced by image patch can naturally represent the group of image patches. A GrC-based GMM learning algorithm was developed to learn the NSS prior from natural images, and an associated weighted sparse coding algorithm was developed for high-performance image denoising. The feasibility of image denoising is demonstrated by eight natural images and measured by PSNR. On the one hand, in the experiment section of this paper, we only discuss the denoising of gray image, the denoising of color image is our future work because the color image includes multiple features, such as RGB image includes red, green, and blue channels, the denoising algorithm can be performed on three channels respectively, the clean color image is combination of the clean images from three channels. On the other hand, the patch granule space is induced by GrC with L∞-norm in the paper, the GrC can be extended to the other shape of granule, such as granule induced by L1 norm and L2 norm, for image denoising.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported in part by the Natural Science Foundation of China (Grant No. 61170202, 61202287), Henan Natural Science Foundation Project (182300410145), and the Teacher Education Curriculum Reform Project of Henan Province in China (2016-JSJYZD-027).
