Image fusion algorithm based on improved K-singular value decomposition and Hadamard measurement matrix

Abstract

To solve the problem of high time and space complexity of traditional image fusion algorithms, this paper elaborates the framework of image fusion algorithm based on compressive sensing theory. A new image fusion algorithm based on improved K-singular value decomposition and Hadamard measurement matrix is proposed. This proposed algorithm only acts on a small amount of measurement data after compressive sensing sampling, which greatly reduces the number of pixels involved in the fusion and improves the time and space complexity of fusion. In the fusion experiments of full-color image with multispectral image, infrared image with visible light image, as well as multispectral image with full-color image, this proposed algorithm achieved good experimental results in the evaluation parameters of information entropy, standard deviation, average gradient, and mutual information.

Keywords

Sparse representation measurement matrix image reconstruction image fusion

Introduction

The image fusion algorithm includes pixel-level fusion, feature-level fusion, and decision-level fusion. After denoising and registration of the original image, pixel-level fusion is used to fuse the pixel value of the original image, which is a basic fusion method for pixel points. Pixel-level fusion algorithm can effectively maintain pixel information of images and is conducive to feature extraction and target recognition. Feature-level fusion first extracts features from original images and then extracts features from texture, edge, corner, line, and specific area, and uses the extracted feature parameters to fuse. After the feature extraction and understanding of the original image, the decision-level fusion is fused according to the understanding result.

The pixel-level fusion algorithm is the main method used to enhance the image processing and is beneficial to extract features. However, the time and space complexity are high with the increasing of image data; real-time fusion algorithm is facing challenges. Compressed sensing theory utilizes the sparsity of images. At the same time, the image is compressed to reduce the sampling frequency and reduce the sampling amount of image data. Meanwhile, the original image signal can be recovered by the reconstruction algorithm and the image fusion algorithm based on compressed sensing theory has become an important research content of image analysis and processing.

The compressive sensing theory breaks through the requirements of traditional Nyquist sampling theorem. The signal is sampled with much lower sampling rate than the Nyquist sampling rate. This reduces the amount of sampled signal data and greatly reduces the requirements for data transfer, processing, and storage. In the theory of compressive sensing, a small amount of data obtained after dimensional reduction measurement can be used to effectively reconstruct the original signal with a certain nonlinear optimization algorithm. Compared with the direct image sparse process, the compressive sensing theory provides a new image acquisition and processing framework, which provides a new research idea for the current large data amount image fusion.

The image fusion process mainly includes image preprocessing, image registration, image feature extraction, image classification and recognition, information understanding and mining. Image fusion can be performed in several stages after image registration. After the registration of the to-be-fused images, the image fusion algorithm based on compressive sensing theory first conducts dimensional reduction transformation on the two images using the designed measurement matrix.¹ The measurement values of the two images are then obtained. And then, the coefficient fusion is conducted on the measured values according to the fusion rules in the compressive sensing domain. Finally, the fused image is reconstructed according to the reconstruction algorithm based on compressive sensing theory.

This paper elaborates the framework of image fusion algorithm based on compressive sensing theory. A new image fusion algorithm based on improved K-SVD and Hadamard measurement matrix is proposed. This proposed algorithm only acts on a small amount of measurement data after compressive sensing sampling, which greatly reduces the number of pixels involved in the fusion and improves the time and space complexity of fusion.

The improved K-SVD algorithm

Aharon et al.² proposed the K-singular value decomposition to construct the dictionary based on K-means algorithm. In the K-SVD algorithm, dictionaries are updated by using image sparse representation coefficients; dictionary training samples and learning is achieved through iterative method. The K-SVD algorithm can effectively reduce the number of atoms in a dictionary; the dictionary atoms can effectively represent the image feature information. In the construction of the dictionary, we will consider the image smoothing, edge contour, and texture features. According to the geometric features of the image, we construct the multicomponent dictionary such as smooth, edge contour, and texture structure.

The algorithm used X represents a sample set of dictionary training, and $x_{i}$ represents a training sample, $X = {x_{i} | x_{i} \in R^{k}, i = 1, 2, … n}$ _, D represents the dictionary, $D = [d^{1}, d^{2}, … d^{m}]$ , $D \in R^{k \times m}$ , where $d^{i}$ is a list of D; $R^{k \times m}$ is an atom; $λ$ is used to balance the reconstruction error; A represents the matrix coefficient; $α^{i}$ represents the row of coefficient matrix, and $α^{i}$ represents the coefficient vector of corresponding image $X^{i}$ . The sparse representation of image is transformed into the optimization problem of formula (1), which can be solved by iterative alternating algorithm

\min_{D, α} | | X - D A | |_{F}^{2} + λ | | A | |_{1, 1} s . t . | | d^{j} | |_{2} \leq 1, \forall j \in 1, 2, … m

(1)

In formula (1), $| | . | |_{F}$ represents the Frobenius norm.² When the dictionary D and the coefficient matrix A are fixed, formula (1) is a convex optimization function. When the dictionary D is fixed, the solution of the coefficient matrix A can be solved by the $l_{1}$ norm, and the formula of the coefficient matrix A is shown in formula (2)

\min_{α^{i}} | | x^{i} - D α^{i} | |_{2}^{2} + λ | | α^{i} | |_{1}

(2)

In the overcomplete sparse representation, D is often optimized by the MOD algorithm.^3,4 First, the partial derivative of the function $| | X - D A | |_{F}^{2}$ is solved, and then the $(X - D A) A^{T} = 0$ equation is obtained; finally, the dictionary D is obtained by the inverse solution $D = X A^{T} {(A A^{T})}^{- 1}$ .

The K-SVD algorithm was proposed by Aharon et al.; this algorithm is a singular value decomposition to update the dictionary^2,3 in order to reduce the error of sparse representation. The K-SVD algorithm includes the sparse encoding and dictionary update.

In this paper, according to the geometric characteristics of the image, we construct the smooth, edge contour, and texture structure of the multicomponent dictionary based on the K-SVD algorithm. In this algorithm, the dictionary D is considered as a multicomponent dictionary and the dictionary updating is implemented by means of classification updating.^5,6

Let $d_{L k}$ represent the k atom in the L class dictionary; $X_{T}^{k}$ represent the sparse coefficient of the K row in the sparse matrix X; $E_{k}$ represent the expressed as the approximation error matrix; and the atoms in the multicomponent overcomplete dictionary D are classified and updated one by one. The objective optimization function in formula (3) can be obtained

\begin{array}{l} ‖ Y - D X {‖_{F}}^{2} = | | Y - (\sum_{j \neq k} d_{L j} {X_{T}}^{j} + d_{L k} {X_{T}}^{k}) | |_{F}^{2} \\ = | | (Y - \sum_{j \neq k} d_{L j} X_{T}^{j}) - d_{L x} X_{T}^{k} | |_{F}^{2} \\ = | | E_{k} - d_{L k} X_{T}^{k} | |_{F}^{2} \end{array}

(3)

Each atom $d_{L k}^{j}$ in the dictionary $D_{L}$ is updated one by one, and the subset of signals represented by $d_{L k}^{j}$ is obtained $Ω_{L k}^{j} = {f_{i} | a^{j} (k, i) \neq 0}, i = 1, 2, …, M$ . The residual error is calculated by formula (4)

R_{L k}^{j} = Ω_{L k}^{j} - d_{L k}^{j} . X^{j} (k)

(4)

In formula (4), $X^{j}$ represents the sparse factorization coefficient matrix. The algorithm selects the column $E_{L K}^{R}$ related to $d_{L k}$ to form a new matrix from $E_{k}$ , and then makes the SVD decomposition of $E_{L K}^{R}$ , and obtains the updated atomic $\bar{d_{L k}}$ (the first column of U). The first column of V and the singular value $Δ (1, 1)$ are multiplied to obtain the new coefficient ${\bar{X}}_{R}^{k}$ , and constructs the gradient operator $\nabla$ , $\nabla = - R_{k}^{j} . p i n v (X^{j} (k))$ . Then, each atom X in the classified dictionary D is updated one by one in which $μ$ represents the set step factor. According to the updated atoms, the gradient operator is constructed iteratively, and the update is completed in an iterative way.⁷

Hadamard measurement matrix

Hadamard matrix is defined as:⁸ a square matrix $H n$ ( $N \times N$ ) with n order, all elements are +1 or −1 in it, if the square $H n$ satisfies the condition of formula (5), the matrix $H n$ is Hadamard matrix

H_{n} {H_{n}}^{T} = {H_{n}}^{T} H_{n} = n I_{n}

(5)

The construction of Hadamard matrix can use the concept of hypercube in graph theory,⁹ and the four-dimensional hypercube structure is shown in Figure 1.

Figure 1.

The four-dimensional hypercube structure.

The construction of hypercube is constructed by $2^{n}$ points in n-dimensional space, and $2^{n}$ points are labeled as ${x_{0}, x_{1}, x_{2}, … x_{2^{n} - 1}}$ . If the distance between points $x_{i}, x_{j}$ satisfies $\sqrt{{(x_{i} - x_{j})}^{2}} = 1$ , it is defined as an edge of hypercube, and the length of this edge is marked as 1. According to the relation between vertex and edge of hypercube, the matrix representation of the following undirected graph is obtained

[\begin{matrix} 0 & 0 & … & 0 & 0 \\ 0 & 0 & … & 0 & 1 \\ 0 & 0 & … & 1 & 0 \\ … & … & … & … & … \\ 1 & 1 & … & 1 & 1 \end{matrix}]

According to the hypercube graph representation matrix X, the calculation of the shortest distance between two points and the distance between two markers represent the $d_{i j}$ if $d_{i j}$ =1 or $d_{i j}$ =2. ${H_{i}}_{j}$ corresponding to the constructed measurement matrix is labeled 1 and $d_{i j}$ is the rest; and then ${H_{i}}_{j}$ is labeled as −1, and the construction of the Hadamard matrix is realized. The four order Hadamard matrix constructed by this algorithm is H as follows

H = [\begin{matrix} - 1 & 1 & 1 & 1 \\ 1 & - 1 & 1 & 1 \\ 1 & 1 & - 1 & 1 \\ 1 & 1 & 1 & - 1 \end{matrix}]

An image fusion algorithm based on improved K-SVD and Hadamard measurement matrix

Based on the study of compressive sensing theory, this paper proposes an image fusion algorithm based on the improved K-SVD and Hadamard measurement matrix. First, the to-be-fused images are sparsely represented with the improved K-SVD algorithm. And then, the dimensional reduction measurement is conducted with the improved Hadamard observation matrix. The image fusion is completed in the compressive sensing domain. Finally, the fused images are reconstructed using the reconstruction algorithm based on the improved Hadamard measurement matrix.^10,11

The description and implementation steps of the fusion algorithm based on improved K-SVD sparse representation and Hadamard measurement matrix are shown in Algorithm ¹.

Algorithm 1. The implementation steps of the fusion algorithm based on improved K-SVD sparse representation and Hadamard measurement matrix

Step 1: Accurate registration for images F1 and F2.

Step 2: Sparse representation for the two images with the improved K-SVD algorithm.

First, an image Y is divided into $\sqrt{n} \times \sqrt{n}$ subblocks. The divided subblocks are rearranged into vectors $Y_{H} \in R^{n}$ . Based on the image geometric characteristics, the subblocks are classified according to different characteristics of smooth, edge contours, texture structure, and more. Different types of subblocks use different sparse dictionaries. One set of base $Φ_{r}$ corresponds to one characteristic.^2,4 The total bases $Φ_{r}$ are combined into a set of redundant base, represented by the dictionary D, $D \in R^{n \times k}, k > n$ . The dictionary D is initialized to ${D^{0}}_{L} = {d_{L k}^{0}; k = 1, 2, …, K}$ .

The image vector $Y_{H}$ is represented sparsely as $Y_{H} \approx D X$ . The error satisfies $| | Y_{H} - D X | |_{p} \leq ε$ . The sparse decomposition of the image signal after subblock division is conducted, $X = O M P (Y_{H}, D^{j})$ . $X$ is the coefficient matrix of $K \times M$ . The sparse coefficient matrix X can also be obtained by solving $\hat{X} = \underset{x}{\arg \min} | | Y_{H} - D X | |_{2}^{2} + μ | | X | |_{0}$ .

Update each element $d_{L k}^{j}$ of the dictionary ${D^{j}}_{L}$ one by one according to the target optimization function. The signal subset represented by $d_{L k}^{j}$ is first obtained, $Ω_{k}^{j} = {f_{i} | X^{j} (k, i) \neq 0}, i = 1, 2, …, M$ . And then, the residual is calculated using formula $R_{L k}^{j} = Ω_{L k}^{j} - d_{L k}^{j} . X^{j} (k)$ . The gradient operator $\nabla$ is then constructed as $\nabla = - R_{L k}^{j} . p i n v (X^{j} (k))$ . Each element $d_{L k}^{j}$ is updated one by one, $d_{L k}^{j + 1} = d_{L k}^{j} - μ . \nabla$ (where $μ$ is the chosen step size factor). The gradient operator is reconstructed repeatedly according to the obtained updated elements.

The number of iteration $i$ increases by 1. And the iteration condition is judged if $| | Y_{H} - D^{(i)} X^{(i)} | |_{F}^{2} \leq δ$ ( $δ$ represents the reconstruction error) or the upper limit of the predefined iteration number M is reached; the iteration is terminated and the training dictionary $D'$ is obtained. Otherwise, Step2 (b) is continued until the iteration termination condition is satisfied.¹²

Step 3: Conduct dimensional reduction measurements on images F1 and F2 with the improved Hadamard measurement matrix.

Step 4: Carry out coefficients fusion with the measured values. In order to overcome the problem that important information may be discarded when the fusion is achieved with the fusion rule of choosing bigger absolute value, the weighted fusion method is selected as the fusion rule of this proposed algorithm. Where the weighting coefficient is determined according to the ratio of the obtained norm of vector $l_{0}$ in the sum of the entire measurement vector norm $l_{0}$ after the projection measurement of subimage.¹³

Step 5: Conduct image reconstruction on the measurement values after the fusion.

Y represents the observation vector; K represents the degree of sparseness; H represents the improved Hadamard measurement matrix; $r$ represents the residual and the residual $r$ is initialized to $r^{0} = Y$ . The iteration number n is set to 0 and the parameters are initialized to $x^{0} = 0, Γ^{0} = \emptyset$ . $x_{0}$ represents the reconstructed sparse solution.^14,15 I represents the index setting I = E. The measured values after the image fusion is reconstructed, and the steps are as follows:

Calculate $g^{n} = < r^{n - 1}, H >;$

Calculate $i^{n} = \underset{i}{\arg \max} | g_{i}^{n} |$ , and update $Γ^{n}$ , $Γ^{n} = Γ^{n - 1} \cup i^{n}$ .

Calculate the update of the direction based on formula $a^{n}$ , $a^{n} = \frac{< r^{n}, H_{Γ^{n}} {d^{n}}_{Γ^{n}}}{| | H_{Γ^{n}} {d^{n}}_{Γ^{n}} | |_{2}^{2}}$ .

Estimate $x_{Γ^{n}}$ , $x_{Γ^{n}}^{n} = x_{Γ^{n}}^{n - 1} + a^{n} d_{Γ^{n}}^{n}$ .

Update $r^{n}$ , $r^{n} = r^{n - 1} - a^{n} H_{Γ^{n}} {d^{n}}_{Γ^{n}}$ .

Judge the iteration condition if $| r^{n} - r^{n - 1} | \leq 10^{- 6}$ , or the iteration number n reaches the predefined upper limit; the iteration is terminated and the results $r^{n}$ and $x_{Γ^{n}}^{n}$ are output. If the iteration termination condition is not satisfied, proceed to Step5 (a) to continue.

Image fusion performance evaluation

Currently, there are mainly two types of evaluation methods for the fused images: subjective evaluation and objective evaluation. The subjective evaluation method is mainly based on the subjective judgment by the image observation with human eye. The judgment results are greatly influenced by subjective factors.¹⁶ The objective evaluation is quantitative evaluation through the statistical characteristics of the image and the relationship with other images.

Subjective evaluation

Subjective visual evaluation is based on the observation of the generated image by the fusion algorithm. The image quality after the fusion is evaluated and the judgment is made.¹⁷ The current evaluation criteria for subjective evaluation are shown in Table 1.

Table 1.

Subjective evaluation criteria.

Score	Grade	Quality description
5	The best	The human eye cannot identify the quality difference between images
4	Good	The human eye can identify the image quality deterioration, but it does not affect the usage
3	OK	The human eye can identify the image quality deterioration; it has small impact on the usage
2	Bad	The human eye can clearly observe the image quality deterioration; it has moderate impact on the usage
1	The worst	The human eye can clearly observe the image quality deterioration; it has serious impact on the usage

The subjective visual evaluation makes judgments based on personal perception. There is a lot of one-sidedness and uncertainty. It should not be used independently in the image fusion evaluation methods. However, it can be used as the auxiliary reference method for the objective evaluation.

Objective evaluation

Because the image fusion lacks standard images as the reference, the fusion result image cannot be compared with the standard image for the purpose of analysis. Therefore, the traditional performance evaluation parameters for image processing, such as mean squared error and peak signal to noise ratio, cannot be used for the objective performance evaluation of fusion experiments.

In this paper, the current commonly used nonreference image evaluation methods are used to analyze the experimental results, including information entropy (H_X), standard deviation ( $S_{F}$ ), cross entropy (C), fusion factor (FF), union entropy (UE), and definition and correlation coefficient (CC). For the calculation of evaluation parameters, please refer to Yutao,¹⁷ Juan,¹⁸ and Qiang.¹⁹

Experimental analysis

Two 256 × 256 images are selected for the experiment, as shown in Figures 2 to 5. Figures 2 and 3 are the registered images (F1 and F2). The gray levels of the images F1 and F2 are both 256.

Figure 2.

Image F1 after the registration.

Figure 3.

Image F2 after the registration.

Figure 4 shows the wavelet sparse transformation of the source image F1. A large number of black pixels (i.e. value 0 points) can be seen from a layer of wavelet decomposition sparse representation in Figure 4. Figure 5 shows the fusion result image R1 of images F1 and F2 with the proposed image fusion algorithm based on the improved K-SVD sparse representation and the Hadamard measurement matrix.

Figure 4.

The wavelet sparse transformation of image F1.

Figure 5.

The fusion result image R1.

For the source images F1 and F2, the fusion result image R1 based on the improved K-SVD sparse representation and the Hadamard measurement matrix is analyzed with the parameters of information entropy $H_{X}$ , standard deviation $S_{F}$ , and cross entropy C. The evaluation parameters for fusion experiments are shown in Table 2.

Table 2.

The evaluation parameters of image fusion.

	Evaluation parameters
Image	Information entropy $H_{X}$	Standard derivation $S_{F}$	The cross entropy C with fused image
Source image F1	7.198	39.893	1.227
Source image F2	6.497	38.127	0.849
Fused image R1	7.372	39.563	–

The experimental results show that the information entropy and standard deviation of the obtained fusion images R1 are improved. The evaluation parameters of the obtained fusion images from the fusion algorithm based on the improved K-SVD sparse representation and the Hadamard measurement matrix are fine. The satisfactory fusion performance is achieved.

In this paper, multisource image fusion experiments are implemented on two image fusion algorithms based on the compressive sensing. In order to verify the effectiveness of the fusion algorithm, the experiments use the principal component analysis method and the proposed fusion method based on the improved K-SVD sparse representation and the Hadamard measurement matrix. The fusion experimental analysis on full-color image and multispectral remote sensing image are implemented. The fusion results are shown in Figure 6. In this figure, (a) is a multispectral remote sensing image F1, (b) is a full-color image F2, and the image pixel size is 256 × 256.

Figure 6.

Experimental results of fusion algorithm. (a) Multispectral image F1, (b) full-color image F2, (c) image fusion method used in pyramid, (d) fusion algorithm based on wavelet transform and Fourier measurement matrix, (e) fused image with the principal component analysis method, and (f) the fusion result image based on the improved K-SVD sparse representation and the Hadamard measurement matrix.

Figure 6(a) is the multispectral image F1, the selected bands are 4, 3, and 2; the image color is reddish and the spectral information is rich. Figure 6(b) is the full-color image F2 and it has high spatial resolution. Figure 6(c) is the fusion result image based on the pyramid-type algorithm. Figure 6(d) is the fusion result image based on the wavelet transform and Fourier measurement matrix. Figure 6(e) is the fusion result image based on the principal component analysis method. Figure 6(f) is the fusion result image obtained by the proposed fusion algorithm based on the improved K-SVD sparse representation and the Hadamard measurement matrix. From the subjective evaluation point of view, the fusion results of the two algorithms both can effectively combine the characteristics of two images. “M” represents the multispectral image, “P” represents the full-color image, and “F” represents the fused image. The objective evaluation parameters of the two algorithms are shown in Table 3.

Table 3.

Comparison of SAM, ISAM, and CC parameters between each fusion result image and multispectral image, full-color image.

Fusion methods	Multispectral bands	SAM_MF	ISAM_MF	CC_MF	SAM_PF	ISAM_PF	CC_PF
Principal component analysis method	4	0.5652	0.5763	0.0574	0.2831	0.2853	0.7612
	3	0.5312	0.5316	0.1542	0.1558	0.1564	0.9213
	2	0.5328	0.5332	0.1481	0.1928	0.1939	0.8813
Pyramid-type algorithm	4	0.5165	0.5187	0.1563	0.1467	0.1502	0.9289
	3	0.4969	0.4989	0.2081	0.1268	0.1318	0.9479
	2	0.4919	0.4928	0.2078	0.1232	0.1309	0.9517
Wavelet transform and Fourier measurement matrix	4	0.5571	0.5376	0.1141	0.1905	0.1905	0.8746
	3	0.5278	0.5382	0.2141	0.0945	0.0923	0.9621
	2	0.5169	0.5169	0.2370	0.1243	0.1245	0.9552
The improved K-SVD sparse representation and the Hadamard measurement matrix	4	0.5273	0.5480	0.1062	0.1897	0.1798	0.8792
	3	0.5122	0.5224	0.2054	0.0617	0.0625	0.9882
	2	0.5173	0.5275	0.1808	0.0406	0.0458	0.9757

CC: correlation coefficient; ISAM: improved spectral angular mapping; K-SVD: K-singular value decomposition; SAM: spectral angular mapping.

In this paper, the fusion of infrared image and visible light image is analyzed experimentally. The principal component analysis method and the proposed fusion algorithm based on improved K-SVD sparse representation and Hadamard measurement matrix are used to make fusion experimental analysis on the two kinds of images. The fusion results are shown in Figure 7.

Figure 7.

The fusion results of infrared image with visible light image. (a) Infrared image F1, (b) visible light imaging image F2, (c) image fusion method used in pyramid, (d) fusion algorithm based on wavelet transform and Fourier measurement matrix, (e) fusion result image with the principal component analysis method, and (f) the fusion result image based on the improved K-SVD sparse representation and the Hadamard measurement matrix.

Figure 7(a) shows the infrared imaging image F1. Figure 7(b) is the visible light image F2 and the image pixel size is 256 × 256. Figure 7(c) is the fusion result image based on pyramid-type algorithm. Figure 7(d) is the fusion result image based on wavelet transform and Fourier measurement matrix. Figure 7(e) is the fusion result image obtained from the principal component analysis method. Figure 7(f) is the fusion result image obtained from the proposed fusion algorithm, which is based on the improved K-SVD sparse representation and Hadamard measurement matrix.

The evaluation parameters of information entropy, average gradient, mutual information, SAM, and ISAM are selected for the comparison and analysis of experimental results. “I,” “V,” and “F” represent the infrared, visible, and fused images, respectively. The evaluation parameters of the obtained fusion result images based on the two achieved fusion algorithms are shown in Table 4.

Table 4.

Fusion quality evaluation parameters of infrared image with visible light remote sensing image.

Fusion methods	Information entropy	Average gradient	Mutual information	SAM_IF	ISAM_IF	SAM_VF	ISAM_VF	Time consumption(s)
Principal component analysis method	6.9138	0.0229	1.8654	0.2936	0.4651	0.1501	0.1762	29.7148
Pyramid-type algorithm	7.0346	0.0339	1.8342	0.2567	0.3489	0.2065	0.2111	93.6812
Wavelet transform and Fourier measurement matrix	6.5272	0.0164	1.721	0.268	0.3594	0.1474	0.1549	54.4332
The improved K-SVD sparse representation and the Hadamard measurement matrix	7.4058	0.0271	2.7264	0.3618	0.4560	0.2081	0.2085	60.042

ISAM: improved spectral angular mapping; K-SVD: K-singular value decomposition; SAM: spectral angular mapping.

In this paper, fusion experimental analysis was implemented on another group of multispectral image with full-color image. Figure 8(a) is a multispectral image, Figure 8(b) is a full-color image, and the image pixel size is 256 × 256. Figure 8(c) is the fusion result image based on pyramid-type algorithm. Figure 8(d) is the fusion result image based on wavelet transform and Fourier measurement matrix. Figure 8(e) is the fusion result image obtained from the principal component analysis method. Figure 8(f) is the fusion result image obtained from the proposed fusion algorithm. The fusion experimental analysis was made on full-color image with multispectral image using the principal component analysis method and the proposed fusion algorithm based on improved K-SVD sparse representation and Hadamard measurement matrix. The fusion results are shown in Figure 8.

Figure 8.

The fusion results of multispectral image with full-color image. (a) Multispectral image F1, (b) full-color image F2, (c) image fusion method used in pyramid, (d) fusion algorithm based on wavelet transform and Fourier measurement matrix, (e) fusion result image with the principal component analysis method, and (f) the fusion result image with the fusion algorithm based on the improved K-SVD sparse representation and the Hadamard measurement matrix.

“M,” “P,” and “F” are used to represent the multispectral, full-color and fusion image, respectively. The objective evaluation parameters of fusion algorithms are shown in Table 5.

Table 5.

Fusion quality evaluation table of multispectral image with full-color remote sensing image.

Fusion method	Information entropy	Average gradient	Mutual information	SAM_MF	ISAM_MF	SAM_PF	ISAM_PF	Time consumption(s)
Principal component analysis method	20.6876	464.6653	6.8821	0.3231	0.6358	0.1286	0.3956	25.84
Pyramid-type algorithm	19.1321	574.4867	7.1167	0.1546	0.5587	0.1923	0.3845	89.58
Wavelet transform and Fourier measurement matrix	20.8245	447.1975	6.7253	0.3879	0.5589	0.3231	0.3878	41.77
The improved K-SVD sparse representation and the Hadamard measurement matrix	20.9454	468.1923	7.4045	0.3167	0.6139	0.1342	0.3743	64.76

ISAM: improved spectral angular mapping; K-SVD: K-singular value decomposition; SAM: spectral angular mapping.

The experimental results show that the proposed fusion algorithm based on the improved K-SVD sparse representation and the Hadamard measurement matrix achieved good evaluation parameters in the multisource image fusion experiments. In the fusion experiments of full-color image with multispectral image, infrared imaging image with visible light image, as well as multispectral image with full-color image, the evaluation parameters of information entropy $H_{X}$ , standard deviation $S_{F}$ , average gradient, mutual information amount, SAM, and ISAM achieved good experimental performance. The fusion algorithm based on compressive sensing theory only acts on a small amount of measurement data after compressive sensing sampling, which greatly reduces the number of pixels involved in the fusion, improves the time and space complexity of the fusion, and the image fusion performance as well.

Conclusions

In this paper, the compressive sensing theory is applied to the image fusion and a fusion algorithm based on improved K-SVD sparse representation and Hadamard measurement matrix is proposed. This proposed algorithm overcomes the problem of high data information amount and high time complexity in the traditional image fusion algorithm. Since the fusion algorithm only acts on a small amount of data after compressive sensing sampling, the image fusion is completed in the compressive sensing domain. The data after fusion is reconstructed with the reconstruction algorithm of the compressive sensing theory. And then, the fusion result images are obtained. Hence the algorithm has achieved good experimental results.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The research work in this paper was partly funded by the Department of Education in Sichuan Provincial (no. 17ZA0453), and partly funded by the Yibin University through the scientific research funding (no. 2016QD12).

References

Wang

A Hadamard matrix construction method based on hypercube graph. Wuyi Univ J 2013; 32: 51–54.

Aharon

Elad

Bruckstein

AMK.

SVD: an algorithm for designing over complete dictionaries for sparse representation. IEEE Trans Signal Process 2006; 15: 4311–4322.

Engan

Aase

Hakon

HJ.

Method of optimal directions for frame design. In: Proceedings of the 1999 IEEE international conference on acoustics, speech, and signal processing, 1999, vol. 5, pp.2443–2446. Phoenix, AZ: IEEE.

Engan

Rao

Kreutz-Delgado

Frame design using FOCUSS with method of optimal directions (MOD). In: Proc. NORSIG, 1999, Oslo, Norway, 1999, pp.65–69.

Aharon

Elad

Bruckstein

K-SVD: an algorithm for designing over complete dictionaries for sparse representation.

IEEE Trans Signal Process 2006; 54: 4311–4322.

Shi

Wang

Super resolution image reconstruction based on improved K-SVD dictionary learning. J Electron 2013; 51: 997–101.

Williams

Zhang

Chen

A new image deconvolution method with fractional regularization. J Algorithms Comput Technol 2016; 10: 265–276.

Zhang

Pseudo random code and Hadamard matrix. J North China Inst Technol 2001; 22: 55–67.

Wang

Hadamard matrix construction method based on hypercube graph. Wuyi College J 2013; 32: 51–54.

10.

Zeng

Yang

et al . Improved compressive sensing reconstruction algorithm and its applications in image fusion. J Nanjing Univ Sci Technol 2014; 38: 260–264.

11.

Kumaraswamy

Srinivasa Rao

Naveen Kumar

Satellite image fusion using fuzzy logic. Acta Univ Sapientiae Inform 2016; 8: 241–253.

12.

Ding

Similarity measures of full polarimetric SAR images fusion for improved SAR image matching. ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2015; XL-7/W4: 13–16.

13.

Madhav

BTP

Pardhasaradhi

Manepalli

RKNR

et al . Image enhancement using virtual contrast image fusion on Fe3O4 and ZnO nanodispersed decyloxy benzoic acid. Liquid Cryst 2015; 42: 1329–1336.

14.

Tamilselvan

Murugesan

SG.

Survey and analysis of various image fusion techniques for clinical CT and MRI images. Int J Imaging Syst Technol 2014; 24: 193–202.

15.

Sun

Jiang

Color multi-focus image fusion algorithm based on fuzzy theory and dual-tree complex wavelet transform. J Algorithms Comput Technol 2017; 11: 163–169.

16.

Yao

Guo

Xin Xin et al . Image fusion by hierarchical joint sparse representation. Cogn Comput 2014; 6: 281–292.

17.

Yutao

Image fusion based on the non-sampling Contourlet. PhD Thesis, Graduate School of Chinese Academy of Science, China, 2012.

18.

Juan

Research on Contourlet domain image fusion technology based on urban images. PhD Thesis, Journal of Chengdu University of Technology, China, 2005.

19.

Qiang

Research on image fusion and super-resolution reconstruction algorithm based on compressed sensing theory. PhD Thesis, Chengdu University of Technology, China, 2015.