Modeling and image quality enhancement for dynamic compressive imaging system

Abstract

Traditional single-pixel imaging systems are aimed mainly at relatively static or slowly changing targets. When there is relative motion between the imaging system and the target, sizable deviations between the measurement values and the real values can occur and result in poor image quality of the reconstructed target. To solve this problem, a novel dynamic compressive imaging system is proposed. In this system, a single-column digital micro-mirror device is used to modulate the target image, and the compressive measurement values are obtained for each column of the image. Based on analysis of the measurement values, a new recovery model of dynamic compressive imaging is given. Differing from traditional reconstruction results, the measurement values of any column of vectors in the target image can be used to reconstruct the vectors of two adjacent columns at the same time. Contingent upon characteristics of the results, a method of image quality enhancement based on an overlapping average algorithm is proposed. Simulation experiments and analysis show that the proposed dynamic compressive imaging can effectively reconstruct the target image; and that when the moving speed of the system changes within a certain range, the system reconstructs a better original image. The system overcomes the impact of dynamically changing speeds, and affords significantly better performance than traditional compressive imaging.

Introduction

Compressive imaging (CI),^1–3 a key sub-field of compressive sensing theory research, obtains high resolution images using only a small number of sensors. Because of this advantage, compressive sensing theory has been widely applied in medical imaging,^4,5 image processing, ⁶ remote sensing imaging, ⁷ wireless communication ⁸ and wireless sensor networks. ⁹

At present, the most typical compressive imaging system described in the research literature is a single pixel camera.^10–13 The camera employs a large-scale array digital micro-mirror device (DMD) and a single sensor to realize the compressive sampling of the scene. However, this mode requires the foreground target to be in a static state or changing only slightly during the process of compressive sampling, otherwise the reconstructed image quality is blurred or distorted. Research interest is now focusing on the motion problems associated with foreground targets and compressive imaging systems.^14–17 Tong Q. et al. reconstructed an image in the infrared (IR) rosette scanning system by analyzing the relationship between the target image and the scene in each frame. ¹⁵ Jiao S. et al. solved the problem of dynamic compressive sampling using prior knowledge of the target motion type and the ultrafast structured illumination. ¹⁶ Yun Li. proposed a parallel complementary compressive sensing imaging system based on a block model ¹⁷ and analyzed various characteristics of dynamic compressive sampling performance.

In this paper, a dynamic compressive imaging system based on push-sweep mode for the space remote sensing is proposed. Different from existing dynamic imaging systems, our system employs a single-column DMD to obtain the compressive measurement values of a foreground image. A method of image enhancement is then proposed that combines the measurement values with characteristics of the reconstructed image. Experimental results verify the feasibility of this compressive imaging system.

The paper consists of the following parts: the next section introduces compressive sensing theory; the Dynamic Compressive imaging section describes our novel dynamic compressive imaging system and performance analysis in detail; the Image Enhancement based on overlapping average algorithm section details the method of image quality enhancement and results of the experimental data analysis; the Conclusion section summarizes the work of this paper.

Theory of compressive sensing

Consider a sparse signal $\text{x}$ whose length is N, which can be expressed as a linear combination of sparse bases $\mathit{\Omega }\in {\text{R}}^{N\times N}$ :

$$\text{x}=\mathit{\Omega }\mathit{\theta }$$

(1)

If there are only $K(K\ll N)$ nonzero terms in vector $\mathit{\theta }\in {\mathit{R}}^N$ , the signal $\mathit{\theta }$ is a sparse representation of $\mathit{x}$ in domain $\mathit{\Omega }$ .^1,18 For sparse signal reconstruction, when we know the measurement values $\text{y}\in {\mathit{R}}^M$ and measurement matrix $\mathit{\Phi }\in {\mathit{R}}^{M\times N}(M\ll N)$ , the coefficient vector $\mathit{\theta }$ can be estimated using the minimum $l_1$ -norm, that is:

$$\widehat{\mathit{\theta }}=\arg \min {\Vert \mathit{\theta }\Vert }_{1}s.t.\text{y}=\mathit{\Phi \Omega \theta }\mathit{\Omega }\mathit{\theta }$$

(2)

With the coefficient vector $\widehat{\mathit{\theta }}$ , we can reconstruct the original signal $\text{x}=\mathit{\Omega }\widehat{\mathit{\theta }}$ .

Taking into consideration the influence of errors, equation (2) can be rewritten as

$$\widehat{\mathit{\theta }}=\arg \min {\Vert \mathit{\theta }\Vert }_{1}s.t.{\Vert \text{y-}\mathit{\Phi }\mathit{\Omega }\mathit{\theta }\Vert }_2\le \epsilon$$

(3)

where: $\epsilon$ is the permissible error.^19,20

When we obtain the compressive measurement values of a static target image, all measurement values contain the same original information. However, when the foreground target is dynamic, each measurement value contains mixed information relative to the target image.

Dynamic compressive imaging

Composition of the imaging system

A schematic diagram of the dynamic compressive imaging system is shown in Figure 1. The system is based on single-column DMD, and uses push-sweep mode to capture the measurement value of the foreground image. The compressive sampling part is composed of an imaging lens, a single-column DMD, a collecting lens, and a detector. The terminal recovers the original image using a reconstruction algorithm.

Figure 1.

Schematic diagram of the dynamic compressive imaging system.

Dynamic compressive imaging recovery model

For clarity, the foreground image is $\mathit{X}=\left[{\mathit{x}}_1,{\mathit{x}}_2,\dots ,{\mathit{x}}_N\right]\in {\mathit{R}}^{N\times N}$ , the measurement matrix is $\mathit{\Phi }\in {\mathit{R}}^{M\times N}(M\ll N)$ . We use the measurement matrix $\mathit{\Phi }$ to obtain the measurement values ${\mathit{y}}_i\in {\mathit{R}}^M$ of the i-th column vector ${\mathit{x}}_i={\left[\begin{array}{ccc}{x}_{1,i}& \cdots & x_{N,i}\end{array}\right]}^T\in {\mathit{R}}^N$ of the image. According to the theory of compressive sensing, the measurement values ${\mathit{y}}_i$ can be written as:

$${\mathit{y}}_i=\left[\begin{array}{c}{y}_{1,i}\\ ⋮\\ y_{M,i}\end{array}\right]\mathbf{=}\mathit{\Phi }{\mathit{x}}_i=\left[\begin{array}{ccc}{\varphi }_{11}& \cdots & {\varphi }_{1N}\\ ⋮& \ddots & ⋮\\ {\varphi }_{M1}& \cdots & {\varphi }_{MN}\end{array}\right]\left[\begin{array}{c}{x}_{1,i}\\ ⋮\\ x_{N,i}\end{array}\right]=\left[\begin{array}{c}{\mathit{\phi }}_1\\ ⋮\\ {\mathit{\phi }}_M\end{array}\right]{\mathit{x}}_i$$

(4)

where:

$${\mathit{\phi }}_m=\left[\begin{array}{cccc}{\varphi }_{m1}& {\varphi }_{m2}& \cdots & {\varphi }_{mN}\end{array}\right]\in {\mathit{R}}^N(1\le m\le M)$$

Because the system adopts push-sweep mode to obtain the measurement values, there is a certain deviation between the current measurement value and the previous one. The dynamic sampling process is shown in Figure 2. To solve the dynamic problem, we need to remodel the recovery model.

Figure 2.

The black area is the foreground image and the red area is the single-column DMD.

Assuming that the size of the DMD micro-mirror is $a\times a$ , the average speed of the foreground image moving on the DMD is $v$ , and the sampling frequency of the system is $f$ . For each sampling, the relative moving distance of the foreground image on the DMD is:

$$△s=\frac{v}{f}$$

(5)

For convenience of expression, we define the relative motion ratio p as:

$$p=\frac{△s}{a}$$

(6)

Assuming that the two adjacent vectors of the foreground image are ${\mathit{x}}_1\in {\mathit{R}}^N$ and ${\mathit{x}}_2\in {\mathit{R}}^N$ , when obtaining the measurement vector ${\mathit{y}}_1={\left[\begin{array}{ccc}{y}_{1,1}& \cdots & y_{M,1}\end{array}\right]}^T\in {\mathit{R}}^M$ of the vector ${\mathit{x}}_1$ , according to equation (4), the first value $y_{1,1}$ can be expressed as:

$$y_{1,1}={\mathit{\phi }}_1\cdot {\mathit{x}}_1$$

(7)

When obtaining the M-th measurement value, due to the relative moving distance $(M-1)\cdot △s$ between DMD and the first column of pixels (i.e., ${\mathit{x}}_1$ ), the measurement value $y_{M,1}$ should be:

$$\begin{array}{c}{y}_{M,1}={\mathit{\phi }}_{\mathit{M}}\cdot \left[1-\left(M-1\right)\cdot p\right]\cdot {\mathit{x}}_1+{\mathit{\phi }}_{\mathit{M}}\cdot \left(M-1\right)\cdot p\cdot {\mathit{x}}_2\\ =\left[\begin{array}{cc}1-\left(M-1\right)\cdot p& \left(M-1\right)\cdot p\end{array}\right]\left[\begin{array}{cc}{\mathit{\phi }}_{\mathit{M}}& \mathbf{0}\\ \mathbf{0}& {\mathit{\phi }}_{\mathit{M}}\end{array}\right]\left[\begin{array}{c}{\mathit{x}}_1\\ {\mathit{x}}_2\end{array}\right]\end{array}$$

(8)

As can be seen from equation (8), the measurement value $y_{M,1}$ contains the next column of vector ${\mathit{x}}_2$ information. After gathering M measurement values, we get the measurement vector:

$${\mathit{y}}_1=\left[\begin{array}{c}{y}_{1,1}\\ ⋮\\ y_{M,1}\end{array}\right]\mathbf{=}\mathbf{\Psi }\left[\begin{array}{c}{\mathit{x}}_1\\ {\mathit{x}}_2\end{array}\right]$$

(9)

where, $\mathbf{\Psi }\in {\mathit{R}}^{M\times \mathrm{2N}}$ :

$$\mathbf{\Psi }=\left[\begin{array}{c}\left[\begin{array}{cc}\mathrm{1}& \mathrm{0}\end{array}\right]\left[\begin{array}{cc}{\mathit{\phi }}_{\mathrm{1}}& \mathbf{0}\\ \mathbf{0}& {\mathit{\phi }}_{\mathrm{1}}\end{array}\right]\\ ⋮\\ \left[\begin{array}{cc}\mathrm{1-(}M\mathrm{-1)}\cdot p& \mathrm{(}M\mathrm{-1)}\cdot p\end{array}\right]\left[\begin{array}{cc}{\mathit{\phi }}_{\mathit{M}}& \mathbf{0}\\ \mathbf{0}& {\mathit{\phi }}_{\mathit{M}}\end{array}\right]\end{array}\right]$$

(10)

Therefore, it can be deduced that the measurement vector ${\mathit{y}}_i$ of the i-th column of the foreground image can be expressed as:

$${\mathit{y}}_i=\mathbf{\Psi }\left[\begin{array}{c}{\mathit{x}}_i\\ {\mathit{x}}_{i+1}\end{array}\right]$$

(11)

Combining equations (2) and (11), we can get a novel recovery model of the dynamic compressive imaging system:

$$\{\begin{array}{l}\widehat{\mathit{\theta }}=\arg \min {\Vert \mathit{\theta }\Vert }_{1}s.t.{\mathbf{y}}_i=\mathbf{\Psi }\mathit{\Omega }\mathit{\theta }\\ {\mathit{y}}_i=\mathbf{\Psi }\left[\begin{array}{c}{\mathit{x}}_i\\ {\mathit{x}}_{i+1}\end{array}\right]\end{array}$$

(12)

where: $\mathbf{\Psi }$ is a new measurement matrix.

It can be seen from equation (12) that the original image can be recovered by using a traditional reconstruction algorithm (such as Orthogonal Matching Pursuit),^21–23 and that only one reconstruction is needed to obtain the two adjacent vectors ${\mathit{x}}_i$ and ${\mathit{x}}_{\mathrm{i+}\mathrm{1}}$ of the image. If we combine all the vectors in order, we get two original images, as shown in Figure 3.

Figure 3.

The order diagram of reconstructed vectors.

Figures 4 to 6 show this result and compare it with the result of a traditional CI recovery model. From Figures 4 and 5, we can see that the image effect of the upper part of the reconstructed image which is reconstructed by dynamic CI worsens with the increase of the relative motion ratio p, while the image effect of the lower part is just the opposite. The main reason for this is that as the relative motion ratio p increases, the information in the image vector ${\mathit{x}}_1$ contained in the measurement value decreases, so the image effect of the upper part will degrade. However, for the lower part, the information in the vector ${\mathit{x}}_2$ will increase, and the image effect will improve. The image which is reconstructed via a traditional CI recovery model also degrades with increasing relative motion ratio p. Figure 6 further verifies this conclusion through the Peak Signal-to-Noise Ratio (PSNR) of the reconstructed image.

Figure 4.

The reconstructed images of Lena: (a) Original image:128 × 128. (b1) M = 100, p = 0. (b2) M = 100, p = 0.003. (b3) M = 100, p = 0.006. (b4) M = 100, p = 0.009. (c1) M = 100, p = 0. (c2) M = 100, p = 0.003. (c3) M = 100, p = 0.006. (c4) M = 100, p = 0.009.

Figure 5.

The reconstructed images of Remote: (a) Original image:256 × 256. (b1) M = 200, p = 0. (b2) M = 200, p = 0.0025. (b3) M = 200, p = 0.0035. (b4) M = 200, p = 0.0045. (c1) M = 200, p = 0. (c2) M = 200, p = 0.0025. (c3) M = 200, p = 0.0035. (c4) M = 200, p = 0.0045.

Figure 6.

Relationship between PSNR and relative motion ratio p: (a) M = 100. (b) M = 200.

In addition, it can be seen from Figure 6 that when the relative motion ratio p is small, the image quality of the reconstructed upper part is almost the same as that of the traditional CI image; but when the relative motion ratio p is large, the image quality of the reconstructed lower part is significantly better than that of the traditional CI image. Therefore, for dynamic CI, regardless of how the value of relative motion ratio p is taken, an original image of goodquality can always be reconstructed. This overcomes the influence of the dynamic change of the system’s speed and reduces the problem of the changing speed. Its performance is obviously better than that of the traditional CI.

Image enhancement based on overlapping average algorithm

To make full use of the reconstructed image information and get a higher quality image, we enhance the quality of the upper and lower parts of the reconstructed image using an overlapping average algorithm, as shown in Figure 7.

Figure 7.

Flow chart of image enhancement.

Let us suppose that the upper part $g_1(m,n)$ and the lower part $g_2(m,n)$ of the reconstructed image are:

$$\{\begin{array}{c}{g}_1(m,n)=f_1(m,n)+e_1(m,n)\\ g_2(m,n)=f_2(m,n)+e_2(m,n)\end{array}$$

(13)

where: $f_i(m,n),i=1,2$ is the original image and $e_i(m,n),i=1,2$ is random white noise. After overlapping and averaging equation (13), the output image is:

$$\begin{array}{c}g(m,n)=\frac{1}{2}{\displaystyle \sum _{i=1}^2{g}_i(m,n)}\\ =\frac{1}{2}{\displaystyle \sum _{i=1}^2{f}_i(m,n)}+\frac{1}{2}{\displaystyle \sum _{i=1}^2{e}_i(m,n)}\end{array}$$

(14)

According to the overlapping average algorithm, to maximize the signal-to-noise ratio (SNR) of output image, the images $f_1(m,n)$ and $f_2(m,n)$ should satisfy the following relationship:

$$f_1(m,n)=f_2(m,n)$$

(15)

From the analysis in the previous section, we know that the values of the parameters M and p in equation (8) directly affect the quality of the images g₁ðm; nþ and g₂ðm; nþ. Three scenarios are discussed in the following:

If $1-\left(M-1\right)\cdot p>\left(M-1\right)\cdot p$ ; then,

$$p<\frac{1}{2\left(M-1\right)}$$

(16)

The proportion of the vector ${\mathit{x}}_1$ in the measurement value $y_{M,1}$ is relatively large, the influence of the vector x₂ on the image g₁ðm; nþ is relatively small;

2. If $1-\left(M-1\right)\cdot p<\left(M-1\right)\cdot p$ ; then,

$$p>\frac{1}{2\left(M-1\right)}$$

(17)

The proportion of the vector x2 in the measurement value yM;1 is relatively large, the influence of the vector x₂ on the image g₁ðm; nþ is relatively large;

• 3. If $1-\left(M-1\right)\cdot p\text{=}\left(M-1\right)\cdot p$ ; then,

  $$p\text{=}\frac{1}{2\left(M-1\right)}$$

  (18)

The proportion of the vectors ${\mathit{x}}_1$ and ${\mathit{x}}_2$ in the measurement value $y_{M,1}$ are the same, so so the influence is equivalent. At this point, the SNR of the output image $g(m,n)$ is greatest, and the output image can be described as:

$$\begin{array}{l}g(m,n)=f_1(m,n)+\frac{1}{2}{\displaystyle \sum _{i=1}^2{e}_i(m,n)}\\ \hspace{1em}\hspace{1em}\hspace{1em}=f_2(m,n)+\frac{1}{2}{\displaystyle \sum _{i=1}^2{e}_i(m,n)}\end{array}$$

(19)

To verify the image enhancement effect, the paper uses the PSNR to compare the performance of the image before and after the enhancing process. Results are shown in Figure 8. It can be seen from Figure 8 that the closer the PSNRs of the upper and lower parts of the reconstructed image are, the more obvious the enhancement effects of the output image quality will be; and conversely, the worse the enhancement effect will be. When, the parameters M and p satisfy equation (18), the PSNR of the output image is the greatest.

Figure 8.

Performance before and after image enhancement (Lena:128 × 128): (a) M = 80. (b) M = 100.

Under the premise of satisfying equation (18), we further analyze the influence of parameter M on the quality of the output image $g(m,n)$ and compare performance with the results of the traditional CI. The results are shown in Table 1. As can be seen from Table 1, with the number of measurements M increasing, the PSNR of each reconstructed image is improved, but the image quality after enhancement is clearly better than that of traditional CI.

Table 1.

Performance comparison between the image enhancement method and traditional CI.

Image	Resolution	Measure	Imaging methods	M = 70	M = 80	M = 90	M = 100
Lena	128 × 128	PSNR(dB)	Traditional CI	22.1825	20.0491	24.8143	25.4453
			The upper part	22.1893	24.0679	24.8311	25.5303
			The lower part	20.4737	21.4550	21.8954	22.1830
			Image enhancement	25.3742	26.2651	27.9289	28.8529
Image	Resolution	Measure	Imaging methods	M = 160	M = 180	M = 200	M = 220
Remote	256 × 256	PSNR(dB)	Traditional CI	18.8979	19.5876	20.5542	21.1541
			The upper part	18.8937	19.5853	20.5340	21.1844
			The lower part	17.5699	17.9898	18.4699	18.8026
			Image enhancement	20.2353	20.9546	21.8160	22.4708

Conclusion

Aiming at the problems of traditional CI, this paper proposes a dynamic imaging system based on push-sweep mode. Unlike traditional imaging systems, the system can reconstruct two adjacent column vectors of the foreground image at one time, and produce two foreground images simultaneously.

Simulation results show that when the dynamic moving speed of the system changes within a certain range, a good quality original image can always be obtained, which effectively overcomes the impact of the dynamic change of the moving speed.

To make full use of the reconstructed image information and improve the output image quality, this paper proposes an image enhancement method based on an overlapping average algorithm. Experimental results show that the method is effective.