Discriminative collaborative representation for multimodal image classification

Abstract

Sparse representation has been widely researched for image-based classification. However, sparse representation classification directly treats training samples as a dictionary, so it needs a large training set and is time consuming, especially for a large training set. To derive a small dictionary, many dictionary learning algorithms are researched. Thus, object recognition problem is transformed to optimize the sparse representation errors on the compact dictionary. The sparse representation optimization is constraint by $l_0$ -norm, which is NP-hard problem. Though we can use $l_1$ -norm minimization instead to work effectively, it is still time consuming for optimization. To make the algorithm discriminative and simultaneously decrease the computational burden, we proposed a fast discriminative collaborative representation–based classification algorithm. The new algorithm incorporated the within-class scatter and the linear classification error terms into the objective function to derive a more discriminative dictionary and simultaneously added collaborative representation mechanism to cut off the time consuming. At the end of this article, we designed two experiments to validate our method using near-infrared and AR visible databases for multimodal face recognition. The results showed that our algorithm outperformance sparse representation–based, collaborative representation–based, and discriminative-KSVD classification algorithms.

Introduction

In the past few years, sparse representation, also called sparse coding, were widely researched by computer community. The technology of sparse representation was used for image denoising ^1,2 image analysis, ³ image super resolution, ⁴ and especially image-based classification such as face recognition, ^{5 –11} automatic target recognition, ¹² and traffic sign recognition. ¹³ Wright et al. ⁶ used sparse representation for classification task and proposed a sparse representation–based classification (SRC) in face recognition task. In SRC framework, training samples of all classes are arranged as columns of a matrix usually called dictionary, and the query image is considered linearly represented by the atoms of the dictionary. Most entries of the linear representation coefficient coding vector are zeros or approximate zeros, that is, the coefficient vectors are sparse. The minimal reconstruction errors are used for classification. However, when the training data set is larger, SRC is time consuming for all training samples are regarded as a dictionary.

In order to cut down the time consuming, many researchers proposed dictionary learning (DL) algorithms ^{3,5,11,14 –18} to derive a smaller dictionary. K-SVD algorithm ¹⁴ is the representative one of DL algorithms. An overcomplete dictionary can be learned from a training data set by K-SVD, which works well for signal representation but not for classification tasks. To address the image classification issue, Mairal et al. ³ added a discriminative reconstruction constraint and optimized both class discrimination and sparse reconstruction components based on K-SVD. In a study by Pham and Venkatesh, ¹⁶ a dictionary construction and joint learning algorithm was proposed for classification task. Later, a discriminative K-SVD (D-KSVD) was proposed for face recognition by Zhang and Li et al.⁵ The classification error was incorporated into the objective function based on extending K-SVD algorithm. The linear classifier was utilized for obtaining the query image’s label in D-KSVD method. Jiang et al. ¹⁷ introduced a new discriminative sparse coding error constraint to jointly learn a single dictionary and a linear classification, the algorithm is called label consistent K-SVD (LC-KSVD). The common character of the methods mentioned above is that a shared dictionary is learned during DL scheme. Different from the above methods, Yang et al. ¹⁸ proposed to learn a structured dictionary. Fisher discrimination dictionary learning method, which made the coding coefficients having big between-class scatter but small within-class scatter, was used in the classification scheme. Xu et al. ¹¹ synthesized the advantages of the methods mentioned above. They proposed supervised within-class similar discriminative DL method, which incorporates linear classification error term and within-class representation coefficients constraint into objective function for face recognition.

The sparsity of the sparse representation coefficients is measured by $l_0$ -norm, which is NP-hard. Using $l_1$ -norm minimization instead is popular. It is still time consuming, though $l_1$ -norm minimization is more efficient. Zhang et al. ¹⁹ indicated that it was the collaborative representation (CR) mechanism that improved the face recognition accuracy. Motivated by this idea, we proposed a discriminative collaborative representation–based classification (DCRC), which incorporated the within-class scatter and the linear classification error terms into the objective function to derive a more discriminative dictionary, and simultaneously added CR mechanism to decrease the time cost. Besides, recently researchers proposed algorithms for multisource image classification. ^20,21 Liu et al. ²⁰ proposed a joint sparse coding model to solve the room-level localization using multiple sets. Li et al. ²² proposed multimodal fusion method for object recognition. Our algorithm is also effective for multimodal image sets.

This article was organized as follows. In the second section, we described the related works about classification on sparse representation and CR, respectively. In the third section, we described the methodology of the DCRC algorithm. Experiments were performed in the fourth section using different well-known databases to prove the validity of the proposed method. The conclusions were given in the fifth section.

Related works

Brief introduction of sparse representation–based classification

SRC was first proposed for face recognition by Wright et al. ⁶ The SRC framework contains two procedures, sparse coding and classification. Suppose the training samples from K different classes are denoted as $\mathit{D}=\left[{ }_1,\mathrm{...},{ }_i,\mathrm{...},{ }_K\right]\in {ℝ}^{d\times n},i=1,\mathrm{...},K$ , where ${\mathit{\text{D}}}_i\in {ℝ}^{d\times {\mathit{\text{n}}}_i}$ is ${\mathit{\text{n}}}_i$ training samples subset from class i, and $n={\displaystyle \sum _{i=1}^K{\mathit{\text{n}}}_i}$ . Given a query image $\mathit{z}\in {ℝ}^d$ , SRC considers the query image as a sparse linear combination of the training data set, and the sparse coding can be described as follows

$\mathit{x}=\text{arg}\underset{x}{\text{min}}{\Vert \mathit{z}-\mathit{Dx}\Vert }_2^{\text{2}},s.t.{\Vert \mathit{x}\Vert }_0\le \tau$

1

where x is the code coefficient vector, τ is a sparsity constant, and $l_0$ -norm counts the number of nonzero elements of x . This optimization is an NP-hard problem, and it is generally transformed to an $l_1$ -norm minimization instead

$\mathit{x}=\text{arg}\underset{x}{\text{min}}{\Vert \mathit{z}-\mathit{Dx}\Vert }_2^{\text{2}},s.t.{\Vert \mathit{x}\Vert }_1\le \tau$

2

or

$\mathit{x}=\text{arg}\underset{x}{\text{min}}\left\{{\Vert \mathit{z}-\mathit{Dx}\Vert }_2^{\text{2}}+\lambda {\Vert \mathit{x}\Vert }_1\right\}$

3

where λ is a scalar constant to balance the sparsity and reconstruction error terms. Equation (3) is equivalent to equation (2).

The coding vector $\mathit{x}=\left[{\mathit{x}}_1,\mathrm{...},{\mathit{\text{x}}}_i,\mathrm{...},{\mathit{\text{x}}}_K\right]$ is obtained, and then we do classification via reconstruction errors minimization:

$\begin{array}{l}\mathit{\text{L}}a\mathit{\text{b}}e\mathit{\text{l}}\left(\mathit{z}\right)=\text{arg}\underset{i}{\text{min}}{\mathit{\text{r}}}_i\left(\mathit{z}\right)\\ =\text{arg}\underset{i}{\text{min}}{\Vert \mathit{z}-{\mathit{\text{D}}}_i{\mathit{\text{x}}}_i\Vert }_2,i=1,\mathrm{...},K\end{array}$

4

where ${\mathit{\text{x}}}_i$ is the representation coefficient vector corresponding to class i.

Dictionary learning model

The SRC framework directly used training data set as the dictionary. In order to learn a smaller dictionary, DL model was proposed. DL model used the training data set to learn a corresponding compact dictionary $\mathit{D}=\left[{\mathit{d}}_1,{\mathit{d}}_2,\mathrm{...},{\mathit{\text{d}}}_m\right]\in {ℝ}^{d\times m}$ with m atoms ( $m<n$ ). Then, DL framework was written as follows ¹¹

$\begin{array}{l}〈\mathit{D},\mathit{X}〉=\text{arg}\underset{D,X}{\text{min}}\left\{{\Vert \mathit{Y}-\mathit{DX}\Vert }_{\text{F}}^2+\lambda {\Vert \mathit{X}\Vert }_1\right\}\\ s.t.{\Vert {\mathit{\text{d}}}_j\Vert }_2^{\text{2}}=1,j=1,2,\mathrm{...},m\end{array}$

5

where $\mathit{Y}=\left[{\mathit{Y}}_1,\mathrm{...},{\mathit{\text{Y}}}_i,\mathrm{...},{\mathit{\text{Y}}}_K\right]\in {ℝ}^{d\times n},i=1,\mathrm{...},K$ is the training data set, $\mathit{X}=\left[{\mathit{x}}_1,{\mathit{x}}_2,\mathrm{...},{\mathit{\text{x}}}_n\right]\in {ℝ}^{m\times n}$ is the sparse representation coefficients matrix of training set Y on dictionary D , ${\Vert \mathbf{·}\Vert }_{\text{F}}$ is the Frobenius norm, and ${\Vert \mathit{X}\Vert }_1={\displaystyle \sum _{i=1}^n{\Vert {\mathit{\text{x}}}_i\Vert }_1}$ , λ is a scalar constant to balance the sparsity and reconstruction error terms as equation (3).

The first term in equation (5) denotes reconstruction errors, so the model is suitable for signal representation tasks but not for classification tasks. The model in equation (5) is an unsupervised DL framework, because the labels of the training set are not taken into account. A supervised DL framework is designed for classification tasks via adding discriminative terms into objective function. Xu et al. ¹¹ synthesized the methods in literature ^5,17,18 and proposed a supervised DL framework called SCDDL.

Classification based on collaborative representation

The DL methods mentioned above are $l_1$ -norm minimizing optimization problem. Though there are many speeding up methods proposed as reviewed in the study by Zhang et al., ²² they are complicated and still time consuming. Zhang et al. ¹⁹ analyzed the mechanism of SRC and indicated that CR played an essential role for classification in SRC but not $l_1$ -norm sparsity. To collaboratively represent the query image z , they proposed to utilize the regularized least square method. The CR-based classification with regularized least square (CRC-RLS) model is as follows

$\mathit{x}=\text{arg}\underset{x}{\text{min}}\left\{{\Vert \mathit{z}-\mathit{Dx}\Vert }_2^{\text{2}}+\lambda {\Vert \mathit{x}\Vert }_2^{\text{2}}\right\}$

6

where λ is the regularization parameter. It is easy to derive the analytical solution of equation (7) as

$\mathit{x}=\mathit{Pz}={\left({\mathit{\text{D}}}^T\mathit{D}+\lambda \mathit{I}\right)}^{-1}{\mathit{\text{D}}}^T\mathit{z}$

7

where P is a projection matrix independent of z . Matrix P can be calculated offline, and this makes CR more effective.

It indicates that equation (6) makes the classification discriminative, while equation (7) makes the classification fast. Can we combine these models to make the classification procedure more discriminative and faster?

Discriminative collaborative representation for image-based classification

Drawing inspiration from literatures, ^{5,11,17 –19} we proposed a DCRC algorithm. The DCRC method was described as follows.

Discriminative collaborative representation–based dictionary learning

Suppose $\mathit{Y}=\left[{\mathit{Y}}_1,\mathrm{...},{\mathit{\text{Y}}}_i,\mathrm{...},{\mathit{\text{Y}}}_K\right]\in {ℝ}^{d\times n},i=1,\mathrm{...},K$ is the set of training samples from K classes. Every training sample is a d-dimensional column vector of Y . Discriminative collaborative representation–based DL is to derive a discriminative dictionary $\mathit{D}\in {ℝ}^{d\times m}$ and a classifier $\mathit{W}\in {ℝ}^{K\times m}$ for classification. Dictionary D consists of m atoms, and $\mathit{X}=\left[{\mathit{X}}_1,\mathrm{...},{\mathit{\text{X}}}_i,\mathrm{...},{\mathit{\text{X}}}_K\right]\in {ℝ}^{m\times n}$ is the coding coefficients matrix of training samples on dictionary D . The discriminative collaborative representation–based DL model can be written as follows

$\begin{array}{l}〈\mathit{D},\mathit{W},\mathit{X}〉=\text{arg}\underset{D,W,X}{\text{min}}\left\{\begin{array}{l}{\Vert \mathit{Y}-\mathit{DX}\Vert }_{\text{F}}^2+{\lambda }_1{\Vert \mathit{X}\Vert }_{\text{F}}^2+\alpha {\Vert \mathit{H}-\mathit{WX}\Vert }_{\text{F}}^2+\\ \beta {\Vert \mathit{W}\Vert }_{\text{F}}^2+{\lambda }_2{\displaystyle \sum _{i=1}^K\left({\Vert {\mathit{\text{X}}}_i-{\mathit{\text{M}}}_i\Vert }_{\text{F}}^2+\eta {\Vert {\mathit{\text{X}}}_i\Vert }_{\text{F}}^2\right)}\end{array}\right\}\\ s.t.{\Vert {\mathit{\text{d}}}_j\Vert }_2^{\text{2}}=1,j=1,\mathrm{...},m\end{array}$

8

where ${\Vert \mathit{X}\Vert }_{\text{F}}^2={\displaystyle \sum _{i=1}^K{\Vert {\mathit{\text{X}}}_i\Vert }_{\text{F}}^2}$ and ${\Vert \mathbf{·}\Vert }_{\text{F}}$ is the Frobenius norm. Then we can rewrite equation (8) as

$\begin{array}{l}〈\mathit{D},\mathit{W},\mathit{X}〉=\text{arg}\underset{D,W,X}{\text{min}}\left\{\begin{array}{l}{\Vert \mathit{Y}-\mathit{DX}\Vert }_{\text{F}}^2+{\lambda }_1{\displaystyle \sum _{i=1}^K{\Vert {\mathit{\text{X}}}_i\Vert }_{\text{F}}^2}+\alpha {\Vert \mathit{H}-\mathit{WX}\Vert }_{\text{F}}^2+\\ \beta {\Vert \mathit{W}\Vert }_{\text{F}}^2+{\lambda }_2{\displaystyle \sum _{i=1}^K\left({\Vert {\mathit{\text{X}}}_i-{\mathit{\text{M}}}_i\Vert }_{\text{F}}^2+\eta {\Vert {\mathit{\text{X}}}_i\Vert }_{\text{F}}^2\right)}\end{array}\right\}\\ \begin{array}{c}=\end{array}\text{arg}\underset{D,W,X}{\text{min}}\left\{\begin{array}{l}{\Vert \mathit{Y}-\mathit{DX}\Vert }_{\text{F}}^2+\left({\lambda }_1+{\lambda }_2\eta \right){\displaystyle \sum _{i=1}^K{\Vert {\mathit{\text{X}}}_i\Vert }_{\text{F}}^2}+\\ \alpha {\Vert \mathit{H}-\mathit{WX}\Vert }_{\text{F}}^2+\beta {\Vert \mathit{W}\Vert }_{\text{F}}^2+{\lambda }_2{\displaystyle \sum _{i=1}^K\left({\Vert {\mathit{\text{X}}}_i-{\mathit{\text{M}}}_i\Vert }_{\text{F}}^2\right)}\end{array}\right\}\\ s.t.{\Vert {\mathit{\text{d}}}_j\Vert }_2^{\text{2}}=1,j=1,\mathrm{...},m\end{array}$

9

Here, we set $\eta =1$ as in Xu et al. ¹¹ for simplicity. Then, equation (9) is rewritten as

$\begin{array}{l}〈\mathit{D},\mathit{W},\mathit{X}〉=\text{arg}\underset{D,W,X}{\text{min}}\left\{\begin{array}{l}{\Vert \mathit{Y}-\mathit{DX}\Vert }_{\text{F}}^2+\left({\lambda }_1+{\lambda }_2\right){\Vert \mathit{X}\Vert }_{\text{F}}^2+\\ \alpha {\Vert \mathit{H}-\mathit{WX}\Vert }_{\text{F}}^2+\beta {\Vert \mathit{W}\Vert }_{\text{F}}^2+{\lambda }_2{\displaystyle \sum _{i=1}^K\left({\Vert {\mathit{\text{X}}}_i-{\mathit{\text{M}}}_i\Vert }_{\text{F}}^2\right)}\end{array}\right\}\\ s.t.{\Vert {\mathit{\text{d}}}_j\Vert }_2^{\text{2}}=1,j=1,\mathrm{...},m\end{array}$

10

where ${\Vert \mathit{Y}-\mathit{DX}\Vert }_{\text{F}}^2$ is the reconstruction errors term of training data set Y on dictionary D , $\left({\lambda }_1+{\lambda }_2\right){\Vert \mathit{X}\Vert }_{\text{F}}^2$ is the coding coefficients restriction term, $\alpha {\Vert \mathit{H}-\mathit{WX}\Vert }_{\text{F}}^2+\beta {\Vert \mathit{W}\Vert }_{\text{F}}^2$ is the classification errors term, and ${\lambda }_2{\displaystyle \sum _{i=1}^K\left({\Vert {\mathit{\text{X}}}_i-{\mathit{\text{M}}}_i\Vert }_{\text{F}}^2\right)}$ is the within-class similar term as in the study by Xu et al. ¹¹ $\mathit{W}\in {ℝ}^{K\times m}$ is the linear classifier, and $\mathit{H}\in {ℝ}^{K\times n}$ is the label matrix corresponding to the training samples. Each column of H is a vector with the form ${\left[0,\mathrm{...0},1,0,\mathrm{...},0\right]}^T\in {ℝ}^K$ , where 1 locates the training samples to the corresponding class. ${\mathit{\text{X}}}_i$ is the coding coefficients corresponding to class i. ${\mathit{\text{M}}}_i$ is the same type matrix of ${\mathit{\text{X}}}_i$ , and each column of ${\mathit{\text{M}}}_i$ is the mean vector of ${\mathit{\text{X}}}_i$ . Parameters ${\lambda }_1$ , ${\lambda }_2$ , α, and β are all nonnegative constants. This framework simultaneously derives the dictionary D and classifier W . Once the classifier is obtained, the query image can be easily classified by the classifier.

Optimization scheme

The dictionary D and classifier W can be optimized simultaneously. We can synthesize D and W to derive an extended matrix as in literature, ^5,11,17 and equation (10) can be transformed to

$〈\mathit{D},\mathit{W},\mathit{X}〉=\text{arg}\underset{D,W,X}{\text{min}}\left\{\begin{array}{l}{\parallel \left(\begin{array}{c}\mathit{Y}\\ \sqrt{\alpha }\mathit{H}\end{array}\right)-\left(\begin{array}{c}\mathit{D}\\ \sqrt{\alpha }\mathit{W}\end{array}\right)\mathit{X}\parallel }_{\text{F}}^2+\left({\lambda }_1+{\lambda }_2\right){\Vert \mathit{X}\Vert }_{\text{F}}^2+\\ \beta {\Vert \mathit{W}\Vert }_{\text{F}}^2+{\lambda }_2{\displaystyle \sum _{i=1}^K\left({\Vert {\mathit{\text{X}}}_i-{\mathit{\text{M}}}_i\Vert }_{\text{F}}^2\right)}\end{array}\right\}$

11

If $\left(\begin{array}{c}\mathit{D}\\ \sqrt{\alpha }\mathit{W}\end{array}\right)$ is normalized column-wise, the regularization penalty can be dropped. ⁵ Denote $\left(\begin{array}{c}\mathit{Y}\\ \sqrt{\alpha }\mathit{H}\end{array}\right)$ as ${\mathit{Y}}^{*}$ and denote $\left(\begin{array}{c}\mathit{D}\\ \sqrt{\alpha }\mathit{W}\end{array}\right)$ as ${\mathit{D}}^{*}$ . Then, the objective function can be simply written as

$\begin{array}{l}〈{\mathit{D}}^{*},\mathit{X}〉=\text{arg}\underset{D^{*},X}{\text{min}}\left\{\begin{array}{l}{\Vert {\mathit{Y}}^{*}-{\mathit{D}}^{*}\mathit{X}\Vert }_{\text{F}}^2+\left({\lambda }_1+{\lambda }_2\right){\Vert \mathit{X}\Vert }_{\text{F}}^2+\\ {\lambda }_2{\displaystyle \sum _{i=1}^K\left({\Vert {\mathit{\text{X}}}_i-{\mathit{\text{M}}}_i\Vert }_{\text{F}}^2\right)}\end{array}\right\}\\ s.t.{\Vert {\mathit{\text{d}}}_j^{*}\Vert }_2^{\text{2}}=1,j=1,\mathrm{...},m\end{array}$

12

We can alternatively update X and ${\mathit{D}}^{*}$ , that is, fix one of them, and update the other.

Now, we fix the value of ${\mathit{D}}^{*}$ and then update X . The function in equation (12) is turned to a coding problem

$〈\mathit{X}〉=\text{arg}\underset{X}{\text{min}}\left\{\begin{array}{l}{\Vert {\mathit{Y}}^{*}-{\mathit{D}}^{*}\mathit{X}\Vert }_{\text{F}}^2+\left({\lambda }_1+{\lambda }_2\right){\Vert \mathit{X}\Vert }_{\text{F}}^2+\\ {\lambda }_2{\displaystyle \sum _{i=1}^k\left({\Vert {\mathit{\text{X}}}_i-{\mathit{\text{M}}}_i\Vert }_{\text{F}}^2\right)}\end{array}\right\}$

13

We can derive X class by class. For example, if we want to obtain ${\mathit{\text{X}}}_i$ of class i, the function in equation (13) can be written by class

$〈{\mathit{\text{X}}}_i〉=\text{arg}\underset{{\mathit{\text{X}}}_i}{\text{min}}\left\{{\Vert {\mathit{\text{Y}}}_i^{*}-{\mathit{D}}^{*}{\mathit{\text{X}}}_i\Vert }_{\text{F}}^2+\left({\lambda }_1+{\lambda }_2\right){\Vert {\mathit{\text{X}}}_i\Vert }_{\text{F}}^2+{\lambda }_2{\Vert {\mathit{\text{X}}}_i-{\mathit{\text{M}}}_i\Vert }_{\text{F}}^2\right\}$

14

The objective function in equation (14) is differentiable, so we can obtain ${\mathit{\text{X}}}_i$ via derivative of objective function and setting to be zero. Denote f as the objective function and then derivative

$\frac{\partial \mathit{f}}{\partial {\mathit{\text{X}}}_i}=\left\{\begin{array}{l}-2{\mathit{D}}^{*T}{\mathit{\text{Y}}}_i^{*}+2{\mathit{\text{X}}}_i+\\ 2\left({\lambda }_1+{\lambda }_2\right){\mathit{\text{X}}}_i+2{\lambda }_2\left({\mathit{\text{X}}}_i-{\mathit{\text{M}}}_i\right)\end{array}\right\}$

15

Set $\frac{\partial \mathit{f}}{\partial {\mathit{\text{X}}}_i}$ to be zero and denote the obtained variables from the k-th to (k+1)-th iteration using the subscripts $\left(k\right)$ and $\left(k+1\right)$ . Thus, ${\mathit{\text{X}}}_i$ can be calculated as follows

${\mathit{X}}_{i\left(k+1\right)}=\frac{{\lambda }_2{\mathit{M}}_{i\left(k\right)}+{\mathit{D}}_{\left(k\right)}^{*\mathit{T}}{Y}_{i\left(k\right)}^{*}}{1+{\lambda }_1+2{\lambda }_2}$

16

After obtaining the coding coefficient X , we fix it and update ${\mathit{D}}^{*}$ . Then the function in equation (12) is reduced to a DL problem

$\begin{array}{l}〈{\mathit{D}}^{*}〉=\text{arg}\underset{{\mathit{D}}^{*}}{\text{min}}{\Vert {\mathit{Y}}^{*}-{\mathit{D}}^{*}\mathit{X}\Vert }_{\text{F}}^2\\ s.t.{\Vert {\mathit{\text{d}}}_j^{*}\Vert }_2^{\text{2}}=1,j=1,\mathrm{...},m\end{array}$

17

The objective function in equation (17) is differentiable, so we can derive ${\mathit{D}}^{*}$ as follows

${\mathit{D}}_{\left(k+1\right)}^{*}={\mathit{Y}}_{\left(k\right)}^{*}{\mathit{X}}_{\left(k+1\right)}^T{({\mathit{X}}_{\left(k+1\right)}{\mathit{X}}_{\left(k+1\right)}^T)}^{-1}$

18

${\mathit{D}}^{*}=\left(\begin{array}{c}\mathit{D}\\ \sqrt{\alpha }\mathit{W}\end{array}\right)$ contains W and D , so we can derive dictionary D and classifier W simultaneously via the lemma in the study by Zhang and Li. ⁵ Suppose ${\mathit{D}}^{*}=\left(\begin{array}{c}{\mathit{d}}_1^{*},{\mathit{d}}_2^{*},\mathrm{...},{\mathit{\text{d}}}_m^{*}\\ {\mathit{w}}_1^{*},{\mathit{w}}_2^{*},\mathrm{...},{\mathit{\text{w}}}_m^{*}\end{array}\right)$ , then we can obtain D and W as follows

$\mathit{D}=\left({\mathit{d}}_1,{\mathit{d}}_2,\mathrm{...},{\mathit{\text{d}}}_m\right)=\left(\frac{{\mathit{d}}_1^{*}}{{\Vert {\mathit{d}}_1^{*}\Vert }_2},\frac{{\mathit{d}}_2^{*}}{{\Vert {\mathit{d}}_2^{*}\Vert }_2},\mathrm{...},\frac{{\mathit{\text{d}}}_m^{*}}{{\Vert {\mathit{\text{d}}}_m^{*}\Vert }_2}\right)$

19

$\mathit{W}=\left({\mathit{w}}_1,{\mathit{w}}_2,\mathrm{...},{\mathit{\text{w}}}_m\right)=\left(\frac{{\mathit{w}}_1^{*}}{\sqrt{\alpha }{\Vert {\mathit{d}}_1^{*}\Vert }_2},\frac{{\mathit{w}}_2^{*}}{\sqrt{\alpha }{\Vert {\mathit{d}}_2^{*}\Vert }_2},\mathrm{...},\frac{{\mathit{\text{w}}}_m^{*}}{\sqrt{\alpha }{\Vert {\mathit{\text{d}}}_m^{*}\Vert }_2}\right)$

20

Classification scheme

To collaboratively represent a given test sample z on normalized D with low computational burden, we utilize the regularized least square method via equation (6). The solution of equation (6) can be analytically and easily derived by equation (7).

The classification of z is based on its CR coefficient x , which contains most discriminative information for classification. We can easily use the linear classifier W with x to derive the label of the test sample z

$\text{Label}\left(\mathit{z}\right)=\text{arg}\underset{i}{\text{max}}{\left(\mathit{Wx}\right)}_i,i=1,2,\mathrm{...},K$

21

The maximum i-th element of Wx locates the class label i of z .

Flow of our algorithm

The DL is proposed for one-dimensional signal processing, while image-based classification is usually two-dimensional signals. Therefore, an image preprocessing is needed. First, we need to arrange the image data in a line as a vector. The transformation makes the original data being high-dimensional data. Thus, a dimensionality reduction (DR) processing is needed, for high-dimensional data lead to the inefficiency of data processing. In our method, the basic DR method called principal component analysis (PCA) ²³ is used, for it is simple and widely used. The procedures of DCRC are summarized in Table 1.

Table 1.

The flow of discriminative collaborative representation classification algorithm.

Algorithm 1. Discriminative collaborative representation classification (DCRC)
Task: Derive the label of the test sample of z Input: A set of n h-dimensional training samples from K classes $\mathit{T}=\left[{\mathit{T}}_1,\mathrm{...},{\mathit{\text{T}}}_i,\mathrm{...},{\mathit{\text{T}}}_K\right]\in {ℝ}^{h\times n}$ , and test sample z
Initialize: Use PCA to initialize T and get the low d-dimensional training set $\mathit{Y}=\left[{\mathit{Y}}_1,\mathrm{...},{\mathit{\text{Y}}}_i,\mathrm{...},{\mathit{\text{Y}}}_K\right]\in {ℝ}^{d\times n}$ Initialize constant parameters: $\alpha ,\lambda ,{\lambda }_1,{\lambda }_2$ Initialize ${\mathit{D}}_{\left(0\right)}$ using random matrix, then ${\mathit{X}}_{\left(0\right)}=\frac{1}{1+{\lambda }_1}{\mathit{D}}_{\left(0\right)}^{\text{T}}\mathit{Y}$ Initialize labels of training samples ${\mathit{H}}_{\left(0\right)}$ Initialize ${\mathit{W}}_{\left(0\right)}={\mathit{H}}_{\left(0\right)}{\mathit{X}}_{\left(0\right)}{ }^{\text{T}}{\left({\mathit{X}}_{\left(0\right)}{\mathit{X}}_{\left(0\right)}{ }^{\text{T}}+\mathit{I}\right)}^{-1}$ Initialize ${\mathit{D}}_{\left(0\right)}^{*}=\left(\begin{array}{c}{\mathit{D}}_{\left(0\right)}\\ \sqrt{\alpha }{\mathit{W}}_{\left(0\right)}\end{array}\right)$
Dictionary Learning Procedure:  While not converged Updating X via Eq. (16) Updating ${\mathit{D}}^{*}$ via Eq. (18) Normalizing D and W via Eq. (20) and Eq. (21)  end while Output: D , W , X
Classification Procedure: CR coding via Eq. (7) Classification via Eq. (21) Output: $\mathit{\text{L}}a\mathit{\text{b}}e\mathit{\text{l}}\left(\mathit{z}\right)$

Experiments for multimodal databases

To verify the validity of DCRC algorithm, we design two experiments with two databases: near-infrared database ²⁴ and AR visible database. ²⁵ We compared our algorithm with SRC, ⁶ CRC, ¹⁹ D-KSVD ¹⁷ algorithms on accurate recognition rate and time for classifying one test sample.

All the experiments were run on Matlab R2011a. The PC is Lenovo E40-80 notebook computer with an Intel Corel i5 2.30 GHz CPU and 8 GB RAM. The GPUs are AMD Radeon(TM) R5 M330 and Intel(R) HD Graphics 5500 with 2 GB RAM. We also employed sparse solver SPAMS ²⁶ to optimize a standard sparse representation.

Experiment with near-infrared database

The near-infrared database contains 50 distinct subjects and 10 different infrared images for each one. Each image is $100\times 80$ pixels. Figure 1 shows some images of the near-infrared database.

Figure 1.

Samples from the near-infrared database.

With this database, we tested SRC, CRC, D-KSVD, and our method DCRC. The samples from the database were average divided into two groups: one group was used as training sample and the other group was used as test sample. In order to reduce the calculation cost, we used PCA ²⁰ to reduce the dimension of the sample vector from ${ℝ}^{100\times 80}$ to ${ℝ}^{100}$ . For the SRC and CRC methods, we used all the training samples as the dictionary atoms (total 250 atoms). For the D-KSVD method, we used 150 atoms for the dictionary learning. For the DCRC method, we used a dictionary with 70 atoms, and with the parameters $\alpha =1$ , $\lambda =0.001$ , ${\lambda }_1=0.001$ , ${\lambda }_2=0.1$ .

Table 1 shows that the proposed DCRC algorithm contains two procedures: DL and classification. In the dictionary training procedure, the representation reconstruction errors are quickly convergent as shown in Figure 2.

Figure 2.

The representative reconstruction error.

The effect on the accurate recognition rate versus dictionary size is shown in Figure 3. The result shows that the curve is boosting quickly with the increasing atoms forepart, and terminal recognition rates change smooth to 99%. Dictionary size is the main factor affecting the recognition rate. The parameters $\lambda ,{\lambda }_1,{\lambda }_2$ affect the recognition rate weakly. The parameter ${\lambda }_1$ affects the sparsity, while the parameter ${\lambda }_2$ affects the within-class errors. The coding coefficients of one test sample for the four methods are shown in Figure 4, and from top to bottom followed by the coefficient of SRC, CRC, D-KSVD, and DCRC. From the results, we can see that SRC and D-KSVD are strong sparse, while CRC and DCRC are weak sparse.

Figure 3.

The accurate recognition rate versus dictionary size.

Figure 4.

The coding coefficients of one test sample for SRC, CRC, D-KSVD, and DCRC. SRC: sparse representation–based classification; CRC: collaborative representation–based classification; D-KSVD: discriminative-KSVD; DCRC: discriminative collaborative representation–based classification.

Compared with SRC, CRC, and D-KSVD, the proposed method DCRC has very competitive recognition rate but with significantly lower complexity. We recorded the time for classifying one test sample of the three methods, and the results were shown in Table 2.

Table 2.

Face recognition rate and time for classifying one test sample of different methods using near-infrared database.

Method	Recognition rate (%)	Time (ms)
SRC	98.8	0.9139
CRC	99.0	0.3928
D-KSVD	98.0	0.04858
DCRC	99.2	0.01918

SRC: sparse representation–based classification; CRC: collaborative representation–based classification; D-KSVD: discriminative-KSVD; DCRC: discriminative collaborative representation–based classification.

The results indicate that the proposed method DCRC has the approximate recognition rate with the SRC, CRC, and D-KSVD methods for this database. The recognition rates are near the same or almost the same as each other, maybe because the near-infrared database is without noise. SRC and CRC directly used the training sample as a dictionary, while D-KSVD and DCRC learned a compact dictionary from the training set. Therefore, the latter two methods are faster for object recognition, and DCRC is the fastest one of all. To validate our method being competitive for recognition accurate rate, we designed a new experiment in subsection “Experiment with AR database.”

Experiment with AR database

The AR database contains images for 126 persons, with 26 images for each one. Each image is 165 × 120 pixels. This database is widely used for face recognition. Figure 5 shows some images of the AR database. We can see that these images are captured with different viewpoints, different illuminations, different facial expressions, and different disguises (sunglass and neckerchief). These interfering factors make it more difficult for face recognition. In order to validate our method is still effective for samples with noise, we designed an experiment with the AR database.

Figure 5.

Samples from the AR database.

In our experiment, we chose 2600 images from 50 males and 50 females. For each person, there are 26 samples; we chose 20 samples for training and the other 6 samples for testing. First, we used PCA algorithm to reduce the character vector dimension from ${ℝ}^{165\times 120}$ to ${ℝ}^{1000}$ . For the SRC and CRC methods, we still used all training samples as a dictionary (2000 atoms). For the DKSVD method, we used 1600 atoms for the dictionary learning. For the DCRC method, we used a dictionary with 100 atoms, and with the parameters $\alpha =1$ , $\lambda =0.001$ , ${\lambda }_1=0.01$ , ${\lambda }_2=0.0001$ . The results are given in Table 3.

Table 3.

Face recognition rate and time for classifying one test sample of different methods using AR visible database.

Method	Recognition rate (%)	Time (ms)
SRC	77.17	205.9
CRC	83.66	115.1
D-KSVD	98.0	1.300
DCRC	99.1	0.4965

SRC: sparse representation–based classification; CRC: collaborative representation–based classification; D-KSVD: discriminative-KSVD; DCRC: discriminative collaborative representation–based classification.

The results show that the recognition rates of the four methods decreased in different degrees compared with the near-infrared results. The recognition rate of DCRC performs better than that of SRC, CRC, and D-KSVD. The occurrence of such results may be the interfering with noise. The D-KSVD and DCRC methods have the discriminative ability, while the SRC and CRC methods focus on representation of raw signals. Our method is robust for face recognition. The proposed method is the fastest one of the three methods.

Conclusions

A fast discriminative CR image classification method DCRC was proposed in this article. It incorporated the within-class scatter and the linear classification error terms into the objective function, so the method had the discriminative ability for classification. In order to decrease the computing time, we added CR mechanism to the objective function. The experimental results showed that DCRC had better recognition performance than the other three methods, and it was suitable for multimodal image (infrared or visible) classification. The calculating speed of our method was improved a lot.