Distributed Face Recognition Using Multiple Kernel Discriminant Analysis in Wireless Sensor Networks

Abstract

This paper proposes a module based distributed wireless face recognition system by integrating multiple kernel discriminant analysis with face recognition in wireless sensor networks. By maximizing the margin maximization criterion (MMC), we separately perform an iterative scheme for kernel parameter optimization for each module. The simulation on the FERET and CMU PIE face databases shows that our multiple kernel framework and the optimization procedure achieve high recognition performance, compared with single-kernel-based KDDA.

1. Introduction

Face recognition (FR) system is one of the most important biometric techniques and is used in a wide range of security applications such as access control, identification systems, and surveillance [1]. FR is a contactless biometric technique and has advantages of being natural and passive over other biometric techniques requiring cooperative subjects, such as fingerprint recognition and iris recognition [2]. A normal framework of FR system is shown in Figure 1, including procedures of enrollment and identification [3].

Figure 1

A normal framework of FR system.

In recent years, FR systems combined with wireless sensor networks (WSNs) [4] have shown great interest, as WSNs are very helpful for contactless biometrics security applications. For example, Kim et al. implement a wireless face recognition system based on ZigBee protocol and principle components analysis (PCA) method with low energy consumption [5]. Muraleedharan et al. propose the use of a specific evolutionary algorithm to optimize routing in distributed time varying network for face recognition [6]. Chang and Aghajan focus on recovering face orientation for more robust face recognition in wireless image sensor networks [7]. Zaeri et al. propose application of face recognition for wireless surveillance systems [8].

As there exist many image variations such as pose, illumination, and facial expression, face recognition is a highly complex and nonlinear problem which could not be sufficiently handled by linear methods, such as principal components analysis (PCA) [9] and linear discriminant analysis (LDA) [10]. Therefore, it is reasonable to assume that a better solution to this inherent nonlinear problem could be achieved using nonlinear methods, such as the so-called kernel machine techniques [11]. Following the success of applying the kernel trick in support vector machines (SVMs) [12], many kernel-based PCA and LDA methods have been developed and applied in pattern recognition tasks, such as kernel PCA (KPCA) [13], kernel Fisher discriminant (KFD) [14], generalized discriminant analysis (GDA) [15], and kernel direct LDA (KDDA) [16].

It has been shown that the kernel-based LDA method is a feasible approach to solve the nonlinear problems in face recognition. However, the performance of the kernel-based LDA method is sensitive to the selection of a kernel function and its parameters. Kernel parameter selection to date can mainly be achieved by cross-validation [17], which is computationally expensive, and the selected kernel parameters cannot be guaranteed optimal. Furthermore, a single and fixed kernel can only characterize the geometrical structure of some aspects for the input data, and, thus, not always be fit for the applications which involve data from multiple, heterogeneous sources [18, 19].

Recent applications and developments based on SVMs [20, 21] have shown that using multiple kernels (i.e., a combination of several “base kernels”) instead of a single fixed one can enhance classifier performance, which raised the so-called multiple kernel learning (MKL) method. With m kernels, input data can be mapped into m feature spaces, where each feature space can be taken as one view of the original input data [19]. Each view is expected to exhibit some geometrical structures of the original data from its own perspective such that all the views can complement for the subsequent learning task. It has been proven that MKL can offer some needed flexibility and well manipulate the case that involves multiple, heterogeneous data sources [18, 22, 23]. However, MKL is proposed for SVMs, and there have been few reports on performance of the kernel-based LDA method with multiple kernels. Liu and Feng propose multiple kernel Fisher discriminant analysis (MKFD) with an iterative scheme for weight optimization [24], in which the constructed kernel is a linear combination of several base kernels with a constraint on their weights.

In this paper, we integrate multiple kernel discriminant analysis with face recognition in wireless sensor networks, and propose a module based distributed wireless face recognition system. We consider separate cluster head for each module, that is, fore head, eye, lips, and nose. Only the local cluster is responsible for internal module processing for both training and recognition.

The rest of this paper is organized as follows. First we describe the module based distributed wireless face recognition system in Section 2. Then in Section 3, the optimization scheme for the multikernels is presented. The simulation results are reported in Section 4, while we draw our conclusion in Section 5.

2. Module Based Distributed Wireless Face Recognition

We present a face recognition system in wireless sensor networks where training and recognition are performed both in distributed environment. The image is divided into four submodules, that is, forehead, eyes, nose, and lips, as shown in Figure 2. For face recognition tasks, enrolment and identification of each submodule are performed in separate cluster heads, and the computations are carried out in kernel feature space [12]. Each cluster head is responsible for processing its submodule and communicating with the sink cluster which performs the score level fusion.

Figure 2

Submodules of a face image.

The following describes the score level fusion criterion at the sink node. Given N images belonging to C subjects. Denote the membership degree of the nth image belonging to the cth subject as ${\mu }_n^c$ , $n=\mathrm{1,2},\dots ,N$ , $c=\mathrm{1,2},\dots ,C$ . ${\mu }_n^c$ is obtained as follows:

$$\begin{array}{cc}& {\mu }_n^c={\alpha }_{\mathrm{1}}{s}_n^c\left(\mathrm{1}\right)+{\alpha }_{\mathrm{2}}{s}_n^c\left(\mathrm{2}\right)+{\alpha }_{\mathrm{3}}{s}_n^c\left(\mathrm{3}\right)+{\alpha }_{\mathrm{4}}{s}_n^c\left(\mathrm{4}\right),\end{array}$$

(1)

where $s_n^c(k)$ denotes the score of the kth submodule from the nth image with regard to the cth subject, and ${\alpha }_k$ is the corresponding weight value, $k=\mathrm{1,2},\mathrm{3,4}$ . The nth image is assigned to the Ith subject if and only if ${\mu }_n^I={\max }_{\mathrm{1}\le c\le C}{\mu }_n^c$ .

Now we explain score $s_n^c(k)$ , taking the forehead module as an example. In the forehead module cluster, there are also N samples belonging to C classes. For the nth sample ${\mathbf{x}}_n$ , we can compute the squared kernel distance between ${\mathbf{x}}_n$ and the center of the cth class ${\mathbf{m}}_c$ as follows:

$$\begin{array}{l}{d}_n^c={\parallel \mathrm{\Phi }\left({\mathbf{x}}_n\right)-\mathrm{\Phi }\left({\mathbf{m}}_c\right)\parallel }^{\mathrm{2}}\hfill \\ \hspace{1em}=\mathrm{\Phi }{\left({\mathbf{x}}_n\right)}^T\mathrm{\Phi }\left({\mathbf{x}}_n\right)-\mathrm{2}\mathrm{\Phi }{\left({\mathbf{x}}_n\right)}^T\mathrm{\Phi }\left({\mathbf{m}}_c\right)+\mathrm{\Phi }{\left({\mathbf{m}}_c\right)}^T\mathrm{\Phi }\left({\mathbf{m}}_c\right)\hfill \\ \hspace{1em}=k({\mathbf{x}}_n,{\mathbf{x}}_n)-\mathrm{2}k({\mathbf{x}}_n,{\mathbf{m}}_c)+k({\mathbf{m}}_c,{\mathbf{m}}_c),\hfill \end{array}$$

(2)

where $\mathrm{\Phi }:\mathbf{x}\in {\mathbf{R}}^t\to \mathrm{\Phi }(\mathbf{x})\in \mathbf{F}$ is a nonlinear mapping, which is implicitly defined by a mercer kernel function $k(\mathbf{x},\mathbf{y})=\mathrm{\Phi }(\mathbf{x}{)}^T\mathrm{\Phi }(\mathbf{y})$ [12, 25]. Then we sort the C distances $\{d_n^c\hspace{0.17em}|\hspace{0.17em}c=\mathrm{1,2},\dots ,C\}$ in ascending order and denote the squared kernel distances after sorting as $\{d_n^{i(c)}\hspace{0.17em}|\hspace{0.17em}c=\mathrm{1,2},\dots ,C\}$ ; that is, $i(c)$ is the order number of the cth largest squared kernel distance in $\{d_n^c\hspace{0.17em}|\hspace{0.17em}c=\mathrm{1,2},\dots ,C\}$ . Let

$$\begin{array}{cc}& s_n^{i\left(c\right)}\left(\mathrm{1}\right)=C-c,\end{array}$$

(3)

$c=\mathrm{1,2},\dots ,C$ , then we get $\{s_n^c(\mathrm{1})\hspace{0.17em}|\hspace{0.17em}c=\mathrm{1,2},\dots ,C\}$ . Thus, given the nth sample, the smaller the kernel distance between it and the center of the cth class, the greater $s_n^c(\mathrm{1})$ is, $c=\mathrm{1,2},\dots ,C$ .

Scores $s_n^c(\mathrm{2})~s_n^c(\mathrm{4})$ can be obtained from the eyes, nose, and lips modules, respectively, using a similar way to score $s_n^c(\mathrm{1})$ .

3. Optimization of Multiple Kernels

From Section 2, we can see the computations in the proposed distributed wireless face recognition frame are based on kernel techniques. The framework has four computing modules, for forehead, eyes, nose, and lips, respectively. These four kinds of data are so different that it is hard to imagine that an excellent classification can be reached by a single kernel. We propose the use of multiple kernels for these four modules. Specifically, we use the Gaussian RBF kernel

$$\begin{array}{cc}& k\left(\mathbf{x},\mathbf{y}\right)=\exp \left(-\frac{{\parallel \mathbf{x}-\mathbf{y}\parallel }^{\mathrm{2}}}{{\sigma }^{\mathrm{2}}}\right),\end{array}$$

(4)

but with different values of parameter σ for the different computing modules.

Every module should have its own optimal kernel parameter. For each module, we separately perform the following kernel parameter optimization procedure.

3.1. Some Notations on Kernel Discriminant Analysis

For a certain module X, there are N samples belonging to C classes. Assume the ith class $X_i$ contains $N_i$ samples; that is, $X_i=\{{\mathbf{x}}_{\mathrm{1}}^i,{\mathbf{x}}_{\mathrm{2}}^i,\dots ,{\mathbf{x}}_{N_i}^i\}$ , $i=\mathrm{1,2},\dots ,C$ , so $N={\sum }_{i=\mathrm{1}}^C{N}_i$ . Denote the kernel matrix ( $N\times N$ ) as

$$\begin{array}{cc}& \mathbf{K}={\left[k({\mathbf{x}}_j^i,{\mathbf{x}}_h^l)\right]}_{\begin{array}{c}i=\mathrm{1},\dots ,C,j=\mathrm{1},\dots ,N_i\\ l=\mathrm{1},\dots ,C,h=\mathrm{1},\dots ,N_l\end{array}},\end{array}$$

(5)

where $k({\mathbf{x}}_j^i,{\mathbf{x}}_h^l)=\mathrm{\Phi }({\mathbf{x}}_j^i{)}^T\mathrm{\Phi }({\mathbf{x}}_h^l)$ ; then the kernel matrix corresponds to the nonlinear mapping.

Under the nonlinear mapping $\mathrm{\Phi }$ , the ith mapped class and the mapped sample set are, respectively, given by

$$\begin{array}{cc}& \mathrm{\Phi }\left(X_i\right)=\left\{\mathrm{\Phi }\left({\mathbf{x}}_{\mathrm{1}}^i\right),\mathrm{\Phi }\left({\mathbf{x}}_{\mathrm{2}}^i\right),\dots ,\mathrm{\Phi }\left({\mathbf{x}}_{N_i}^i\right)\right\},\\ \mathrm{\Phi }\left(X\right)=\left\{\mathrm{\Phi }\left(X_{\mathrm{1}}\right),\mathrm{\Phi }\left(X_{\mathrm{2}}\right),\dots ,\mathrm{\Phi }\left(X_C\right)\right\}.\end{array}$$

(6)

Also, the mean of the mapped class $\mathrm{\Phi }(X_i)$ and that of the mapped sample set $\mathrm{\Phi }(X)$ are, respectively, given by

$$\begin{array}{cc}& {\mathbf{m}}_i=\frac{\mathrm{1}}{N_i}\sum _{j=\mathrm{1}}^{N_i}\mathrm{\Phi }\left({\mathbf{x}}_j^i\right),\hspace{1em}\hspace{1em}\mathbf{m}=\frac{\mathrm{1}}{N}\sum _{i=\mathrm{1}}^C\sum _{j=\mathrm{1}}^{N_i}\mathrm{\Phi }\left({\mathbf{x}}_j^i\right).\end{array}$$

(7)

In kernel feature space F (let f be dimensionality of F), the within-class scatter matrix ${\mathbf{S}}_w^{\mathrm{\Phi }}$ and between-class scatter matrix ${\mathbf{S}}_b^{\mathrm{\Phi }}$ are, respectively, defined as

$$\begin{array}{cc}& {\mathbf{S}}_w^{\mathrm{\Phi }}=\frac{\mathrm{1}}{N}\sum _{i=\mathrm{1}\hspace{0.17em}}^C\sum _{\mathbf{x}\in X_i}(\mathrm{\Phi }\left(\mathbf{x}\right)-{\mathbf{m}}_i){\left(\mathrm{\Phi }\left(\mathbf{x}\right)-{\mathbf{m}}_i\right)}^T={\mathrm{\Phi }}_w{\mathrm{\Phi }}_w^T,\\ {\mathbf{S}}_b^{\mathrm{\Phi }}=\frac{\mathrm{1}}{N}\sum _{i=\mathrm{1}}^C{N}_i({\mathbf{m}}_i-\mathbf{m}){\left({\mathbf{m}}_i-\mathbf{m}\right)}^T={\mathrm{\Phi }}_b{\mathrm{\Phi }}_b^T,\end{array}$$

(8)

where

$$\begin{array}{cc}& {\mathrm{\Phi }}_w={\left[{\varphi }_{\mathrm{1}}^{\mathrm{1}},\dots ,{\varphi }_{N_{\mathrm{1}}}^{\mathrm{1}},{\varphi }_{\mathrm{1}}^{\mathrm{2}},\dots ,{\varphi }_{N_{\mathrm{2}}}^{\mathrm{2}},\dots \dots ,{\varphi }_{\mathrm{1}}^C,\dots ,{\varphi }_{N_C}^C\right]}_{df\times N},\\ {\varphi }_j^i=\frac{\mathrm{1}}{\sqrt{N}}\left(\mathrm{\Phi }\left({\mathbf{x}}_j^i\right)-{\mathbf{m}}_i\right),\\ {\mathrm{\Phi }}_b={\left[{\phi }_{\mathrm{1}},\dots ,{\phi }_C\right]}_{df\times C},\hspace{1em}\hspace{1em}{\phi }_i=\sqrt{\frac{N_i}{N}}\left({\mathbf{m}}_i-\mathbf{m}\right).\end{array}$$

(9)

The kernel Fisher criterion is defined as

$$\begin{array}{cc}& J^{\mathrm{\Phi }}\left(\mathbf{W}\right)=\frac{\mathrm{tr}\left({\mathbf{W}}^T{\mathbf{S}}_b^{\mathrm{\Phi }}\mathbf{W}\right)}{\mathrm{tr}\left({\mathbf{W}}^T{\mathbf{S}}_w^{\mathrm{\Phi }}\mathbf{W}\right)},\end{array}$$

(10)

where $\mathbf{W}=\{{\mathbf{w}}_{\mathrm{1}},\dots ,{\mathbf{w}}_q\}$ is a $f\times q\mathrm{\hspace{0.17em}\hspace{0.17em}}(f>q)$ projection matrix. Kernel discriminant analysis is to find an optimal projection matrix ${\mathbf{W}}^{*}:{\mathbf{R}}^{df}\to {\mathbf{R}}^q$ in mapped feature space F, such that ${\mathbf{W}}^{*}={\text{argmax}}_{\mathbf{W}}{J}^{\mathrm{\Phi }}(\mathbf{W})$ .

3.2. Diagonalization Strategy

We use the same diagonalization strategy as KDDA [16] to deal with the small sample size (SSS) problem; that is, first diagonalizing ${\mathbf{S}}_b^{\mathrm{\Phi }}$ to I (identical matrix) and then diagonalizing ${\mathbf{S}}_w^{\mathrm{\Phi }}$ to ${\mathrm{\Lambda }}_w$ , which is briefly expressed as follows.

3.2.1. Eigenanalysis of ${\mathbf{S}}_b^{\mathrm{\Phi }}$ in the Feature Space

${\mathrm{\Phi }}_b^T{\mathrm{\Phi }}_b$ can be expressed using the kernel matrix K as follows:

$$\begin{array}{l}{\mathrm{\Phi }}_b^T{\mathrm{\Phi }}_b=\frac{\mathrm{1}}{N}\mathbf{D}·({\mathbf{A}}_{NC}^T·\mathbf{K}·{\mathbf{A}}_{NC}-\frac{\mathrm{1}}{N}{\mathbf{A}}_{NC}^T·\mathbf{K}·{\mathbf{1}}_{NC}\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{0.5em}-\frac{\mathrm{1}}{N}{\mathbf{1}}_{NC}^T·\mathbf{K}·{\mathbf{A}}_{NC}+\frac{\mathrm{1}}{N^{\mathrm{2}}}{\mathbf{1}}_{NC}^T·\mathbf{K}·{\mathbf{1}}_{NC})·\mathbf{D},\end{array}$$

(11)

where $\mathbf{D}=\mathrm{diag}(\sqrt{N_{\mathrm{1}}},\dots ,\sqrt{N_C})$ ( $C\times C$ diagonal matrix), ${\mathbf{1}}_{NC}$ is a $N\times C$ matrix with terms all equal to one, ${\mathbf{A}}_{NC}=\mathrm{diag}({\mathbf{a}}_{N_{\mathrm{1}}},\dots ,{\mathbf{a}}_{N_C})$ is a $N\times C$ block diagonal matrix, and ${\mathbf{a}}_{N_i}$ is a $N_i\times \mathrm{1}$ vector with all terms equal to $\mathrm{1}/N_i$ .

Let ${\lambda }_i$ and ${\mathbf{e}}_i\mathrm{\hspace{0.17em}\hspace{0.17em}}(i=\mathrm{1},\dots ,C)$ be the ith largest eigenvalue and corresponding eigenvector of ${\mathrm{\Phi }}_b^T{\mathrm{\Phi }}_b$ . Let $r(\le C-\mathrm{1})$ be the rank of ${\mathbf{S}}_b^{\mathrm{\Phi }}(={\mathrm{\Phi }}_b{\mathrm{\Phi }}_b^T)$ (also the rank of ${\mathrm{\Phi }}_b^T{\mathrm{\Phi }}_b$ ). Denote ${\mathbf{E}}_r=({\mathbf{e}}_{\mathrm{1}},\dots ,{\mathbf{e}}_r)$ and $\mathbf{V}=({\mathbf{v}}_{\mathrm{1}},\dots ,{\mathbf{v}}_r)={\mathrm{\Phi }}_b{\mathbf{E}}_r$ . It can be derived that ${\mathbf{V}}^T{\mathbf{S}}_b^{\mathrm{\Phi }}\mathbf{V}={\mathrm{\Lambda }}_b$ , with ${\mathrm{\Lambda }}_b=\mathrm{diag}({\lambda }_{\mathrm{1}}^{\mathrm{2}},\dots ,{\lambda }_r^{\mathrm{2}})$ , a nonsingular diagonal matrix. Let $\mathbf{U}=\mathbf{V}{\mathrm{\Lambda }}_b^{-\mathrm{1}/\mathrm{2}}$ . Then ${\mathbf{U}}^T{\mathbf{S}}_b^{\mathrm{\Phi }}\mathbf{U}=\mathbf{I}$ .

3.2.2. Eigenanalysis of ${\mathbf{S}}_w^{\mathrm{\Phi }}$ in the Feature Space

Based on the analysis in Section 3.2.1, it can be seen that

$$\begin{array}{cc}& {\mathbf{U}}^T{\mathbf{S}}_w^{\mathrm{\Phi }}\mathbf{U}={\left({\mathbf{E}}_r{\mathrm{\Lambda }}_b^{-\mathrm{1}/\mathrm{2}}\right)}^T\left({\mathrm{\Phi }}_b^T{\mathbf{S}}_w^{\mathrm{\Phi }}{\mathrm{\Phi }}_b\right)\left({\mathbf{E}}_r{\mathrm{\Lambda }}_b^{-\mathrm{1}/\mathrm{2}}\right),\end{array}$$

(12)

where ${\mathrm{\Phi }}_b^T{\mathbf{S}}_w^{\mathrm{\Phi }}{\mathrm{\Phi }}_b$ can be expressed using K, with similar details to that seen in [16].

Let ${\mathbf{z}}_j$ be the eigenvector of ${\mathbf{U}}^T{\mathbf{S}}_w^{\mathrm{\Phi }}\mathbf{U}$ corresponding to the jth smallest eigenvalue ${\lambda }_j^{\prime }$ , $j=\mathrm{1},\dots ,r$ . Denote $\mathbf{Z}=({\mathbf{z}}_{\mathrm{1}},\dots ,{\mathbf{z}}_r)$ . Defining $\mathbf{Y}=\mathbf{U}\mathbf{Z}$ , it can be derived that ${\mathbf{Y}}^T{\mathbf{S}}_w^{\mathrm{\Phi }}\mathbf{Y}={\mathrm{\Lambda }}_w$ , with ${\mathrm{\Lambda }}_w=\mathrm{diag}({\lambda }_1^{\prime },\dots ,{\lambda }_r^{\prime })$ .

Based on the derivation presented in Sections 3.2.1 and 3.2.2, an optimal projection matrix for kernel discriminant analysis is obtained as

$$\begin{array}{cc}& {\mathbf{W}}^{*}=\mathbf{Y}{\mathrm{\Lambda }}_w^{-\mathrm{1}/\mathrm{2}}={\mathrm{\Phi }}_b{\mathbf{E}}_r{\mathrm{\Lambda }}_b^{-\mathrm{1}/\mathrm{2}}\mathbf{Z}{\mathrm{\Lambda }}_w^{-\mathrm{1}/\mathrm{2}}.\end{array}$$

(13)

Certainly, as the nonlinear mapping $\mathrm{\Phi }$ is implicitly defined by the kernel function (or matrix), ${\mathrm{\Phi }}_b$ (defined by (9)) remains unknown, and ${\mathbf{W}}^{*}$ can not be evaluated. The real meaning of (13) is obtaining matrix ${\mathbf{E}}_r{\mathrm{\Lambda }}_b^{-\mathrm{1}/\mathrm{2}}\mathbf{Z}{\mathrm{\Lambda }}_w^{-\mathrm{1}/\mathrm{2}}$ , which can be computed from the kernel matrix K. This is the core result of diagonalization process.

3.3. Optimization Criterion and Objective

We adopt the maximum margin criterion (MMC) [26] as the objective function to optimize the kernel parameter σ for each specific module (module of forehead, eyes, nose, or lips):

$$\begin{array}{cc}& F\left(\mathbf{W},\sigma \right)=\mathrm{tr}\left({\mathbf{W}}^T{\mathbf{S}}_b^{\mathrm{\Phi }}\mathbf{W}\right)-\mathrm{tr}\left({\mathbf{W}}^T{\mathbf{S}}_w^{\mathrm{\Phi }}\mathbf{W}\right),\end{array}$$

(14)

where W is a projection matrix, and σ is the parameter for the Gaussian RBF kernel as in (4).

Based on the result of (13) in Section 3.2, the optimal projection matrix ${\mathbf{W}}^{*}={\mathrm{\Phi }}_b{\mathbf{E}}_r{\mathrm{\Lambda }}_b^{-\mathrm{1}/\mathrm{2}}\mathbf{Z}{\mathrm{\Lambda }}_w^{-\mathrm{1}/\mathrm{2}}$ . Denoting $\mathbf{G}={\mathbf{E}}_r{\mathrm{\Lambda }}_b^{-\mathrm{1}/\mathrm{2}}\mathbf{Z}{\mathrm{\Lambda }}_w^{-\mathrm{1}/\mathrm{2}}$ , which can be computed from the kernel matrix K. Then the objective function (14) can be reformulated as

$$\begin{array}{l}F\left(\sigma \right)=\mathrm{tr}({\mathbf{W}}^{*T}{\mathbf{S}}_b^{\mathrm{\Phi }}{\mathbf{W}}^{*}-{\mathbf{W}}^{*T}{\mathbf{S}}_w^{\mathrm{\Phi }}{\mathbf{W}}^{*})\hfill \\ \hspace{1em}\hspace{1em}=\mathrm{tr}\left({\mathbf{G}}^T{\mathrm{\Phi }}_b^T{\mathrm{\Phi }}_b{\mathrm{\Phi }}_b^T{\mathrm{\Phi }}_b\mathbf{G}-{\mathbf{G}}^T{\mathrm{\Phi }}_b^T{\mathrm{\Phi }}_w{\mathrm{\Phi }}_w^T{\mathrm{\Phi }}_b\mathbf{G}\right)\hfill \\ \hspace{1em}\hspace{1em}=\mathrm{tr}({\mathbf{G}}^T\mathbf{P}{\mathbf{P}}^T\mathbf{G}-{\mathbf{G}}^T\mathbf{Q}{\mathbf{Q}}^T\mathbf{G}),\hfill \end{array}$$

(15)

where $\mathbf{P}={\mathrm{\Phi }}_b^T{\mathrm{\Phi }}_b$ and $\mathbf{Q}={\mathrm{\Phi }}_b^T{\mathrm{\Phi }}_w$ can be expressed in terms of the kernel matrix K as follows:

$$\begin{array}{l}\mathbf{P}=\frac{\mathrm{1}}{N}\mathbf{D}·({\mathbf{A}}_{NC}^T·\mathbf{K}·{\mathbf{A}}_{NC}-\frac{\mathrm{1}}{N}{\mathbf{A}}_{NC}^T·\mathbf{K}·{\mathbf{1}}_{NC}\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{0.5em}-\frac{\mathrm{1}}{N}{\mathbf{1}}_{NC}^T·\mathbf{K}·{\mathbf{A}}_{NC}+\frac{\mathrm{1}}{N^{\mathrm{2}}}{\mathbf{1}}_{NC}^T·\mathbf{K}·{\mathbf{1}}_{NC})·\mathbf{D},\end{array}$$

(16)

with D, ${\mathbf{A}}_{NC}$ , and ${\mathbf{1}}_{NC}$ defined the same as in (11);

$$\begin{array}{l}\mathbf{Q}=\frac{\mathrm{1}}{N}\mathbf{D}·(\phantom{\frac{\mathrm{1}}{N}}{\mathbf{A}}_{NC}^T·\mathbf{K}-{\mathbf{A}}_{NC}^T·\mathbf{K}·{\mathbf{H}}_{NN}\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{0.5em}-\frac{\mathrm{1}}{N}{\mathbf{1}}_{NC}^T·\mathbf{K}+\frac{\mathrm{1}}{N}{\mathbf{1}}_{NC}^T·\mathbf{K}·{\mathbf{H}}_{NN}),\end{array}$$

(17)

where ${\mathbf{H}}_{NN}=\mathrm{diag}({\mathbf{h}}_{N_{\mathrm{1}}},\dots ,{\mathbf{h}}_{N_C})$ is a $N\times N$ block diagonal matrix and ${\mathbf{h}}_{N_i}$ is a $N_i\times N_i$ matrix with all terms equal to $\mathrm{1}/N_i$ .

Therefore, regarding G as constant, the objective function $F(\sigma )$ is an explicit function of the kernel parameter σ. To maximize the objective function with ${\sigma }_{\mathrm{0}}$ as the initial value of the parameter, an iterative procedure based on Newton's method is developed in our method to update the kernel parameter, as shown in the following section.

3.4. Solving the Optimization Problem

Assume that G is constant. To obtain the extremum of the objective function $F(\sigma )$ , we need to differentiate $F(\sigma )$ with respect to σ.

For the sake of clarity, we denote the differentiation of the kernel matrix K with respect to σ as $d{\mathbf{K}}_{\sigma }$ , which is an $N\times N$ matrix expressed as follows:

$$\begin{array}{cc}& d{\mathbf{K}}_{\sigma }={\left[\frac{dk({\mathbf{x}}_j^i,{\mathbf{x}}_h^l)}{d\sigma }\right]}_{\begin{array}{c}i=\mathrm{1},\dots ,C,j=\mathrm{1},\dots ,N_i\\ l=\mathrm{1},\dots ,C,h=\mathrm{1},\dots ,N_l\end{array}}.\end{array}$$

(18)

Each element in matrix $d{\mathbf{K}}_{\sigma }$ is the differentiation of the corresponding element in the kernel matrix K with respect to σ and can be formulated as

$$\begin{array}{c}\frac{dk({\mathbf{x}}_j^i,{\mathbf{x}}_h^l)}{d\sigma }=\frac{d}{d\sigma }\exp (-\frac{{\parallel {\mathbf{x}}_j^i-{\mathbf{x}}_h^l\parallel }^{\mathrm{2}}}{{\sigma }^{\mathrm{2}}})\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}=\frac{\mathrm{2}{\parallel {\mathbf{x}}_j^i-{\mathbf{x}}_h^l\parallel }^{\mathrm{2}}}{{\sigma }^{\mathrm{3}}}\exp (-\frac{{\parallel {\mathbf{x}}_j^i-{\mathbf{x}}_h^l\parallel }^{\mathrm{2}}}{{\sigma }^{\mathrm{2}}}).\hfill \end{array}$$

(19)

From (16) and (17), matrices P and Q can be differentiated with respect to σ as follows:

$$\begin{array}{l}\frac{d\mathbf{P}}{d\sigma }=\frac{\mathrm{1}}{N}\mathbf{D}·({\mathbf{A}}_{NC}^T·d{\mathbf{K}}_{\sigma }·{\mathbf{A}}_{NC}-\frac{\mathrm{1}}{N}{\mathbf{A}}_{NC}^T·d{\mathbf{K}}_{\sigma }·{\mathbf{1}}_{NC}\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}-\frac{\mathrm{1}}{N}{\mathbf{1}}_{NC}^T·d{\mathbf{K}}_{\sigma }·{\mathbf{A}}_{NC}+\frac{\mathrm{1}}{N^{\mathrm{2}}}{\mathbf{1}}_{NC}^T·d{\mathbf{K}}_{\sigma }·{\mathbf{1}}_{NC})\\ \hspace{1em}\hspace{1em}\hspace{1em}·\mathbf{D},\\ \frac{d\mathbf{Q}}{d\sigma }=\frac{\mathrm{1}}{N}\mathbf{D}·(\phantom{\frac{\mathrm{1}}{N}}{\mathbf{A}}_{NC}^T·d{\mathbf{K}}_{\sigma }-{\mathbf{A}}_{NC}^T·d{\mathbf{K}}_{\sigma }·{\mathbf{H}}_{NN}\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{0.17em}\hspace{1em}\hspace{0.5em}-\frac{\mathrm{1}}{N}{\mathbf{1}}_{NC}^T·d{\mathbf{K}}_{\sigma }+\frac{\mathrm{1}}{N}{\mathbf{1}}_{NC}^T·d{\mathbf{K}}_{\sigma }·{\mathbf{H}}_{NN}),\end{array}$$

(20)

where $d\mathbf{P}/d\sigma$ is a $C\times C$ matrix and $d\mathbf{Q}/d\sigma$ is a $C\times N$ matrix.

Then, the derivative of F with respect to σ can be formulated as

$$\begin{array}{l}{F}^{\prime }\left(\sigma \right)=\mathrm{tr}(({\mathbf{G}}^T\frac{d\mathbf{P}}{d\sigma }{\mathbf{P}}^T\mathbf{G}+{\mathbf{G}}^T\mathbf{P}\frac{d{\mathbf{P}}^T}{d\sigma }\mathbf{G})\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{0.5em}\hspace{0.5em}\hspace{1em}-({\mathbf{G}}^T\frac{d\mathbf{Q}}{d\sigma }{\mathbf{Q}}^T\mathbf{G}+{\mathbf{G}}^T\mathbf{Q}\frac{d{\mathbf{Q}}^T}{d\sigma }\mathbf{G})).\end{array}$$

(21)

Thus the derivative of F with respect to σ can be expressed in terms of matrix K and matrix $d{\mathbf{K}}_{\sigma }$ . To achieve the maximum of $F(\sigma )$ , we set the derivative to zero:

$$\begin{array}{cc}& F^{\prime }\left(\sigma \right)=\mathrm{0}.\end{array}$$

(22)

We use Newton's method to solve (22) with the initial value of the kernel parameter

$$\begin{array}{cc}& {\sigma }_{\mathrm{0}}=\sqrt{\frac{\mathrm{1}}{N^{\mathrm{2}}}\sum _{i=\mathrm{1}}^N\sum _{j=\mathrm{1}}^N{\parallel {\mathbf{x}}_i-{\mathbf{x}}_j\parallel }^{\mathrm{2}}};\end{array}$$

(23)

that is, ${\sigma }_{\mathrm{0}}^{\mathrm{2}}$ is the average squared distance of all the samples in the given module. And the iteration formula is

$$\begin{array}{cc}& {\sigma }_{k+\mathrm{1}}={\sigma }_k-\frac{F\left({\sigma }_k\right)}{F^{\prime }\left({\sigma }_k\right)}.\end{array}$$

(24)

$k=\mathrm{0,1},\mathrm{2},\dots$ .

3.5. Identification Using Multiple Kernels

For each of the four modules (forehead, eyes, nose, and lips), the optimization of the Gaussian kernel parameter runs and finds the optimal value as above separately. Then the four submodules of a testing image are fed into the corresponding kernel discriminant classifiers to compute the membership degree of the image belonging to the every subject according to (1). Finally the image is assigned to the subject which shows the greatest membership degree of the image.

4. Simulation

To evaluate the performance of our multiple kernel framework for distributed wireless face recognition, we have made experimental comparisons with KDDA based on Gaussian RBF kernel, in terms of recognition accuracy. Images are from two face databases, namely, the FERET and the CMU PIE databases.

In our experiments, for the weight value in the fusion criterion (1), we set ${\alpha }_{\mathrm{1}}=\mathrm{0.1}$ , ${\alpha }_{\mathrm{2}}=\mathrm{0.3}$ , ${\alpha }_{\mathrm{3}}=\mathrm{0.4}$ , and ${\alpha }_{\mathrm{4}}=\mathrm{0.2}$ , meaning that we give larger weights to the nose module and the eyes module.

4.1. Face Image Datasets

From the FERET database [27], we select 72 people, with 6 frontal-view images for each individual. Face image variations in these 432 images include illumination, facial expression, wearing glasses, and aging. All the images are aligned by the centers of the eyes and the mouth and then normalized with a resolution of 92 × 112. The pixel value of each image is normalized between 0 and 1. The original images with resolution 92 × 112 are reduced to wavelet feature faces with resolution 49 × 59 after 1-level Daubechies-4 (Db4) wavelet decomposition. Images from one individual are shown in Figure 3.

Figure 3

Images of one person from the FERET database.

In the CMU PIE face database [28], there are a total of 68 people, and each person has 13 pose variations ranged from the full right profile image to the full left profile image and 43 different lighting conditions, 21 flashes with ambient light on or off. In our experiments, for each person, we select 56 images including 13 poses with neutral expression and 43 different lighting conditions in the frontal view. For all frontal-view images, we apply alignment based on two-eye center and nose center points, and no alignment is applied on the other images with poses. All the segmented images are rescaled to the resolution of 92 × 112, and then reduced to wavelet feature faces with resolution 49 × 59 after 1-level Daubechies-4 (Db4) wavelet decomposition. Some images of one person are shown in Figure 4.

Figure 4

Some images of one person from the CMU PIE face database.

4.2. Recognition Results

This section reports the recognition results of the proposed multiple kernel framework and KDDA with a single Gaussian RBF kernel on the FERET and the CMU PIE datasets. For KDDA, the parameter of Gaussian RBF kernel is optimized via grid search. For each subject in the FERET dataset, we randomly select n ( $n=2$ to 5) out of 6 images for training, with the rest for testing. In the CMU PIE dataset, the number of randomly selected training images is ranged from 10 to 18 out of 56 for each individual, while the rest are testing images. The average recognition accuracies over 10 runs on the FERET and CMU PIE datasets are shown in Figures 5(a) and 5(b), respectively.

Figure 5

Comparison of accuracies obtained by multiple kernel framework and KDDA with a single kernel.

Table 1 shows the average and standard deviation of the accuracies for FERET (n = 4: 4 images per subject for training with the rest for testing) and CMU-PIE (n = 14: 14 images per subject for training with the rest for testing), respectively.

Table 1

Performance comparison between multiple kernel framework and KDDA with a single kernel.

Type of kernel	FERET ( $n=\mathit{4}$ )	CMU PIE ( $n=\mathit{14}$ )
RBF kernel
Mean	87.54%	71.99%
Std	0.042	0.013
Multi kernels
Mean	91.16%	78.96%
Std	0.011	0.006

The bold value means the higher average accuracy of the two methods.

From the results in Figure 5 and Table 1, it can be seen that the proposed multiple kernel framework can achieve higher accuracies than KDDA with an optimized parameter.

5. Conclusion

In this paper, on the assumption that multiple kernels can characterize geometrical structures of the original data from multiple views which can complement to improve recognition performance, we integrate multiple kernel discriminant analysis with face recognition in wireless sensor networks and propose a module based distributed wireless face recognition system. For each module, we separately perform an iterative scheme based on Newton's method for kernel parameter optimization, by maximizing the margin maximization criterion. The multiple kernel framework and the optimization procedure yield high recognition accuracy on the FERET and CMU PIE face database, compared with a single kernel.