Abstract
In this article, a novel spatially constrained clustering approach is proposed for ensemble image registration. We use a spatially constrained Gaussian mixture model, which is based on a joint Gaussian mixture model and Markov random field, to model the joint intensity scatter plot of the unregistered images. The spatially constrained Gaussian mixture model has the capability of performing the correlation among neighboring observations. A cost function of reducing the dispersion in the joint intensity scatter plot is proposed using the spatially constrained Gaussian mixture model to simultaneously register a group of images. We derive an expectation maximization algorithm for the proposed model. Computer simulations demonstrate the effectiveness of the proposed method.
Keywords
Introduction
Image registration is a fundamental operation in image analysis. It has been widely used in many applied fields, 1 –5 including remote sensing, computer vision, medical image processing, and robotics.
Many algorithms have been proposed to deal with the problem of image registration in the last decades. 6 –11 They can be roughly categorized as feature-based, Fourier-based, and intensity-based methods. Traditional ways for registering are choosing one image as a template, and every other image is registered to it. This kind of way is widely used. However, the disadvantage is obvious which image should be chosen as the template. It has the problem of selection dependency and internal inconsistency. 12 So, many algorithms were proposed to perform the group image registration. Minimizing the sum of squared differences was applied to register group images. 13 A sum of entropies criterion was proposed to group images registration. 14 A sum of Jensen–Shannon divergence is considered as cost function in the study of Qian et al. 15 A joint intensity space, where each axis corresponds to intensity from each of the images, was proposed for ensemble image registration. 12 It is assumed that each object in an image corresponds to a coherent collection. The density of joint intensity space of group images is modelled by a Gaussian mixture model (GMM). The likelihood of GMM maximized to reduce dispersion of the joint intensity scatter plot (JISP). 12 An ensemble clustering method with a regularization term based on mean elastic energy of B-spline was proposed to register nonrigid multisensor ensemble images. 16 Considering without choosing a proper number of clusters in advance, infinite GMM was proposed to simultaneously register a group of images. 17 However, the aforementioned methods were not considering the relationship between neighboring pixels in an image.
To address this issue, we propose a clustering method for the registration of ensemble images. The JISP of the unregistered images is modelled using a spatially constrained Gaussian mixture model (SCGMM). 18 The expectation maximization (EM) algorithm is employed to inference the parameters in SCGMM.
The article is organized as follows. The proposed SCGMM model is presented in section “Model formulation.” In section “EM for SCGMM,” EM is proposed to estimate the parameters in SCGMM. Simulation results based on various data sets are given in “Results” section to validate the performance of the proposed method. The final section gives the conclusion.
Model formulation
Consider an image group consisting of D images. Let
The SCGMM model is proposed to model the density of D images. So, the density function can be written as
where
From the study of Yasutomi and Tanaka, 19 the spatial correlation among label values was modelled by Markov random field distribution. We get
where Z is a normalizing constant, T is a temperature constant, and U(Π) is the smoothing prior. The complete data log-likelihood of our proposed method is
The proposed method for ensemble image registration is based on maximizing complete data log-likelihood in equation (4) of the images by (1) density estimation and (2) minimizing the dispersion of the scatter plot by moving the images with parameters θ.
EM for SCGMM
To obtain a maximum likelihood estimate of the parameters, the EM algorithm is proposed. The EM algorithm consists of the following two steps:
where
From the study of Nguyen and Wu, 18 the smoothing prior U(Π) is given as
where Gijis defined as
where zmj and β denote the posterior probability and the temperature value that controls the smoothing prior, respectively. Pi denotes the number of neighboring pixels around the pixel xi. In this article, the parameters β and Pi are chosen as 12 and 25, respectively. 18 The log-likelihood function in equation (4) can be rewritten as
The conditional expectation of equation (7) can be given as
The conditional expectation values zij of the hidden variables can be computed as follows
For simplify, set Z = 1, T = 1. 18 So, we get
To maximize equation (10), for mean
Set it to zero, we have
For the covariance
For the prior distribution, in order to enforce the constraint
Thus, setting the derivative of the function in equation (10) with πij to zero, we have
To optimize
To determine the motion parameters,
It becomes
It can be rewritten as
where
For example, consider the problem of registering three images a, b, and c. Let
The same linear approximation is proposed to images b and c. Then, we have
We get
Results
In order to demonstrate the performance of the proposed method, the normalized mutual information (NMI) and traditional GMM method 12 are compared. For each experiment, the image data was initialized by applying known displacements. The performance of each registration method was obtained by comparing the estimated transformations to the gold standard transformations, which is defined as
where err means the average pixel displacement error, ‖.‖ denotes the standard Euclidean norm, and ta and tb are the gold standard transformation and the inverse of the estimated transformations, respectively. Therefore, a good registration has a small err. If the err is larger than 3 pixels, then the registration is considered a failure. 12
Satellite images
We begin with a set of Landsat 7 satellite images to demonstrate the performance of the proposed method for the affine registration. The set of satellite images are given in Figure 1. A total of 30 trial images group is generated by applying displacements with six parameters (two scales, one shear, one rotation, and two translations). Parameters are chosen uniformly from the range [0.95–1.05] for scales, [−0.2 to 0.2] for the shear, [−5 to 5] degrees for the rotation, and [−5 to 5] pixels for the translations. The number of Gaussian mixture in these registration methods is chosen as 6. The overlap region in Figure 1(a) was performed for registration. The JISP of satellite images is given in Figure 2. The results of the satellite image experiment are given in Table 1. In Table 1, it is shown that that the proposed method has better performance than the NMI method. The proposed method has almost the same performance as that of the GMM method.

The set of satellite images.

The joint intensity scatter plot of satellite images.
Average err of satellite images.
GMM, Gaussian mixture model; NMI, normalized mutual information.
Face images
A data set of facial images from the Extended Yale Face Database B 20 is shown in Figure 3. The images are the same face illuminated with five different light positions ranging from far left to far right. Ten trial ensembles are generated by applying rigid-body displacements. The parameters of displacements are chosen uniformly from the following range [−10 to 10] pixels or degrees. The overlap region in Figure 3(a) was performed for registration. The number of Gaussian mixture in these registration methods is chosen as 6. The JISP of satellite images is given in Figure 4. The results of the facial image experiment are given in Table 2. It is observed the proposed method again has best registration performance.

The set of face images.

The joint intensity scatter plot of facial images.
Average err of facial images.
NMI, normalized mutual information; GMM Gaussian mixture model.
Road images
A data set of road images is shown in Figure 5. The image is obtained by a vehicle travelling data recorder. Ten trial ensembles are generated by applying rigid-body displacements. The parameters of displacements are chosen uniformly from the following range: [−1 to 1] degrees for the rotation and [−5 to 5] pixels for the translations. The number of Gaussian mixture in these registration methods is chosen as 6. The results of the road image experiment are given in Table 3. It is shown that the proposed method has better performance than NMI and GMM methods. Furthermore, the JISP of road images is given in Figure 6. From Figure 6, different Gaussian components model coherent structures in the image.

The set of road images.
Average err of road images.
NMI, normalized mutual information; GMM, Gaussian mixture model.

The joint intensity scatter plot of road images.
Conclusion
In this article, a spatially constrained clustering approach is proposed for group image registration using SCGMM. Considering the relationship of neighboring pixels of images, the SCGMM is applied to model the intensity of the JISP of unregistered images. The EM algorithm is then proposed to estimate the parameters in SCGMM. Using the experiments of satellite images, face images, and road images, the proposed method has better ensemble image registration performance than other conventional methods.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work is jointly supported by the Open Foundation of first level Zhejiang Province Key in Key Discipline of Control Science and Engineering and by the National Natural Science Foundation of China (Grant no. 61301033).
