A common problem in statistical modelling is to distinguish between finite mixture distribution and a homogeneous non-mixture distribution. Finite mixture models are widely used in practice and often mixtures of normal densities are indistinguishable from homogenous non-normal densities. This paper illustrates what happens when the EM algorithm for normal mixtures is applied to a distribution that is a homogeneous non-mixture distribution. In particular, a population-based EM algorithm for finite mixtures is introduced and applied directly to density functions instead of sample data. This algorithm is used to find finite mixture approximations to common homogeneous distributions. An example regarding the nature of a placebo response in drug treated depressed subjects is used to illustrate ideas.
Azzalini A
(2006) R package sn: the skew-normal and skew-t distributions (version 0.4-0).
Universit di Padova, Italia
. Available at http://cran.r-project.org/src/contrib/Descriptions/sn.html
2.
Azzalini A
and
Capitanio A
(1999)
Statistical applications of the multivariate skew-normal distribution
. Journal of the Royal Statistical Society, Series B, 61,
579–602
.
3.
Bauer DJ
and
Curran PJ
(2003)
Distributional assumptions of growth mixture models: implications for over extraction of latent trajectory classes
. Psychological Methods, 8,
338–63
.
4.
Box GEP
(1979) Robustness in the strategy of scientific model building. In
Launer RL
and
Wilkinson GN
eds. Statistics.
New York: Academic Press
.
5.
Dempster AP
,
Laird NM
and
Rubin DB
(1977)
Maximum likelihood from incomplete data via the EM algorithm
. Journal of the American Statistical Association, 39,
1–38
.
6.
Dunn G
,
Sham PC
and
Hand DJ
(1993)
Statistics and the nature of depression
. Journal of the Royal Statistical Society, Series A, 156,
63–87
.
7.
Everitt BS
(1981)
Bimodality and the nature of depression
. British Journal of Psychiatry, 138,
336–39
.
8.
Fang KT
and
Wang Y
(1994) Number-theoretic method in statistics.
London: Chapman and Hall
.
9.
Flury B
(1990)
Principal points
. Biometrika, 77,
33–41
.
10.
Flury B
(1993)
Estimation of principal points
. Applied Statistics, 42,
139–51
.
11.
Forgy EW
(1965)
Cluster analysis of multivariate data: efficiency vs interpretability of classifications
. Biometrics, 21,
768–69
.
12.
Hartigan JA
(1978)
Asymptotic distributions for clustering criteria
. Annals of Statistics, 6,
117–31
.
13.
Hartigan JA
and
Wong MA
(1979)
A k-means clustering algorithm
. Applied Statistics, 28,
100–08
.
14.
Li L
and
Flury B
(1995)
Uniqueness of principal points for univariate distributions
. Statistics and Probability Letters, 25,
323–27
.
15.
MacQueen J
(1967)
Some methods for classification and analysis of multivariate observations
. Proceedings of 5th Berkeley Symposium on Mathematics, Statistics and Probability, 3,
281–97
.
16.
Marriott FHC
(1971)
Practical problems in a method of cluster analysis
. Biometrics, 27,
501–14
.
17.
McLachlan GJ
and
Krishnan T
(1997) The EM algorithm and extensions.
New York: Wiley
.
18.
Mease D
and
Nair VN
(2006)
Unique optimal partitions of distributions and connections to hazard rates and stochastic ordering
. Statistica Sinica, 16,
1299–312
.
19.
Murphy EA
(1964)
One cause? Many causes? The argument from the bimodal distribution
. Journal of Chronic Diseases, 17,
301–24
.
20.
Pearson K
(1895)
Contributions to the mathematical theory of evolution. ii. skew variation in homogeneous material
. Philisophical Transactions of the Royal Society of London, 186,
343–414
.
21.
Piessens R
,
de Doncker Kapenga E
,
Uberhuber C
and
Kahaner D
(1983) QUADPACK, A Quadrature Subroutine Package, Series in Computational Mathematics.
Berlin: Springer-Verlag
.
22.
R Development Core Team (2003) R: A language and environment for statistical computing.
Vienna: R Foundation for Statistical Computing
.
23.
Scott DW
and
Szewczyk WF
(2001)
From kernels to mixtures
. Technometrics, 43,
323–35
.
24.
Swartz T
and
Evans M
(2000) Approximating integrals via Monte Carlo and deterministic methods.
New York: Oxford University Press
.
25.
Tarpey T
(1994)
Two principal points of symmetric, strongly unimodal distributions
. Statistics and Probability Letters, 20,
253–57
.
26.
Tarpey T
(1998)
Self-consistent patterns for symmetric multivariate distributions
. Journal of Classification, 15,
57–79
.
27.
Tarpey T
and
Petkova E
(2007) Latent regression analysis. Submitted for publication.
28.
Tarpey T
,
Petkova E
and
Ogden RT
(2003)
Profiling placebo responders by self-consistent partitions of functional data
. Journal of the American Statistical Association, 98,
850–58
.
29.
Titterington DM
,
Smith AFM
and
Makov UE
(1985) Statistical analysis of finite mixture distributions.
New York: Wiley
.
30.
Topsøe F
(2001)
Basic concepts, identities and inequalities—the toolkit of information theory
. Entropy, 3,
162–90
.
