This article presents a general framework for univariate non-parametric density estimation, based on mixture models. Similar to kernel-based estimation, the proposed approach uses bandwidth to control the density smoothness, but each density estimate for a fixed bandwidth is determined by non-parametric likelihood maximization, with bandwidth selection carried out as model selection. This leads to simple models, yet with higher accuracy, especially in terms of the Kullback–Leibler or the Hellinger risk. The particular problem of estimating a symmetric density function is investigated. Both simulation study and real-world data examples suggest that the mixture-based estimators outperform their kernel-based counterparts.
BöhningD (1995) A review of reliable algorithms for the semi-parametric maximum likelihood estimator of a mixture distribution. Journal of Statistical Planning & Inference, 47, 5–28.
2.
BöhningD (2000) Computer-assisted analysis of mixtures and applications: meta-analysis, disease mapping, and others. London: Chapman and Hall/CRC.
3.
BöhningD (2003) The EM algorithm with gradient function update for discrete mixtures with known (fixed) number of components. Statistics and Computing, 13, 257–65.
4.
ChenJKhaliliA (2009) Order selection in finite mixture models with nonsmooth penalty. Journal of the American Statistical Association, 104, 187–96.
5.
ChristofidesLNStengosT (2002) The symmetry of the wage-change distribution: Survey and contract data. Empirical Economics, 27, 705–23.
6.
DaxA (1990) The smallest point of a polytope. Journal of Optimization Theory and Applications, 64, 429–32.
7.
EfronB (2004) Large-scale simultaneous hypothesis testing: the choice of a null hypothesis. Journal of the American Statistical Association, 99, 96–104.
8.
Frühwirth-SchnatterS (2006) Finite mixture and markov switching models. New York: Springer.
9.
HärdleWMüllerMSperlichSWerwatzA (2004) Nonparametric and semiparametric models. Berlin: Springer.
10.
HedenfalkIDugganDChenYRadmacherM. (2001) Gene-expression profiles in hereditary breast cancer. New England Journal of Medicine, 344, 539–48.
11.
HürlimannW (2001) Financial data analysis with two symmetric distributions. Astin Bulletin, 31, 187–211.
12.
JakiTWestRW (2010) Symmetric maximum kernel likelihood estimation. Journal of Statistical Computation and Simulation, 81, 193–206.
13.
KieferJWolfowitzJ (1956) Consistency of the maximum likelihood estimator in the presence of infinitely many incidental parameters. Annals of Mathematical Statistics, 27, 886–906.
14.
KraftCHLepageYvan EedenC (1985) Estimation of a symmetric density function. Communications in Statistics: Theory and Methods, 14, 273–88.
15.
LairdNM (1978) Nonparametric maximum likelihood estimation of a mixing distribution. Journal of the American Statistical Association, 73, 805–11.
16.
LawsonCLHansonRJ (1974) Solving least squares problems. Englewood Cliffs: Prentice-Hall, Inc.
17.
LindsayBG (1983) The geometry of mixture likelihoods: A general theory. Annals of Statistics, 11, 86–94.
18.
LindsayBG (1995) Mixture models: theory, geometry and applications, Volume 5 of NSF-CBMS Regional Conference Series in Probability and Statistics. Institute for Mathematical Statistics: Hayward, CA.
19.
LindsayBGLesperanceML (1995) A review of semi-parametric mixture models. Journal of Statistical Planning & Inference, 47, 29–39.
20.
LoaderCR (1999) Bandwidth selection: classical or plug-in. Annals of Statistics, 27, 415–38.
21.
McLachlanGJKrishnanT (1997) The EM algorithm and extensions. New York: Wiley.
22.
McLachlanGJPeelD (2000) Finite mixture models. New York: Wiley.
23.
R Development Core Team (2009) R: a language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. ISBN 3-900051-07-0.
24.
RoederK (1990) Density estimation with confidence sets exemplified by superclusters and voids in the galaxies. Journal of the American Statistical Association, 85, 617–24.
ScottDWSzewczykWF (2001) From kernels to mixtures. Technometrics, 43, 323–35.
27.
ScottDWTerrellGR (1987) A reliable data-based bandwidth selection method for kernel density estimation. Journal of the American Statistical Association, 82, 1131–146.
28.
SheatherSJ (2004) Density estimation. Statistical Science, 19, 588–97.
29.
SheatherSJJonesMC (1991) A reliable data-based bandwidth selection method for kernel density estimation. Journal of the Royal Statistical Society, Ser. B, 53, 683–90.
30.
SilvermanBW (1986) Density estimation for statistics and data analysis. London: Chapman and Hall.
31.
SimonoffJS (1996) Smoothing methods in statistics. New York: Springer.
32.
TitteringtonDMSmithAFMMarkovUE (1985) Statistical analysis of finite mixture distributions. Chichester: Wiley.
33.
WandMPJonesMC (1995) Kernel smoothing. London: Chapman and Hall.
34.
WangY (2007) On fast computation of the non-parametric maximum likelihood estimate of a mixing distribution. Journal of the Royal Statistical Society, Ser. B, 69, 185–98.
35.
WangY (2010) Maximum likelihood computation for fitting semi-parametric mixture models. Statistics and Computing, 20, 75–86.
