Likelihood inference with hierarchical models is often complicated by the fact that
the likelihood function involves intractable integrals. Numerical integration (e.g.
quadrature) is an option if the dimension of the integral is low but quickly becomes
unreliable as the dimension grows. An alternative approach is to approximate the
intractable integrals using Monte Carlo averages. Several different algorithms based
on this idea have been proposed. In this paper we discuss the relative merits of
simulated maximum likelihood, Monte Carlo EM, Monte Carlo Newton-Raphson and
stochastic approximation.
Abramowitz M
,
Stegun IA
(1972) Handbook of mathematical tables.
New York: Dover
.
2.
Booth JG
,
Hobert JP
(1999)
Maximizing generalized linear mixed model likelihoods with an automated
Monte Carlo EM algorithm
. Journal of the Royal Statistical Society, Series B,
61,
265-285
.
3.
Breslow NE
,
Clayton DG
(1993)
Approximate inference in generalized linear mixed models
. Journal of the American Statistical Association,
88,
9-25
.
4.
Breslow NE
,
Lin X.
(1995)
Bias correction in generalized linear mixed models with a single component
of dispersion
. Biometrika, 82,
81-91
.
5.
Cochran WG
(1977) Sampling techniques, 3rd edn.New York: Wiley
.
6.
Conte SD
,
de Boor C
(1972) Elementary numerical analysis: an algorithmic
approach, 2nd edn.New York: McGraw Hill
.
7.
Coull BA
,
Hobert JP
,
Ryan LM
,
Holmes LB
(2001)
Crossed random effect models for multiple outcomes in a study of teratogenesis
. Journal of the American Statistical Association,
96,
1194-1204
.
8.
Crepon B
,
Duguet E
(1997)
Research and development, competition and innovation: Pseudo-maximum
likelihood and simulated maximum likelihood methods applied to count data
models with heterogeneity
. Journal of Econometrics, 79,
355-378
.
9.
Crouch EAC
,
Spiegelman D
(1990)
The evaluation of integrals of the form (See PDF for Formula): Application to
logistic-normal models
. Journal of the American Statistical Association,
85,
464-469
.
10.
Danielsson J
(1994)
Stochastic volatility in asset prices. Estimation with simulated maximum likelihood
. Journal of Econometrics, 64,
375-400
.
11.
Delyon B
,
Lavielle M
,
Moulines E
(1999)
Convergence of a stochastic approximation version of the EM algorithm
. The Annals of Statistics, 27,
94-128
.
12.
Dempster AP
,
Laird NM
,
Rubin DB
(1977)
Maximum likelihood from incomplete data via the EM algorithm (with discussion)
. Journal of the Royal Statistical Society, Series B,
39,
1-38
.
13.
Evans M
,
Swartz T
(1995)
Methods for approximating integrals in statistics with special emphasis on
Bayesian integration problems
. Statistical Science, 10,
254-272
.
14.
Geyer CJ
(1994)
On the convergence of Monte Carlo maximum likelihood calculations
. Journal of the Royal Statistical Society, Series B,
56,
261-274
.
15.
Geyer CJ
,
Thompson EA
(1992)
Constrained Monte Carlo maximum likelihood for dependent data
. Journal of the Royal Statistical Society, Series B,
54,
657-699
.
16.
Gouriéroux C
,
Monfort A
(1993)
Simulation-based inference: A survey with special reference to panel data models
. Journal of Econometrics, 59,
5-33
.
17.
Gu MG
,
Li S
(1998)
A stochastic approximation algorithm for maximum-likelihood estimation with
incomplete data
. Canadian Journal of Statistics, 26,
567-582
.
18.
Gu MG
,
Zhu H-T
(2001)
Maximum likelihood estimation for spatial models by Markov chain Monte
Carlo stochastic approximation
. Journal of the Royal Statistical Society, Series B,
63,
339-355
.
19.
Hobert JP
(2000)
Hierarchical models: A current computational perspective
. Journal of the American Statistical Association,
95,
1312-1316
.
20.
Jank W (2001) Stochastic estimation methods in general hierarchical
models, PhD thesis, University of Florida, Department of Statistics.
21.
Karim MR
,
Zeger SL
(1992)
Generalized linear models with random effects; salamander mating revisited
. Biometrics, 48,
631-644
.
22.
Kuk AYC
(1995)
Asymptotically unbiased estimation in generalized linear models with random effects
. Journal of the Royal Statistical Society, Series B,
57,
395-407
.
23.
Lange K
(1995)
A gradient algorithm locally equivalent to the EM algorithm
. Journal of the Royal Statistical Society, Series B,
57,
425-437
.
24.
Lee L-F
(1999)
Statistical inference with simulated likelihood functions
. Econometric Theory, 15(3),
337-360
.
25.
Lin X
,
Breslow NE
(1996)
Bias correction in generalized linear mixed models with multiple components
of dispersion
. Journal of the American Statistical Association,
91,
1007-1016
.
26.
McCullagh P
,
Nelder JA
(1989) Generalized linear models, 2nd edn,
London: Chapman & Hall
.
27.
McCulloch CE
(1997)
Maximum likelihood algorithms for generalized linear mixed models
. Journal of the American Statistical Association,
92,
162-170
.
28.
McCulloch CE
,
Searle SR
(2001) Generalized, linear, and mixed models.
New York: Wiley
.
29.
Meng X-L
,
Rubin DB
(1991)
Using EM to obtain asymptotic variance-covariance matrices: The SEM algorithm
. Journal of the American Statistical Association,
86,
899-909
.
30.
Moon C-G
,
Stotsky JG
(1993)
Testing the differences between the determinants of Moody’s and
Standard & Poor’s ratings. An application of smooth
simulated maximum likelihood estimation
. Journal of Applied Econometrics, 8,
51-69
.
31.
Oakes D
(1999)
Direct calculation of the information matrix via the EM algorithm
. Journal of the Royal Statistical Society, Series B,
61,
479-482
.
32.
Owen AB
,
Zhou Y
(2000)
Safe and effective importance sampling
. Journal of the American Statistical Association,
95,
135-143
.
33.
Robbins H
,
Monro S.
(1951)
A stochastic approximation method
. The Annals of Mathematical Statistics, 22,
400-407
.
34.
Robert CP
,
Casella G
(1999) Monte Carlo statistical methods,
New York: Springer
.
35.
Ruppert D
(1991) Handbook of sequential analysis.
New York: Marcel Dekker
, 503-529.
36.
Searle SR
,
Casella G
,
McCulloch CE
(1992) Variance components.
New York: John Wiley & Sons
.
37.
Tanner MA
(1996) Tools for statistical inference: methods for the
exploration of posterior distributions and likelihood functions. 3rd edn.New York: Springer
.
38.
Wei GCG
,
Tanner MA
(1990)
A Monte Carlo implementation of the EM algorithm and the poor
man’s data augmentation algorithms
. Journal of the American Statistical Association,
85,
699-704
.
39.
Wolfinger R
,
O’Connell M
(1993)
Generalized linear mixed models: a pseudo-likelihood approach
. Journal of Statistical Computation and Simulation,
48,
233-243
.
