What should a researcher do when statistical analysis software terminates before completion with a message that the Hessian is not invertible? The standard textbook advice is to respecify the model, but this is another way of saying that the researcher should change the question being asked. Obviously, however, computer programs should not be in the business of deciding what questions are worthy of study. Although noninvertable Hessians are sometimes signals of poorly posed questions, nonsensical models, or inappropriate estimators, they also frequently occur when information about the quantities of interest exists in the data through the likelihood function. The authors explain the problem in some detail and lay out two preliminary proposals for ways of dealing with noninvertable Hessians without changing the question asked.
Amemiya, Takeshi. 1985. Advanced Econometrics. Cambridge, MA: Harvard University Press.
2.
Berger, James O.1976. “Admissible Minimax Estimation of a Multivariate Normal Mean With Arbitrary Quadratic Loss.”Annals of Statistics4: 223-226.
3.
Davidson, Russell and James G. MacKinnon. 1993. Estimation and Inference in Econometrics. New York: Oxford University Press.
4.
Efron, Bradley and C. Morris. 1972. “Limiting the Risk of Bayes and Empirical Bayes Estimators—Part II: The Empirical Bayes Case.”Journal of the American Statistical Association67: 130-139.
5.
Efron, Bradley and Robert J. Tibshirani. 1993. An Introduction to the Bootstrap. New York: Chapman & Hall.
6.
Fiacco, A. V. and G. P. McCormick. 1968. Nonlinear Programming: Sequential Unconstrained Minimization Techniques. New York: John Wiley.
7.
Freedman, D. A.1981. “Bootstrapping Regression Models.”Annals of Statistics9: 1218-1228.
8.
Gelman, Andrew, John B. Carlin, Hal S. Stern, and Donald B. Rubin. 1995. Bayesian Data Analysis. New York: Chapman & Hall.
9.
Gill, Phillip E., G. H. Golub, Walter Murray, and M. A. Sanders. 1974. “Methods for Modifying Matrix Factorizations.”Mathematics of Computation28: 505-535.
10.
Gill, Phillip E. and Walter Murray. 1974. “Newton-Type Methods for Unconstrained and Linearly Constrained Optimization.”Mathematical Programming7: 311-350.
11.
Gill, Phillip E., Walter Murray, and M. H. Wright. 1981. Practical Optimization. London: Academic Press.
12.
Greene, William H.1993. Econometric Analysis. 2d ed.New York: Macmillan.
13.
Hinkley, David V.1988. “Bootstrap Methods.”Journal of the Royal Statistical Society, Series B50: 321-337.
14.
Hoerl, A. E. and R. W. Kennard. 1970a. “Ridge Regression: Biased Estimation for Non-orthogonal Problems.”Technometrics12: 55-67.
15.
Hoerl, A. E. and R. W. Kennard. 1970b. “Ridge Regression: Applications to Nonorthogonal Problems.”Technometrics12: 69-82.
16.
James, W. and C. Stein. 1961. “Estimation With Quadratic Loss.” In Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, edited by Jerzy Neyman. Berkeley: University of California Press.
17.
King, Gary. 1997. A Solution to the Ecological Inference Problem: Reconstructing Individual Behavior From Aggregate Data. Princeton, NJ: Princeton University Press.
18.
King, Gary. 1998. Unifying Political Methodology: The Likelihood Theory of Statistical Inference. Ann Arbor: University of Michigan Press.
19.
King, Gary, James Honaker, Anne Joseph, and Kenneth Scheve. 1998. “Listwise Deletion Is Evil: What to Do About Missing Data in Political Science.” Paper presented at the annual meetings of the American Political Science Association, Boston.
20.
King, Gary, Michael Tomz, and Jason Wittenberg. 2000. “Making the Most of Statistical Analyses: Improving Interpretation and Presentation.”American Journal of Political Science44: 347-361.
21.
Leamer, Edward E.1973. “Multicollinearity: A Bayesian Interpretation.”Review of Economics and Statistics55(3): 371-380.
22.
Marquardt, D. W.1970. “Generalized Inverses, Ridge Regression, Biased Linear Estimation, and Nonlinear Estimation.”Technometrics12: 591-612.
23.
Matthews, A. and D. Davies. 1971. “A Comparison of Modified Newton Methods for Unconstrained Optimization.”Computer Journal14: 213-294.
24.
McCullagh, P. and J. A. Nelder. 1989. Generalized Linear Models. New York: Chapman & Hall.
25.
Moore, E. H.1920. “On the Reciprocal of the General Algebraic Matrix (Abstract). ”Bulletin of the American Mathematical Society26: 394-395.
26.
Penrose, R. A.1955. “A Generalized Inverse for Matrices.” Proceedings of the CambridgePhilosophical Society51: 406-413.
27.
Rao, C. Radhakrishna and Sujit Kumar Mitra. 1971. Generalized Inverse of Matrices and Its Applications. New York: Johnn Wiley.
28.
Riley, James. 1955. “Solving Systems of Linear Equations With a Positive Definite, Symmetric but Possibly Ill-Conditioned Matrix.”Mathematical Tables and Other Aides to Computation9: 96-101.
29.
Rubin, D. B.1987. Multiple Imputation for Nonresponse in Surveys. New York: John Wiley.
30.
Schnabel, Robert B. and Elizabeth Eskow. 1990. “A New Modified Cholesky Factorization.”SIAM Journal of Scientific Statistical Computing11(6): 1136-1158.
31.
Searle, S. R.1971. Linear Models. New York: John Wiley.
32.
Shao, Jun and Dongsheng Tu.1995. The Jackknife and Bootstrap. New York: Springer.
33.
Smith, Gary and Frank Campbell. 1980. “A Critique of Some Ridge Regression Methods (With Comments). ”Journal of the American Statistical Association75: 74-103.
34.
Strawderman, W. E.1978. “Minimax Adaptive Generalized Ridge Regression Estimators.”Journal of the American Statistical Association72: 890-891.
35.
Tanner, M. A.1996. Tools for Statistic Inference: Methods for the Exploration of Posterior Distributions and Likelihood Functions. New York: Springer.
36.
Wei, G. C. G. and M. A. Tanner. 1990. “A Monte Carlo Implementation of the EM Algorithm and the Poor Man’s Data Augmentation Algorithm.”Journal of the American Statistical Association85: 699-704.
