Sage Journals: Discover world-class research

Abstract

This article introduces an entropy-based measure of data–model fit that can be used to assess the quality of logistic regression models. Entropy has previously been used in mixture-modeling to quantify how well individuals are classified into latent classes. The current study proposes the use of entropy for logistic regression models to quantify the quality of classification and separation of group membership. Entropy complements preexisting measures of data–model fit and provides unique information not contained in other measures. Hypothetical data scenarios, an applied example, and Monte Carlo simulation results are used to demonstrate the application of entropy in logistic regression. Entropy should be used in conjunction with other measures of data–model fit to assess how well logistic regression models classify cases into observed categories.

Keywords

data–model fit logistic regression classification

Get full access to this article

View all access options for this article.

References

Celeux

Soromenho

(1996). An entropy criterion for assessing the number of clusters in a mixture model. Journal of Classification, 13, 195-212.

Clark

Muthén

(2009). Relating latent class analysis results to variables not included in the analysis. Retrieved from http://www.statmodel.com/download/relatinglca.pdf

Cohen

West

S. G.

Aiken

L. S.

(2003). Applied multiple regression/correlation analysis for the behavioral sciences (3rd ed.). Hillsdale, NJ: Lawrence Erlbaum.

Cox

D. R.

Snell

E. J.

(1989). Analysis of binary data (2nd ed.). London, England: Chapman & Hall.

Dawson

R. M.

(1995). The “unusual episode” data revised. Journal of Statistics Education, 3(3). Retrieved from http://www.amstat.org/publications/jse/v3n3/datasets.dawson.html

Eaton

J. P.

Haas

C. A.

(1994). Titanic: Triumph and tragedy. New York, NY: W. W. Norton.

Fawcett

(2005). An introduction to ROC analysis. Pattern Recognition Letters, 27, 861-874.

Harrell

F. E.

(2001). Regression modeling strategies with applications to linear models, logistic regression, and survival analysis. New York, NY: Springer-Verlag.

Henson

J. M.

Reise

S. P.

Kim

K. H.

(2007). Detecting mixtures from structural model differences using latent variable mixture modeling: a comparison of relative model fit statistics. Structural Equation Modeling, 14, 202-226.

10.

Hosmer

D. W.

Lemeshow

(2000). Applied logistic regression (2nd ed.). New York, NY: Wiley.

11.

Hosmer

D. W.

Hosmer

Le Cessie

Lemeshow

(1997). A comparison of goodness-of-fit tests for the logistic regression model. Statistics in Medicine, 16, 965-980.

12.

Lomax

R. G.

Hahs-Vaughn

D. L.

(2012). An introduction to statistical concepts. 3rd ed.New York, NY: Routledge.

13.

Long

J. S.

(1997). Regression models for categorical and limited dependent variables. Thousand Oaks, CA: Sage.

14.

Nagelkerke

N. J. D.

(1991). A note on a general definition of the coefficient of determination, Biometrika, 78, 691-692.

15.

Nylund

K. L.

Asparouhov

Muthén

B. O.

(2007). Deciding on the number of classes in latent class analysis and growth mixture modeling: A Monte Carlo simulation study. Structural Equation Modeling, 14, 535-569.

16.

O’Connell

A. A.

Amico

K. R.

(2010). Logistic regression. In Hancock

G. R.

Mueller

R. O.

(Eds.), The reviewer’s guide to quantitative methods in the social sciences (pp. 221-239). New York, NY: Routledge.

17.

Pastor

D. A.

Gagne

(2013). Mean and covariance structure mixture models. In Hancock

G. R.

Mueller

R. O.

(Eds.), Structural equation modeling: A second course (2nd ed., pp. 197-224). Charlotte, NC: Information Age.

18.

Peng

C. Y.

T. H.

Stage

F. K.

St. John

E. P.

(2002). The use and interpretation of logistic regression in higher education journals: 1988-1999. Research in Higher Education, 43, 259-293.

19.

Ramaswamy

Desarbo

W. S.

Reibstein

D. J.

Robinson

W. T.

(1993). An empirical pooling approach for estimating marketing mix elasticities with PIMS data. Marketing Science, 12, 103-124.

20.

Streiner

D. L.

Cairney

(2007). What’s under the ROC? An introduction to receiver operating characteristic curves. Canadian Journal of Psychiatry, 52, 121-128.

An Entropy-Based Measure for Assessing Fuzziness in Logistic Regression

Abstract

Keywords

Get full access to this article

References