Conclusions about changes in categorical characteristics based on observed panel data can be incorrect when (even a small amount of) measurement error is present. Random measurement errors, referred to as independent classification errors, usually lead to over-estimation of the total amount of gross change, whereas systematic, correlated errors usually cause underestimation of the transitions. Furthermore, the patterns of true change may be seriously distorted by independent or systematic classification errors. Latent class models and directed log-linear analysis are excellent tools to correct for both independent and correlated measurement errors. An extensive example on labor market states taken from the Survey of Income and Program Participation panel is presented.
Get full access to this article
View all access options for this article.
References
1.
Abowd, John M.
and Arnold Zellner. 1985. “Estimating Gross Labor Force Flows.”Journal of Business and Economic Statistics3:254-283.
2.
Agresti, Alan
. 1990. Categorical Data Analysis. New York: Wiley.
3.
Bartholomew, David J.
and Karl F. Schuessler. 1991. “Reliability of Attitude Scores Based on a Latent Trait Model.” Pp. 97-123 in Sociological Methodology 1991, edited by Peter V. Marsden. Oxford: Blackwell.
4.
Bassi, Francesca
, Marcel Croon, Jacques A. Hagenaars, and Jeroen K. Vermunt. 1995. “Estimating Latent Turnover Tables When Data Are Affected by Correlated and Uncorrelated Classification Errors.” WORC Paper No. 95.12, Tilburg University.
5.
Becker, Mark P.
and Ilsoon Yang. 1998. “Latent Class Marginal Models for Cross-Classifications of Counts.” Pp. 293-326 in Sociological Methodology 1998, edited by Adrian E. Raftery. Washington, DC: American Sociological Association.
6.
Bergsma, Wicher P.
1997. Marginal Models for Categorical Data. Tilburg: Tilburg University Press.
7.
Biemer, Paul P.
and Dennis Trewin. 1997. “A Review of Measurement Error Effects on the Analysis of Survey Data.” Pp. 603-632 in Survey Measurement and Process Quality, edited by Lars Lyberg, Paul Biemer, Martin Collins, Edith de Leeuw, Cathryn Dippo, Norbert Schwarz, and Dennis Trewin. New York: Wiley.
8.
Bishop, Yvonne M. M.
, Stephen E. Fienberg, and Paul W. Holland. 1975. Discrete Multivariate Analysis: Theory and Practice. Cambridge, MA: MIT Press.
9.
Bollen, Kenneth A.
1989. Structural Equations With Latent Variables. New York: Wiley.
10.
Bryk, Anthony S.
and Stephen W. Raudenbush. 1992. Hierarchical Linear Models: Applications and Data Analysis Methods. Newbury Park, CA: Sage.
11.
Burkhead, Dan
and John Coder. 1985. “Gross Changes in Income Recipiency From the Survey of Income and Program Participation.” Pp. 351-356 in Proceedings of the Social Statistics Section. Washington, DC: American Statistical Association.
12.
Chua, Tin C.
and Wayne A. Fuller. 1987. “A Model for Multinomial Response Error Applied to Labor Flows.”Journal of the American Statistical Association82:46-51.
13.
Citro, C. F.
and Graham Kalton. 1993. The Future of the SIPP. Washington, DC: National Academy Press.
14.
Clogg, Clifford C.
1981. “New Developments in Latent Structure Analysis.” Pp. 215-246 in Factor Analysis and Measurement in Sociological Research, edited by David J. Jackson and Edgar F. Borgatta. Beverly Hills, CA: Sage.
15.
Collins, Linda M.
, Penny L. Fidler, and Stuart E. Wugalter. 1996. “Some Practical Issues Related to the Estimation of Latent Class and Latent Transition Parameters.” Pp. 133-146 in Categorical Variables in Developmental Research: Methods of Analysis, edited by Alexander von Eye and Clifford C. Clogg. San Diego, CA: Academic Press.
16.
Collins, Linda M.
, Penny L. Fidler, Stuart E. Wugalter, and Jeffrey D. Long. 1993. “Goodness-of-Fit Testing for Latent Class Models.”Multivariate Behavioral Research28:375-389.
17.
Collins, Linda M.
and Stuart E. Wugalter. 1992. “Latent Class Models for Stage-Sequential Dynamic Latent Variables.”Multivariate Behavioral Research27:131-157.
18.
Cox, D. R.
and Nanny Wermuth. 1996. Multivariate Dependencies: Models, Analysis and Interpretations. London: Chapman & Hall.
19.
Davidson, Russell
and James G. MacKinnon. 1993. Estimation and Inference in Econometrics. New York: Oxford University Press.
20.
De Leeuw, Jan
, Peter G. Van der Heijden, and Peter Verboon. 1990. “A Latent Time-Budget Model.”Statistica Neerlandica44:1-22.
21.
DeSarbo, Wayne S.
and Michel Wedel. 1993. A Review of Recent Developments in Latent Class Regression Models. Research Memorandum No. 521. Groningen: University of Groningen.
22.
Diggle, Peter J.
, Kung-Yee Liang, and Scott L. Zeger. 1996. Analysis of Longitudinal Data. Oxford: Clarendon.
23.
Evers, Mark
and N. Krishnan Namboodiri. 1978. “On the Design Matrix Strategy in the Analysis of Categorical Data.” Pp. 86-111 in Sociological Methodology 1979, edited by Karl F. Schuessler. San Francisco: Jossey-Bass.
24.
Gelman, Andrew
, John B. Carlin, Hal S. Stern, and Donald B. Rubin. 1995. Bayesian Data Analysis. London: Chapman & Hall.
25.
Glymour, Clark
, Richard Scheines, Peter Spirtes, and Kevin Kelly. 1987. Discovering Causal Structure. Orlando, FL: Academic Press.
26.
Goodman, Leo A.1973. “The Analysis of a Multidimensional Contingency Table When Some Variables Are Posterior to the Others.”Biometrika60:179-192.
27.
Goodman, Leo A.1974a. “Exploratory Latent Structure Analysis Using Both Identifiable and Unidentifiable Models.”Biometrika61:215-231.
28.
Goodman, Leo A.1974b. “The Analysis of Systems of Qualitative Variables When Some of the Variables Are Unobservable: Part I. A Modified Latent Structure Approach.”American Journal of Sociology79:1179-1259.
29.
Haberman, Shelby J.
1978. Analysis of Qualitative Data: Vol. 1. Introductory Topics. New York: Academic Press.
30.
Haberman, Shelby J.
1979. Analysis of Qualitative Data: Vol. 2. New Developments. New York: Academic Press.
31.
Hadgu, Alula
and Yinsheng Qu. 1998. “A Biomedical Application of Latent Class Models With Random Effects.”Applied Statistics47:603-616.
32.
Hagenaars, Jacques A.1988. “Latent Structure Models With Direct Effects Between the Indicators, Local Dependence Models.”Sociological Methods & Research16:379-405.
33.
Hagenaars, Jacques A.
1990. Categorical Longitudinal Data Log-Linear Panel, Trend and Cohort Analysis. Newbury Park, CA: Sage.
34.
Hagenaars, Jacques A.
1993. Loglinear Models With Latent Variables. Newbury Park, CA: Sage.
35.
Hagenaars, Jacques A.
1994. “Latent Variables in Log-Linear Models of Repeated Observations.” Pp. 329-352 in Latent Variable Analysis: Applications for Developmental Research, edited by Alexander von Eye and Clifford C. Clogg. Thousand Oaks, CA: Sage.
36.
Hagenaars, Jacques A.1998. “Categorical Causal Modeling: Latent Class Analysis and Directed Log-Linear Models With Latent Variables.”Sociological Methods & Research26:436-487.
37.
Heckman, James J.
and Burton Singer. 1982. “Population Heterogeneity in Demographic Models.” Pp. 567-599 in Multidimensional Mathematical Demography, edited by Kenneth Land and Andrei Rogers. New York: Academic Press.
38.
Heinen, Ton G.
1996. Latent Class and Discrete Latent Trait Models: Similarities and Differences. Thousand Oaks, CA: Sage.
39.
Heise, David R.1969. “Separating Reliability and Stability in Test-Retest Correlation.”American Sociological Review34:93-101.
40.
Hubble, Dan L.
and David R. Judkins. 1987. “Measuring the Bias in Gross Flows in the Presence of Autocorrelated Measurement Error.” Survey of Income and Program Participation Working Paper No. 8712.
41.
Hughes, James P.
, Peter Guttorp, and Stephen P. Charles. 1999. “A Non-Homogeneous Hidden Markov Model for Precipitation Occurrence.”Applied Statistics48:15-30.
42.
Juang, B. H.
and L. R. Rabiner. 1991. “Hidden Markov Models for Speech Recognition.”Technometrics33:251-272.
43.
Kalton, Graham
and C. F. Citro. 1994. “Panel Surveys: Adding the Fourth Dimension.”Survey Methodology19:205-215.
44.
Kasprzyk, Daniel
, Greg J. Duncan, Graham Kalton, and M. P. Singh. 1989. Panel Surveys. New York: Wiley.
45.
Kendall, Patricia
. 1954. Conflict and Mood: Factors Affecting Stability and Response. New York: Free Press.
46.
Kuha, Jouni
and Chris Skinner. 1997. “Categorical Data and Misclassification.” Pp. 633-670 in Survey Measurement and Process Quality, edited by Lars Lyberg, Paul Biemer, Martin Collins, Edith de Leeuw, Cathryn Dippo, Norbert Schwarz, and Dennis Trewin. New York: Wiley.
47.
Lang, Joseph B.
and Alan Agresti. 1994. “Simultaneous Modeling Joint and Marginal Distributions of Multivariate Categorical Responses.”Journal of the American Statistical Association89:625-632.
48.
Langeheine, Rolf
, Jeroen Pannekoek, and Frank Van de Pol. 1996. “Bootstrapping Goodness-of-Fit Measures in Categorical Data Analysis.”Sociological Methods & Research24:492-516.
49.
Lauritzen, Steffen L.
1996. Graphical Models. Oxford: Clarendon.
50.
Lazarsfeld, Paul F.
1972. “The Problem of Measuring Turnover.” Pp. 115-125 in Continuities in the Language of Social Research, edited by Paul F. Lazarsfeld, Ann K. Pasanella, and Morris Rosenberg. New York: Free Press.
51.
Lazarsfeld, Paul F.
and Neil W. Henry. 1968. Latent Structure Analysis. Boston: Houghton Mifflin.
52.
Lazarsfeld, Paul F.
and Morris Rosenberg. 1955. The Language of Social Research. Glencoe, IL: Free Press.
53.
Lemaitre, Georges E.
1988. “The Measurement and Analysis of Gross Flows.”Working Paper Series No. SSMD-88-1-E, Statistics Canada.
54.
Lord, Frederic M.
, and Melvin R. Novick. 1968. Statistical Theories of Mental Test Scores. Reading, MA: Addison-Wesley.
55.
Maccoby, Eleanor E.1956. “Pitfalls in the Analysis of Panel Data: A Research Note on Some Technical Aspects of Voting.”American Journal of Sociology61:359-362.
56.
Martini, Alberto
. 1988. “Retrospective Versus Panel Data in Estimating Labor Force Gross Flows: Comparing SIPP and CPS.” Paper presented at the annual meeting of the American Statistical Association, Social Science Section, August 22-25, New Orleans.
57.
Martini, Alberto
. 1989. “Seam Effect, Recall Bias and the Estimation of Labor Force Transition Rates From SIPP.” Pp. 387-392 in Proceedings of the Section of Survey Research Methods. Washington, DC: American Statistical Association.
58.
Mooijaart, Ab
and Peter G. M. Van der Heijden. 1992. “The EM Algorithm for Latent Class Analysis With Equality Constraints.”Psychometrika57:261-269.
59.
O'Muircheartaigh, Colm
. 1996. “Measurement Errors in Panel Surveys: Implications for Survey Design and for Survey Instruments.” Pp. 207-218 in Proceedings of the Scientific Reunion of the Italian Statistical Society. Rimini: Maggioli.
60.
Pearl, Judea
. 1995. “Causal Diagrams for Empirical Research.”Biometrika82:669-710.
61.
Plewis, Ian
. 1985. Analysing Change: Measurement and Explanation Using Longitudinal Data. Chichester, UK: Wiley.
62.
Poterba, James M.
and Lawrence H. Summers. 1986. “Reporting Errors and Labor Market Dynamics.”Econometrica54:1319-1338.
63.
Poulsen, Chris S. 1982. “Latent Structure Analysis With Choice Modeling Applications.” Ph.D. dissertation, Wharton School, University of Pennsylvania.
64.
Qu, Y.
, M. Tan and M. H. Kutner. 1996. “Random Effects Models in Latent Class Analysis for Evaluating Accuracy of Diagnostic Tests.”Biometrics52:797-810.
65.
Raftery, Adrian E.
1993. “Bayesian Model Selection in Structural Equation Models.” Pp. 163-180 in Testing Structural Equation Models, edited by Kenneth A. Bollen and J. Scott Long. Newbury Park, CA: Sage.
66.
Raftery, Adrian E.
1995. “Bayesian Model Selection in Social Research.” Pp. 111-164 in Sociological Methodology 1995, edited by Peter V. Marsden. Washington, DC: American Sociological Association.
67.
Raftery, Adrian E., ed. 1998. “Special Issue: Causality in the Social Sciences: In Honor of Herbert L. Costner.”Sociological Methods & Research27:139-348.
68.
Rubin, Donald B.1974. “Estimating Causal Effects of Treatments in Randomized and Nonrandomized Studies.”Journal of Educational Psychology66:688-701.
69.
Rubin, Donald B.
and Hal S. Stern. 1994. “Testing in Latent Class Models Using a Posterior Predictive Check Distribution.” Pp. 420-438 in Latent Variables Analysis: Applications for Developmental Research, edited by Alexander von Eye and Clifford C. Clogg. Thousand Oaks, CA: Sage.
70.
Schafer, Joseph L.
1997. Analysis of Incomplete Multivariate Data. London: Chapman & Hall.
71.
Skinner, Chris
and Nicola Torelli. 1993. “Measurement Error and the Estimation of Gross Flows From Longitudinal Economic Data.”Statistica3:391-405.
72.
Sobel, Michael E.
1994. “Causal Inference in Latent Variable Models.” Pp. 3-35 in Latent Variables Analysis: Applications for Developmental Research, edited by Alexander von Eye and Clifford C. Clogg. Thousand Oaks, CA: Sage.
73.
Sobel, Michael E.
. 1995. “Causal Inferences in the Social and Behavioral Sciences.” Pp. 1-38 in Handbook of Statistical Modeling for the Social and Behavioral Sciences, edited by Gerhard Arminger, Clifford C. Clogg, and Michael E. Sobel. New York: Plenum.
74.
Sutcliffe, J. P.1965a. “A Probability Model for Errors of Classification: I. General Considerations.”Psychometrika30:73-96.
75.
Sutcliffe, J. P.1965b. “A Probability Model for Errors of Classification: II. Particular Cases.”Psychometrika30:129-155.
76.
U.S. Department of Commerce
. 1991. SIPP's User Guide. Washington, DC: U.S. Department of Commerce.
77.
Van de Pol, Frank
and Rolf Langeheine. 1990. “Mixed Markov Latent Class Models.” Pp. 213-247 in Sociological Methodology, edited by C. Clogg. New York: Blackwell.
78.
Van der Heijden
, Peter, Harm 't Hart, and Jos Dessens. 1997. “A Parametric Bootstrap Procedure to Perform Statistical Tests in a LCA of Anti-Social Behavior.” Pp. 196-208 in Applications of Latent Trait and Latent Class Models in the Social Sciences, edited by Jürgen Rost and Rolf Langeheine. New York: Waxmann.
79.
Vermunt, Jeroen K.
1996. Loglinear Event History Analysis: A General Approach With Missing Data, Latent Variables, and Unobserved Heterogeneity. Tilburg: Tilburg University Press.
80.
Vermunt, Jeroen K.
1997a. “EM: A General Program for the Analysis of Categorical Data.” WORC paper, Tilburg University.
81.
Vermunt, Jeroen K.
1997b. Loglinear Models for Event History Analysis. Thousand Oaks, CA: Sage.
82.
Weakliem, David L.1999. “A Critique of the Bayesian Information Criterion for Model Selection [With Discussion].”Sociological Methods & Research27:359-397.
83.
Whittaker, Joe
. 1990. Graphical Models in Applied Multivariate Statistics. New York: Wiley.
84.
Wiggins, Lee M.
1955. Mathematical Model for the Analysis of Multi-Wave Panels. Ann Arbor, MI: University Microfilms.
85.
Wiggins, Lee M.
1973. Panel Analysis: Latent Probability Models for Attitude and Behavior Change. Amsterdam: Elsevier.
86.
Young, Nathan
. 1989. “Wave Seam Effect in SIPP.” Pp. 393-398 in Proceedings of the Survey Research Section. Washington, DC: American Statistical Association.
