The problems created by measurement error are entirely ignored in the vast majority of statistical analyses. To adjust for the effects of measurement error requires both a theory, or model, of measurement and estimates of the relevant measurement parameters (e.g., reliability coefficients). A fairly well-developed measurement theory for interval level data has been known for quite some time. A corresponding measurement theory for categorical data is not widely known even though such data are at least as important in the social sciences as interval data. Nevertheless, such a theory exists in the statistical journals. The primary purpose of this article is pedagogical: that is, to present the foundation of this theory for binary variables, the simplest type of categorical variable, and to demonstrate that the consequences of measurement errors in binary data are different from and probably more serious than the effects of measurement errors in interval level data. The principal reason for this is that measurement errors in a binary variable are likely to have a nonzero mean and will always be negatively correlated with the underlying true scores. The former has the effect of biasing the sample estimate of the mean, often to such a degree that the likelihood that a 95% confidence interval will contain the population mean is almost nil.
Get full access to this article
View all access options for this article.
References
1.
ASSAKUL, K.
and C. H. PROCTOR (1967) “Testing independence in two-way contingency tables with data subject to misclassification.”Psychometrika32: 67-76.
2.
ARMOR, D.J.
(1974) “Theta reliability and factor scaling,” pp. 17-50 in H. L. Constner (ed.) Sociological Methodology 1973-1974. San Francisco: Jossey-Bass.
3.
BIELBY, W. T.
, R. M. HAUSER, and D. L. FEATHERMAN (1977) “Response errors of black and non-black males in models of the intergenerational transmission of socioeconomic status.”Amer. J. of Sociology82: 1242-1288.
4.
BROSS, I.
(1954) “Misclassification in 2×2 tables.”Biometrics10: 478-486.
5.
CHEN, T. T.
(1979) “Log-linear models for categorical data with misclassification and double sampling.”J. of the Amer. Stat. Assn.74: 481-488.
6.
CHIACCHIERINI, R.P.
and J.C. ARNOLD (1977) “A two-sample test for independence in 2×2 contingency tables with both margins subject to misclassification.”J. of the Amer. Stat. Assn.72: 170-174.
7.
CLOGG, C.C.
(1981) “New developments in latent structure analysis,” pp. 215-246 in D.J. Jackson and E. F. Borgatta (eds.) Factor Analysis and Measurement in Sociological Research. Beverly Hills, CA: Sage.
8.
CLOGG, C.C.
and D. O. SAWYER (1980) “A comparison of alternative models for analyzing the scalability of response patterns,” pp. 240-280 in K. Schuessler (ed.) Sociological Methodology 1981. San Francisco: Jossey-Bass.
9.
COCHRAN, W. G.
(1968) “Errors of measurement in statistics.”Technometrics10: 637-666.
10.
CRONBACH, L. J.
(1951) “Coefficient alpha and the internal structure of tests.”Psychometrica16: 297-334.
11.
DIAMOND, E. L.
and A. M. LILIENFIELD (1962) “Effects of errors in classification and diagnosis in various types of epidemiological studies.”Amer. J. of Public Health52: 1137-1144.
12.
ERIKSON, R.
(1983) “Changes in social mobility in industrial nations: the case of Sweden,” pp. 165-195 in D. Treiman and R. Robinson (eds.) Research in Social Stratification and Mobility. Greenwich, CT:JAI.
13.
ERIKSON, R.
(1979) “On measuring errors in comparative surveys.”Statistisk Tidskrift2: 107-110.
14.
FLEISS, J.L.
(1981) Statistical Methods for Rates and Proportions. New York: John Wiley.
15.
GOLDBERG, J.D.
(1975) “The effects of misclassification on the bias in the difference between two proportions and the relative odds in the fourfold table.”J. of the Amer. Stat. Assn.70: 561-567.
16.
GOLDBERG, J.D.(1972) “The effects of misclassification on the analysis of datas in 2×2 tables.” Sc.D. dissertation, Boston, Harvard University School of Public Health.
17.
GOODMAN, L. A.
(1974) “The analysis of systems of qualitative variables when some of the variables are unobservable. Part I-A: modified latent structure analysis.”Amer. J. of Sociology79: 1179-1259.
18.
GULLEN, W.H.
, J.E. BEARMAN, and E. A. JOHNSON (1968) “Effects of misclassification in epidemiologic studies.”Public Health Reports83: 914-918.
19.
HANSEN, M. H.
, W. N. HURWITZ, and M. A. BERSHAD (1961) “Measurement errors in censuses and surveys.”Bulletin de l'Institute International de Statistiques38: 359-374.
20.
HEISE, D.R.
(1969) “Separating reliability and stability in test-retest correlations.”Amer. Soc. Rev.34: 93-101.
21.
HEISE, D.R.
and G. W. BOHRNSTEDT (1970) “Validity, invalidity, and reliability,” pp. 104-129 in E. F. Borgatta and G. W. Bohrnstedt (eds.) Sociological Methodology 1970. San Francisco: Jossey-Bass.
22.
HOCHBERG, Y.
(1977) “On the use of double sampling schemes in analyzing categorical data with misclassification errors.”J. of the Amer. Stat. Assn.72: 914-921.
23.
HOEM, J.M.
(1984) “Weighting, misclassification, and other issues in the analysis of survey samples of life histories,” in J. J. Heckman and B. Singer (eds.) Longitudinal Studies of the Labor Market. New York: Academic.
24.
HOPE, K.
, J.E. SCHWARTZ, and S. GRAHAM (1982) “Uncovering the pattern of social stratification: a two-year test-retest inquiry.”Stockholm: Swedish Institute for Social Research.
25.
JÖRESKOG, K. G.
and B. SÖRBOM (1981) LISREL V: Analysis of Linear Structural Relationships by Maximum Likelihood and Least Squares Methods. Chicago: National Educational Resources.
26.
KEYS, A.
and J. K. KIHLBERG (1963) “Effect of misclassification on estimated relative prevalence of a characteristic: Parts I and II.”Amer. J. of Public Health53: 1656-1665.
27.
KMENTA, J.
(1971) Elements of Econometrics. New York: Macmillan.
28.
KOCH, G.G.
(1969) “The effect of non-sampling errors on measures of association in 2×2 contingency tables.”J. of the Amer. Stat. Assn.64: 852-863.
29.
KRISTIANSSON, K. E.
(1982) “Resultat fr˙an n˙agra studier av kvaliteten i flödesdata fr˙an AKU (An investigation of the quality of labor mobility data in the Swedish Labor Force Survey).”Stockholm: Swedish Statistical Central Bureau.
30.
KRISTIANSSON, K. E.
(1979) “Försök till bedömning av tillfōrlitligheten i flödesdata ur AKU (Attempt to judge the reliability of labor mobility data in the Swedish Labor Force Survey).”Stockholm: Swedish Statistical Central Bureau.
31.
LAZARSFELD, P.F.
and N. W. HENRY (1968) Latent Structure Analysis. Boston: Houghton Mifflin.
32.
LINDSTRÖM, H.
, M. THORSLUND, and B. WÄRNERYD (1981) “Förändringsskatttninger i undersökningarna av levandsförh˙allanden (ULF): en redovisning av metodstudier och rekommanderad metodik (Estimating trends from the level of living survey: a report of methodological investigations with recommendations).” Appendix 5 to Levnadsförh˙allanden. Stockholm: Swedish Statistical Central Bureau.
33.
LORD, F. M.
and M. R. NOVICK (1968) Statistical Theories of Mental Test Scores. Reading, MA: Addison-Wesley.
34.
MADDALA, G.S.
(1977) Econometrics. New York: McGraw-Hill.
35.
NORLÉN, U.
(1977) “Response errors in the answers to retrospective questions.”Statistisk Tidskrift1: 331-341.
36.
NUNNALLY, J.C.
(1967) Psychometric Theory. New York: McGraw-Hill.
37.
SCHUESSLER, K.
(1971) Analyzing Social Data: A Statistical Orientation. Boston: Houghton Mifflin.
38.
SCHWARTZ, J.E.
(1983) “Estimating misclassification error rates for retrospective social survey data.”Stockholm: Swedish Institute for Social Research.
39.
SELÉN, J.
(1979) “Some effects of misclassification in the analysis of partly and purely categorical data,” pp. 71-96 in J. Selén, Tre bidrag till välfärdsmätningarnas metodik. Stockholm: Institutet för social forskning.
40.
STALLONES, R. A.
(1980) “The rise and fall of ischemic heart disease.”Scientific Amer.243, 5: 43-49.
41.
SUTCLIFFE, J. P.
(1965a) “A probability model for errors of classification. I. general considerations.”Psychometrika30: 73-96.
42.
SUTCLIFFE, J. P.
(1965b) “A probability model for errors of classification. II. particular cases.”Psychometrika30: 129-155.
43.
TENENBEIN, A.
(1972) “A double sampling scheme for estimating from misclassified multinomial data with applications to sampling inspection.”Technometrics14: 187-202.
44.
WILLLIAMS, W.H.
and C.L. MALLOWS (1970) “Systematic biases in panel surveys due to differential nonresponse.”J. of the Amer. Stat. Assn.65: 1338-1349.
45.
ZELLER, R. A.
and E. G. CARMINES (1980) Measurement in the Social Sciences. Cambridge: Cambridge Univ. Press.
