Estimating the Effect of Nonignorable Nonresponse in Sample Surveys

Abstract

Rubin (1977) developed a method for estimating, in a subjective sense, the effect of nonignorable nonresponse in sample surveys. Based on Bayesian techniques, this method produces a subjective probability interval for the statistic, such as the mean of a response variable, that would have been calculated if all nonrespondents had responded. Demographic and socioeconomic background information that is recorded for both respondents and nonrespondents plays an important role in sharpening the subjective interval - through the adjustment of a regression equation that uses this information. In this article, Rubin's method-sometimes called the mixture modeling approach to drawing inferences from self-selected samples-is reviewed and applied to real survey and experimental data on community standards for sexually explicit material in which respondents were asked to judge the material's appeal to prurient interest and patent offensiveness (two of the three legal criteria for a determination of obscenity). A critically important substantive issue in this context is whether or not the sample self-selection processes governing the willingness of individuals to participate in the experiment have so truncated the frequency distributions of participant judgments about obscenity that they are grossly biased and inaccurate. It is shown how the mixture modeling approach sheds light on the possible extent of such biases.

Get full access to this article

View all access options for this article.

References

Berk, Richard A. 1983. “An Introduction to Sample Selection Bias in Sociological Data.” American Sociological Review 48:386-398.

Berk, Richard A. and Subhash C. Ray . 1982. “Selection Biases in Sociological Data.” Social Science Research 11:301-340.

Donnerstein, Edward and Daniel Linz . 1990. Final Report of the Federal Western District of Tennessee Film Evaluation Project. Santa Barbara: University of California, Santa Barbara, Department of Communication.

Glynn, Robert J. , Nan M. Laird , and Donald B. Rubin . 1986. “Selection Modeling Versus Mixture Modeling With Nonignorable Nonresponse.” Pp. 115-142 in Drawing Inferences From Self-Selected Samples, edited by Howard Wainer . New York: Springer-Verlag.

Goodman, Leo A. 1975. “The Relationship Between Modified and Usual Multiple-Regression Approaches to the Analysis of Dichotomous Variables.” Sociological Methodology 7:83-110.

Groves, Robert M. 1989. Survey Errors and Survey Costs. New York: Wiley.

Heckman, James J. 1976. “The Common Structure of Statistical Models of Truncation, Sample Selection and Limited Dependent Variables and a Simple Estimator for Such Models.” Annals of Economic and Social Measurement 5:475-492.

Heckman, James J. 1979. “Sample Selection Bias as a Specification Error.” Econometrica 45:153-161.

Lillard, Lee , James P. Smith , and Finis Welch . 1986. “What Do We Really Know About Wages? The Importance of Nonreporting and Census Imputation.” Journal of Political Economy 94:489-506.

10.

Linz, Daniel , Edward Donnerstein , Kenneth C. Land , Patricia L. McCall , Joseph Scott , Bradley J. Shafer , Lee J. Klein , and Lawrence Lance . 1991. “Estimating Community Standards: The Use of Social Science Evidence in an Obscenity Prosecution.” Public Opinion Quarterly 55:80-112.

11.

Little, Roderick J. A. and Donald B. Rubin . 1987. Statistical Analysis With Missing Data. New York: Wiley.

12.

Miller v. California. 1973. 413 U.S. 15, 93 S. Ct. 2607, 37 L.Ed.2d 419.

13.

Press, S. James . 1989. Bayesian Statistics: Principles, Models, and Applications. New York: Wiley.

14.

Rubin, Donald B. 1976. “Inference and Missing Data.” Biometrika 63:581-592.

15.

Rubin, Donald B. 1977. “Formalizing Subjective Notions About the Effect of Nonrespondents in Sample Surveys.” Journal of the American Statistical Association 72:538-543.

16.

Rubin, Donald B. 1987. Multiple Imputation for Nonresponse in Surveys. New York: Wiley.

17.

Stinchcombe, Arthur L. , Calvin Jones , and Paul Sheatsley . 1981. “Nonresponse Bias for Attitude Questions.” Public Opinion Quarterly 45:359-375.

18.

Stolzenberg, Ross M. and Daniel A. Relles . 1990. “Theory Testing in a World of Constrained Research Design: The Significance of Heckman's Censored Sampling Bias Correction for Nonexperimental Research.” Sociological Methods & Research 18:395-415.

19.

Wainer, Howard . 1986. “Introduction and Overview.” Pp. 1-6 in Drawing Inferences From Self-Selected Samples, edited by Howard Wainer . New York: Springer-Verlag.