Theory Testing in a World of Constrained Research Design

Abstract

Because censored sampling is often unavoidable in much sociological data analysis, computationally simple corrections of censoring bias would be useful. Heckman's correction is simple to compute, widely used, and proven asymptotically correct under certain assumptions, but its limitations in practical situations are not well known in sociology. Here, we overview prior criticisms of Heckman's estimator, and we consider the case in which its normality assumptions are satisfied, censoring rates are high, and sample sizes are small. Results of 14,400 analyses of computer-generated simulation data suggest that Heckman's method performs well under certain circumstances, but that it very frequently worsens estimates, especially under conditions that are likely to be present in sociological data. Thus, the technique is probably not a general cure for censoring bias in sociology, except perhaps where strong theory permits certain strong assumptions. We reconsider censored sampling correction strategies in the context of statistical analysis as a theory-building tool, with emphasis on research strategy in the presence of irremediable censoring bias.

Get full access to this article

View all access options for this article.

References

ALLISON, P. D. and J. S. LONG (1987) “Interuniversity mobility of academic scientists.” Amer. Soc. Rev. 52: 643-652.

BERK, R. (1983) “An introduction to sample selection bias in sociological data.” Amer. Soc. Rev. 48: 386-398.

BERK R. A. , W. P. BRIDGES , and A. SHIH (1981) “Does IQ really matter — a study of the use of IQ scores for the tracking of the mentally retarded.” Amer. Soc. Rev. 46: 58-71.

CRONBACH, L. (1970) Essentials of Psychological Testing (3rd ed.). New York: Harper & Row.

DUAN, N. , W. MANNING , C. MORRIS , and J. NEWHOUSE (1984) “Choosing between the sample selection model and the multi-part model.” J. of Business and Econ. Statistics 2: 283-289.

ENGLAND, P. , B. S. KILBOURNE , G. FARKAS , and T. DOU (1988) “Explaining occupational sex segregation and wages — findings from a model with fixed effects.” Amer. Soc. Rev. 53: 544-558.

GOLDBERGER, A. (1980) “Abnormal selection bias.” Madison: University of Wisconsin Department of Economics. (unpublished)

GRONAU, R. (1974) “Wage comparisons — a selectivity bias.” J. of Pol. Economy 82: 1119-1144.

HAGAN, J. and P. PARKER (1985) “White-collar crime and punishment — the class structure and legal sanctioning of securities violations.” Amer. Soc. Rev. 50: 302-316.

10.

HAMMERSLEY, J. and D. HANDSCOMB (1964) Monte Carlo Methods. London: Methuen.

11.

HECKMAN, J. (1976) “The common structure of statistical models of truncation, sample selection and limited dependent variables and a simple estimator for such models.” Annals of Econ. and Social Measurement 5: 475-492.

12.

HECKMAN, J. (1979) “Sample selection bias as a specification error.” Econometrica 45: 153-161.

13.

HECKMAN, J. (1980) “Sample selection bias as a specification error,” pp. 206-248 in J. P. Smith (ed.), Female Labor Supply. Princeton, NJ: Princeton Univ. Press.

14.

JASSO, G. (1985) “Marital coital frequency and the passage of time — estimating the separate effects of spouses' ages and marital duration, birth and marriage cohorts, and period influences.” Amer. Soc. Rev. 50: 224-241.

15.

KENDALL, D. and W. BUCKLAND (1971) A Dictionary of Statistical Terms. New York: Hafner.

16.

KENNEDY, W. and J. GENTLE (1980) Statistical Computing. New York: Decker.

17.

LEWIS, H. G. (1974) “Comments on selectivity biases in wage comparisons.” J. of Pol. Economy 82: 1145-1155.

18.

LIEBERSON, S. (1985) Making it Count. Berkeley: Univ. of California Press.

19.

LILLARD, L. , J. P. SMITH , and F. WELCH (1986) “What do we really know about wages? the importance of nonreporting and census imputation.” J. of Pol. Economy 94: 489-506.

20.

LITTLE, R. (1983) “A note about models for selectivity bias.” Los Angeles, CA: UCLA School of Medicine Department of Biomathematics. (unpublished)

21.

LITTLE, R. and D. RUBIN (1987) Statistical Analysis with Missing Data. New York: Wiley.

22.

MARINI, M. M. (1984) “Women's educational attainment and the timing of entry into parenthood.” Amer. Soc. Rev. 49: 491-511.

23.

MINER, J. and J. PEARSON (1983) “An evaluation of business and economics doctoral student selection.” Collegiate News and Views (Fall): 15-19.

24.

NELSON, F. (1984) “Efficiency of the two-step estimator for models with endogenous sample selection.” J. of Econometrics 24: 181-196.

25.

PAARSCH, H. (1984) “A Monte Carlo comparison of estimators for censored regression models.” J. of Econometrics 24: 197-213.

26.

PILIAVIN, I. , R. GARTNER , C. THORNTON , and R. L. MATSUEDA (1986) “Crime, deterrence, and rational choice.” Amer. Soc. Rev. 51: 101-119.

27.

SANDERS, J. M. and V. NEE (1987) “Limits of ethnic solidarity in the enclave economy.” Amer. Soc. Rev. 52: 745-767.

28.

SNIPP, C. M. and G. D. SANDEFUR (1988) “Earnings of American Indians and Alaskan natives — the effects of residence and migration.” Social Forces 66: 994-1008.

29.

SORENSEN, A. (1976) “Estimating rates from retrospective questions,” pp. 209-223 in D. Heise (ed.), Sociological Methodology 1977. San Francisco: Jossey-Bass.

30.

TIENDA, M. and D. T. LII (1987) “Minority concentration and earnings inequality — blacks, Hispanics, and Asians compared.” Amer. Soc. Rev. 93: 141-165.

31.

TIENDA, M. , S. A. SMITH , and V. ORTIZ (1987) “Industrial restructuring, gender segregation, and sex differences in earnings.” Amer. Soc. Rev. 52: 195-210.

32.

TUMA, N. and M. HANNAN (1978) “Approaches to the censoring problem in analysis of event histories,” pp. 209-240 in K. Schuessler (ed.), Sociological Methodology 1979. San Francisco: Jossey-Bass.

33.

TUMA, N. and M. HANNAN (1984) Social Dynamics: Models and Methods. New York: Academic Press.

34.

WONNACOTT, R. J. and T. H. WONNACOTT (1970) Econometrics. New York: John Wiley.