Effectiveness of Different Missing Data Treatments in Surveys with Likert-Type Data: Introducing the Relative Mean Substitution Approach

Abstract

This article introduces a new approach to the substitution of missing values in surveys with Likert-type scales: relative mean substitution. The effectiveness of this method is demonstrated in comparison with three other commonly used methods for dealing with missing values, making use of actual field data. The emphasis is on two aspects of global effectiveness: (a) the accuracy in estimating various parameters at the same time and (b) the accuracy in estimating, for Likert-type scales with different psychometric characteristics, these various parameters under different conditions, such as different numbers of respondents (1,674; 400; and 100) and different distributions of missing values (two random and three nonrandom situations). The results indicated that this new relative mean substitution approach globally produced the most accurate estimates, mainly because of the more accurate estimation of the variances and the sensitivity to items with deviating means, provided that the Likert-type scales are sufficiently homogeneous.

Get full access to this article

View all access options for this article.

References

Anderson, A. B. , Basilevsky, A. , & Hum, D.P.J. (1983). Missing data: A review of the literature. In P. H. Rossi , J. D. Wright , & A. B. Anderson (Eds.), Handbook of survey research (pp. 415-494). London: Academic Press.

Buck, S. F. (1960). A method of estimation of missing values in multivariate data suitable for use with an electronic computer. Journal of the Royal Statistical Society, B22, 302-307.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.

Ferber, R. (1966). Item nonresponse in a consumer survey. Public Opinion Quaterly, 30, 399-415.

Ford, B. L. (1976). Missing data procedures: A comparative study. Washington, DC: Statistical Reporting Service, U. S. Department of Agriculture.

Frane, J. W. (1976). Some simple procedures for handling missing data in multivariate analysis. Psychometrika, 41, 409-415.

Judge, G. G. , Hill, R. C. , Griffiths, W. E. , Lütkepohl, H. , & Lee, T. (1988). Introduction to the theory and practice of econometry (2nd ed.). New York: John Wiley.

Kromrey, J. D. , & Hines, C. V. (1994). Nonrandomly missing data in multiple regression: An empirical comparison of common missing-data treatments. Educational and Psychological Measurement, 54, 573-593.

Little, R.J.A. (1988). Missing-data adjustments in large surveys. Journal of Business & Economic Statistics, 6, 287-296.

10.

Little, R.J.A. , & Rubin, D. B. (1987). Statistical analysis with missing data. New York: John Wiley.

11.

Little, R.J.A. , & Su, H. L. (1989). Item nonresponse in panel surveys. In D. Kasprzyk , G. Duncan , & M. P. Singh (Eds.), Panel surveys (pp. 400-425). New York: John Wiley.

12.

Madow, W. G. , Nisselson, H. , Olkin, I. , & Rubin, D. B. (1983). Incomplete data in sample surveys (Vols. 1-3). New York: Academic Press.

13.

Meeus, W. , & ‘t Hart, H. (1993). Jongeren in Nederland [Youth in the Netherlands]. Amersfoort, the Netherlands: Acco.

14.

Nie, N. H. , Hull, C. H. , Jenkins, J. G. , Steinbrenner, K. , & Bent, D. H. (1975). Statistical package for the social sciences (2nd ed.). New York: McGraw-Hill.

15.

Pigott, Th. D. (1994). Methods for handling missing data in research synthesis. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 163-175). New York: Russell Sage.

16.

Pregibon, D. (1975). Typical survey data: Estimation and imputation. University of Waterloo: Statistics Canada.

17.

Raaijmakers, Q.A.W. (1997). Midpoint responses on Likert-type items: Neutral or missing values? Manuscript in preparation, University of Utrecht, the Netherlands.

18.

Raymond, M. R. (1986). Missing data in evaluation research. Evaluation & The Health Professions, 9, 395-420.

19.

Raymond, M. R. , & Roberts, D. M. (1987). A comparison of methods for treating incomplete data in selection research. Educational and Psychological Measurement, 49, 13-26.

20.

Roth, P. L. , Campion, J. E. , & Jones, S. D. (1996). The impact of four missing data techniques on validity estimates in human resource management. Journal of Business and Psychology, 11, 101-112.