The accuracy of eight missing data techniques (MDTs) under conditions of systematically missing data was tested using a Monte Carlo analysis. Data were generated from a population correlation matrix, then deleted using several patterns that might be found in a human resource management (HRM) selection validation study. The results indicated that listwise and pairwise deletion were the most accurate methods, followed closely by imputation methods such as regression and hot-deck. Mean substitution was substantially inferior to the other methods tested. Future research that examines different missing data patterns is recommended.
Get full access to this article
View all access options for this article.
References
1.
Beale, E. M. L.
, & Little, R. J. A.1975. Missing values in multivariate analysis. Journal of the Royal Statistical Society, Series B, 37 (1): 129-145.
2.
Buck, S. F.1960. A method of estimation of missing values in multivariate data suitable for use with electronic computers. Journal of the Royal Statistical Society, B (22): 302-306.
3.
Chan, L. S.
, & Dunn, O. J.1972. The treatment of missing values in discriminant analysis: The sampling experiment. Journal of the American Statistical Association, 67 (338): 473-477.
4.
Cohen, J.
, & Cohen, P.1983. Applied multiple regressionñcorrelation analysis for the behavioral science. Hilldsdale, NJ: Lawrence Erlbaum.
5.
Cortina, J. M.1993. Interaction, nonlinearity, and multicollinearity: Implications for multiple regression. Journal of Management, 19 (4): 915-922.
6.
Gleason, T. C.
, & Staelin, R.1975. A proposal for handling missing data. Psychometrika, 40 (2): 229-252.
7.
Graham, J. W.
, & Donaldson, S. W.1993. Evaluating interventions with differential attrition: The importance of nonresponse mechanisms and use of follow-up data. Journal of Applied Psychology, 78 (1): 119-128.
8.
Hunter, J. E.
1994. Commentary on Roth, Switzer, Campion, and Jones. In F. L. Schmidt (chair), Advances in construct and criterion related validity research. Symposium presented at the Ninth Annual Conference for Industrial and Organizational Psychology, Nashville, TN.
9.
Kim, J. O.
, & Curry, J.1977. The treatment of missing data in multivariate analysis. Sociological Methods & Research, 6 (2): 215-241.
10.
Little, R. J. A.
, & Ruhin, D. B.1987. Statistical analysis with missing data. New York: Wiley.
11.
Malhorta, N. K.1987. Analyzing marketing research data with incomplete information on the dependent variable. Journal of Market Research, 24: 74-84.
12.
McClelland, G. H.
, & Judd, C. M.1993. Statistical difficulties of detecting interactions and moderator effects. Psychological Bulletin, 114 (2): 376-390.
13.
Raymond, M. R.1986. Missing data in evaluation research. Evaluation & the Health Professions, 9 (4): 395-420.
14.
Raymond, M. R.
, & Roberts, D. M.1987. A comparison of methods for treating incomplete data in selection research. Education and Psychological Measurement, 47:13-26.
15.
Rizvi, M. H.1983. Hot-deck procedures: Introduction. In W. G. Madow & I. Olkin. (Eds.), Incomplete data in sample surveys. Volume 111: Proceedings of the symposium. New York: Academic Press.
16.
Roth, P. L.1994. Missing data: A conceptual review for applied psychologists. Personnel Psychology, 47: 537-560.
17.
Roth, P. L.
, Campion, J. E., & Jones, S. D.1996. The impact of four missing data techniques on validity estimates in Human Resource Management. Journal of Business & Psychology, 11 (1): 101-112.
18.
Roth, P. L.
, & Switzer, F. S. III. 1995. A Monte Carlo analysis of missing data techniques in an HRM setting. Journal of Management, 21: 1003-1023.
19.
Wilkinson, L.1990. Systat: Statistics. Evanston, IL: Systat, Inc.
20.
Young, W. Y.
, Houston, J. S., Harris, J. H., Hoffman, R. G., & Wise, L. L.1990. Large-scale predictor validation in Project A: Data collection procedures and data base preparation. Personnel Psychology, 43 (2): 301-311.
