A model of the latent structure of performance ratings based on Wherry's (19.52) theory of rating was developed which sought to explain relationships between multiple ratings, each of multiple performance dimensions, obtained from multiple rating sources. The model was tested using data obtained from first-and second-level supervisors who rated the performance of approximately 700 entry level clerical and professional/technical employees of 96 insurance, financial, and service organizations. Results only partially confirmed the a priori model, supporting specifications of dimensional true performance and overall rater bias effects, but failing to support specifications of areal (dimensional) bias effects on ratings.
Get full access to this article
View all access options for this article.
References
1.
Ackerman, P. L. & Humphreys. L. G. (1990). Individual differences theory in industrial and organizational psychology. Pp. 223-282 in M. D. Dunnette & L. M. Hough (eds.), Handbook of industrial and organizational psychology, Vol. 1. 2nd ed.Palo Alto, CA: Consulting Psychologists.
2.
Bartlett, C. J. (1983). What’s the difference between valid and invalid halo? Forced-choice measurement without forcing a choice. Journal of Applied Psychology, 68: 218-226.
3.
Bellows, R. M. (1941). Procedures for evaluating vocational criteria. Journal of Applied Psychology, 25: 499-513.
4.
Bentler. P. M. & Mooijaart. A. (1989). Choice of structural model via parsimony: A rationale based on precision. Psychological Bulletin, 106: 315-317.
5.
Binning, J. F. & Barrett, G. V. (1989). Validity of personnel decisions: A conceptual analysis of the inferential and evidential bases. Journal of Applied Psychology, 74: 478-494.
6.
Borman. W. C. (1978). Exploring upper limits of reliability and validity in job performance ratings. Journal of’ Applied Psycho&v, 63: 135-144.
7.
Borman. W. C. (1991). Job behavior, performance, and effectiveness. Pp. 271-326 in M. D. Dunnette & L. M. Hough (eds.), Handhook of industrial and organizational psychology. Vol. 2, 2nd ed.Palo Alto, CA: Consulting Psychologists.
8.
Bray, D. W. & Grant, D. L. (1966). The assessment center in the measurement of potential for business management. Psychological Monographs: General and Applied, 80(17, Whole No. 625).
9.
Campbell. J. P., McHenry, J. J. & Wise, L. L. (1990). Modeling job performance in a population of jobs. Personnel Psychology, 43: 313-333.
10.
Cudeck. R. (1989). Analysis of correlation matrices using covariance structure models. Psychological Bulletin, 105: 317-327.
11.
Cudeck. R. & Henley, S. J. (1991). Model selection in covariance structures analysis and the “problem” of samp!e size: A clarification. Psychological Bulletin, 109: 520-523.
12.
Dcadrick, D. L. & Madigan. R. M. (1991). The validity of performance measures: A conceptual analysis. Paper presented at the annual meeting of the Academy of Management, Miami, FL.
13.
DeCotiir. T. & Petit, A. (1978). The performance appraisal process: A model and some testable propositions. Academy of Managemmt Review, 3: 635-646.
14.
DeNisi, A. S.Cafferty, T. P. & Meglino. B. M. (1984). A cognitive view of the performance appraisal process: A model and research propositions. Organizational Behavior and Human Performance, 33: 360-396.
15.
Duncan, O. D. (1975). Introduction of structural equation models. New York: Academic.
16.
Fisicaro. S. A. & Lance. C. E. (1990). Implications of three causal models for the measurement of halo error. Applied Psychological Measurement, 14: 419-429.
17.
Guilford. J. P. (1954). Psychometric methods. 2nd ed.New York: McGraw-Hill.
18.
Harris, M. M. & Schaubroeck, J. (1988). A meta-analysis of self-supervisor, self-peer. and peer-supervisor ratings. Personnel Psychology, 41: 43-62.
19.
Heneman, R. L. (1986). The relationship between supervisory ratings and results-oriented measures 01 performance: A meta-analysis. Personnel Psychology, 39: 811-826.
20.
Hosper-s, J. (1967). An introduction to philosophical analysis. 2nd ed.Englewood Cliffs, NJ: Prentice-Hall.
21.
Ilgen. D. R. & Feldman, J. M. (1983). Performance appraisal: A process focus. Pp. 141-197 in B.M. Staw & L.I.. Cummings (cds.). Research in organizational behavior, Vol. 5. Greenwich, CT: JAI Press.
22.
James. I.. R. (1973). Criterion models and construct validity for criteria. Psychological Bulletin, 80: 75-83.
23.
Joreskog. K. G. & S&born, D. (1986). LISREL.- VI user's guide. Mooresville, IN: Scientific Software.
24.
Kegan. R. (1982). The evolving self: Problem and process in human development. Cambridge, MA: Harvard University Press.
25.
Kenny, D. A. & Berman. .I. S. (1980). Statistical approaches to the correction of correlational bias. Psychological Bulletin, 88: 288-295.
26.
Lance, C. E.LaPomte, J. A. & Fisicaro. S. A. (1994) Tests of three causal models of halo rater error. Organizational Behavior and Human Decision Processes, 57: 83-96.
27.
Lancc, C. F.Teachout. M. S. & Donnelly. T. M. (1992). Specification of the criterion construct space: An application of hierarchical confirmatory factor analysis. Journal Applied Psychology, 77: 437-452.
28.
Lance, C. E.Woehr, D. J. & Fisicaro. S. A. (1991). Cognitive categorization processes in performance evaluation: Confirmatory tests 01 two models. Journal of Organizational Behavior, 12: 1-20.
29.
Landy. F. .I. & Farr. J. L. (1980). Performance rating. Psychological Bulletin, 87: 72-107.
30.
Landy. F. .I. & Farr. J. L. (1983). The measurement of work performance. New York: Academic.
31.
Li, P-C. (1937). Statistical analysis of personality rating. Journal of Experimental Education, 5: 229-245.
32.
Marsh, H. W.Balla. J. R. & McDonald. R. P. (1989). Goodness-of-fit indexes in confirmatory factor analysis: The effect of sample size. Psychological Bulletin, 103: 391-410.
33.
Mulaik, S. A., James, L. R., VanAlstine, J., Bennett, N., Lind, S. & Stilwell, C. D. (1989). Evaluation of goodness-of-fit indices for structural equation models. Psychological Bulletin, 105: 430-445.
34.
Ronan, W. W. & Schwartz, A. P. (1974). Ratings as performance criteria. International Review of Applied Psychology, 23: 71-81.
35.
Schneier, C. E. (1977). Multiple rater groups and performance appraisal. Public Personnel Management, 6: 13-20.
36.
Sharon. A. T. & Bartlett, C. J. (1969). Effect of instructional conditions in producing leniency on two types of rating scales. Personnel Psychology, 22: 251-263.
37.
Sobel, M. E. & Bohrnstedt, G. W. (1985). Use of null models in evaluating the fit of covariance structure models. In N.B. Tuma (ed.), Sociological methodology 1985. San Francisco: Jossey-Bass.
38.
Thorndike, R. L. (1986). The role of general ability in prediction, Journal of Vocational Behavior, 2%332-339.
39.
Vance, R. J., MacCallum, R. C., Coovert, M. D. & Hedge, J. W. (1988). Construct validity of multiple job performance measures using confirmatory factor analysis. Journal of Applied Psychology, 73: 74-80.
40.
Waldman, D. A. & Spangler, W. D. (1989). Putting together the pieces: A closer look at the determinants of job performance. Human Performance, 2: 29-59.
41.
Wheaton, B. (1987). Assessment of fit in overidentified models with latent variables, Sociological Methods & Research, 16: 118-154.
42.
Wherry, R. J. (1952). The control of bias in ratings: VIII. A theory of rating. PRB Report No. 922, Contract No. DA-49-083 0SA69, Department of the Army.
43.
Wherry, R. J. (1957). The past and future of criterion evaluation. Personnel Psychology, 10: 1-5.
44.
Wherry. R. J. Sr. & Bartlett, C. J. (1982). The control of bias in ratings: A theory of rating. Personnel Psychology, 35: 521-551.
45.
Widaman. K. F. (1985). Hierarchically nested covariance structure models for multitrait-multimethod data. Applied Psychological Measurement, 9: 1-26.
46.
Zedeck, S. & Cascio, W. F. (1984). Psychological issues in personnel decisions, Annual Review of Psychology, 35: 461-518.
