Sage Journals: Discover world-class research

Abstract

Participants used either a behaviorally anchored rating scale format or a mixed standard scale format in evaluating seven dimensions of interviewer performance as exhibited in videotapes of eight different managers. Based on the correlation between observed ratings and a set of known true or target scores, overall accuracy scores were calculatedfor each subject. In Study 1, subjects (undergraduate students) who used behaviorally anchored rating scales were significantly more accurate than subjects who used mixed standard scales in overall rating accuracy. In Study 2, subjects (MBA students with a minimum of 2 years supervisory experience) who used behaviorally anchored rating scales were significantly more accurate than subjects who used mixed standard scales in overall rating accuracy. This suggests that use of the behaviorally anchored rating scaleformat results in more accurate ratings of performance than does use of the mixed standard scale format. Results are discussed in terms of implicationsfor methods used in rating the behavior of individuals, especially in performance appraisal of employees.

Get full access to this article

View all access options for this article.

References

Banks, C.G. , & Murphy, K.R. (1985). Toward narrowing the research-practice gap in performance appraisal. Personnel Psychology, 38, 335-345.

Barnes-Farrell, J.L. , & Weiss, H.M. (1984). Effects of standard extremity on mixed standard scale performance ratings. Personnel Psychology, 37, 301-316.

Benson, PG. , & Buckley, M.R. (1987). Course grades and practical skills of personnel/industrial relations students: Reappraising management training. Journal of Education for Business, 62, 153-159.

Benson, PG. , Buckley, M.R. , & Dickinson, TL . (1984, March). The mixed standard scale: Implications of scoring methods. Paper presented at the Southeastern Psychological Association, New Orleans.

Benson, PG. , & Dickinson, T.L. (1983). Mixed standard scale response inconsistencies as reliability indices. Educational and Psychological Measurement, 43, 781-789.

Bernardin, H.J. , & Beatty, R.W (1984). Performance appraisal: Assessing human behavior at work. Boston, MA: Kent Publishing Co.

Bernardin, H.J. , & Buckley, M.R. (1981). Strategies in rater training. Academy of Management Review, 6, 205-212.

Bernardin, H.J. , Carlyle, J. , & Elliot, L. (1980). A critical assessment of mixed standard rating scales. Academy of Management Proceedings, 40, 308-312.

Bernardin, H.J. , & Pence, E.C. (1980). Rater training: Creating new response sets and decreasing accuracy. Journal of Applied Psychology, 65, 60-66.

10.

Blanz, F , & Ghiselli, E.E. (1972). The mixed standard scale: A new rating system. Personnel Psychology, 25, 185-199.

11.

Borman, WC . (1979a). Format and training effects on rating accuracy and rater errors. Journal of Applied Psychology, 64, 410-421.

12.

Borman, WC . (1979b). Individual differences correlates of accuracy in evaluating others' performance effectiveness. Applied Psychological Measurement, 3, 103-115.

13.

Borman, WC. , Hough, L.M. , & Dunnette, M.D. (1976). Performance ratings: An investigation of reliability, accuracy, and relationships between individual differences and rater error. Final report to the Army Research Institute for the Behavioral and Social Sciences, Alexandria, VA.

14.

Charter, R.A. , & Larsen, B.S. (1983). Fisher's z to r. Educational and Psychological Measurement, 43, 41-42.

15.

Cascio, WE , & Bernardin, H.J. (1981). Implications of performance appraisal litigation for personnel decisions. Personnel Psychology, 34, 211-226.

16.

Cronbach, L.J. (1955). Processes affecting scores on "understanding of others" and "assuming similarity." Psychological Bulletin, 52, 177-193.

17.

Dickinson, T.L. , & Zellinger, P.M. (1980). A comparison of the behaviorally anchored rating and mixed standard scale formats. Journal of Applied Psychology, 65, 147-154.

18.

Feild, H.S. , & Holley, WH . (1982). The relationship of performance appraisal system characteristics to verdicts in selected employment discrimination cases. Academy of Management Journal, 25, 392-406.

19.

Funder, D.C. (1987). Errors and mistakes: Evaluating the accuracy of social judgment. Psychological Bulletin, 101, 75-90.

20.

Landy, FJ. , & Barnes, J.L. (1979). Scaling behavioral anchors. Applied Psychological Measurement, 3, 193-200.

21.

Landy, FJ. , & Farr, J.L. (1980). Performance rating. Psychological Bulletin, 87, 72-107.

22.

Landy, FJ. , & Farr, J.L. (1983). The measurement of work performance: Methods, theory, and applications. New York: Academic Press.

23.

Murphy, K.R. , Garcia, M. , Kerkar, S. , Martin, C. , & Balzer, W (1982). Relationship between observational accuracy and accuracy in evaluating performance. Journal of Applied Psychology, 67, 320-325.

24.

Prien, E. R , Jones, M.A. , & Miller, L.M. (1977). A job related performance rating system. Personnel Administrator, 22 (9), 37-42.

25.

Saal, FE . (1979). Mixed standard rating scale: A consistent system for numerically coding inconsistent response combinations. Journal of Applied Psychology, 64, 422-428.

26.

Saal, FE. , & Landy, FJ . (1977). The mixed standard rating scale: An evaluation. Organizational Behavior and Human Performance, 18, 19-35.

27.

Waters, L.K. , Edwards, J.E. , & Reardon, M. (1983). Effects of rating inconsistency on mixed standard scales. Psychological Reports, 52, 135-138.

The Impact of Rating Scale Format on Rater Accuracy: An Evaluation of the Mixed Standard Scale

Abstract

Get full access to this article

References