Behaviorally anchored rating scales (BARS) have traditionally required a great amount of time and effort to develop. Two variations of an alternative method to construct BARS are suggested. The traditional and the two simplified methods were used to develop scales for the evaluation of teacher performance. The scales were used by 231 undergraduate students to evaluate seven psychology instructors. The scales were compared across five content dimensions in terms of reliabilities, means, intercorrelations, and expert judgments. The three procedures were found to produce similar scales using these criteria. On the basis of this study, the shortcut derivation method is recommended as preferable for applied use because it requires considerably less construction time. Since this represents the first attempt to use these shortcut methods, additional experimental and field research is necessary to confirm this recommendation.
Get full access to this article
View all access options for this article.
References
1.
Arvey, R.D. and Hoyle, J.C.A Guttman approach to the development of behaviorally based rating scales for systems analysts and programmer/analysts. Journal of Applied Psychology, 1974, 59, 61-68.
2.
Bernardin, H.J.Behavioral expectation scales versus summated scales: A fairer comparison . Journal of Applied Psychology, 1977, 62, 442-447.
3.
Bernardin, H.J., Alvares, K.M., and Cranny, C.J.A recomparison of behavioral expectation scales to summated scales . Journal of Applied Psychology, 1976, 61, 564-570.
4.
Bernardin, H.J., LaShells, M.B., Smith, P.C., and Alvares, K.M.Behavioral expectation scales: Effects of developmental procedures and formats . Journal of Applied Psychology, 1976, 61, 75-79.
5.
Borman, W.C. and Dunnette, M.D.Behavior-based versus trait-oriented performance ratings: An empirical study . Journal of Applied Psychology, 1975 , 60, 561-565.
6.
Borman, W.C. and Vallon, W.R.A view of what can happen when behavioral expectation scales are developed in one setting and used in another. Journal of Applied Psychology , 1974, 59, 197-201.
7.
Burnaska, R.F. and Hollmann, T.D.An empirical comparison of the relative effects of rater response biases on three rating scale formats. Journal of Applied Psychology , 1974, 59, 307-312.
8.
Campbell, J.P., Dunnette, M.D., Arvey, R.D., and Hellervik, L.V.The development and evaluation of behaviorally based rating scales . Journal of Applied Psychology, 1973, 57, 15-22.
9.
Campbell, J.P., Dunnette, M.D., Lawler, E.E., III, and Weick, K.E.Managerial behavior, performance, and effectiveness. New York: McGraw-Hill, 1970.
10.
Cascio, W.F. and Valenzi, E.R.Behaviorally anchored rating scales: Effects of education and job experience of raters and ratees. Journal of Applied Psychology , 1977, 62, 278-282.
11.
Cascio, W.F. and Valenzi, E.R.Relations among criteria of police performance. Journal of Applied Psychology, 1978, 63, 22-28.
12.
DeCotiis, T.A.A Critique and suggested revision of behaviorally anchored rating scales developmental procedures. EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT, 1978, 38, 681-690.
13.
Dickinson, T.L. and Tice, T.E.A multitrait-multimethod analysis of scales developed by retranslation . Organizational Behavior and Human Performance, 1973, 9, 421-438.
14.
Dickinson, T.L. and Tice, T.E.The discriminant validity of scales developed by retranslation. Personnel Psychology, 1977, 30, 217-228.
15.
Dunnette, M.D.Personnel selection and placement. Belmont, Calif. : Brooks/Cole, 1966.
16.
Fogli, L., Hulin, C.L., and Blood, M.R.Development of first-level behavioral job criteria. Journal of Applied Pscyhology, 1971, 55, 3-8.
17.
Friedman, B.A. and Cornelius, E.T., III.Effect of rater participation in scale construction on the psychometric characteristics of the two rating scale formats. Journal of Applied Psychology, 1976, 61, 210-216.
18.
Goodale, J.B. and Burke, R.J.Behaviorally based rating scales need not be job specific. Journal of Applied Psychology, 1975, 60, 389-391.
19.
Harari, O. and Zedeck, S.Development of behaviorally anchored scales for the evaluation of faculty teaching. Journal of Applied Psychology, 1973, 58, 261-265.
20.
Keaveny, T.J. and McGann, A.F.A comparison of behavioral expectation scales and graphic rating scales . Journal of Applied Psychology, 1975, 60, 695-703.
21.
Landy, F.J., Farr, J.L., Saal, F.E., and Freytag, W.R.Behaviorally anchored scales for rating the performance of police officers . Journal of Applied Psychology, 1976, 61, 750-758.
22.
Landy, F.J. and Guion, R.M.Development of scales for the measurement of work motivation. Organizational Behavior and Human Performance, 1970 , 5, 93-103.
23.
Sauser, W.I., Jr.A comparative evaluation of the effects of rater participation and rater training on characteristics of employee performance appraisal ratings and related mediating variables (Doctoral dissertation, Georgia Institute of Technology, 1978). Dissertation Abstracts International, 1979, 39, 5116-B. (University Microfilms No. 7906831).
24.
Sauser, W.I., Jr., Evans, K.L., and Champion, L.C.H.Two-hundred and fifty scaled incidents of college classroom teaching behavior. Resources in Education , 1980, 15 (1), 30.
25.
Schwab, D.P., Heneman, H.G., and DeCotiis, T.A.Behaviorally anchored rating scales: A review of the literature. Personnel Psychology, 1975, 28, 549-562.
26.
Shrout, P.E. and Fleiss, J.L.Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 1979, 86, 420-428.
27.
Smith, P.C. and Kendall, L.M.Retranslation of expectations: An approach to the construction of unambiguous anchors for rating scales. Journal of Applied Psychology , 1963, 47, 149-155.
28.
Vance, R.J., Kuhnert, K.W., and Farr, J.L.Interview judgments: Using external criteria to compare behavioral and graphic scale ratings. Organizational Behavior and Human Performance , 1978, 22, 279-294.
29.
Williams, W.E. and Seiler, D.A.Relationship between measures of effort and job performance. Journal of Applied Psychology, 1973, 57, 49-54.
30.
Zedeck, S. and Baker, H.T.Nursing performance as measured by behavioral expectation scales: A multitrait-multirater analysis. Organizational Behavior and Human Performance, 1972, 7, 457-466.
31.
Zedeck, S., Imparato, N., Krausz, M., and Oleno, T.Development of behaviorally anchored rating scales as a function of organizational level. Journal of Applied Psychology, 1974 , 59, 249-252.
