The Reliability and Construct Validity of a Behaviorally Anchored Rating Scale Used To Measure Nursing Assistant Performance

Abstract

Data are presented that demonstrate the reliability and construct validity of a 27-item Behaviorally Anchored Rating Scale (BARS) used to rate nursing assistant performance. A sample of 757 nursing assistants from 44 randomly selected long-term care facilities were rated with the BARS instrument by a team of independent observers, and later by the nursing assistants'supervisors. Interrater reliability between the independent observers and supervisors ranged from 0.34 to 0.96 and averaged 0.78. The internal consistency reliability measured by Cronbach's alpha was 0.71 for the independent observers, 0.76 for the supervisors. Construct validity was assessed by exploratory factor analysis. The underlying factor structure had 3 factors that accounted for 84.0% of the variance. Results were interpreted as supporting the reliability and indicating construct validity of this BARS for nursing assistants. Because BARSs have been shown to reduce subjectivity and personal bias, they are recommended for evaluating nursing assistant performance.

Get full access to this article

View all access options for this article.

References

Bernardin, H. 1979. The predictability of discrepancy measure of role constructs . Personnel Psychology 32:139-53.

Bynner, J. 1988. Factor analysis and the construct mdicator relationship . Human Relations 41:389-405.

Campbell, J.P. , M.D. Dunnette , R.D. Arvey , and L.V. Hellervick . 1973. The development and evaluation of behaviorally based rating scales. Journal of Applied Psychology 57:15-22.

Hom, P.W. , A.S. DeNisi , A.J. Kinicki , and B.D. Bannister. 1982. Effectiveness of performance feedback from behaviorally anchored rating scales. Journal of Applied Psychology 67: 568-76.

Jacobs, R. , D. Kafry , and S. Zedeck. 1980. Expectations of behaviorally anchored rating scales. Personnel Psychology 33:595-640.

Kavanagh, M. , and J. Duffy. 1978. An extension and field test of the retranslation method for developing rating scales. Personnel Psychology 31:461-70.

Kelly, M. 1989. The Omnibus Budget Reconciliation Act of 1987: A policy analysis. Nursing Clinics of North America 24:791-94.

Kim, J.O. , and C.W. Mueller. 1978. Factor analysis: Statistical methods and practical issues. Beverly Hills, CA: Sage.

Kinicki, A.J. , and B.D. Bannister. 1988. A test of the measurement assumptions underlying behaviorally anchored rating scales. Educational and Psychological Measurement 48:17-27.

10.

Landy, F.J. , and R. Guion. 1970. Development of scales for the measurement of work motivation. Organizational Behavior and Human Performance 5:72-107.

11.

Landy, F.J. , J.L. Farr , F.E. Saal , and W.R. Fretag. 1976. Behaviorally anchored rating scales for rating the performance of police officers. Journal of Applied Psychology 61:750-68.

12.

Lemke, S. , and R.H. Moos. 1988. Quality of residential settings for elderly adults. Journal of Gerontology 67:268-76.

13.

Marradi, A. 1981. Factor analysis as an aid in the formation and refinement of empirically useful concepts. In Factor analysis and measurement in sociological research, edited by D. Lerner and H. D. Lasswell. London: Sage.

14.

McKenzie, I. 1985. Being objective about appraisal. Nursing Times 25:25-26.

15.

Niemi, R.G. , E.G. Carmines , and J.P. McIver. 1986. The impact of scale length on reliability and validity. Quality and Quantity 20:371-76.

16.

Nunnally, J.C. 1978. Psychometric Theory. New York : McGraw-Hill.

17.

Rew, L. , D. Stuppy , and H. Becker. 1988. Construct validity in instrument development: A vital link between nursing practice, research, and theory. Advances in Nursing Science 10 (July): 10-22.

18.

Smith, P.C. , and L.M. Kendall. 1963. Retranslation of expectations: An approach to the construction of unambiguous anchors for rating scales. Journal of Applied Psychology 47:149-55.

19.

Whitehurst, G.J. 1984. Interrater agreement for journal manuscript reviews . American Psychologist 39:22-28.

20.

Zedeck, S. , M. Imparato , M. Krausz , and T. Oleno. 1974. Development of behaviorally anchored scales as a function of organizational level Journal of Applied Psychology 59:249-52.