Sage Journals: Discover world-class research

Abstract

Standard five-point rating scales often do not allow raters to capture perceived differences between objects or individuals within a relatively narrow band of the evaluative dimension. In the frequency domain, using a longer rating scale or packing the rating scale with labels from a particular portion of the dimension of interest have both been shown by Hancock and Klockars in 1991 to increase rating validity for differentiating among a narrow range of performances. The present study investigates the effect of similar manipulations on the validity of ratings from evaluative rating scales. The correlations of evaluative ratings with experimentally manipulated (and hence known) performance tended to be fairly high regardless of the evaluative scale used.

Get full access to this article

View all access options for this article.

References

Bendig

A. W.

(1952) The use of student rating scales in the evaluation of instructors in introductory psychology. Journal of Educational Psychology, 43, 167–175.

Bendig

A. W.

(1953) The reliability of self-ratings as a function of the amount of verbal anchoring and of the number of categories on the scale. Journal of Applied Psychology, 37, 38–41.

Bendig

A. W.

(1954) Reliability and the number of rating scale categories. Journal of Applied Psychology, 38, 38–40.

Dawes

R. M.

(1972) Fundamentals of attitude measurement. New York: Wiley.

Dunham

T. C.

Davison

M. L.

(1991) Effects of scale anchors on student ratings of instructors. Applied Measurement in Education, 4, 23–35.

Fisher

R. A.

(1921) On the “probable error” of a coefficient of correlation. Metron, 1, 1–32.

French-Lazovik

Gibson

C. L.

(1984) Effects of verbally labeled anchor points on the distributional parameters of rating measures. Applied Psychological Measurement, 8, 49–57.

Freyd

(1923) The graphic rating scale. Journal of Educational Psychology, 14, 83–102.

Guilford

J. P.

(1936) Psychometric methods. New York: McGraw-Hill.

10.

Hancock

G. R.

Klockars

A. J.

(1991) The effect of scale manipulations on validity: targeting frequency rating scales for anticipated performance levels. Applied Ergonomics, 22, 147–154.

11.

Klockars

A. J.

Yamagishi

(1988) The influence of labels and positions in rating scales. Journal of Educational Measurement, 25, 85–96.

12.

Komorita

S. S.

(1963) Attitude content, intensity, and the neutral point on a Likert scale. Journal of Social Psychology, 61, 327–334.

13.

Lam

T. C. M.

Klockars

A. J.

(1982) Anchor point effects on the equivalence of questionnaire items. Journal of Educational Measurement, 19, 317–322.

14.

Matell

M. S.

Jacoby

(1971) Is there an optimal number of alternatives for Likert scale items? Study 1. Reliability and validity. Educational and Psychological Measurement, 31, 657–674.

15.

Symonds

P. M.

(1924) On the loss of reliability in ratings due to coarseness of the scale. Journal of Experimental Psychology, 7, 456–461.

16.

Symonds

P. M.

(1931) Diagnosing personality and conduct. New York: Appleton-Century.

Manipulations of Evaluative Rating Scales to Increase Validity

Abstract

Get full access to this article

References