Reliability and validity of 4-point and 6-point scales were assessed using a new model-based ap proach to fit empirical data. Different measurement models were fit by confirmatory factor analyses of a multitrait-multimethod covariance matrix. 165 gradu ate students responded to nine items measuring three quantitative attitudes. Separation of method from trait variance led to greater reduction of reliability and heterotrait-monomethod coefficients for the 6-point scale than for the 4-point scale. Criterion-related valid ity was not affected by the number of scale points. The issue of selecting 4- versus 6-point scales may not be generally resolvable, but may rather depend on the empirical setting. Response conditions theorized to in fluence the use of scale options are discussed to pro vide directions for further research. Index terms: Likert-type scales, multitrait-multimethod matrix, reli ability, scale options, validity.
Get full access to this article
View all access options for this article.
References
1.
Alliger, G.M., & Williams, K.J. (1992). Relating the internal consistency of scales to rater response tendencies. Educational and Psychological Measurement, 52, 337-343.
2.
Bendig, A.W. (1953). Reliability of self-ratings as a function of the amount of verbal anchoring and of the number of categories on the scale. Journal of Applied Psychology, 37, 38-41.
3.
Bendig, A.W. (1954a). Reliability and the number of rating scale categories . Journal of Applied Psychology, 38, 38-40.
4.
Bendig, A.W. (1954b). Reliability of short rating scales and the heterogeneity of the rated stimuli. Journal of Applied Psychology, 38, 167-170.
5.
Bentler, P.M., & Bonett, D.G. (1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin,88, 588-606.
6.
Bollen, K.M. (1989). Structural equations with latent variables. New York: Wiley.
7.
Boote, A.S. (1981). Reliability testing of psychographic scales: Five-point or seven-point? Anchored or labeled ?Journal of Advertising Research, 21, 53-60.
8.
Brown, G., Widing, R.E., II, & Coulter, R.L. (1991). Customer evaluation of retail salespeople utilizing the SOCO scale: A replication, extension, and application. Journal of the Academy of Marketing Science, 9, 347-351.
9.
Chang, L. (1993, April). Using confirmatoryfactor analysis of multitrait-multimethod data to assess the psychometric equivalence of 4-point and 6-point Likert-type scales. Paper presented at the Annual Meeting of the National Council on Measurement in Education, Atlanta .
10.
Chang, L. (1994). Quantitative Attitudes Questionnaire: Instrument development and validation. Manuscript submitted for publication.
11.
Cicchetti, D.V., Showalter, D., & Tyrer, P.J. (1985). The effect of number of rating scale categories on levels of interrater reliability: A monte carlo investigation. Applied Psychological Measurement, 9, 31-36.
12.
Cohen, J. (1983). The cost of dichotomization. Applied Psychological Measurement, 7, 249-253.
13.
Comrey, A.L., & Montag, I. (1982). Comparison of factor analytic results with two-choice and seven-choice personality item formats. Applied Psychological Measurement, 6, 285-289.
14.
Cronbach, L.J. (1950). Further evidence on response sets and test design . Educational and Psychological Measurement, 10, 3-31.
15.
Finn, R.H. (1972). Effect of some variations in rating scale characteristics on the means and reliabilities of ratings. Educational and Psychological Measurement, 34, 885-892.
16.
Goldberg, L.R. (1981). Unconfounding situational attributions from uncertain, neutral, and ambiguous ones: A psychometric analysis of descriptions of oneself and various types of others. Journal of Personality and Social Psychology, 41, 517-552.
17.
Hocevar, D., Zimmer, J., & Chen, C.Y. (1990, April). A multitrait-multimethod analysis of the worry/emotionality component in the measurement of test anxiety. Paper presented at a joint session of the American Educational Research Association and the National Council on Measurement in Education , Boston.
18.
Jenkins, G.D., Jr., & Taber, T.D. (1977). A monte carlo study of factors affecting three indices of composite scale reliability. Journal of Applied Psychology, 62, 392-398.
19.
Joe, V.C., & Jahn, J.C. (1973). Factor structure of the Rotter I-E Scale. Journal of Clinical Psychology, 29, 66-68.
20.
Jöreskog, K.G. (1971). Statistical analysis of sets of con-generic tests . Psychometrika, 36, 109-132.
21.
Jöreskog, K.G., & Sörbom, D. (1988). LISREL 7: A guide to the program and applications [Computer program manual]. Chicago: SPSS, Inc.
22.
Kenny, D.A. (1979). Correlation and causality. New York: Wiley.
Komorita, S.S. (1963). Attitude content, intensity, and the neutral point on a Likert scale. Journal of Social Psychology, 61, 327-334.
25.
Komorita, S.S., & Graham, W.K. (1965). Number of scale points and the reliability of scales. Educational and Psychological Measurement, 25, 987-995.
26.
Likert, R. (1932). A technique for the measurement of attitudes. Archives of Psychology, 140, 5-55.
27.
Lissitz, R.W., & Green, S.B. (1975). Effect of the number of scale points on reliability: A monte carlo approach. Journal of Applied Psychology, 60, 10-13.
28.
Marsh, H.W. (1989). Confirmatory factor analyses of multitrait-multimethod data: Many problems and a few solutions. Applied Psychological Measurement, 13, 335-361.
29.
Marsh, H.W. (1993). Stability of individual differences in multiwave panel studies: Comparison of simplex models and one-factor model. Journal of Educational Measurement, 30, 157-183.
30.
Marsh, H.W., Balla, J.R., & McDonald, R.P. (1988). Cloodness-of flt indexes in confirmatory factor analysis: The effect of sample size. Psychological Bulletin,103, 391-410.
31.
Marsh, H.W., & Hocevar, D. (1983). Confirmatory factor analysis of multitrait-multimethod matrices. Journal of Educational Measurement, 20, 231-248.
32.
Martin, W.S. (1973). The effects of scaling on the correlation coefficient: A test of validity. Journal of Marketing Research, 10, 316-318.
33.
Martin, W.S. (1978). Effects of scaling on the correlation coefficient: Additional considerations. Journal of Marketing Research, 15, 304-308.
34.
Masters, J.R. (1974). The relationship between number of response categories and reliability of Likert-type questionnaires. Journal of Educational Measurement, 11, 49-53.
35.
Matell, M.S., & Jacoby, J. (1971). Is there an optimal number of alternatives for Likert scale items? Study I: Reliability and validity. Educational and Psychological Measurement, 31, 657-674.
36.
McKelvie, S.J. (1978). Graphic rating scales—How many categories?British Journal of Psychology, 69, 185-202.
Muthén, B., & Kaplan, D. (1985). A comparison of some methodologies for the factor analysis of non-normal Likert variables. British Journal ofmathematical and StatisticalPsychology, 38, 171-189.
39.
Nunnally, J.C. (1967). Psychometric theory. New York : McGraw-Hill.
40.
Nunnally, J.C. (1970). Introduction to psychological measurement. New York: McGraw-Hill.
41.
Oswald, W.T., & Velicer, W.F. (1980). Item format and the structure of the Eysenck Personality Inventory: A replication. Journal of Personality Assessment , 44, 283-288.
42.
Peabody, D. (1962). Two components in bipolar scales: Direction and extremeness. Psychological Review, 69, 65-73.
43.
Ramsay, J.O. (1973). The effect of number of categories in rating scales on precision of estimation of scale values. Psychometrika , 38, 513-533.
44.
Remington, M., Tyrer, P.J., Newson-Smith, J., & Cicchetti, D.V. (1979). Comparative reliability of categorical and analogue rating scales in the assessment of psychiatric symptomatology. Psychological Medicine, 9, 765-770.
45.
Remmers, H.H., & Ewart, E. (1941). Reliability of multiple-choice measuring instruments as a function of the Spearman-Brown prophecy formula. Journal of Educational Psychology, 32, 61-66.
Symonds, P.M. (1924). On the loss of reliability in ratings due to coarseness of the scale. Journal of ExperimentalPsychology, 7, 456-461.
48.
Torgerson, W.J. (1958). Theory and methods of scaling. New York: Wiley.
49.
Tucker, L.R., & Lewis, C. (1973). The reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38, 1-10.
50.
Velicer, W.F., DiClemente, C.C., & Corriveau, D.P. (1984). Item format and the structure of the personal orientation inventory. Applied Psychological Measurement, 8, 409-419.
51.
Velicer, W.F., & Stevenson, J.F. (1978). The relation between item format and the structure of the Eysenck Personality Inventory. Applied Psychological Measurement, 2, 293-304.
