Two aspects of the reliability of multidimensional measures can be distinguished: the amount of scale score variance that is accounted for by all underlying factors (composite reliability) and the degree to which the scale score reflects one particular factor (construct reliability). Confidence intervals for composite and construct reliabilities can be estimated by bootstrap methods. The authors demonstrate the application of these methods by analyzing the reliability of an eight-factor, nested-factor model that represents the structure of 45 tasks in an intelligence test (N= 1,233). Composite reliabilities ranged between .78 and .93, whereas construct reliabilities ranged between .17 and .68 when the scale indicators were equally weighted to compute the scale scores and between .52 and .90 with weights based on pattern coefficients. The results indicate the importance of distinguishing diagnostic from research applications when judging whether the reliability values of multidimensional measures are substantial.
Allen, M. P. (1974). Construction of composite measures by the canonical-factor-regression method. In H. L. Costner (Ed.), Sociological methodology(pp. 51-78). San Francisco: Jossey-Bass.
2.
Bacon, D. R., Sauer, P. L.,& Young, M. (1995). Composite reliability in structural equationmodeling. Educational and Psychological Measurement, 55, 394-406.
3.
Benito, J. G. (1989). Estimates of the internal consistency of a factorially complex composite. Educational and Psychological Measurement, 49, 571-578.
4.
Bentler, P. M. (1995). EQS: A structural equations program manual.Encino, CA: Multivariate Software.
5.
Bentler, P. M., & Dudgeon, P. (1996). Covariance structure analysis: Statistical practice, theory, and directions. Annual Review of Psychology, 47, 563-592.
6.
Bollen, K. A. (1989). Structural equations with latent variables. New York: John Wiley.
7.
Bollen, K. A., & Lennox, R. (1991). Conventional wisdom on measurement: A structural equation perspective. Psychological Bulletin, 110, 305-314.
8.
Carroll, J. B. (1993). Human cognitive abilities—A survey of factor analytic studies. New York: Cambridge University Press.
9.
Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. New York: Holt, Rinehart & Winston.
10.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297-334.
11.
DeShon, R. P. (1998). A cautionary note on measurement error in corrections in structural equation models. Psychological Methods, 3, 412-423.
12.
Gustafsson, J.,& Balke, G. (1993). General and specific abilities as predictors of school achievement. Multivariate Behavioral Research, 28, 407-434.
13.
Hancock, G. R., & Mueller, R. O. (2001). Rethinking construct reliability within latent variable systems. In R. Cudeck, S. du Toit,& D. Soerbom (Eds.), Structural equation modeling: Present and future—A festschrift in honor of Karl Jöreskog(pp. 195-216). Lincolnwood, IL: Scientific Software International.
14.
Hogan, T. P., Benjamin, A., & Brezinski, K. L. (2000). Reliability methods: A note on the frequency of use of various types. Educational and Psychological Measurement, 60, 523-531.
15.
Hu, L.-T., & Bentler, P. M. (1998). Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods, 3, 424-453.
16.
Hu, L.-T.,& Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55.
17.
Jäger, A. O., Süβ, H.-M., & Beauducel, A. (1997). Berlin Intelligence Structure—Test, Form 4. Göttingen, Germany: Hogrefe.
18.
Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores.Reading, MA: Addison-Wesley.
19.
MacCallum, R. (1986). Specification searches in covariance structure modeling. Psychological Bulletin, 100, 107-120.
20.
MacCallum, R. C., Roznowski, M., & Necowitz, L. B. (1992). Model modifications in covariance structure-analysis—The problem of capitalization on chance. Psychological Bulletin, 111, 490-504.
21.
McDonald, R. P. (1970). The theoretical foundations of principal factor analysis, canonical factor analysis, and alpha factor analysis. British Journal of Mathematical and Statistical Psychology, 23, 1-21.
22.
McDonald, R. P. (1981). The dimensionality of tests and items. British Journal of Mathematical and Statistical Psychology, 34, 100-117.
23.
McDonald, R. P. (1999). Test theory: A unified treatment.Mahwah, NJ: Lawrence Erlbaum.
24.
Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory. New York: McGraw-Hill.
25.
Raines-Eudy, R. (2000). Using structural equation modeling to test for differential reliability and validity: An empirical demonstration. Structural Equation Modeling, 7, 124-141.
26.
Raykov, T. (1997). Estimation of composite reliability for congenericmeasures. Applied Psychological Measurement, 21, 173-184.
27.
Raykov, T. (1998a). Coefficient alpha and composite reliability with interrelated nonhomogeneous items. Applied Psychological Measurement, 22, 375-385.
28.
Raykov, T. (1998b). Amethod for obtaining standard errors and confidence intervals of composite reliability for congeneric items. Applied Psychological Measurement, 22, 369-374.
29.
Raykov, T.,& Shrout, P. E. (2002). Reliability of scales with general structure: Point and interval estimation using a structural equation modeling approach. Structural Equation Modeling, 9, 195-212.
30.
Süβ, H.-M., Oberauer, K., Wittmann, W. W., Wilhelm, O., & Schulze, R. (2002). Workingmemory capacity explains reasoning ability—And a little bit more. Intelligence, 30, 261-288.
31.
Werts, C. E., Rock, D. A., Linn, R. L.,& Joreskog, K. G. (1978). Ageneral method of estimating the reliability of a composite. Educational and Psychological Measurement, 38, 933-938.
32.
West, S. G., Finch, J. F.,& Curran, P. J. (1995). Structural equation models with nonnormal variables: Problems and remedies. In R. H. Hoyle (Ed.), Structural equation modeling: Concepts, issues, and applications(pp. 56-75). Thousand Oaks, CA: Sage.
33.
Wilhelm, O.,& Schulze, R. (2002). The relation of speeded and unspeeded reasoning with mental speed. Intelligence, 30, 537-554.
