Psychometric theory focuses primarily on tests that are homogeneous, measuring only one attribute of a psychosocial entity. However, the complexity of psychosocial behavior often requires tests that are heterogeneous, measuring more than one attribute. In this presentation, reliability and internal consistency are extended to heterogeneous tests under the rubric of congeneric test theory. The extensions show that reliability and internal consistency have very similar properties. Reliability and internal consistency are shown to be unique up to a linear transformation. Whereas internal consistency is a lower bound to reliability in the homogeneous case, it is a strict lower bound in the heterogeneous case. Reliability equals internal consistency if and only if the test is homogeneous and true-score equivalent. An analytic argument shows that neither reliability nor internal consistency can detect whether a test is homogeneous or heterogeneous. The reliability (internal consistency) of a heterogeneous test is the sum of the sub-tests’ reliabilities (internal consistencies) plus the sum of reliabilities (internal consistencies) involving the correlations among the sub-tests. Higher-order tests are tests in which items measure first-order attributes, first-order attributes reflect second-order attributes, and so on. Reliability (internal consistency) can be partitioned into the sums of direct reliability (direct internal consistency) for the first-order attributes and indirect reliabilities (indirect internal consistencies) for the second- and higher-order attributes. Simple examples demonstrate that internal consistency may be a poor approximator to the reliability of congeneric tests. Given that reliability for congeneric tests can now be estimated by modern statistical software, the need for internal consistency is diminished. However, internal consistency may be useful in bounding or approximating reliability in general linear latent variable models.
Get full access to this article
View all access options for this article.
References
1.
Anderson, J. C., & Gerbing, D. W. (1988). Structural equation modeling in practice: A review and recommended two-step approach. Psychological Bulletin, 103(3), 411-423.
2.
Anderson, J. C., & Gerbing, D. W. (1992). Assumptions and comparative strengths of the two-step approach: Comment on Fornell and Yi. Sociological Methods & Research, 20(3), 321-333.
3.
Bartholomew, D.J. (1996). The statistical approach to social measurement. San Diego: Academic Press.
4.
Bartholomew, D. J., & Knott, M. (1999). Latent variable models and factor analysis (2nd ed.). London: Arnold.
5.
Bartholomew, D. J., & Schuessler, K. (1991). Reliability of attitude scores based on a latent trait model. Sociological Methodology, 21, 97-123.
6.
Bock, R. D. (1975). Multivariate statistical methods in behavioral research. New York: McGraw-Hill.
7.
Boisvert, J., McCreary, D., Wright, K., & Asmundson, G. (2003). Factorial validity of the Center for Epidemiologic Studies Depression (CES-D) Scale in military peacekeepers. Depression and Anxiety, 17(1), 19-25.
8.
Bollen, K. A. (1989). Structural equations with latent variables. New York: John Wiley.
9.
Bollen, K. A. (2002). Latent variables in psychology and the social sciences. Annual Review of Psychology, 53(1), 605-634.
10.
Bollen, K. A., & Lennox, R. (1991). Conventional wisdom on measurement: A structural equation perspective. Psychological Bulletin, 110, 305-314.
11.
Borsboom, D., Mellenbergh, G. J., & van Heerden, J. (2003). The theoretical status of latent variables. Psychological Review, 110(2), 203-219.
12.
Boyd, R. (1999). Homeostasis, species, and higher taxa. In R. A. Wilson (Ed.), Species: New inter-disciplinary essays (pp. 141-185). Cambridge, MA: MIT Press.
13.
Brookings, J. B., & Bolton, B. (1988). Confirmatory factor analysis of the Interpersonal Support Evaluation List. American Journal of Community Psychology, 16(1), 137-147.
14.
Carmines, E. G., & McIver, J. P. (1981). Analyzing models with unobservable variables: Analysis of covariance structures. In G. W. Bohrnstedt & E. F. Borgatta (Eds.), Social measurement (pp. 65-115). Beverly Hills, CA: Sage.
15.
Cohen, S., & Hoberman, H. M. (1983). Positive events and social supports as buffers of life change stress. Journal of Applied Social Psychology, 13(2), 99-125.
16.
Cortina, J. M. (1993). What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 78(1), 98-104.
17.
Corwyn, R. F. (2000). The factor structure of global self-esteem among adolescents and adults. Journal of Research in Personality, 34, 357-379.
18.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297-334.
19.
DeVellis, R. F. (2003). Scale development: Theory and applications (2nd ed.). Thousand Oaks, CA: Sage.
20.
Edwards, J. R., & Bagozzi, R. P. (2000). On the nature and direction of relationships between constructs and measures. Psychological Methods, 5(2), 155-174.
21.
Graybill, F. A. (1969). Matrices with applications in statistics. Belmont, CA: Wadsworth International.
22.
Green, S. B., & Herschberger, S. L. (2000). Correlated errors in true score models and their effect on coefficient alpha. Structural Equation Modeling, 7(2), 251-270.
23.
Green, S. B., Lissitz, R. W., & Mulaik, S. A. (1977). Limitations of coefficient alpha as an index of test unidimensionality. Educational and Psychological Measurement, 37, 827-838.
24.
Guttman, L. (1945). A basis for analyzing test-retest reliability. Psychometrika, 10(4), 255-282.
25.
Hattie, J. (1985). Methodology review: Assessing unidimensionality of tests and items. Applied Psychological Measurement, 9(2), 139-164.
26.
Heise, D. R., & Bohrnstedt, G. W. (1970). Validity, invalidity, and reliability. In E. F. Borgatta & G. W. Bohrnstedt (Eds.), Sociological methodology 1970 (Vol. 2, pp. 104-129). San Francisco: Jossey-Bass.
27.
Iacobucci, D., & Duhachek, A. (2003). Advancing alpha: Measuring reliability with confidence. Journal of Consumer Psychology, 13(4), 478-487.
28.
Jöreskog, K. G. (1971). Statistical analysis of sets of congeneric tests. Psychometrika, 36, 109-133.
29.
Jöreskog, K. G., & Sörbom, D. (1996). LISREL 8 user’s guide. Chicago: Scientific Software International.
30.
Knight, R., Williams, S., McGee, R., & Olaman, S. (1997). Psychometric properties of the Centre for Epidemiologic Studies Depression Scale (CES-D) in a sample of women in middle life. Behaviour Research and Therapy, 35(4), 373-380.
31.
Komaroff, E. (1997). Effect of simultaneous violations of essential τ-equivalence and uncorrelated error on coefficient α. Applied Psychological Measurement, 21(4), 337-348.
32.
Konig, A. J., & Franses, P. H. (2003). Confidence intervals for Cronbach’s coefficient alpha values (Tech. Rep.). Erasmus Institute of Management. Retrieved March 24, 2004, from www.erim.eur.nl
33.
Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores with contributions by Allan Birnbaum. Reading, MA: Addison-Wesley.
34.
McDonald, R. P. (1970). The theoretical foundations of common factor analysis, principal factor analysis, and alpha factor analysis. British Journal of Mathematical and Statistical Psychology, 23, 1-21.
35.
McDonald, R. P. (1981). The dimensionality of tests and items. British Journal of Mathematical and Statistical Psychology, 34, 100-117.
36.
McDonald, R. P. (1985). Factor analysis and related methods. Hillsdale, NJ: Lawrence Erlbaum.
37.
McDonald, R. P. (1999). Test theory: A unified approach. Mahwah, NJ: Lawrence Erlbaum.
38.
Mellenbergh, G. J. (1994). Generalized linear item response theory. Psychological Bulletin, 115(2), 300-307.
39.
Miller, M. B. (1995). Coefficient alpha: A basic introduction from the perspectives of classical test theory and structural equation modeling. Structural Equation Modeling, 2(3), 255-273.
40.
Moustaki, I., & Knott, M. (2000). Generalized latent trait models. Psychometrika, 65(3), 391-411.
41.
Muthén, B. O. (2002). Beyond SEM: General latent variable modeling. Behaviormetrika, 29(1), 81-117.
42.
Muthén, L. K., & Muthén, B. O. (2001). Mplus user’s guide: Statistical analysis with latent variables (2nd ed.). Los Angeles: Author.
43.
Novick, M. R., & Lewis, C. (1967). Coefficient alpha and the reliability of composite measurements. Psychometrika, 32(1), 1-13.
44.
Radloff, L. S. (1977). The CES-D scale: A self-report depression scale for research in the general population. Applied Psychological Measurement, 1(3), 385-401.
45.
Raykov, T. (1997a). Estimation of composite reliability for congeneric measures. Applied Psychological Measurement, 21(2), 173-184.
46.
Raykov, T. (1997b). Scale reliability, Cronbach’s coefficient alpha, and violations of essential tau-equivalence with fixed congeneric components. Multivariate Behavioral Research, 32(4), 329-353.
47.
Raykov, T. (1998a). Coefficient alpha and composite reliability with interrelated non homogeneous items. Applied Psychological Measurement, 22(4), 375-385.
48.
Raykov, T. (1998b). A method for obtaining standard errors and confidence intervals of composite reliability for congeneric items. Applied Psychological Measurement, 22(4), 369-374.
49.
Raykov, T. (2001a). Bias of coefficient α for fixed congeneric measures with correlated errors. Applied Psychological Measurement, 25(1), 69-76.
50.
Raykov, T. (2001b). Estimation of congeneric scale reliability using covariance structure analysis with nonlinear constraints. British Journal of Mathematical and Statistical Psychology, 54, 315-323.
51.
Raykov, T. (2002). Analytic estimation of standard error and confidence interval for scale reliability. Multivariate Behavioral Research, 37(1), 89-103.
52.
Raykov, T., & Penev, S. (2001). The problem of equivalent structural models: An individual residual perspective. In G. A. Marcoulides & R. E. Schumacker (Eds.), New developments and techniques in structural equation modeling (pp. 297-321). Mahwah, NJ: Lawrence Erlbaum.
53.
Raykov, T., & Shrout, P. E. (2002). Reliability of scales with general structure: Point and interval estimation using a structural equation modeling approach. Structural Equation Modeling, 9(2), 195-212.
54.
Reuterberg, S.-E., & Gustafsson, J.-E. (1992). Confirmatory factor analysis and reliability: Testing measurement model assumptions. Educational and Psychological Measurement, 52, 795-811.
55.
Rosenberg, M. (1986). Conceiving the self. Malabar, FL: Krieger.
56.
Rozeboom, W. W. (1989). The reliability of a linear composite of nonequivalent subtests. Applied Psychological Measurement, 13(3), 277-283.
57.
Sheehan, T. J., Fifield, J., Reisine, S., & Tennan, H. (1995). The measurement structure of the Center for Epidemiologic Studies Depression Scale. Journal of Personality Assessment, 64(3), 507-521.
58.
van Zyl, J. M., Neudecker, H., & Nel, D. G. (2000). On the distribution of the maximum likelihood estimator of Cronbach’s alpha. Psychometrika, 65(3), 271-280.
59.
Zimmerman, D. W., Zumbo, B. D., & Lalonde, C. (1993). Coefficient alpha as an estimate of test reliability under violations of two assumptions. Educational and Psychological Measurement, 53, 33-49.
