This study presents a general framework for single-administration reliability methods, such as Cronbach’s alpha, Guttman’s lambda-2, and method MS. This general framework was used to derive a new approach to estimating test-score reliability by means of the unrestricted latent class model. This new approach is the latent class reliability coefficient (LCRC). Unlike other single-administration reliability methods, LCRC places few restrictions on the item scores. A simulation study showed that if data are multidimensional or if double monotonicity does not hold, then LCRC is less biased relative to the true reliability than Cronbach’s alpha, Guttman’s lambda-2, method MS, and the split-half reliability coefficient.
AndrewsR. L.CurrimI. S. (2003). A comparison of segment retention criteria for finite mixture logit models. Journal of Marketing Research, 40, 235-243.
2.
BentlerP. M. (2009). Alpha, dimension-free, and model-based internal consistency reliability. Psychometrika, 74, 137-143.
3.
BentlerP. M.WoodwardJ. A. (1980). Inequalities among lower bounds to reliability: With applications to test construction and factor analysis. Psychometrika, 45, 249-267.
4.
BozdoganH. (1987). Model selection and Akaike’s Information Criterion (AIC): The general theory and its analytical extensions. Psychometrika, 52, 345-370.
5.
BozdoganH. (1992). Choosing the number of component clusters in the mixture-model using a new informational complexity criterion of the inverse-Fisher information matrix. In OpitzO.LausenB.KlarR. (Eds.), Information and classification: Concepts, methods and applications (pp. 40-54). New York, NY: Springer.
6.
CortinaJ. M. (1993). What is coefficient alpha? An examination of theory and application. Journal of Applied Psychology, 78, 98-104.
7.
CronbachL. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297-334.
8.
De AyalaR. J. (1994). The influence of multidimensionality on the graded response model. Applied Psychological Measurement, 18, 155-170.
9.
DiasJ. G. (2006). Model selection for the binary latent class model: A Monte Carlo simulation. In BatageljV.BockH.-H.FerligojA.ŽibernaA. (Eds.), Studies in classification, data analysis, and knowledge organization (pp. 91-199). Berlin, Germany: Springer.
10.
GreenS. B.YangY. (2009). Commentary on coefficient alpha: A cautionary tale. Psychometrika, 74, 121-135.
11.
GuttmanL. (1945). A basis for analyzing test-retest reliability. Psychometrika, 10, 255-282.
12.
HagenaarsJ. A. P.McCutcheonA. L. (Eds.). (2002). Applied latent class analysis. Cambridge, UK: Cambridge University Press.
13.
KangT.CohenA. S. (2007). IRT model selection methods for dichotomous items. Applied Psychological Measurement, 31, 331-358.
14.
KangT.CohenA. S.SungH.-J. (2009). Model selection indices for polytomous items. Applied Psychological Measurement, 33, 499-518.
15.
KoehlerK. J.LarntzK. (1980). An empirical investigation of goodness-of-fit statistics for sparse multinomials. Journal of the American Statistical Association, 75, 336-344.
16.
LazarsfeldP. F. (1950). The logical and mathematical foundation of latent structure analysis. In StoufferS. A.GuttmanL.SuchmanE. A.LazarsfeldP. F.StarS. A.ClausenJ. A. (Eds.), Measurement and prediction (pp. 362-412). Princeton, NJ: Princeton University Press.
17.
LiF.CohenA. S.KimS.-H.ChoS.-J. (2009). Model selection methods for mixture dichotomous IRT models. Applied Psychological Measurement, 33, 353-373.
18.
LordF. M.NovickM. R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
19.
LukočienėO.VermuntJ. K. (2010). Determining the number of components in mixture models for hierarchical data. In FinkA.LausenB.SeidelW.UltschA. (Eds.), Advances in data analysis, data handling and business intelligence (pp. 241-250). Berlin, Germany: Springer.
20.
MokkenR. J. (1971). A theory and procedure of scale analysis. The Hague, Netherlands: Mouton; Berlin, Germany: De Gruyter.
21.
MolenaarI. W. (1997). Nonparametric models for polytomous items. In van der LindenW. J.HambletonR. K. (Eds.), Handbook of modern item response theory (pp. 369-380). New York, NY: Springer.
22.
MolenaarI. W.SijtsmaK. (1988). Mokken’s approach to reliability estimation extended to multicategory items. Kwantitatieve Methoden, 9(28), 115-126. Retrieved from http://arno.uvt.nl/show.cgi?fd=81058
23.
MolenaarI. W.SijtsmaK. (2000). MSP5 for Windows. A program for Mokken scale analysis for polytomous items. Groningen/, Netherlands: iec ProGAMMA.
24.
R Development Core Team. (2006). R: A language and environment for statistical computing [computer programming language]. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from http://www.R-project.org
25.
RaykovT. (1997). Estimation of composite reliability for congeneric measures. Applied Psychological Measurement, 21, 173-184.
26.
RaykovT.ShroutP. E. (2002). Reliability of scales with general structure: Point and interval estimation using a structural equation modeling approach. Structural Equation Modeling, 9, 195-212.
27.
RevelleW. (1979). Hierarchical cluster analysis and the internal structure of tests. Multivariate Behavioral Research, 14, 57-74.
28.
SchmittN. (1996). Uses and abuses of coefficient alpha. Psychological Assessment, 8, 350-353.
29.
SchwarzG. (1978). Estimating the dimension of a model. Annals of Statistics, 6, 461-464.
30.
SedereM. U.FeldtL. S. (1977). The sampling distributions of the Kristof reliability coefficient, the Feldt coefficient, and Guttman’s lambda-2. Journal of Educational Measurement, 14, 53-62.
31.
SijtsmaK. (1988). Contributions to Mokken’s nonparametric item response theory (Unpublished doctoral dissertation). Amsterdam, Netherlands: Vrije Universiteit.
32.
SijtsmaK. (2009). On the use, the misuse, and the very limited usefulness of Cronbach’s alpha. Psychometrika, 74, 107-120.
33.
SijtsmaK.MolenaarI. W. (1987). Reliability of test score in nonparametric item response theory. Psychometrika, 52, 79-97.
34.
Ten BergeJ. M. F.SnijdersT. A. B.ZegersF. E. (1981). Computational aspects of the greatest lower bound to the reliability and constrained minimum trace factor analysis. Psychometrika, 46, 201-213.
35.
Ten BergeJ. M. F.ZegersF. E. (1978). A series of lower bounds to the reliability of a test. Psychometrika, 43, 575-579.
36.
ThompsonB. L.GreenS. B.YangY. (2010). Assessment of the maximal split-half coefficient to estimate reliability. Educational and Psychological Measurement, 70, 232-251.
37.
Van der ArkL. A. (2007). Mokken scale analysis in R. Journal of Statistical Software, 20(11), 1-19.
38.
Van der ArkL. A. (2010). Computation of the Molenaar Sijtsma statistic. In FinkA.LausenB.SeidelW.UltschA. (Eds.), Advances in data analysis, data handling and business intelligence (pp. 775-784). Berlin, Germany: Springer.
39.
ZinbargR. E.RevelleW.YovelI.LiW. (2005). Cronbach’s α, Revelle’s β, and McDonald’s ω:Their relations with each other and two alternative conceptualizations of reliability. Psychometrika, 70, 123-133.
