Cronbach’s coefficient alpha is a widely used reliability measure in social, behavioral, and education sciences. It is reported in nearly every study that involves measuring a construct through multiple items. With non-tau-equivalent items, McDonald’s omega has been used as a popular alternative to alpha in the literature. Traditional estimation methods for alpha and omega often implicitly assume that data are complete and normally distributed. This study proposes robust procedures to estimate both alpha and omega as well as corresponding standard errors and confidence intervals from samples that may contain potential outlying observations and missing values. The influence of outlying observations and missing data on the estimates of alpha and omega is investigated through two simulation studies. Results show that the newly developed robust method yields substantially improved alpha and omega estimates as well as better coverage rates of confidence intervals than the conventional nonrobust method. An R package coefficientalpha is developed and demonstrated to obtain robust estimates of alpha and omega.
BentlerP. M. (2009). Alpha, dimension-free, and model-based internal consistency reliability. Psychometrika, 74, 137-143. doi:10.1007/s11336-008-9100-1
2.
BentlerP. M. (2010). EQS 6 structural equations program manual. Encino, CA: Multivariate Software.
3.
CheungM. W.-L. (2009). Constructing approximate confidence intervals for parameters with structural constructing approximate confidence intervals for parameters with structural equation models. Structural Equation Modeling, 16, 267-294.
4.
ChristmannA.Van AelstS. (2006). Robust estimation of Cronbach’s alpha. Journal of Multivariate Analysis, 97, 1660-1674.
5.
CronbachL. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297-334. doi:10.1007/BF02310555
6.
CronbachL. J. (1988). Internal consistency of tests: Analyses old and new. Psychometrika, 53, 63-70. doi:10.1007/BF02294194
7.
CronbachL. J.ShavelsonR. J. (2004). My current thoughts on coefficient alpha and successor procedures. Educational and Psychological Measurement, 64, 391-418. doi:10.1177/0013164404266386
8.
EndersC. K. (2003). Using the EM algorithm to estimate coefficient alpha for scales with item-level missing data. Psychological Methods, 8, 322-337. doi:10.1037/1082-989X.8.3.322
9.
EndersC. K. (2004). The impact of missing data on sample reliability estimates: Implications for reliability reporting practices. Educational and Psychological Measurement, 64, 419-436.
10.
FanX.ThompsonB. (2001). Confidence intervals about score reliability coefficients, please: An EPM guidelines editorial. Educational and Psychological Measurement, 61, 517-531. doi:10.1177/0013164401614001
11.
FeldtL. S. (1965). The approximate sampling distribution of Kuder-Richardson reliability coefficient twenty. Psychometrika, 30, 357-337. doi:10.1007/BF02289499
12.
GreenS.YangY. (2009). Reliability of summed item scores using structural equation modeling: An alternative to coefficient alpha. Psychometrika, 74, 155-167. doi:10.1007/s11336-008-9099-3
13.
HeadrickT. C.ShengY. (2013). A proposed measure of internal consistency reliability: Coefficient L-alpha. Behaviormetrika, 40, 57-68.
14.
IacobucciD.DuhachekA. (2003). Advancing alpha: Measuring reliability with confidence. Journal of Consumer Psychology, 13, 478-487. doi:10.1207/S15327663JCP1304_14
15.
KristofW. (1963). The statistical theory of stepped-up reliability when a test has been divided into several equivalent parts. Psychometrika, 28, 221-238. doi:10.1007/BF02289571
16.
LittleR. J. A. (1988). Robust estimation of the mean and covariance matrix from data with missing values. Journal of the Royal Statistical Society: Series C (Applied Statistics), 37, 23-38.
17.
LittleR. J. A.RubinD. B. (2002). Statistical analysis with missing data (2nd ed.). New York, NY: Wiley-Interscience.
18.
LiuY.WuA. D.ZumboB. D. (2010). The impact of outliers on Cronbach’s coefficient alpha estimate of reliability: Ordinal/rating scale item responses. Educational and Psychological Measurement, 70, 5-21. doi:10.1177/0013164409344548
19.
LiuY.ZumboB. D. (2007). The impact of outliers on Cronbach’s coefficient alpha estimate of reliability: Visual analogue scales. Educational and Psychological Measurement, 67, 620-634. doi:10.1177/0013164406296976
20.
Maydeu-OlivaresA.CoffmanD.Garcia-ForeroC.Gallardo-PujolD. (2010). Hypothesis testing for coefficient alpha: An SEM approach. Behavior Research Methods, 42, 618-625.
21.
Maydeu-OlivaresA.CoffmanD. L.HartmannW. M. (2007). Asymptotically distribution-free (ADF) interval estimation of coefficient alpha. Psychological Methods, 12, 157-176. doi:10.1037/1082-989X.12.2.157
22.
McDonaldR. P. (1999). Test theory: A unified treatment. Mahwah, NJ: Lawrence Erlbaum.
23.
PadillaM. A.DiversJ. (2013a). Bootstrap interval estimation of reliability via coefficient omega. Journal of Modern Applied Statistical Methods, 12(1), 78-89.
RaykovT. (1997). Scale reliability, Cronbach’s coefficient alpha, and violations of essential tau equivalence with fixed congeneric components. Multivariate Behavioral Research, 33, 343-363. doi:10.1207/s15327906mbr3303_2
26.
RaykovT. (1998). A method for obtaining standard errors and confidence intervals of composite reliability for congeneric items. Applied Psychological Measurement, 22, 369-374.
27.
RaykovT. (2001). Bias of coefficient α for fixed congeneric measures with correlated errors. Applied Psychological Measurement, 25, 69-76. doi:10.1177/01466216010251005
28.
RaykovT. (2002). Analytic estimation of standard error and confidence interval for scale reliability. Multivariate Behavioral Research, 37, 89-103.
29.
RaykovT.MarcoulidesG. A. (2010). Introduction to psychometric theory. New York, NY: Routledge.
30.
RaykovT.MarcoulidesG. A. (2015). Scale reliability evaluation with heterogeneous populations. Educational and Psychological Measurement, 75, 875-892.
31.
RaykovT.MarcoulidesG. A.PatelisT. (2015). The importance of the assumption of uncorrelated errors in psychometric theory. Educational and Psychological Measurement, 75, 634-647.
32.
RaykovT.ShroutP. E. (2002). Reliability of scales with general structure: Point and interval estimation using a structural equation modeling approach. Structural Equation Modeling, 9, 195-212. doi:10.1207/S15328007SEM0902_3
33.
RevelleW.ZinbargR. E. (2009). Coefficients alpha, beta, omega, and the GLB: Comments on Sijtsma, Psychometrika, 74, 145-154. doi:10.1007/s11336-008-9102-z
34.
ShengY.ShengZ. (2013). Is coefficient alpha robust to non-normal data?Frontiers in psychology, 3, 34.
35.
ShroutP. E.FleissJ. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86, 420-428. doi:10.1037/0033-2909.86.2.420
36.
StipekD. (1998). Differences between Americans and Chinese in the circumstances evoking pride, shame, and guilt. Journal of Cross-Cultural Psychology, 29, 616-629. doi:10.1177/0022022198295002
37.
TongX.ZhangZ.YuanK.-H. (2014). Evaluation of test statistics for robust structural equation modeling with nonnormal missing data. Structural Equation Modeling, 21, 553-565.
38.
van ZylJ. M.NeudeckerH.NelD. G. (2000). On the distribution of the maximum likelihood estimator of Cronbach’s alpha. Psychometrika, 65, 271-228. doi:10.1007/BF02296146
39.
WilcoxR. R. (1992). Robust generalizations of classical test reliability and Cronbach’s alpha. British Journal of Mathematical and Statistical Psychology, 45, 239-254.
40.
YangY.GreenS. B. (2010). A Note on structural equation modeling estimates of reliability. Structural Equation Modeling, 17, 66-81. doi:10.1080/10705510903438963
41.
YuanK.-H.BentlerP. M. (2002). On robustness of the normal-theory based asymptotic distributions of three reliability coefficients. Psychometrika, 67, 251-259. doi:10.1007/BF022948450
42.
YuanK.-H.ChanW.TianY. (2015). Expectation-robust algorithm and estimating equations for means and dispersion matrix with missing data. Advance online publication. doi:10.1007/s10463-014-0498-1
43.
YuanK.-H.GuarnacciaC. A.HayslipB. (2003). A study of the distribution of sample coefficient alpha with the Hopkins symptom checklist: Bootstrap versus asymptotic. Educational and Psychological Measurement, 63, 5-23. doi:10.1177/0013164402239314
44.
YuanK.-H.ZhangZ. (2012a). Robust structural equation modeling with missing data and auxiliary variables. Psychometrika, 77, 803-826. doi:10.1007/s11336-012-9282-4
45.
YuanK.-H.ZhangZ. (2012b). Structural equation modeling diagnostics using R package semdiag and EQS. Structural Equation Modeling, 19, 683-702. doi:10.1080/10705511.2012.713282
46.
YuanK.-H.ZhongX. (2008). Outliers, leverage observations and influential cases in factor analysis: Minimizing their effect using robust procedures. Sociological Methodology, 38, 329-368. doi:10.1111/j.1467-9531.2008.00198.x
47.
ZinbargR. E.RevelleW.YovelI.LiW. (2005). Cronbach’s α, Revelle’s β, and McDonald’s ω: Their relations with each other and two alternative conceptualizations of reliability. Psychometrika, 70, 123-133. doi:10.1007/s11336-003-0974-7
