Composite reliability, or coefficient omega, can be estimated using structural equation modeling. Composite reliability is usually estimated under the basic independent clusters model of confirmatory factor analysis (ICM-CFA). However, due to the existence of cross-loadings, the model fit of the exploratory structural equation model (ESEM) is often found to be substantially better than that of ICM-CFA. The present study first illustrated the method used to estimate composite reliability under ESEM and then compared the difference between ESEM and ICM-CFA in terms of composite reliability estimation under various indicators per factor, target factor loadings, cross-loadings, and sample sizes. The results showed no apparent difference in using ESEM or ICM-CFA for estimating composite reliability, and the rotation type did not affect the composite reliability estimates generated by ESEM. An empirical example was given as further proof of the results of the simulation studies. Based on the present study, we suggest that if the model fit of ESEM (regardless of the utilized rotation criteria) is acceptable but that of ICM-CFA is not, the composite reliability estimates based on the above two models should be similar. If the target factor loadings are relatively small, researchers should increase the number of indicators per factor or increase the sample size.
BandalosD. L. (2002). The effects of item parceling on goodness-of-fit and parameter estimate bias in structural equation modeling. Structural Equation Modeling, 9(1), 78-102. https://doi.org/10.1207/S15328007SEM0901_5
3.
BentlerP. M. (2009). Alpha, dimension-free, and model-based internal consistency reliability. Psychometrika, 74(1), 137-143. https://doi.org/10.1007/s11336-008-9100-1
4.
BoomsmaA. (1982). On the robustness of LISREL against small sample size in facto analysis models. North-Holland.
5.
ChenB. B.WiiumN.DimitrovaR. (2018). Factor structure of positive youth development: Contributions of exploratory structural equation modeling. Personality and Individual Differences, 124, 12-15. https://doi.org/10.1016/j.paid.2017.11.039
6.
GrahamJ. M. (2006). Congeneric and (essentially) tau-equivalent estimates of score reliability: What they are and how to use them. Educational and Psychological Measurement, 66(6), 930-944. https://doi.org/10.1177/0013164406288165
GuH.WenZ.FanX. (2017). Examining and controlling for wording effect in a self-report measure: A Monte Carlo simulation study. Structural Equation Modeling, 24(4), 1-11. https://doi.org/10.1080/10705511.2017.1286228
9.
HoganT. P.BenjaminA.BrezinskiK. L. (2000). Reliability methods: A note on the frequency of use of various types. Educational and Psychological Measurement, 60(4), 523-531. https://doi.org/10.1177/00131640021970691
10.
HooglandJ. J.BoomsmaA. (1998). Robustness studies in covariance structure modeling: An overview and a meta-analysis. Sociological Methods & Research, 26(3), 329-368. https://doi.org/10.1177/0049124198026003003
11.
HuL. T.BentlerP. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1-55. https://doi.org/10.1080/10705519909540118
12.
JöreskogK. G. (1971). Statistical analysis of sets of congeneric tests. Psychometrika, 36(2), 109-133. https://doi.org/10.1007/BF02291393
13.
LordF. M.NovickM. R. (1968). Statistical theories of mental test scores. Addison-Wesley.
14.
MarshH. W. (2007). Application of confirmatory factor analysis and structural equation modeling in sport/exercise psychology. In TenenbaumG.EklundR. C. (Eds.), Handbook of sport psychology (3rd ed., pp. 774-798). Wiley.
15.
MarshH. W.HauK. T.WenZ. (2004). In search of golden rules: Comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler’s (1999) findings. Structural Equation Modeling, 11(3), 320-341. https://doi.org/10.1207/s15328007sem1103_2
16.
MarshH. W.LüdtkeO.MuthénB.AsparouhovT.MorinA. J. S.TrautweinU.NagengastB. (2010). A new look at the big five factor structure through exploratory structural equation modeling. Psychological Assessment, 22(3), 471-491. https://doi.org/10.1037/a0019227
17.
MarshH. W.LüdtkeO.NagengastB.MorinA. J. S.Von DavierM. (2013). Why item parcels are (almost) never appropriate: Two wrongs do not make a right: Camouflaging misspecification with item parcels in CFA models. Psychological Methods, 18(3), 257-284. https://doi.org/10.1037/a0032773
18.
MarshH. W.MorinA. J.ParkerP. D.KaurG. (2014). Exploratory structural equation modeling: An integration of the best features of exploratory and confirmatory factor analysis. Annual Review of Clinical Psychology, 10, 85-110. https://doi.org/10.1146/annurev-clinpsy-032813-153700
19.
MarshH. W.MuthénB.AsparouhovT.LüdtkeO.RobitzschA.MorinA. J. S.TrautweinU. (2009). Exploratory structural equation modeling, integrating CFA and EFA: Application to students’ evaluations of university teaching. Structural Equation Modeling, 16(3), 439-476. https://doi.org/10.1080/10705510903008220
20.
McDonaldR. P. (1985). Factor analysis and related methods. Lawrence Erlbaum.
21.
MorinA. J. S.MarshH. W.NagengastB. (2013). Exploratory structural equation modeling: An introduction. In HancockG. R.MuellerR. O. (Eds.), Structural equation modeling: A second course (2nd ed., pp. 395-436). IAP.
22.
MuthénB.AsparouhovT. (2012). Bayesian structural equation modeling: A more flexible representation of substantive theory. Psychological Methods, 17(3), 313-335. https://doi.org/10.1037/a0026802
23.
NovickM. R.LewisC. (1967). Coefficient alpha and the reliability of composite measurements. Psychometrika, 32(1), 1-13. https://doi.org/10.1007/BF02289400
24.
RaykovT. (1997). Estimation of composite reliability for congeneric measures. Applied Psychological Measurement, 21(2), 173-184. https://doi.org/10.1177/01466216970212006
25.
RaykovT. (1998). A method for obtaining standard errors and confidence intervals of composite reliability for congeneric items. Applied Psychological Measurement, 22(4), 369-374. https://doi.org/10.1177/014662169802200406
26.
RaykovT. (2002). Analytic estimation of standard error and confidence interval for scale reliability. Multivariate Behavioral Research, 37(1), 89-103. https://doi.org/10.1207/S15327906MBR3701_04
27.
RaykovT.MarcoulidesG. A. (2011). Introduction to psychometric theory. Routledge.
28.
RaykovT.MarcoulidesG. A. (2019). Thanks coefficient alpha, we still need you!Educational and Psychological Measurement, 79(1), 200-210. https://doi.org/10.1177/0013164417725127
29.
RaykovT.ShroutP. E. (2002). Reliability of scales with general structure: Point and interval estimation using a structural equation modeling approach. Structural Equation Modeling, 9(2), 195-212. https://doi.org/10.1207/S15328007SEM0902_3
30.
ReiseS. P.ScheinesR.WidamanK. F.HavilandM. G. (2013). Multidimensionality and structural coefficient bias in structural equation modeling: A bifactor perspective. Educational and Psychological Measurement, 73(1), 5-26. https://doi.org/10.1177/0013164412449831
31.
SassD. A.SchmittT. A. (2010). A comparative investigation of rotation criteria within exploratory factor analysis. Multivariate Behavior Research, 45(1), 1-33. https://doi.org/10.1080/00273170903504810
32.
SchmittT. A.SassD. A. (2011). Rotation criteria and hypothesis testing for exploratory factor analysis: Implications for factor pattern loadings and interfactor correlations. Educational and Psychological Measurement, 71(1), 95-113. https://doi.org/10.1177/0013164410387348
33.
SijtsmaK. (2009). On the use, the misuse, and the very limited usefulness of Cronbach’s alpha. Psychometrika, 74(1), 107-120. https://doi.org/10.1007/s11336-008-9101-0
34.
Tóth-KirályI.BőtheB.RigóA.OroszG. (2017). An illustration of the exploratory structural equation modeling (ESEM) framework on the Passion Scale. Frontiers in Psychology, 8, Article 1968. https://doi.org/10.3389/fpsyg.2017.01968
35.
Tóth-KirályI.MorinA. J. S.BőtheB.OroszG.RigóA. (2018). Investigating the multidimensionality of need fulfillment: A bifactor exploratory structural equation modeling representation. Structural Equation Modeling, 25(2), 1-20. https://doi.org/10.1080/10705511.2017.1374867
36.
WangQ.PomerantzE. M.ChenH. (2007). The role of parents’ control in early adolescents’ psychological functioning: A longitudinal investigation in the United States and China. Child Development, 78(5), 1592-1601. https://doi.org/10.1111/j.1467-8624.2007.01085.x
37.
XiaoY.LiuH.HauK. T. (2019). A comparison of CFA, ESEM, and BSEM in test structure analysis. Structural Equation Modeling, 26(5), 665-677. https://doi.org/10.1080/10705511.2018.1562928
38.
YangY.GreenS. B. (2010). A note on structural equation modeling estimates of reliability. Structural Equation Modeling, 17(1), 66-81. https://doi.org/10.1080/10705510903438963
ZimmermanD. W.ZumboB. D.LalondeC. (1993). Coefficient alpha as an estimate of test reliability under violation of two assumptions. Educational and Psychological Measurement, 53(1), 33-49. https://doi.org/10.1177/0013164493053001003
41.
ZinbargR. E.RevelleW.YovelI.LiW. (2005). Cronbach’s α, Revelle’s β, and McDonald’s ωh: Their relations with each other and two alternative conceptualizations of reliability. Psychometrika, 70, 123-133. https://doi.org/10.1007/s11336-003-0974-7
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
