The maximal reliability of a congeneric measure is achieved by weighting item scores to form the optimal linear combination as the total score; it is never lower than the composite reliability of the measure when measurement errors are uncorrelated. The statistical method that renders maximal reliability would also lead to maximal criterion validity. Using a career satisfaction measure as an example, the present article calculated the maximal reliability and maximal criterion validity and compared them with the composite reliability and the scale criterion validity, respectively. The improvement of reliability and validity indicated that the optimal linear combination is preferred when forming a total score of a measure. The Mplus codes for analyzing maximal reliability, maximal criterion validity, and related parameters are provided.
AnastasiA. (1988). Psychological testing (6th ed.). New York, NY: Macmillan.
2.
Armstrong-StassenD. M.UrselN. D. (2009). Perceived organizational support, career satisfaction, and the retention of older workers. Journal of Occupational and Organizational Psychology, 82, 201-220.
3.
BakkerA.DemeroutiE.TarisT. W.SchaufeliW. B.SchreursP. J. (2003). A multigroup analysis of the job demands-resources model in four home care organizations. International Journal of Stress Management, 10, 16-38.
4.
BartholomewD. J. (1996). The statistical approach to social measurement. San Diego, CA: Academic Press.
5.
BrunnerM.SübH.-M. (2005). Analyzing the reliability of multidimensional measures: An example from intelligence research. Educational and Psychological Measurement, 65, 227-240.
6.
CohenR. J.SwerdlikM. E. (2009). Psychological testing and assessment: An introduction to tests and measurement (7th ed.). New York, NY: McGraw-Hill Primis.
7.
DimitrovD. M.RaykovT.AlqataeeA. A. (2015). Developing a measure of general academic ability: An application of maximal reliability and optimal linear combination to high school students’ scores. Educational and Psychological Measurement, 75, 475-490.
8.
EbyL. T.ButtsM.LockwoodA. (2003). Predictors of success in the era of the boundaryless career. Journal of Organizational Behavior, 24, 689-708.
9.
FinneyS. J.DiStefanoC. (2006). Non-normal and categorical data in structural equation modeling. In HancockG. R.MuellerR. O. (Eds.), Structural equation modeling: A second course (pp. 269-314). Charlotte, NC: Information Age.
10.
GablerS.RaykovT. (2016). Evaluation of maximal reliability for unidimensional measuring instruments with correlated errors. Structural Equation Modeling, 24, 104-111. doi:10.1080/10705511.2016.1159916
11.
GreenhausJ. H.ParasuramanS.WormleyW. M. (1990). Effects of race on organizational experiences, job performance evaluations, and career outcomes. Academy of Management Journal, 33, 64-86.
12.
GuH.WenZ.FanX. (2017). Examining and controlling for wording effect in a self-report measure: A Monte Carlo simulation study. Structural Equation Modeling, 24, 545-555. doi:10.1080/10705511.2017.1286228
13.
GuilfordJ. P. (1954). Psychometric methods. New York, NY: McGraw-Hill.
14.
HuL. T.BentlerP. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55. doi:10.1080/10705519909540118
15.
JöreskogK. G. (1971). Statistical analysis of sets of congeneric tests. Psychometrika, 36, 109-133. doi:10.1007/BF02291393
16.
KaratepeO. M. (2012). Perceived organizational support, career satisfaction, and performance outcomes. International Journal of Contemporary Hospitality Management, 24, 735-752.
17.
KaratepeO. M.OlugbadeO. A. (2016). The effects of work social support and career adaptability on career satisfaction and turnover intentions. Journal of Management & Organization, 23, 337-355.
18.
KongH. Y.CheungC.SongH. Y. (2012). From hotel career management to employees’ career satisfaction: The mediating effect of career competency. International Journal of Hospitality Management, 31(1), 76-85.
19.
LiH. (1997). A unifying expression for the maximal reliability of a linear composite. Psychometrika, 62, 245-249.
20.
LiH. (2003). The resolution of some paradoxes related to reliability and validity. Journal of Educational and Behavioral Statistics, 28, 89-95.
21.
LiH.RosenthalR.RubinD. B. (1996). Reliability of measurement in psychology: From Spearman-Brown to maximal reliability. Psychological Methods, 1, 98-107.
22.
MarcoulidesK. M.YuanK. H. (2016). New ways to evaluate goodness of fit: A note on using equivalence testing to assess structural equation models. Structural Equation Modeling, 24, 148-153.
23.
MarshH. W.HauK.-T.WenZ. (2004). In search of golden rules: Comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler’s (1999) findings. Structural Equation Modeling, 11, 320-341.
24.
McDonaldR. P. (1970). The theoretical foundations of principal factor analysis, canonical factor analysis, and alpha factor analysis. British Journal of Mathematical and Statistical Psychology, 23(1), 1-21.
25.
McDonaldR. P. (1999). Test theory. A unified treatment. Mahwah, NJ: Erlbaum.
26.
MosierC. I. (1943). On the reliability of a weighted composite. Psychometrika, 8, 161-168.
27.
MuthénL. K.MuthénB. O. (1998-2015). Mplus use’s guide (7th ed.). Los Angeles, CA: Muthén & Muthén.
28.
PenevS.RaykovT. (2006). On the relationship between maximal reliability and maximal validity of linear composites. Multivariate Behavioral Research, 41, 105-126.
29.
RaykovT. (2004). Estimation of maximal reliability: a note on a covariance structure modelling approach. British Journal of Mathematical and Statistical Psychology, 57, 21–27.
30.
RaykovT.GablerS.DimitrovD. M. (2015). Maximal reliability and composite reliability: Examining their difference for multicomponent measuring instruments using latent variable modeling. Structural Equation Modeling, 23, 384-391. doi:10.1080/10705511.2014.966369
31.
RaykovT.GablerS.DimitrovD. M. (2016). Maximal criterion validity and scale criterion validity: A latent variable modeling approach for examining their difference. Structural Equation Modeling, 23, 544-554. doi:10.1080/10705511.2016.1155414
32.
RaykovT.GraysonD. (2003). A test for change of composite reliability in scale development. Multivariate Behavioral Research, 38, 143-159. doi:10.1207/s15327906mbr3802_1
33.
RaykovT.MarcoulidesG. A.GablerS. (2017). Improved estimation of maximal reliability for unidimensional multicomponent measuring instruments in repeated measure studies. Structural Equation Modeling, 24, 755-767. doi:10.1080/10705511.2016.1183491
34.
RodriguezA.ReiseS. P.HavilandM. G. (2015). Evaluating bifactor models: Calculating and interpreting statistical indices. Psychological Methods, 21, 137-150.
35.
WestS. G.FinchJ. F.CurranP. J. (1995). Structural equation models with nonnormal variables: Problems and remedies. In HoyleR. H. (Ed.), Structural equation modeling: Concepts, issues, and applications (pp. 56-75). Thousand Oaks, CA: Sage.
36.
WilloughbyM. T.PekJ.BlairC. B. (2013). Measuring executive function in early childhood: A focus on maximal reliability and the derivation of short forms. Psychological Assessment, 25, 664-670.
37.
YuanK. H.ChanW.MarcoulidesG. A.BentlerP. M. (2015). Assessing structural equation models by equivalence testing with adjusted fit indexes. Structural Equation Modeling, 23, 319-330.
