This article reviews and compares recently proposed factor analytic and item response theory approaches to the study of invariance across groups. Two methods are described and contrasted. The alignment method considers the groups as a fixed mode of variation, while the random-intercept, random-loading two-level method considers the groups as a random mode of variation. Both maximum likelihood and Bayesian analyses are applied. A survey of close to 50,000 subjects in 26 countries is used as an illustration. In addition, the two methods are studied by Monte Carlo simulations. A list of considerations for choosing between the two methods is presented.
AsparouhovT.MuthénB.. 2015. “General random effect latent variable modeling: Random subjects, items, contexts, and parameters.” in Advances in multilevel modeling for educational research: Addressing practical issues found in real-world applications, edited by HarringJ. R.StapletonL. M.BeretvasS. N.. Charlotte, NC: Information Age Publishing, Inc.
4.
BeierleinC.DavidovE.SchmidtP.SchwartzS. H.. 2012. “Testing the Discriminant Validity of Schwartz’ Portrait Value Questionnaire Items—A Replication and Extension of Knoppen and Saris (2009).” Survey Research Methods6:25–36.
5.
BouJ. C.SatorraA.. 2010. “A Multigroup Structural Equation Approach: A Demonstration by Testing Variation of Firm Profitability across EU Samples.” Organizational Research Methods13:738–66. First published on January 26, 2010.
6.
CronbachL. J.1976. “Research on Classrooms and Schools: Formulation of Questions, Design, and Analysis.” Unpublished manuscript, Stanford University, Stanford Evaluation Consortium, School of Education, Stanford, CA.
7.
De BoeckP.2008. “Random Item IRT Models.” Psychometrika73:533–59.
8.
de JongM. G.SteenkampJ. B. E. M.. 2010. “Finite Mixture Multilevel Multidimensional Ordinal IRT Models for Large-scale Cross-cultural Research.” Psychometrika75:3–32.
9.
De JongM. G.SteenkampJ. B. E. M.FoxJ. P.. 2007. “Relaxing Measurement Invariance in Cross-national Consumer Research Using a Hierarchical IRT Model.” Journal of Consumer Research34:260–78.
10.
FoxJ. P.2005. “Multilevel IRT Using Dichotomous and Polytomous Response Data.” British Journal of Mathematical and Statistical Psychology58:145–72.
11.
FoxJ. P.2010. Bayesian Item Response Theory. New York: Springer.
12.
FoxJ. P.GlasC. A. W.. 2001. “Bayesian Estimation of a Multilevel IRT Model Using Gibbs.” Psychometrika66:269–86.
13.
FoxJ. P.VerhagenJ.. 2011. “Random Item Effects Modeling for Cross-national Survey Data.” Pp. 461–82 in Cross-cultural Analysis: Methods and Applications, edited by DavidovE.SchmidtP.BillietJ.. New York: Routledge.
14.
FrederickxS.TuerlinckxF.De BoeckP.MagisD.. 2010. “RIM: A Random Item Mixture Model to Detect Differential Item Functioning.” Journal of Educational Measurement47:432–57.
15.
GoldsteinH.McDonaldR. P.. 1988. “A General Model for the Analysis of Multilevel Data.” Psychometrika53:455–67.
16.
HärnqvistK.1978. “Primary Mental Abilities of Collective and Individual Levels.” Journal of Educational Psychology70:706–16.
17.
HärnqvistK.GustafssonJ. E.MuthénB.NelsonG.. 1994. “Hierarchical Models of Ability at Class and Individual Levels.” Intelligence18:165–87.
18.
JakS.OortF. J.DolanC. V.. 2013. “A Test for Cluster Bias: Detecting Violations of Measurement Invariance across Clusters in Multilevel Data.” Structural Equation Modeling20:265–82.
19.
JakS.OortF. J.DolanC. V.. 2014. “Measurement Bias in Multilevel Data.” Structural Equation Modeling: A Multidisciplinary Journal21:31–39. doi:10.1080/10705511.2014.856694.
20.
JennrichR. I.2006. “Rotation to Simple Loadings Using Component Loss Functions: The Oblique Case.” Psychometrika71:173–91.
21.
KimE. S.YoonM.. 2011. “Testing Measurement Invariance: A Comparison of Multiple-group Categorical CFA and IRT.” Structural Equation Modeling18:212–28.
22.
LeeJ.LittleT. D.PreacherK. J.. 2010. “Methodological Issues in Using Structural Equation Models for Testing Differential Item Functioning.” Pp.55–84 in Cross-cultural Analysis. Methods and Applications, edited by DavidovE.SchmidtP.BillietJ.. New York: Routledge.
23.
LongfordN. T.MuthénB.. 1992. “Factor Analysis for Clustered Observations.” Psychometrika57:581–97.
24.
McDonaldR. P.GoldsteinH.. 1989. “Balanced versus Unbalanced Designs for Linear Structural Relations in Two-level Data.” British Journal of Mathematical and Statistical Psychology42:215–32.
25.
MillsapR. E.2011. Statistical Approaches to Measurement Invariance. New York: Taylor & Francis.
26.
MuthénB.1994. “Multilevel Covariance Structure Analysis.” Pp. 376–98, Vol. 22 in Multilevel Modeling, A Special Issue of Sociological Methods & Research, edited by HoxJ.KreftI.. Thousand Oakes, CA: Sage Publications.
StarkS.ChernyshenkoO. S.DrasgowF.. 2006. “Detecting Differential Item Functioning with Confirmatory Factor Analysis and Item Response Theory: Toward a Unified Strategy.” Journal of Applied Psychology91:1292–306.
29.
ThissenD.SteinbergL.WainerH.. 1993. “Detection of Differential Item Functioning Using the Parameters of Item Response Models.” Pp. 67–113 in Differential Item Functioning, edited by HollandP. W.WainerH.. Hillsdale, NJ: Lawrence Erlbaum.
30.
VerhagenA. J.2013. “Bayesian Item Response Theory Models for Measurement Invariance.” Doctoral dissertation, University of Twente, the Netherlands.
31.
VerhagenA. J.FoxJ. P.. 2013. “Bayesian Tests of Measurement Invariance.” The British Journal of Mathematical and Statistical Psychology66:383–401.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
