This article illustrates the Bayesian approximate measurement invariance (MI) approach in Mplus with longitudinal data and small sample size. Approximate MI incorporates zero-mean small variance prior distributions on the differences between parameter estimates over time. Contrary to traditional invariance testing methods, where exact invariance is tested, this method allows for some “wiggle room” in the parameter estimates over time. The procedure is illustrated using longitudinal data on college students’ academic stress as it changes in the period leading up to and right after an important midterm. Results show that traditional invariance testing methods come to a standstill due to the small sample size. Bayesian approximate MI testing was able to identify non-invariant parameters, after which a partially invariant model could be estimated.
AsparouhovT.MuthénB.MorinA. J. (2015). Bayesian structural equation modeling with cross-loadings and residual covariances: Comments on Stromeyer et al. Journal of Management, 41, 1561–1577. doi:10.1177/0149206315591075
5.
BayesT. (1764). An essay toward solving a problem in the doctrine of chances. Philosophical Transactions of the Royal Society of London, 53, 370–418.
6.
BentlerP. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107, 238–246.
7.
BentlerP.BonettD. G. (1980). Significance tests and goodness-of-fit in analysis of covariance structures. Psychological Bulletin, 88, 558–606. doi:10.1037/0033-2909.88.3.588
8.
BraekenJ.BlömekeS. (2016). Comparing future teachers’ beliefs across countries: Approximate measurement invariance with Bayesian elastic constraints for local item dependence and differential item functioning. Assessment & Evaluation in Higher Education, 41, 733–749. doi:10.1080/02602938.2016.1161005
9.
BrooksS. P.GelmanA. (1998). General methods for monitoring convergence of iterative simulations. Journal of Computational and Graphical Statistics, 7, 434–455.
10.
BujaczA.VittersøJ.HutaV.KaczmarekL. D. (2014). Measuring hedonia and eudaimonia as motives for activities: Cross-national investigation through traditional and Bayesian structural equation modeling. Frontiers in Psychology, 5, 1–10. doi:10.3389/fpsyg.2014.00984
11.
ByrneB. M.ShavelsonR. J.MuthénB. (1989). Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. Psychological Bulletin, 105, 456–466. doi:10.1037/0033-2909.105.3.456
12.
ChanD. K. C.IvarssonA.StenlingA.YangX. S.ChatzisarantisN. L. D. (2015). Response-order effects in survey methods: A randomized controlled crossover study in the context of sport injury prevention. Journal of Sport and Exercise Psychology, 37, 666–673. doi:10.1123/jsep.2015-0045
13.
ChenF. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 14, 464–504. doi:10.1080/10705510701301834
14.
CheungG. W.RensvoldR. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 9, 233–255. doi:10.1207/S15328007SEM0902_5
15.
ChiorriC.DayT.MalmbergL. E. (2014). An approximate measurement invariance approach to within-couple relationship quality. Frontiers in Psychology, 5. doi:10.3389/fpsyg.2014.00983
16.
CieciuchJ.DavidovE.AlgesheimerR.SchmidtP. (2018). Testing for approximate measurement invariance of human values in the European Social Survey. Sociological Methods & Research, 47, 665–686. doi:10.1177/0049124117701478
17.
CieciuchJ.DavidovE.SchmidtP.AlgesheimerR.SchwartzS. H. (2014). Comparing results of an exact vs. an approximate (Bayesian) measurement invariance test: A cross-country illustration with a scale to measure 19 human values. Frontiers in Psychology, 5, 982. doi:10.3389/fpsyg.2014.00982
18.
DavidovE.CieciuchJ.MeulemanB.SchmidtP.AlgesheimerR.HausherrM. (2015). The comparability of measurements of attitudes toward immigration in the European Social Survey: Exact versus approximate measurement equivalence. Public Opinion Quarterly, 79, 244–266. doi:10.1093/poq/nfv008
19.
DepaoliS.van de SchootR. (2017). Improving transparency and replication in Bayesian statistics: The WAMBS-Checklist. Psychological Methods, 22, 240–261. doi:10.1037/met0000065
20.
DesaD.van de VijverF.CarstenR.SchulzW. (2019). Measurement invariance in international large-scale assessments: Integrating theory and method. In JohnsonT. P.PennellB. E.StoopI.DorerB. (Eds.), Advances in comparative survey methodology (pp. 881–910). New York, NY: Wiley.
21.
EdossaA. K.SchroedersU.WeinertS.ArteltC. (2018). The development of emotional and behavioral self-regulation and their effects on academic achievement in childhood. International Journal of Behavioral Development, 42, 192–202. doi:10.1177/0165025416687412
22.
ElsworthG. R.BeauchampA.OsborneR. H. (2016). Measuring health literacy in community agencies: A Bayesian study of the factor structure and measurement invariance of the health literacy questionnaire (HLQ). BMC Health Services Research, 16, 508. doi:10.1186/s12913-016-1754-2
23.
FalkenströmF.HatcherR. L.SkjulsvikT.LarssonM. H.HolmqvistR. (2015). Development and validation of a 6-item working alliance questionnaire for repeated administrations during psychotherapy development and validation of a 6-item working alliance questionnaire for repeated administrations during psychotherapy. Psychological Assessment, 27, 169–183. doi:10.1037/pas0000038
24.
FerrerE.BalluerkaN.WidamanK. F. (2008). Factorial invariance and the specification of second-order latent growth models. Methodology, 4, 22–36. doi:10.1027/1614-2241.4.1.22
25.
FlakeJ. K.McCoachD. B. (2017). An investigation of the alignment method with polytomous indicators under conditions of partial measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 25, 56–70. doi:10.1080/10705511.2017.1374187
26.
FongT. C. T.HoR. T. H. (2014). Testing gender invariance of the Hospital Anxiety and Depression Scale using the classical approach and Bayesian approach. Quality of Life Research, 23, 1421–1426. doi:10.1007/s11136-013-0594-3
Garnier-VillarrealM.JorgensenT. D. (2019). Adapting fit indices for Bayesian structural equation modeling: Comparison to maximum likelihood. Psychological Methods. Epub ahead of print 10 June 2019. doi:10.1037/met0000224
29.
GeiserC.KellerB.LockhartG. (2013). First versus second order latent growth curve models: Some insights from latent state-trait theory. Structural Equation Modeling: A Multidisciplinary Journal, 20, 479–503. doi:10.1080/10705511.2013.797832
30.
GeisserS.EddyW. F. (1979). A predictive approach to model selection. Journal of the American Statistical Association, 74, 153–160. doi:10.1080/01621459.1979.10481632
31.
GelfandA. E.DeyD. K. (1994). Bayesian model choice: Asymptotics and exact calculations. Journal of the Royal Statistical Society. Series B (Methodological), 56, 501–514.
32.
GelfandA. E.SmithA. F. M. (1990). Sampling-based approaches to calculating marginal densities. Journal of the American Statistical Association, 85, 398–409.
33.
GelmanA.CarlinJ. B.SternH. S.DunsonD. B.VehtariA.RubinD. B. (2013). Bayesian data analysis (3rd ed.). New York, NY: Chapman & Hall/CRC.
34.
GelmanA.HwangJ.VehtariA. (2014). Understanding predictive information criteria for Bayesian models. Statistics and Computing, 24, 997–1016. doi:10.1007/s11222-013-9416-2
35.
GelmanA.MengX. L.SternH. (1996). Posterior predictive assessment of model fitness via realized discrepancies. Statistica Sinica, 6, 733–760.
36.
GrimmK. J.RamN. (2009). A second-order growth mixture model for developmental research. Research in Human Development, 6, 121–143. doi:10.1080/15427600902911221
37.
GucciardiD. F.ZhangC. Q.PonnusamyV.SiG.StenlingA. (2016). Cross-cultural invariance of the mental toughness inventory among Australian, Chinese, and Malaysian athletes: A Bayesian estimation approach. Journal of Sport & Exercise Psychology, 38, 187–202. doi:10.1123/jsep.2015-0320
38.
HastingsW. K. (1970). Monte Carlo sampling methods using Markov chains and their applications. Biometrika, 57, 97.
39.
HawesS. W.ByrdA. L.KelleyS. E.GonzalezR.EdensJ. F.PardiniD. A. (2018). Psychopathic features across development: Assessing longitudinal invariance among Caucasian and African American youths. Journal of Research in Personality, 73, 180–188. doi:10.1016/j.jrp.2018.02.003
40.
HeJ.KubackaK. (2015). Data comparability in the teaching and learning international survey (TALIS) 2008 and 2013. OECD Education Working Papers, No. 124, OECD Publishing, Paris, https://doi.org/10.1787/5jrp6fwtmhf2-en.
41.
HigdonJ. D. (2015). Measuring and modeling intercultural attitudes among adolescents across Europe: A multi-level, multiple-group analysis examining student attitudes, intergroup contact, and school climate. Doctoral dissertation, Harvard Graduate School of Education.
42.
HoijtinkH.van de SchootR. (2017). Testing small variance priors using prior-posterior predictive p values. Psychological Methods, 23, 561–569. doi:10.1037/met0000131
43.
HoofsH.van de SchootR.JansenN. W. H.KantI. (2018). Evaluating model fit in Bayesian confirmatory factor analysis with large samples: Simulation study introducing the BRMSEA. Educational and Psychological Measurement, 78, 537–568. doi:10.1177/0013164417709314
44.
JordalenG.LemyreP.SolstadB. E.IvarssonA. (2018). The role of self-control and motivation on exhaustion in youth athletes: A longitudinal perspective. Frontiers in Psychology, 9, 1–13. doi:10.3389/fpsyg.2018.02449
45.
JöreskogK. G. (1971). Simultaneous factor analysis in several populations. Psychometrika, 36, 409–426. doi:10.1007/BF02291366
46.
JorgensenT. D.Garnier-VillarrealM.PornprasermanitS.LeeJ. (2019). Small-variance priors can prevent detecting important misspecifications in Bayesian confirmatory factor analysis. In WibergM.CulpepperS.JanssenR.GonzálezJ.MolenaarD. (Eds.), Quantitative Psychology: IMPS 2017. Springer Proceedings in Mathematics & Statistics (pp. 255–263). Cham, Switzerland: Springer. doi:10.1007/978-3-030-01310-3_23
47.
KaplanD. (2014). Bayesian statistics for the social sciences. New York, NY: Guilford Press.
48.
KelceyB.McGinnD.HillH. (2014). Approximate measurement invariance in cross-classified rater-mediated assessments. Frontiers in Psychology, 5, 1–13. doi:10.3389/fpsyg.2014.01469
49.
KimE. S.CaoC.WangY.NguyenD. T. (2017). Measurement invariance testing with many groups: A comparison of five approaches. Structural Equation Modeling: A Multidisciplinary Journal, 24, 524–544. doi:10.1080/10705511.2017.1304822
50.
KruschkeJ. K. (2013). Bayesian estimation supersedes the t test. Journal of Experimental Psychology: General, 142, 573–603. doi:10.1037/a0029146
51.
LakaevN. (2009). Validation of an Australian academic stress questionnaire. Australian Journal of Guidance and Counselling, 19, 56–70. doi:10.1375/ajgc.19.1.56
52.
LaveniaM. (2016). Mathematics formative assessment system: Testing the theory of action based on the results of a randomized field trial. Tallahassee, FL: Florida State University.
53.
LiangX.YangY.HuangJ. (2018). Evaluation of structural relationships in autoregressive cross-lagged models under longitudinal approximate invariance: A Bayesian analysis. Structural Equation Modeling: A Multidisciplinary Journal, 25, 558–572. doi:10.1080/10705511.2017.1410706
54.
LiuY.WestS. G. (2018). Longitudinal measurement non-invariance with ordered-categorical indicators: How are the parameters in second-order latent linear growth models affected?Structural Equation Modeling: A Multidisciplinary Journal, 25, 762–777. doi:10.1080/10705511.2017.1419353
55.
LunnD.JacksonC.BestN.ThomasA.SpiegelhalterD. J. (2012). The BUGS book: A practical introduction to Bayesian analysis. New York, NY: Chapman and Hall/CRC.
MaccallumR. C.EdwardsM. C.CaiL. (2012). Hopes and cautions in implementing Bayesian structural equation modeling. Psychological Methods, 17, 340–345. doi:10.1037/a0027131
58.
MeadeA. W.JohnsonE. C.BraddyP. W. (2008). Power and sensitivity of alternative fit indices in tests of measurement invariance. The Journal of Applied Psychology, 93, 568–592. doi:10.5465/AMBPP.2006.27182124
59.
MeadeA. W.LautenschlagerG. J.HechtJ. E. (2005). Establishing measurement equivalence and invariance in longitudinal data with item response theory. International Journal of Testing, 5, 279–300. doi:10.1207/s15327574ijt0503
60.
MengX. L. (1994). Posterior predictive p values. The Annals of Statistics, 22, 1142–1160.
61.
MeredithW. (1964a). Notes on factorial invariance. Psychometrika, 29, 177–185.
62.
MeredithW. (1964b). Rotation to achieve factorial invariance. Psychometrika, 29, 187–206.
MerkleE. C.RosseelY. (2018). Blavaan: Bayesian structural equation models via parameter expansion. Journal of Statistical Software, 85, 1–30. doi:10.18637/jss.v085.i04
65.
MetropolisN.RosenbluthA. W.RosenbluthM. N.TellerA. H.TellerE. (1953). Equation of state calculations by fact computing machines. Journal of Chemical Physics, 21, 1087–1092.
66.
MeulemanB. (2012). When are item intercept differences substantively relevant in measurement invariance testing? In SalzbornS.DavidovE.ReineckeJ. (Eds.), Methods, theories, and empirical applications in the social sciences: Festschrift for Peter Schmidt (pp. 97–104). Heidelberg: Springer VS. doi:10.1007/978-3-531-18898-0
MuthénL. K.MuthénB. O. (2017). Mplus user’s guide (8th ed.). Los Angeles, CA: Muthén & Muthén.
69.
PokropekA.DavidovE.SchmidtP. (2019). A Monte Carlo simulation study to assess the appropriateness of traditional and newer approaches to test for measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 26, 724–744. doi:10.1080/10705511.2018.1561293
70.
PriceR. (1765). A demonstration of the second rule in the essay toward the solution of a problem in the doctrine of chances. Philosophical Transactions of the Royal Society of London, 54, 296–325.
71.
RammstedtB.DannerD.SotoC. J.JohnO. P. (2018). Validation of the short and extra-short forms of the Big Five Inventory-2 (BFI-2) and their German adaptations. European Journal of Psychological Assessment. Epub ahead of print 3 August 2018. doi:10.1027/1015-5759/a000481
72.
RhemtullaM.Brosseau-LiardP. É.SavaleiV. (2012). When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods, 17, 354–373. doi:10.1037/a0029315.supp
73.
RobertC.CasellaG. (2011). A short history of Markov chain Monte Carlo: Subjective recollections from incomplete data. Statistical Science, 26, 102–115. doi:10.1214/10-STS351
74.
RummelJ.SteindorfL.MarevicI.DannerD. (2017). A validation study of the German complex-span tasks and some general considerations on task translation procedures in cognitive psychology. European Journal of Psychological Assessment, 1–12. Epub ahead of print 15 December 2017. doi:10.1027/1015-5759/a000444
75.
RutkowskiL.SvetinaD. (2014). Assessing the hypothesis of measurement invariance in the context of large-scale international surveys. Educational and Psychological Measurement, 74, 31–57. doi:10.1177/0013164413498257
76.
ScheinesR.HoijtinkH.BoomsmaA. (1999). Bayesian estimation and testing of structural equation models. Psychometrika, 64, 37–52.
77.
SeddigD.LeitgöbH. (2018). Approximate measurement invariance and longitudinal confirmatory factor analysis: Concept and application with panel data. Survey Research Methods, 12, 29–41. doi:10.18148/srm/2018.v12i1.7210
SokolovB. (2018). The index of emancipative values: Measurement model misspecifications. American Political Science Review, 112, 395–408. doi:10.1017/S0003055417000624
80.
SolstadB. E. (2017). Towards a better understanding of the dynamics of sports coaching at the youth level: The coach’s perspective. Oslo: Norwegian School of Sport Sciences.
81.
SpiegelhalterD. J.BestN. G.CarlinB. P.van der LindeA. (2002). Bayesian measures of model complexity and fit. Journal of the Royal Statistical Society. Series B (Statistical Methodology), 64, 583–639.
82.
Stan Development Team (2018). RStan: The R interface to Stan. R package version 2.18.2. Retrieved fromhttp://mc-stan.org/
83.
SvetinaD.RutkowskiL.RutkowskiD. (2019). Multiple-group invariance with categorical outcomes using updated guidelines: An illustration using Mplus and the lavaan/semtools packages. Structural Equation Modeling: A Multidisciplinary Journal. Epub ahead of print 29 April 2019. doi:10.1080/10705511.2019.1602776
84.
VandenbergR. J.LanceC. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4–70. doi:10.1177/109442810031002
85.
van de SchootR.DepaoliS. (2014). Bayesian analyses: Where to start and what to report. The European Health Psychologist, 16, 75–84.
86.
van de SchootR.KluytmansA.TummersL.LugtigP.HoxJ.MuthénB. (2013). Facing off with Scylla and Charybdis: A comparison of scalar, partial, and the novel possibility of approximate measurement invariance. Frontiers in Psychology, 4, 1–15. doi:10.3389/fpsyg.2013.00770
87.
van de SchootR.WinterS. D.RyanO.Zondervan-ZwijnenburgM.DepaoliS. (2017). A systematic review of Bayesian articles in psychology: The last 25 years. Psychological Methods, 22, 217–239.
88.
VehtariA.GelmanA.GabryJ. (2017). Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC. Statistics and Computing, 27, 1413–1432. doi:10.1007/s11222-016-9696-4
89.
Von SuchodoletzA.FäscheA.SkuballaI. T. (2017). The role of attention shifting in orthographic competencies: Cross-sectional findings from 1st, 3rd, and 8th grade students. Frontiers in Psychology, 8, 1665. doi:10.3389/fpsyg.2017.01665
90.
WatanabeS. (2010). Asymptotic equivalence of Bayes cross validation and widely applicable information criterion in singular learning theory. Journal of Machine Learning Research, 11, 3571–3594.
91.
WilliamsB. D.ChandolaT.PendletonN. (2018). An application of Bayesian measurement invariance to modelling cognition over time in the English Longitudinal Study of Ageing. International Journal of Methods in Psychiatric Research, 27, e1749. doi:10.1002/mpr.1749
92.
WinterS. D.DepaoliS. (in press). Measurement Invariance of Continuous and Dichotomous Indicators in Second-Order Latent Growth Models: Assessing Frequentist and Bayesian Approaches.
93.
YuanK. H.ChanW. (2016). Measurement invariance via multigroup SEM: Issues and solutions with chi-square-difference tests. Psychological Methods, 21, 405. doi:10.1037/met0000080
94.
ZercherF.SchmidtP.CieciuchJ.DavidovE. (2015). The comparability of the universalism value over time and across countries in the European Social Survey: Exact vs. approximate measurement invariance. Frontiers in Psychology, 6, 1–11. doi:10.3389/fpsyg.2015.00733
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
