A multilevel latent class model (MLCM) is a useful tool for analyzing data arising from hierarchically nested structures. One important issue for MLCMs is determining the minimum sample sizes needed to obtain reliable and unbiased results. In this simulation study, the sample sizes required for MLCMs were investigated under various conditions. A series of design factors, including sample sizes at two levels, the distinctness and the complexity of the latent structure, and the number of indicators were manipulated. The results revealed that larger samples are required when the latent classes are less distinct and more complex with fewer indicators. This study also provides recommendations about the minimum required sample sizes that satisfied all four criteria—model selection accuracy, parameter estimation bias, standard error bias, and coverage rate—as well as rules of thumb for sample size requirements when applying MLCMs in data analysis.
AitkinM. (1999). A general maximum likelihood analysis of variance components in generalized linear models. Biometrics, 55, 117-128.
2.
AsparouhovT.MuthénB. (2008). Multilevel mixture models. In HancockG. R.SamuelsonK. M. (Eds.), Advances in latent variable mixture models (pp. 27-51). Charlotte, NC: Information Age.
3.
AustinP. C. (2010). Estimating multilevel logistic regression models when the number of clusters is low: A comparison of different statistical software procedures. International Journal of Biostatistics, 6, Article 16.
4.
BaldwinS. A.FellinghamG. W. (2013). Bayesian methods for the analysis of small sample multilevel data with a complex variance structure. Psychological Methods, 18, 151-164.
5.
BartolucciF.PennoniF.VittadiniG. (2011). Assessment of school performance through a multilevel latent Markov Rasch model. Journal of Educational and Behavioral Statistics, 36, 491-522.
6.
BassiF. (2009). Latent class models for marketing strategies: An application to the Italian pharmaceutical market. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 5, 40-45.
7.
BijmoltT. H. A.PaasL. J.VermuntJ. K. (2004). Country and consumer segmentation: Multi-level latent class analysis of financial product ownership. International Journal of Research in Marketing, 21, 323-334.
8.
BrowneW. J.DraperD. (2006). A comparison of Bayesian and likelihood-based methods for fitting multilevel models. Bayesian Analysis, 1, 473-514.
9.
BrykA. S.RaudenbushS. W. (1992). Application of hierarchical linear models to assessing change. Psychological Bulletin, 101, 147-158.
10.
ChungH.AnthonyJ. C.SchaferJ. L. (2011). Latent class profile analysis: An application to stage sequential processes in early onset drinking behaviours. Journal of the Royal Statistical Society, Series A, 174, 689-712.
11.
CohenM. P. (1998). Determining sample sizes for surveys with data analyzed by hierarchical linear models. Journal of Official Statistics, 14, 267-275.
12.
da CostaL. P.DiasJ. G. (2014). What do Europeans believe to be the causes of poverty? A multilevel analysis of heterogeneity within and between countries. Social Indicators Research, 122, 1-20.
13.
DiC. Z.Bandeen-RocheK. (2011). Multilevel latent class models with dirichlet mixing distribution. Biometrics, 67, 86-96.
14.
DziakJ. J.LanzaS. T.TanX. (2014). Effect size, statistical power and sample size requirements for the bootstrap likelihood ratio test in latent class analysis. Structural Equation Modeling, 21, 534-552. doi:10.1080/10705511.2014.919819
15.
FinchW. H.FrenchB. F. (2013). Multilevel latent class analysis: Parametric and nonparametric models. Journal of Experimental Education, 82, 307-333.
16.
FinchW. H.MarchantG. J. (2013). Application of multilevel latent class analysis to identify achievement and socio-economic typologies in the 20 wealthiest countries. Journal of Educational and Developmental Psychology, 3, 201-221.
17.
GoldsteinH. (1995). Multilevel statistical models. New York, NY: Halsted.
18.
GoodmanL. A. (1974). The analysis of systems of qualitative variables when some of the variables are unobservable. Part I: A modified latent structure approach. American Journal of Sociology, 79, 1179-1259.
19.
GudichaD. W.SchmittmannV. D.VermuntJ. K. (2016). Power computation for likelihood ratio tests for the transition parameters in latent Marko models. Structural Equation Modeling, 23, 234-245.
20.
GudichaD. W.TekleF. B.VermuntJ. K. (2016). Power and sample size computation for Wald tests in latent class models. Journal of Classification, 33, 30-51.
21.
HedekerD. (2003). A mixed-effects multinomial logistic regression model. Statistics in Medicine, 22, 1433-1446.
22.
HenryK. L.MuthénB. (2010). Multilevel latent class analysis: An application of adolescent smoking typologies with individual and contextual predictors. Structural Equation Modeling, 17, 193-215.
23.
HoxJ. J. (2010). Multilevel analysis. Techniques and applications (2nd ed.). New York, NY: Routledge.
24.
HensonJ. M.ReiseS. P.KimK. H. (2007). Detecting mixtures from structural model differences using latent variable mixture modeling: A comparison of relative model fit statistics. Structural Equation Modeling, 14, 202-226.
25.
KaplanD.KellerB. (2011). A note on cluster effects in latent class analysis. Structural Equation Modeling, 18, 525-536.
26.
KaplanD.KimJ.-S.KimS.-Y. (2009). Multilevel latent variable modeling: Current research and recent developments. In MillsapR. E.Maydeu-OlivaresA. (Eds.), Handbook of quantitative methods in psychology (pp. 592-612). Thousand Oaks, CA: Sage.
27.
KimS.-Y. (2012). Sample size requirements in single- and multiphase growth mixture models: A Monte Carlo simulation study. Structural Equation Modeling, 19, 457-476. doi:10.1080/10705511.2012.687672
28.
KreftI. G. G. (1996). Are multilevel techniques necessary? An overview, including simulation studies. Unpublished manuscript, California State University, Los Angeles.
29.
LairdN. M. (1978). Nonparametric maximum likelihood estimation of a mixing distribution. Journal of the American Statistical Association, 73, 805-811.
LukočienéO.VarrialeR.VermuntJ. K. (2010). The simultaneous decision(s) about the number of lower- and higher-level classes in multilevel latent class analysis. Sociological Methodology, 40, 247-283.
32.
LukočienéO.VermuntJ. K. (2010). Determining the number of components in mixture models for hierarchical data. In FinkA.BertholdL.SeidelW.UltschA. (Eds.), Advances in data analysis, data handling and business intelligence (pp. 241-249). Berlin-Heidelberg, Germany: Springer.
33.
MaasC. J. M.HoxJ. J. (2005). Sufficient sample sizes for multilevel modeling. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 1, 85-91.
34.
McLachlanG.PeelD. (2000). Finite mixture models. New York, NY: Wiley.
35.
McNeishD.HarringJ. R. (2017). Clustered data with small sample sizes: Comparing the performance of model-based and design-based approaches. Communications in Statistics: Simulation and Computation, 46, 855-869.
36.
McNeishD.StapletonL. M. (2016). The effect of small sample size on two level model estimates: A review and illustration. Educational Psychology Review, 28, 295-314.
37.
MoineddinR.MathesonF. I.GlazierR. H. (2007). A simulation study on the sample size for multilevel logistic regression models. BMC Medical Research Methodology, 7, 1-10.
38.
MuthénL. K.MuthénB. O. (2002). How to use a Monte Carlo study to decide on sample size and determine power. Structural Equation Modeling, 9, 599-620.
39.
NylundK. L.AsparouhovT.MuthénB. O. (2007). Deciding on the number of classes in latent class analysis and growth mixture modeling: A Monte Carlo simulation study. Structural Equation Modeling, 14, 535-569. doi:10.1080/10705510701575396
40.
OnwezenM. C.ReindersM. J.LansV. D. I.SijtsemaS. J.JasiulewiczA.GuardiaM. D.GuerreroL. (2012). A cross-national consumer segmentation based on contextual differences in food choice benefits. Food Quality and Preference, 24, 276-286.
41.
PalardyG.VermuntJ. K. (2010). Multilevel growth mixture models for classifying groups. Journal of Educational and Behavioral Statistics, 35, 532-565.
42.
PiraniE. (2011). Evaluating contemporary social exclusion in Europe: A hierarchical latent class approach. Quality and Quantity, 47, 923-941.
43.
R Development Core Team. (2010). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from http://www.R-project.org/
44.
RamaswamyV.DeSarboW. S.ReibsteinD. J.RobinsonW. T. (1993). An empirical pooling approach for estimating marketing mix elasticities with PIMS data. Marketing Science, 12, 103-124.
45.
RightsJ. D.SterbaS. K. (2016). The relationship between multilevel models and nonparametric multilevel mixture models: Discrete approximation of intraclass correlation, random coefficient distributions, and residual heteroscedasticity. British Journal of Mathematical and Statistical Psychology, 69, 316-343.
46.
RindskopfD. (2006). Heavy alcohol use in the “Fighting Back” survey sample: Separating individual and community level influences using multilevel latent class analysis. Journal of Drug Issues, 36, 441-462.
47.
RüdigerM.Hans-DieterD. (2013). University and student segmentation: Multilevel latent class analysis of students’ attitudes toward research methods and statistics. British Journal of Educational Psychology, 83. 280-304.
48.
ScherbaumC. A.FerreterJ. M. (2009). Estimating statistical power and required sample sizes for organizational research using multilevel modeling. Organizational Research Methods, 12, 347-367.
49.
SchwarzG. (1978). Estimating the dimension of a model. Annals of Statistics, 6, 461-464.
SnijdersT. A. B.BoskerR. J. (2012). Multilevel analysis: An introduction to basic and advanced multilevel modeling (2nd ed.). Thousand Oaks, CA: Sage.
52.
StegmuellerD. (2013). How many countries for multilevel modeling? A comparison of frequentist and Bayesian approaches. American Journal of Political Science, 57, 748-761.
53.
TeinJ.-Y.CoxeS.ChamH. (2013). Statistical power to detect the correct number of classes in latent profile analysis. Structural Equation Modeling, 20, 640-657.
54.
TekleF. B.GudichaD. W.VermuntJ. K. (2016). Power analysis for the bootstrap likelihood ratio test for the number of classes in latent class models. Advances in Data Analysis and Classification, 10, 209-224.
55.
TuellerS. J.LubkeG. H. (2011). Evaluation of structural equation mixture models: Parameter estimates and correct class assignment. Structural Equation Modeling, 17, 165-192.
56.
Van HornM. L.FaganA. A.JakiT.BrownE. C.HawkinsJ. D.ArthurM. W.… CatalanoR. F. (2008). Using multilevel mixtures to evaluate intervention effects in group randomized trials. Multivariate Behavioral Research, 43, 289-326.
VermuntJ. K. (2003). Multilevel latent class models. Sociological Methodology, 33, 213-239.
59.
VermuntJ. K. (2004). An EM algorithm for the estimation of parametric and nonparametric hierarchical nonlinear models. Statistica Neerlandica, 58, 220-233.
VermuntJ. K.MagidsonJ. (2013). Technical guide for Latent GOLD 5.0: Basic, advanced, and syntax. Belmont, MA: Statistical Innovations.
62.
WolfE. J.HarringtonK. M.ClarkS. L.MillerM. W. (2013). Sample size requirements for structural equation models: An evaluation of power, bias, and solution propriety. Educational and Psychological Measurement, 73, 913-934.
63.
WurptsI. C.GeiserC. (2014). Is adding more indicators to a latent class analysis beneficial or detrimental? Results of a Monte-Carlo study. Frontiers in Psychology, 5, 1-15.
64.
YangC. C. (2006). Evaluating latent class analyses in qualitative phenotype identification. Computational Statistics & Data Analysis, 50, 1090-1104.
65.
YangC. C.YangC. C. (2007). Separating latent classes by information criteria. Journal of Classification, 24, 183-203.
66.
YuH.-T.ParkJ. (2014). Simultaneous decision on the number of latent clusters and classes for multilevel latent class models. Multivariate Behavioral Research, 49, 232-244.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
