Recent work on measuring growth with categorical outcome variables has combined the item response theory (IRT) measurement model with the latent growth curve model and extended the assessment of growth to multidimensional IRT models and higher order IRT models. However, there is a lack of synthetic studies that clearly evaluate the strength and limitations of different multilevel IRT models for measuring growth. This study aims to introduce the various longitudinal IRT models, including the longitudinal unidimensional IRT model, longitudinal multidimensional IRT model, and longitudinal higher order IRT model, which cover a broad range of applications in education and social science. Following a comparison of the parameterizations, identification constraints, strengths, and weaknesses of the different models, a real data example is provided to illustrate the application of different longitudinal IRT models to model students’ growth trajectories on multiple latent abilities.
AndersenE. B. (1985). Estimating latent correlations between repeated testings. Psychometrika, 50, 3–16.
2.
AndradeD. F.TavaresH. R. (2005). Item response theory for longitudinal data: Population parameter estimation. Journal of Multivariate Analysis, 95, 1–22.
3.
BollenK. A.CurranP. J. (2006). Latent curve models: A structural equation perspective. Hoboken, NJ: Wiley-Interscience.
4.
BreslowN. E.ClaytonD. G. (1993). Approximate inference in generalized linear mixed models. Journal of the American Statistical Association, 88, 9–25.
5.
CaiL. (2010). A two-tier full-information item factor analysis model with applications. Psychometrika, 75, 581–612.
de la TorreJ.HongY. (2010). Parameter estimation with small sample size: A higher-order IRT model approach. Applied Psychological Measurement, 34, 267–285.
10.
de la TorreJ.SongH. (2009). Simultaneous estimation of overall and domain abilities: A higher-order IRT model approach. Applied Psychological Measurement, 33, 620–639.
11.
DeYoungC. (2006). Higher-order factors of the Big Five in a multi-information sample. Journal of Personality and Social Psychology, 91, 1138–1151.
12.
DuncanT. E.DuncanS. C.StryckerL. A. (Eds.). (2006). An introduction to latent variable growth curve modeling: Concepts, issues, and applications. Mahwah, NJ: Lawrence Erlbaum Associates.
13.
EdwardsM.WirthR. (2009). Measurement and study of change. Research in Human Development, 6, 74–96.
14.
EmbretsonS. E. (1991). A multidimensional latent trait model for measuring learning and change. Psychometrika, 56, 495–515.
15.
ForeroC. G.Maydeu-OlivaresA. (2009). Estimation of IRT graded response models: Limited versus full information methods. Psychological Methods, 14, 275–299.
16.
GelmanA.RubinD. (1992). Inference from iterative simulation using multiple sequences. Statistical Science, 7, 457–472.
GolayP.LecerfT. (2011). Orthogonal higher order structure and confirmatory factor analysis of the French Wechsler Adult Intelligence Scale (WAIS-III). Psychological Assessment, 23, 143–52.
19.
GoldsteinH. (1991). Nonlinear multilevel models, with an application to discrete response data. Biometrika, 78, 45–51.
20.
GoldsteinH.RasbashJ. (1996). Improved approximations for multilevel models with binary responses. Journal of the Royal Statistical Society, Series A: Statistics in Society, 159, 505–513.
21.
GrimmK. J.KuhlA. P.ZhangZ. (2013). Measurement models, estimation, and the study of change. Structural Equation Modeling, 20, 504–517.
22.
GustafssonJ.-E.BalkeG. (1993). General and specific abilities as predictors of school achievement. Multivariate Behavioral Research, 28, 407–434.
23.
HsiehC.-A.von EyeA. A.MaierK. S. (2010). Using a multivariate multilevel polytomous item response theory model to study parallel processes of change: The dynamic association between adolescents’ social isolation and engagement with delinquent peers in the national youth survey. Multivariate Behavioral Research, 45, 508–552.
24.
HuangH. Y. (2013). Measuring latent growth under the multilevel higher-order item response theory model. Paper presented at 2013 Annual Meeting of National Council on Measurement in Education, San Francisco, CA.
25.
HuangH. Y. (2015). A multilevel higher order item response theory model for measuring latent growth in longitudinal data. Applied Psychological Measurement, 39, 362–372.
26.
HuangH. Y.WangW. C. (2014). Multilevel higher-order item response theory models. Educational and Psychological Measurement, 74, 495–515.
27.
IsiordiaM.FerrerE. (2018). Curve of factors model: A latent growth modeling approach for education research. Educational and Psychological Measurement, 78, 203–231.
28.
KaplanD.DepaoliS. (2012). Bayesian structural equation modeling. In HoyleR. H. (Eds.), Handbook of structural equation modeling (pp. 650–673). New York, NY: Guilford Press.
29.
KohliN.HughesJ.WangC.ZopluogluC.DavisonM. L. (2015). Fitting a linear-linear piecewise growth mixture model with unknown knots: A comparison of two common approaches to inference. Psychological Methods, 20, 259–275.
30.
KolenM. J.BrennanR. L. (2004). Test equating, scaling, and linking: Methods and practices. New York, NY: Springer.
31.
LairdN. M. (1978). Empirical Bayes methods for two-way contingency tables. Biometrika, 65, 581–590.
32.
LiuY.MillsapR. E.WestS. G.TeinJ. Y.TanakaR.GrimmK. J. (2017). Testing measurement invariance in longitudinal data with ordered-categorical measures. Psychological Methods, 22, 486–506. doi:10.1037/met0000075
33.
LunnD. J.ThomasA.BestN.SpiegelhalterD. (2000). WinBUGS—A Bayesian modelling framework: Concepts, structure, and extensibility. Statistics and Computing, 10, 325–337.
34.
McArdleJ. J. (1988). Dynamic but structural equation modeling of repeated measures data. In NesselroadeJ. R.CattellR. B. (Eds.), The handbook of multivariate experimental psychology (Vol. 2, pp. 561–614). New York, NY: Plenum Press.
35.
McArdleJ. J. (2001). A latent difference score approach to longitudinal dynamic structural analysis. In CudeckR.du ToitS.SorbomD. (Eds.), Structural equation modeling: Present and future (pp. 342–380). Lincolnwood, IL: Scientific Software International.
36.
McArdleJ. J.GrimmK. J.HamagamiF.BowlesR. P.MeredithW. (2009). Modeling life-span growth curves of cognition using longitudinal data with multiple samples and changing scales of measurement. Psychological Measurement, 14, 126–149.
37.
McArdleJ. J.HamagamiF. (2001). Latent difference score structural models for linear dynamic analyses with incomplete longitudinal data. In CollinsL. M.SayerA. G. (Eds.), New methods for the analysis of change (pp. 137–175). Washington, DC: American Psychological Association.
38.
McCullaghP.NelderJ. A. (1989). Generalized linear models. New York, NY: Chapman & Hall.
39.
MengX.-L.van DykD. (1997). The EM algorithm—An old folk-song sung to a fast new tune. Journal of the Royal Statistical Society. Series B (Methodological), 59, 511–567.
MurrayA. L.JohnsonW. (2013). The limitations of model fit in comparing the bi-factor versus higher-order models of human cognitive ability structure. Intelligence, 41, 407–422.
42.
MuthénB.AsparouhovT. (2015). Growth mixture modeling with non-normal distributions. Statistics in Medicine, 34, 1041–1058.
MuthénL. K.MuthénB. O. (1998–2011). Mplus user’s guide(6th ed.). Los Angeles, CA: Author.
45.
PaekI.LiZ.ParkH. (2016). Specifying ability growth models using a multidimensional item response model for repeated measures categorical ordinal item response data. Multivariate Behavioral Research, 51, 569–581.
46.
PatzR. J.JunkerB. W. (1999). A straightforward approach to Markov chain Monte Carlo methods for item response models. Journal of Educational and Behavioral Statistics, 24, 146–178.
47.
PlummerM. (2003). JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling. Proceedings of the 3rd Internaitonal Workshop on Distributed Statistical Computing, March20–22, Vienna, Austria.
48.
ReckaseM. D. (2009). Multidimensional item response theory. New York, NY: Springer.
49.
RijmenF.VansteelandtK.De BoeckP. (2008). Latent class models for diary method data: Parameter estimation by local computations. Psychometrika, 73, 167–182.
50.
RindskopfD.RoseT. (1988). Some theory and applications of confirmatory second-order factor analysis. Multivariate Behavioral Research, 23, 51–67.
51.
RodriguezG.GoldmanN. (1995). An assessment of estimation procedures for multi-level models with binary responses. Journal of the Royal Statistical Society (Series A), 158, 73–90.
SawakiY.StrickerL. J.OranjeA. H. (2009). Factor structure of the TOEFL Internet-based test. Language Testing, 26, 5–30.
54.
ShengY.WikleC. (2008). Bayesian multidimensional IRT models with a hierarchical structure. Educational and Psychological Measurement, 68, 413–430.
55.
SpiegelhalterD. J.BestN. G.CarlinB. P.van der LindeA. (2002). Bayesian measures of model complexity and fit. Journal of Royal Statistical Society (Series B), 64, 583–639.
56.
TakaneY.de LeeuwJ. (1987). On the relationship between item response theory and factor analysis of discretized variables. Psychometrika, 52, 393–408.
57.
TeresiJ. A. (2006). Overview of quantitative measurement methods: Equivalence, invariance, and differential item functioning in health outcomes. Medical Care, 44, S39–S49.
58.
TuerlinckxF.RijmenF.VerbekeG.De BoeckP. (2006). Statistical inference in generalized linear mixed models: A review. British Journal of Mathematical and Statistical Psychology, 59, 225–255.
59.
von DavierM.XuX.CarstensenC. H. (2011). Measuring growth in a longitudinal large-scale assessment with a general latent variable model. Psychometrika, 76, 318–336.
60.
WangC. (2014). Improving measurement precision of hierarchical latent traits using adaptive testing. Journal of Equational and Behavioral Statistics, 39, 452–477.
61.
WangC.KohliN.HennL. (2016). A second-order longitudinal model for binary outcomes: Item response theory versus structural equation modeling. Structural Equation Modeling: A Multidisciplinary Journal, 23, 455–465.
62.
WangC.NydickS. (2015). Comparing two algorithms for calibrating the restricted non-compensatory multidimensional IRT model. Applied Psychological Measurement, 39, 119–134.
63.
WangC.XuG. (2015). A mixture hierarchical model for response times and response accuracy. British Journal of Mathematical and Statistical Psychology, 68, 456–477.
64.
WidamanK. F.FerrerE.CongerR. (2010). Factor invariance within longitudinal structural equation models: measuring the same construct across time. Child Development Perspectives, 1, 10–18.
65.
WidamanK. F.ReiseS. P. (1997). Exploring the measurement invariance of psychological instruments: Applications in the substance use domain. In BryantK. J.WindleM.WestS. G. (Eds.), The science of prevention: Methodological advances from alcohol and substance abuse research (pp. 281–324). Washington, DC: American Psychological Association.
66.
WilsonM.ZhengX.McGuireL. W. (2012). Formulating latent growth using an explanatory item response model appraoch. Journal of Applied Measurement, 13, 1–22.
67.
ZhangX.TaoJ.WangC. (2019). Bayesian model selection methods for multilevel IRT models: A comparison of five DIC-based indices. Journal of Educational Measurement, 56, 3–27.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
