In psychological and educational measurement, a testlet-based test is a common and popular format, especially in some large-scale assessments. In modeling testlet effects, a standard bifactor model, as a common strategy, assumes different testlet effects and the main effect to be fully independently distributed. However, it is difficult to establish perfectly independent clusters as this assumption. To address this issue, correlations among testlets could be taken into account in fitting data. Moreover, one may desire to maintain a good practical interpretation of the sparse loading matrix. In this paper, we propose data-driven learning of significant correlations in the covariance matrix through a latent variable selection method. Under the proposed method, a regularization is performed on the weak correlations for the extended bifactor model. Further, a stochastic expectation maximization algorithm is employed for efficient computation. Results from simulation studies show the consistency of the proposed method in selecting significant correlations. Empirical data from the 2015 Program for International Student Assessment is analyzed using the proposed method as an example.
AkaikeH. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19(6), 716–723. https://doi.org/10.1109/tac.1974.1100705
2.
BirnbaumA. (1969). Statistical theory for logistic mental test models with a prior distribution of ability. Journal of Mathematical Psychology, 6(2), 258–276. https://doi.org/10.1016/0022-2496(69)90005-4
3.
BockR. D.AitkinM. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika, 46(4), 443–459. https://doi.org/10.1007/bf02293801
4.
BozdoganH. (1987). Model selection and akaike’s information criterion (aic): The general theory and its analytical extensions. Psychometrika, 52(3), 345–370. https://doi.org/10.1007/bf02294361
5.
BradlowE. T.WainerH.WangX. (1999). A bayesian random effects model for testlets. Psychometrika, 64(2), 153–168. https://doi.org/10.1007/bf02294533
6.
BurnhamK. P.AndersonD. R. (2004). Model selection and multimodel inference: A practical information-theoretic approach. (2nd Edition, 2). New York: Springer-Verlag.
7.
CaiL. (2010a). High-dimensional exploratory item factor analysis by a metropolis–hastings robbins–monro algorithm. Psychometrika, 75(1), 33–57. https://doi.org/10.1007/s11336-009-9136-x
8.
CaiL. (2010b). Metropolis-hastings robbins-monro algorithm for confirmatory item factor analysis. Journal of Educational and Behavioral Statistics, 35(3), 307–335. https://doi.org/10.3102/1076998609353115
CeleuxG.ChauveauD.DieboltJ. (1996). Stochastic versions of the em algorithm: An experimental study in the mixture case. Journal of Statistical Computation and Simulation, 55(4), 287–314. https://doi.org/10.1080/00949659608811772
11.
ChenF. F.WestS. G.SousaK. H. (2006). A comparison of bifactor and second-order models of quality of life. Multivariate Behavioral Research, 41(2), 189–225. https://doi.org/10.1207/s15327906mbr4102_5
12.
ChenY. (2016). Latent variable modeling and statistical learning. PhD Thesis. Columbia University.
13.
ChenY.LiX.LiuJ.YingZ. (2017). Regularized latent class analysis with application in cognitive diagnosis. Psychometrika, 82(3), 660–692. https://doi.org/10.1007/s11336-016-9545-6
14.
FanJ.LiR. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 96(456), 1348–1360. https://doi.org/10.1198/016214501753382273
15.
FanY.TangC. Y. (2013). Tuning parameter selection in high dimensional penalized likelihood. Journal of the Royal Statistical Society: Series B, 75(3), 531–552. https://doi.org/10.1111/rssb.12001
16.
FangG.XuX.GuoJ.YingZ.ZhangS. (2021). Identifiability of bifactor models. Statistica Sinica, 31, 2309–2330.
17.
FoxJ. P. (2003). Stochastic em for estimating the parameters of a multilevel irt model. British Journal of Mathematical and Statistical Psychology, 56(Pt 1), 65–81. https://doi.org/10.1348/000711003321645340
18.
FoxJ. P.WenzelJ.KlotzkeK. (2021). The bayesian covariance structure model for testlets. Journal of Educational and Behavioral Statistics, 46(2), 219–243. https://doi.org/10.3102/1076998620941204
19.
FujimotoK. A. (2019). The bayesian multilevel trifactor item response theory model. Educational and Psychological Measurement, 79(3), 462–494. https://doi.org/10.1177/0013164418806694
GreenS.YangY. (2018). Empirical underidentification with the bifactor model: A case study. Educational and Psychological Measurement, 78(5), 717–736. https://doi.org/10.1177/0013164417719947
22.
GuM. G.KongF. H. (1998). A stochastic approximation algorithm with Markov chain monte-carlo method for incomplete data estimation problems. Proceedings of the National Academy of Sciences, 95(13), 7270–7274. https://doi.org/10.1073/pnas.95.13.7270
23.
GuoJ.XuX.YingZ.ZhangS. (2022). Modeling not-reached items in timed tests: A response time censoring approach. psychometrika, 1–33. https://doi.org/10.1007/s11336-021-09810-0
24.
HerbornK.StadlerM.MustafićM.GreiffS. (2020). The assessment of collaborative problem solving in pisa 2015: Can computer agents replace humans?Computers in Human Behavior, 104, 105624. https://doi.org/10.1016/j.chb.2018.07.035
HuangJ. Z.LiuN.PourahmadiM.LiuL. (2006). Covariance matrix selection and estimation via penalised normal likelihood. Biometrika, 93(1), 85–98. https://doi.org/10.1093/biomet/93.1.85
27.
IpE. H. (2002). On single versus multiple imputation for a class of stochastic algorithms estimating max. Computational Statistics, 17(4), 517–524. https://doi.org/10.1007/s001800200124
JiaoH.WangS.KamataA. (2005). Modeling local item dependence with the hierarchical generalized linear model. Journal of Applied Measurement.
32.
KangH. A.HanS.BettsJ.MunteanW. (2022). Computerized adaptive testing for testlet-based innovative items. British Journal of Mathematical and Statistical Psychology, 75(1), 136–157. https://doi.org/10.1111/bmsp.12252
33.
LevinaE.RothmanA.ZhuJ. (2008). Sparse estimation of large covariance matrices via a nested lasso penalty. Annals of Applied Statistics, 2(1), 245–263. https://doi.org/10.1214/07-aoas139
34.
LiuH.WangL.ZhaoT. (2014). Sparse covariance matrix estimation with eigenvalue constraints. Journal of Computational & Graphical Statistics, 23(2), 439–459. https://doi.org/10.1080/10618600.2013.782818
35.
LordF. (1952). A theory of test scores. Psychometric monographs.
36.
LordF. M. (1951). A theory of test scores and their relation to the trait measured. PhD Thesis. Princeton University.
37.
Lorenzo-SevaU.FerrandoP. J. (2019). A general approach for fitting pure exploratory bifactor models. Multivariate Behavioral Research, 54(1), 15–30. https://doi.org/10.1080/00273171.2018.1484339
38.
McDonaldR. P. (1997). Normal-ogive multidimensional model. In van der LindenWim J.HambletonRonald K. (Eds.), Handbook of modern item response theory (pp. 257–269). Springer. https://doi.org/10.1007/978-1-4757-2691-6_15
39.
OECD (2016). Pisa 2015 assessment and analytical framework: Science, reading, mathematic and financial literacy.
40.
OlsonE. D. (2020). A multilevel testlet joint model of responses and response time. PhD Thesis. University of Maryland.
41.
ParshallC. G.DaveyT.PashleyP. J. (2000). Innovative item types for computerized testing. In LindenWim J.GlasGees A. W. (Eds.), Computerized adaptive testing: Theory and practice (pp. 129–148). Springer. https://doi.org/10.1007/0-306-47531-6_7
42.
ReckaseM. D. (1997). A linear logistic multidimensional model for dichotomous item response data. In Handbook of modern item response theory (pp. 271–286). Springer. https://doi.org/10.1007/978-1-4757-2691-6_16
43.
ReiseS.MooreT.Maydeu-OlivaresA. (2011). Target rotations and assessing the impact of model violations on the parameters of unidimensional item response theory models. Educational and Psychological Measurement, 71(4), 684–711. https://doi.org/10.1177/0013164410378690
44.
ReckaseM. D. (2009). Multidimensional item response theory models. In ReckaseM. D. (Ed.), Multidimensional item response theory (pp. 79–112). Springer. https://doi.org/10.1007/978-0-387-89976-3_4
ReiseS. P.MooreT. M.HavilandM. G. (2010). Bifactor models and rotations: Exploring the extent to which multidimensional data yield univocal scale scores. Journal of Personality Assessment, 92(6), 544–559. https://doi.org/10.1080/00223891.2010.496477
47.
RijmenF. (2011). Hierarchical factor item response theory models for pirls: Capturing clustering effects at multiple levels. IERI monograph series: Issues and methodologies in large-scale assessments, 4, 59–74.
48.
RijmenF.JeonM.von DavierM.Rabe-HeskethS. (2014). A third-order item response theory model for modeling the effects of domains and subdomains in large-scale educational assessment surveys. Journal of Educational and Behavioral Statistics, 39(4), 235–256. https://doi.org/10.3102/1076998614531045
49.
RobertsonS. (2019). Bifactor models and factor collapse: A Monte Carlo study. PhD Thesis. Clemson University.
SongX. Y.LeeS. Y. (2005). A multivariate probit latent variable model for analyzing dichotomous responses. Statistica Sinica, 645–664.
54.
SugiuraN. (1978). Further analysis of the data by akaike’s information criterion and the finite corrections: Further analysis of the data by Akaike’s. Communications in Statistics - Theory and Methods, 7(1), 13–26. https://doi.org/10.1080/03610927808827599
55.
SunJ.ChenY.LiuJ.YingZ.XinT. (2016). Latent variable selection for multidimensional item response theory models via l1 regularization. Psychometrika, 81(4), 921–939. https://doi.org/10.1007/s11336-016-9529-6
56.
ThoemmesF.RoseN. (2014). A cautious note on auxiliary variables that can increase bias in missing data problems. Multivariate Behavioral Research, 49(5), 443–459. https://doi.org/10.1080/00273171.2014.931799
WainerH.BradlowE. T.DuZ. (2000). Testlet response theory: An analog for the 3pl model useful in testlet-based adaptive testing. In Computerized adaptive testing: Theory and practice (pp. 245–269). Springer. https://doi.org/10.1007/0-306-47531-6_13
59.
WainerH.BradlowE. T.WangX. (2007). Testlet response theory and its applications. Cambridge University Press.
ZhangS.ChenY. (2022). Computation for latent variable model estimation: A unified stochastic proximal framework. Psychometrika, 1–30. https://doi.org/10.1007/s11336-022-09863-9
64.
ZhangS.ChenY.LiuY. (2020). An improved stochastic em algorithm for large-scale full-information item factor analysis. British Journal of Mathematical and Statistical Psychology, 73(1), 44–71. https://doi.org/10.1111/bmsp.12153
65.
ZhengC. (2013). Examination of the parameter estimate bias when violating the orthogonality assumption of the Bifactor model. PhD Thesis. University of Kansas.
66.
ZouH.HastieT. (2005). Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society: Series B, 67(2), 301–320. https://doi.org/10.1111/j.1467-9868.2005.00503.x
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
