The authors present a generalization of the multiple-group bifactor model that extends the classical bifactor model for categorical outcomes by relaxing the typical assumption of independence of the specific dimensions. In addition to the means and variances of all dimensions, the correlations among the specific dimensions are allowed to differ between groups. By including group-specific difficulty parameters, the model can be used to assess differential item functioning (DIF) for testlet-based tests. The model encompasses various item response models for polytomous data by allowing for different link functions, and it includes testlet and second-order models as special cases. Importantly, by assuming that the testlet dimensions are conditionally independent given the general dimension, the authors show, using a graphical model framework, that the integration over all latent variables can be carried out through a sequence of computations in two-dimensional subspaces, making full-information maximum likelihood estimation feasible for high-dimensional problems and large datasets. The importance of relaxing the orthogonality assumption and allowing for a different covariance structure of the dimensions for each group is demonstrated in the context of the assessment of DIF. Through a simulation study, it is shown that ignoring between-group differences in the structure of the multivariate latent space can result in substantially biased estimates of DIF.
ClauserB. E.NungesterR.MazorK.RipkeyD. (1996). A comparison of alternative matching strategies for DIF detection in tests that are multidimensional. Journal of Educational Measurement, 33, 202–214.
8.
De la TorreJ.SongH. (2009). Simultaneous estimation of overall and domain abilities: A higher-order IRT model approach. Applied Psychological Measurement, 33, 620–639.
9.
DeMarsC. E. (2006). Application of the bi-factor multidimensional item response theory model to testlet-based tests. Journal of Educational Measurement, 43, 145–168.
10.
DeMarsC. E. (2009). Modification of the Mantel-Haenszel and logistic regression DIF procedures to incorporate the SIBTEST regression correlation. Journal of Educational and Behavioral Statistics, 34, 149–170.
11.
DoransN. J.HollandP. H. (1993). DIF detection and description: Mantel-Haenszel and standardization. In HollandP. W.WainerH. (Eds.), Differential item functioning (pp. 67–113). Hillsdale, NJ: Lawrence Erlbaum.
12.
FahrmeirL.TutzG. (2001). Multivariate statistical modelling based on generalized linear models. New York, NY: Springer.
13.
FukuharaH.KamataA. (2011). A bifactor multidimensional item response theory model for differential item functioning analysis on testlet-based items. Applied Psychological Measurement, 35, 604–622.
GibbonsR. D.GrochocinskiV. J.WeissD. J.BhaumikD. K.KupferD. J.StoverA. (2008). Using computerized adaptive testing to reduce the burden of mental health assessment. Psychiatric Services, 59, 361–368.
GierlM. J.BisanzJ.BisanzG.BoughtonK.KhaliqS. (2001). Illustrating the utility of differential bundle functioning analyses to identify and interpret group differences on achievement tests. Educational Measurement: Issues and Practice, 20, 26–36.
18.
GlasC. A. W.WainerH.BradlowE. T. (2000). MML and EAP estimation in testlet-based adaptive testing. In van der LindenW. J.GlasC. A. W. (Eds.), Computerized adaptive testing: Theory and practice (pp. 271–287). Boston, MA: Kluwer-Nijhoff.
19.
HollandP. W.ThayerD. T. (1988). Differential item performance and the Mantel-Haenszel procedure. In WainerH.BraunH. (Eds.), Test validity (pp. 129–146). Hillsdale, NJ: Lawrence Erlbaum.
20.
HolzingerK. J.SwinefordF. (1937). The bi-factor method. Psychometrika, 2, 41–54.
21.
HunterM.BattersbyR.WhiteheadM. (1986). Relationships between psychological symptoms, somatic complaints and menopausal status. Maturitas, 8, 217–228.
22.
IpE. H. (2002). Locally dependent latent trait model and the Dutch identity revisited. Psychometrika, 67, 367–386.
23.
IpE. H. (2010). Empirically indistinguishable multidimensional IRT and locally dependent unidi-mensional item response models. British Journal of Mathematical and Statistical Psychology, 63, 395–416.
24.
LauritzenS. L. (1995). The EM algorithm for graphical association models with missing data. Computational Statistics and Data Analysis, 19, 191–201.
25.
LeeY.-S.CohenA.ToroM. (2009). Examining Type I error and power for detection of differential item and testlet functioning. Asia Pacific Education Review, 10, 365–375.
26.
LiY.BoltD. M.FuJ. (2006a). A comparison of alternative models for testlets. Applied Psychological Measurement, 30, 3–21.
27.
LiY.BoltD. M.FuJ. (2006b). A multiple-group testlet model and its application to DIF assessment. Paper presented at the Annual Meeting of the National Council on Measurement in Education, San Francisco, CA.
28.
MastersG. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149–174.
29.
MazorK. M.KanjeeA.ClauserB. E. (1995). Using logistic regression and the Mantel-Haenszel with multiple ability estimates to detect differential item functioning. Journal of Educational Measurement, 32, 131–144.
30.
McLachlanG.KrishnanT. (1997). The EM algorithm and extensions. New York, NY: Wiley.
31.
MeilijsonI. (1989). A fast improvement to the EM algorithm on its own terms. Journal of the Royal Statistical Society, Series B, 51, 127–138.
MillsapR. E.EversonH. T. (1993). Methodology review: Statistical approaches for assessing measurement bias. Applied Psychological Measurement, 17, 297–334.
34.
NatarajanR.KassR. E. (2000). Reference Bayesian methods for generalized linear mixed models. Journal of the American Statistical Association, 95, 227–237.
35.
OshimaT. CRajuN. S.FlowersC. P. (1997). Development and demonstration of multidimensional IRT-based internal measures of differential functioning of items and tests. Journal of Educational Measurement, 34, 253–272.
36.
PenfieldR. D. (2007). An approach for categorizing DIF in polytomous items. Applied Measurement In Education, 20, 335–355.
37.
Rabe-HeskethS.SkrondalA.PicklesA. (2005). Maximum likelihood estimation of limited and discrete dependent variable models with nested random effects. Journal of Econometrics, 128, 301–323.
38.
RajuN. S. (1988). The area between two item characteristic curves. Psychometrika, 53, 495–502.
39.
ReiseS. P.MorizotJ.HaysR. D. (2007). The role of the bifactor model in resolving dimensionality issues in health outcomes measures. Quality of Life Research, 16, 19–31.
40.
RijmenF. (2006). BNL: A Matlab toolbox for Bayesian networks with logistic regression (Technical Report). VrijeUniversiteit Medical Center, Amsterdam.
41.
RijmenF. (2009). An efficient EM algorithm for multidimensional IRT models: Full information maximum likelihood estimation in limited time(ETS Research Report RR0903). Princeton, NJ: Educational Testing Service.
42.
RijmenF. (2010). Formal relations and an empirical comparison between the bi-factor, the testlet, and a second-order multidimensional IRT model. Journal of Educational Measurement, 47, 361–372.
43.
RijmenF.VansteelandtK.De BoeckP. (2008). Latent class models for diary method data: Parameter estimation by local computations. Psychometrika, 73, 167–182.
44.
RogersH. J.SwaminathanH. (1993). A comparison of logistic regression and Mantel-Haenszel procedures for detecting differential item functioning. Applied Psychological Measurement, 17, 105–116.
45.
RoussosL. A.StoutW. F. (1996). Simulation studies of the effects of small sample size and studied item parameters on SIBTEST and Mantel-Haenszel Type I error performance. Journal of Educational Measurement, 33, 215–230.
46.
ShealyR. T. (1989). An item response theory-based statistical procedure for detecting concurrent internal bias in ability tests. Unpublished Ph.D dissertation, University of Illinois, Urbana-Champaign.
47.
ShealyR. T.StoutW. F. (1991). An item response theory model for test bias. Washington, DC: Office of Naval Research.
48.
ShealyR. T.StoutW. F. (1993). A model-based standardization approach that separates true bias/DIF from group ability differences and detects test bias/DTF as well as item bias/DIF. Psychometrika, 58, 159–194.
49.
SinharayS.DoransN. J. (2010). Two simple approaches to overcome a problem with the Mantel-Haenszel statistic: Comments on Wang, Bradlow, Wainer, and Muller. Journal of Educational and Behavioral Statistics, 35, 474–488.
50.
SkrondalA. (2000). Design and analysis of Monte Carlo experiments: Attacking the conventional wisdom. Multivariate Behavioral Research, 35, 137–167.
SunX.SunD. (2005). Estimation of the Cholesky decomposition of the covariance matrix for a conditional independent normal model. Statistics and Probability Letters, 73, 1–12.
53.
SwaminathanH.RogersH. J. (1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27, 361–370.
54.
SwansonD. B.ClauserB. E.CaseS. M.NungesterR. J.FeathermanC. (2002). Analysis of differential item functioning (DIF) using hierarchical logistic regression models. Journal of Educational and Behavioral Statistics, 27, 53–75.
55.
ThissenD.SteinbergL.WainerH. (1988). Use of item response theory in the study of group differences in trace lines. In WainerH.BraunH. I. (Eds.), Test validity (pp. 147–169). Hillsdale, NJ: Lawrence Erlbaum.
56.
Van den NoortgateW.De BoeckP. (2005). Assessing and explaining differential item functioning using logistic mixed models. Journal of Educational and Behavioral Statistics, 30, 443–464.
57.
WainerH.BradlowE.WangX. (2007). Testlet response theory and its applications. New York, NY: Cambridge University Press.
WangX.BradlowE.WainerH.MullerE. (2008). A Bayesian method for studying DIF: A cautionary tale filled with surprises and delights. Journal of Educational and Behavioral Statistics, 33, 363–384.
60.
WolfingerR. D. (1999). Fitting non-linear mixed models with the new NLMIXED procedure (Technical Report). Cary, NC: SAS Institute.
61.
ZumboB. D. (1999). A handbook on the theory and methods for differential item functioning: Logistic regression modeling as a unitary framework for binary and likert-type (ordinal) item scores. Ottawa, Canada: Directorate of Human Resources Research and Evaluation, Department of National Defense.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
