Psychometricians have devoted much research and attention to categorical item responses, leading to the development and widespread use of item response theory for the estimation of model parameters and identification of items that do not perform in the same way for examinees from different population subgroups (e.g., differential item functioning [DIF]). With the increasing use of computer-based measurement, use of items with a continuous response modality is becoming more common. Models for use with these items have been developed and refined in recent years, but less attention has been devoted to investigating DIF for these continuous response models (CRMs). Therefore, the purpose of this simulation study was to compare the performance of three potential methods for assessing DIF for CRMs, including regression, the MIMIC model, and factor invariance testing. Study results revealed that the MIMIC model provided a combination of Type I error control and relatively high power for detecting DIF. Implications of these findings are discussed.
BrownT.A. (2015). Confirmatory factor analysis for applied research. New York: The Guilford Press.
3.
ChenF. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14(3), 464–504.
4.
CheungG. W.RensvoldR. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9(2), 233–255.
5.
DoransN. J.HollandP. W. (1993). DIF detection and description: Mantel-Haenszel and standardization. In HollandP. W.WainerH. (Eds.), Differential item functioning (pp. 35–66). Lawrence Erlbaum Associates.
6.
DoransN. J.KulickE. (1986). Demonstrating the utility of the standardization approach to assessing unexpected differential item performance on the Scholastic Aptitude Test. Journal of Educaitonal Measurement, 23(4), 355–368.
7.
FerrandoP. J. (2002). Theoretical and empirical comparisons between two models for continuous item responses. Multivariate Behavioral Research, 37(4), 521–542.
8.
FerrandoP. J. (2009). Difficulty, discrimination, and information indices in the linear factor analysis model for continuous item responses. Applied Psychological Measurement, 33(1), 9–24.
9.
FerrandoP. J. (2013). A general linear framework for modeling continuous responses with error in persons and items. Methodology, 9, 150–161.
10.
FerrandoP. J.Navarro-GonzalezD. (2021). A multidimensional item response theory model for continuous and graded responses with error in persons and items. Educational and Psychological Measurement, 81(6), 1029–1053.
11.
FinchH. (2005). The MIMIC model as a method for detecting DIF: Comparison with Mantel-Haenszel, SIBTEST, and the IRT likelihood ratio. Applied Psychological Measurement, 29, 278–295.
12.
FinchW.H.FrenchB.F. (2018). A Simulation Investigation of the Performance of Invariance Assessment using Equivalence Testing Procedures. Structural Equation Modeling, 25(5), 673–686.
13.
FreedleR. (2003). Correcting the SAT’s ethnic and social-class bias: A method for reestimating SAT scores. Harvard Educational Review, 73, 1–43.
14.
FrenchB. F.FinchW. H. (2006). Confirmatory factor analytic procedures for the determination of measurement invariance. Structural Equation Modeling, 13, 378–402.
15.
HambletonR. K.SwaminathanH. (1985). Item response theory: Principles and applications. Springer.
16.
HollandP. W.ThayerD. T. (1988). Differential item performance and the Mantel-Haenszel procedure. In WainerH.BrownH. I. (Eds.), Test validity (pp. 129–145). Lawrence Erlbaum Associates.
17.
JinK.-Y.ChenH.-F. (2020). MIMIC approach to assessing differential item functioning with control of extreme response style. Behavior Research Methods, 52, 23–35.
18.
JodoinM. G.GierlM. J. (2001). Evaluating type I error and power rates using an effect size measure with the logistic regression procedure for DIF detection. Applied Measurement in Education, 14(4), 329–349.
19.
JöreskogK. G. (1971). Statistical analysis of sets of congeneric tests. Psychometrika, 36, 109–123.
20.
KiteB. A.JorgensenT. D.ChenP.-Y. (2018). Random permutation testing applied to measurement invariance testing with ordered-categorical indicators. Structural Equation Modeling: A Multidisciplinary Journal, 25(4), 573–587.
21.
KlineR. B. (2016). Principles and practice of structural equation modeling. New York: The Guilford Press.
22.
LinnR. L. (2005). Evaluating the validity of assessments: The consequences of use. Educational Measurement Issues and Practice, 16(2), 14–16.
23.
LordF. M. (1980). Applications of item response theory to practical testing problems. Lawrence Erlbaum Associates.
24.
LordF. M.NovickM. R. (1968). Statistical theories of mental test scores. Addison-Wesley.
25.
LubkeG. H.MuthénB. O. (2004). Applying Multigroup Confirmatory Factor Models for Continuous Outcomes to Likert Scale Data Complicates Meaningful Group Comparisons. Structural Equation Modeling, 11(4), 514–534.
26.
MarcoulidesK. M.YuanK.-H. (2016). New ways to evaluate goodness of fit: A note on using equivalence testing to assess structural equation models. Structural Equation Modeling: A Multidisciplinary Journal, 24(1), 1–6.
27.
MarcoulidesK.M.YuanK-H. (2017). New ways to evaluate goodness of fit: A note on using equivalence testing to assess structural equation models. Structural Equation Modeling: A multidisciplinary journal, 34(1): 1–6.
28.
McAllisterP. H. (1993). Testing, DIF, and public policy. In HollandP. W.WainerH. (Eds.), Differential item functioning (pp. 389–396). Lawrence Erlbaum Associates.
29.
McDonaldR. P. (1999). Test theory: A unified treatment. Lawrence Erlbaum Associates.
30.
MellenberghG. J. (2016). Models for continuous responses. In van der LindenW. J. (Ed.), Handbook of item response theory (pp. 153–163). CRC Press/Taylor & Francis Group.
MessickS. (1989). Validity. In LinnR. L. (Ed.), Educational measurement (3rd ed., pp. 13–103). Macmillan.
33.
MichaelidesM. P. (2010). A review of the effects on IRT item parameter estimates with a focus on misbehaving common items in test equating. Frontiers in Psychology, 1, Article 167.
34.
MillsapR. E. (2011). Statistical approaches to measurement invariance. Routledge/Taylor & Francis Group.
MinchenN. D.de la TorreJ.LiuY. (2017). A cognitive diagnosis model for continuous response. Journal of Educational and Behavioral Statistics, 42(6), 651–677.
37.
MontoyaA. K.JeonM. (2020). MIMIC models for uniform and nonuniform DIF as moderated mediated models. Applied Psychological Measurement, 44(2), 118–136.
38.
MulaikS. A. (2009). Linear causal modeling with structural equations. Chapman & Hall.
39.
MüllerH. (1987). A Rasch model for continuous ratings. Psychometrika, 52, 165–181.
40.
MurakiE. (1992). A generalized partial credit model: Application of an EM algorithm. ETS Research Report Series, 1992(1), 1–30.
41.
NarayanonP.SwaminathanH. (1996). Identification of items that show nonuniform DIF. Applied Psychological Measurement, 20(3), 257–274.
42.
NoelY.DauvierB. (2007). A Beta item response model for continuous bounded responses. Applied Psychological Measurement, 31(1), 47–73.
43.
OsterlindS. J.EversonH. T. (2009). Differential item functioning. SAGE.
44.
PutnickD. L.BornsteinM. H. (2016). Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Developmental Review, 41, 71–90.
45.
RajuN. S.van der LindenW J.FleerP. F. (1995). An IRT-based internal measure of test bias with applications for differential item functioning. Applied Psychological Mea surement, 19, 353–368.
46.
RaschG. (1980). Probabilistic model for some intelligence and achievement tests. University of Chicago Press.
47.
R Core Team. (2020). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org/
48.
ReiseS. P.WidamanK. F.PughR. H. (1993). Confirmatory factor analysis and item response theory: Two approaches for exploring measurement invariance. Psychological Bulletin, 114(3), 552–566.
49.
RogersH. J.SwaminathanH. (1993). A comparison of logistic regression and Mantel-Haenszel procedures for detecting differential item functioning. Applied Psychological Measurement, 17(2), 105–116.
50.
RoussosL. A.StoutW. F. (1996). Simulation studies of the effects of small sample size and studied item parameters on SIBTEST and Mantel-Haenszel type I error performance. Journal of Educational Measurement, 33(2), 215–230.
51.
RutkowskiL.SvetinaD. (2017). Measurement invariance in international surveys: Categorical indicators and fit measure performance. Applied Measurement in Education, 30(1), 39–51.
SamejimaF. (1973). Homogeneous case of the continuous response model. Psychometrika, 38, 203–219.
54.
ScherbaumC. A.GoldsteinH. W. (2008). Examining the relationship between race-based differential item functioning and item difficulty. Educational and Psychological Measurement, 68(4), 537–553.
55.
ShealyR.StoutW. (1993). A model-based standardization approach that separates true bias/DIF from group ability differences and detects test bias/DTF as well as item bias/DIF. Psychometrika, 58(2), 159–194.
56.
ShojimaK. (2005). A noniterative item parameter solution in each EM cycle of the continuous response model. Educational Technology Research, 28, 11–22.
57.
SwaminathanH.RogersH. J. (1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27, 361–370.
58.
ThissenD.SteinbergL.WainerH. (1988). Use of item response theory in the study of group differences in trace lines. In WainerH.BraunH. I. (Eds.), Test validity (pp. 147–169). Lawrence Erlbaum Associates.
59.
WangT.ZengL. (1998). Item parameter estimation for a continuous response model using an EM algorithm. Applied Psychological Measurement, 22(4), 333–344.
60.
WichertsJ. M.DolanC. V. (2010). Measurement invariance in confirmatory factor analysis: An illustration using IQ test performance of minorities. Educational Measurement Issues and Practices, 29(3), 39–47.
61.
WoodsC. M. (2009). Empirical selection of anchors for tests of differential item functioning. Applied Psychological Measurement, 33, 42–57.
62.
WoodsC. M.GrimmK. J. (2011). Testing for nonuniform differential item functioning with multiple indicator multiple cause models. Applied Psychological Measurement, 35(5), 339–361.
63.
YuanK.-H.BentlerP. M. (2004). On chi-square difference and z tests in mean and covariance structure analysis when the base model is misspecified. Educational and Psychological Measurement, 64(5), 737–757.
64.
YuanK.-H.ChanW.MarcoulidesG. A.BentlerP. M. (2016). Assessing structural equation models by equivalence testing with adjusted fit indexes. Structural Equation Modeling: A Multidisciplinary Journal, 23(3), 319–330.
65.
ZopluogluC. (2012). EstCRM: An R package for Samejima’s continuous IRT model. Applied Psychological Meausrement, 36(2), 149–150.
66.
ZopluogluC. (2020). A finite mixture item response theory model for continuous measurement outcomes, 80(2), 346–364.
67.
ZumboB. D. (1999). A handbook on the theory and methods of differential item functioning (DIF): Logistic regression modeling as a unitary framework for Binary and Likert-Type (Ordinal) Item Scores. Directorate of Human Resources Research and Evaluation, Department of National Defense.
