Methods for testing differential item functioning (DIF) require that the reference and focal groups are linked on a common scale using group-invariant anchor items. Several anchor-selection strategies have been introduced in an item response theory framework. However, popular strategies often utilize likelihood ratio testing with all-others-as-anchors that requires multiple model fittings. The current study explored alternative anchor-selection strategies based on a modified version of the Wald χ2 test that is implemented in flexMIRT and IRTPRO, and made comparisons with methods based on the popular likelihood ratio test. Accuracies of anchor identification of four different strategies (two testing methods combined with two selection criteria), along with the power and Type I error associated with respective follow-up DIF tests, will be presented. Implications for applied researchers and suggestions for future research will be discussed.
AggenS. H.NealeM. C.KendlerK. S. (2005). DSM criteria for major depression: Evaluating symptom patterns using latent-trait item response models. Psychological Medicine, 35, 475-487. doi:10.1017/S0033291704003563
2.
AllisonK. C.EngelS. G.CrosbyR. D.de ZwaanM.O’ReardonJ. P.WonderlichS. A.. . . StunkardA. J. (2008). Evaluation of diagnostic criteria for night eating syndrome using item response theory analysis. Eating Behaviors, 9, 398-407.
3.
AnkenmannR. D.WittE. A.DunbarS. B. (1999). An investigation of the power of the likelihood ratio goodness-of-fit statistic in detecting differential item functioning. Journal of Educational Measurement, 36, 277-300.
4.
CaiL. (2008). SEM of another flavour: Two new applications of the supplemented EM algorithm. British Journal of Mathematical and Statistical Psychology, 61, 309-329. doi:10.1348/000711007X249603
5.
CaiL. (2015). flexMIRT® 3.0: Flexible multilevel and multidimensional item response theory analysis and test scoring [Computer software]. Chapel Hill, NC: Vector Psychometric Group.
6.
CaiL.ThissenD.du ToitS. H. C. (2011). IRTPRO: Flexible, multidimensional, multiple categorical IRT modeling [Computer software]. Lincolnwood, IL: Scientific Software International.
7.
ChildsR. A.DahlstromW. G.KempS. M.PanterA. T. (2000). Item response theory in personality assessment: A demonstration using the MMPI-2 depression scale. Assessment, 7, 37-54.
8.
EmbretsonS. E.ReiseS. P. (2000). Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum.
9.
FinchH. (2005). The MIMIC model as a method for detecting DIF: Comparison with Mantel-Haenszel, SIBTEST, and the IRT likelihood ratio. Applied Psychological Measurement, 29, 278-295. doi:10.1177/0146621605275728
10.
GomezR.VanceA.GomezA. (2011). Item response theory analyses of parent and teacher ratings of the ADHD symptoms for recoded dichotomous scores. Journal of Attention Disorders, 15, 269-285. doi:10.1177/1087054709356404
11.
HansonB. A.BéguinA. A. (2002). Obtaining a common scale for item response theory item parameters using separate versus concurrent estimation in the common-item equating design. Applied Psychological Measurement, 26, 3-24.
12.
HollandW. P.ThayerD. T. (1988). Differential item performance and the Mantel-Haenszel procedure. In WainerH.BraunH. (Eds.), Test validity (pp. 129-145). Hillsdale, NJ: Lawrence Erlbaum.
13.
KimS.CohenA. S.ParkT. (1995). Detection of differential item functioning in multiple groups. Journal of Educational Measurement, 32, 261-276.
14.
KolenM. J.BrennanR. L. (2014). Test equating, scaling, and linking: Methods and practices (3rd ed.). New York, NY: Springer Science + Business Media. doi:10.1007/978-1-4939-0317-7
15.
LangerM. M. (2008). A reexamination of Lord’s Wald test for differential item functioning using item response theory and modern error estimation (Unpublished doctoral dissertation). University of North Carolina, Chapel Hill.
16.
Lopez RivasG. E.StarkS.ChernyshenkoO. S. (2009). The effects of referent item parameters on differential item functioning detection using the free baseline likelihood ratio test. Applied Psychological Measurement, 33, 251-265. doi:10.1177/0146621608321760
17.
LordF. M. (1968). An analysis of the verbal scholastic aptitude test using Birnbaum’s three-parameter logistic model. Educational and Psychological Measurement, 28, 989-1020.
18.
LordF. M. (1977). A study of item bias using item characteristic curve theory. In PoortingaY. H. (Ed.), Basic problems in cross-cultural psychology (pp. 19-29). Amsterdam, The Netherlands: Swets & Zeitlinger.
19.
LordF. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.
20.
MantelN.HaenszelW. (1959). Statistical aspects of the analysis of data from retrospective studies of disease. Journal of the National Cancer Institute, 22, 719-748.
21.
Maydeu-OlivaresA.CaiL. (2006). A cautionary note on using G2(dif) to assess relative model fit in categorical data analysis. Multivariate Behavioral Research, 41, 55-64. doi:10.1207/s1532790 6mbr4101_4
22.
MeadeA. W.WrightN. A. (2012). Solving the measurement invariance anchor item problem in item response theory. Journal of Applied Psychology, 97, 1016-1031. doi:10.1037/a0027934
23.
R Core Team. (2015). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Available from http://www.R-project.org/
24.
StarkS.ChernyshenkoO. S.DrasgowF. (2006). Detecting differential item functioning with CFA and IRT: Toward a unified strategy. Journal of Applied Psychology, 91, 1292-1306. doi:10.1037/0021-9010.91.6.1292
25.
SwaminathanH.RogersH. J. (1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27, 361-370. doi:10.1111/j.1745-3984.1990 .tb00754.x
26.
ThissenD. (2001). IRTLRDIF v2.0b: Software for the computation of the statistics involved in item response theory likelihood-ratio tests for differential item functioning. Documentation for computer program [Computer software and manual]. Chapel Hill: L.L. Thurstone Psychometric Laboratory, University of North Carolina.
27.
ThissenD.SteinbergL.WainerH. (1993). Detection of differential item functioning using the parameters of item response models. In HollandP. W.WainerH. (Eds.), Differential item functioning (pp. 67-111). Hillsdale, NJ: Lawrence Erlbaum.
28.
WaldA. (1943). Tests of statistical hypotheses concerning several parameters when the number of observations is large. Transactions of the American Mathematical Society, 54, 426-482.
29.
WangW.YehY. (2003). Effects of anchor item methods on differential item functioning detection with the likelihood ratio test. Applied Psychological Measurement, 27, 479-498. doi:10.1177/0146621603259902
30.
WoodsC. M. (2009). Empirical selection of anchors for tests of differential item functioning. Applied Psychological Measurement, 33, 42-57. doi:10.1177/0146621607314044
31.
WoodsC. M.CaiL.WangM. (2013). The Langer-improved Wald test for DIF testing with multiple groups: Evaluation and comparison to two-group IRT. Educational and Psychological Measurement, 73, 532-547. doi:10.1177/0013164412464875
