Restricted accessResearch articleFirst published online 2011-12
Formulating the Rasch Differential Item Functioning Model Under the Marginal Maximum Likelihood Estimation Context and Its Comparison With Mantel–Haenszel Procedure in Short Test and Small Sample Conditions
This study elaborates the Rasch differential item functioning (DIF) model formulation under the marginal maximum likelihood estimation context. Also, the Rasch DIF model performance was examined and compared with the Mantel–Haenszel (MH) procedure in small sample and short test length conditions through simulations. The theoretically known relationship of the DIF estimators between the Rasch DIF model and the MH procedure was confirmed. In general, the MH method showed a conservative tendency for DIF detection rates compared with the Rasch DIF model approach. When there is DIF, the z test (when the standard error of the DIF estimator is estimated properly) and the likelihood ratio test in the Rasch DIF model approach showed higher DIF detection rates than the MH chi-square test for sample sizes of 100 to 300 per group and test lengths ranging from 4 to 39. In addition, this study discusses proposed Rasch DIF classification rules that accommodate statistical inference on the direction of DIF.
AdamsR. J.WilsonM.WangW. C. (1997). The multidimensional random coefficients multinomial logit model. Applied Psychological Measurement, 21, 1-23.
2.
AdamsR. J.WilsonM.WuM. (1997). Multilevel item response models: An approach to errors in variables regression. Journal of Educational and Behavioral Statistics, 22, 47-76.
3.
BirnbaumA. (1968). Some latent trait models and their use in inferring an examinee’s ability. In LordF. M.NovickM. R. (Eds.), Statistical theories of mental test scores (pp. 397-479). Reading, MA: Addison-Wesley.
4.
BohrerR. (1974). Multiple three-decision rules for parametric signs. Journal of the American Statistical Association, 74, 432-437.
5.
BrykA. S.RaudenbushS. W.CongdonR. T. (1996). HLM: Hierarchical linear and nonlinear modeling with the HLM/2L and HLM/3L programs. Chicago, IL: Scientific Software.
6.
CamilliG.ShepardL. A. (1994). Methods for identifying biased test items. Thousand Oaks, CA: Sage.
7.
DavidianM.GiltinanD. M. (1998). Nonlinear models for repeated measurement data. New York, NY: Chapman & Hall/CRC.
8.
De BoeckP.WilsonM. (2004). Explanatory item response models: A generalized linear and nonlinear approach. New York, NY: Springer-Verlag.
9.
DonoghueJ. R.HollandP. W.ThayerD. T. (1993). A Monte Carlo study of factors that affect the Mantel–Haenszel and standardization measures of differential item functioning. In HollandP. W.WainerH. (Eds.), Differential item functioning (pp. 137-166). Hillsdale, NJ: Lawrence Erlbaum.
10.
DoransN. J.HollandP. W. (1993). DIF detection and description: Mantel-Haenszel and Standardization. In HollandP. W.WainerH. (Eds.), Differential item functioning (pp. 35-66). Hillsdale, NJ: Lawrence Erlbaum.
11.
FidalgoA. M.FerreresD.MunizJ. (2004). Utility of the Mantel–Haenszel procedure for detecting differential item functioning in small samples. Educational and Psychological Measurement, 64, 925-936.
12.
FischerG. H. (1973). The linear logistic test model as an instrument in educational research. Acta Psychologica, 37, 357-374.
13.
FischerG. H. (1983). Logistic latent trait models with linear constraints. Psychometrika, 48, 3-26.
14.
GlasC. A. W. (1998). Detection of differential item functioning using Lagrange multiplier tests. Statistica Sinica, 8, 647-667.
15.
HansonB. A. (1998). Uniform DIF and DIF defined by differences in item response functions. Journal of Educational and Behavioral Statistics, 3, 244-253.
16.
HarrisR. J. (1997). Reforming significance testing via three-valued logic. In HarlowL. L.MulaikS. A.SteigerJ. H. (Eds.), What if there were no significance tests? (pp. 145-174). Hillsdale, NJ: Erlbaum.
17.
HollandP. W.ThayerD. T. (1988). Differential item performance and the Mantel–Haenszel procedure. In WainerH.BraunH. I. (Eds.), Test validity (pp. 129-145). Hillsdale, NJ: Lawrence Erlbaum.
18.
HollandP. W.WainerH. (1993). Differential item functioning. Hillsdale, NJ: Lawrence Erlbaum.
19.
KaiserH. F. (1960). Directional statistical decisions. Psychological Review, 67, 160-167.
20.
KamataA. (2001). Item analysis by the hierarchical generalized linear model. Journal of Educational Measurement, 38, 79-93.
21.
LinacreJ. M.WrightB. D. (1987). Item bias: Mantel–Haenszel and the Rasch model (MESA Psychometric Laboratory, Memorandum No. 39). Chicago, IL: University of Chicago Press.
22.
LinacreJ. M.WrightB. D. (1989). Mantel–Haenszel DIF and PROX are equivalent!Rasch Measurement Transactions, 3(2), 52-53.
23.
LongfordN. T.HollandP. W.ThayerD. T. (1993). Stability of the MH D-DIF statistics across populations. In HollandP. W.WainerH. (Eds.), Differential item functioning (pp. 171-196). Hillsdale, NJ: Lawrence Erlbaum.
24.
LordF. M. (1977). A study of item bias, using item characteristic curve theory. In PoortingaY. H. (Ed.), Basic problems in cross-cultural psychology (pp. 19-29). Amsterdam, Netherlands: Swets & Zitlinger.
25.
LordF. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.
26.
MantelN.HaenszelW. (1959). Statistical aspects of the analysis of data from retrospective studies of disease. Journal of National Cancer Institution, 22, 719-768.
27.
MapurangaR.DoransN. J.MiddletonK. (2008). A review of recent developments in differential item functioning (ETS Research Rep. No. 08-43). Princeton, NJ: Educational Testing Service.
28.
MazorK. M.ClauserB. E.HambletonR. K. (1992). The effect of sample size on the functioning of the Mantel–Haenszel statistic. Educational and Psychological Measurement, 52, 443-452.
29.
MellenberghG. J. (1982). Contingency table methods for assessing item bias. Journal of Educational Statistics, 7, 105-118.
30.
MeredithW.MillsapR. E. (1992). On the misuse of manifest variables in the detection of measurement bias. Psychometrika, 57, 289-311.
PaekI. (2002). Investigation of differential item functioning: Comparisons among approaches, and extension to a multidimensional context (Unpublished doctoral dissertation). University of California, Berkeley.
35.
ParshallC. G.MillerT. R. (1995). Exact versus asymptotic Mantel–Haenszel DIF statistics: A comparison of performance under small-sample conditions. Journal of Educational Measurement, 32, 302-316.
36.
RaschG. (1980). Probabilistic models for some intelligence and attainment tests. Chicago, IL: University of Chicago Press. (Original work published 1960)
37.
RijimenF.TuerlinckxF.De BoeckP.KuppensP. (2003). A nonlinear mixed model framework for item response theory. Psychological Methods, 8, 185-205.
38.
RoussosL. A.SchnipkeD. L.PashleyP. J. (1999). A generalized formula for the Mantel–Haenszel differential item functioning parameter. Journal of Educational and Behavioral Statistics, 24, 293-322.
39.
SAS Institute. (1999). SAS online doc (Version 8). Cary, NC: Author.
40.
SchulzE. M.PerlmanC.RiceW. K.WrightB. D. (1996). An empirical comparison of Rasch and Mantel–Haenszel procedures for assessing differential item functioning. In EngelhardG.Jr.WilsonM. (Eds.), Objective measurement: Theory into practice (Vol. 3, pp. 65-82). Norwood, NJ: Ablex.
41.
ShafferJ. P. (1972). Directional statistical hypotheses and comparisons among means. Psychological Bulletin, 77, 195-197.
42.
ShafferJ. P. (1974). Bidirectional unbiased procedures. Journal of the American Statistical Association, 69, 437-439.
43.
ShealyR.StoutW. F. (1993). A model-based standardization approach that separates true bias/DIF from group differences and detects test bias/DIF as well as item bias/DIF. Psychometrika, 58, 159-194.
44.
SkrondalA.Rabe-HeskethS. (2004). Generalized latent variable modeling: Multilevel, longitudinal and structural equation models. New York, NY: Chapman & Hall/CRC.
45.
ThissenD.SteinbergL.GerrardM. (1986). Beyond group mean differences: The concept of item bias. Psychological Bulletin, 99, 118-128.
46.
ThissenD.SteinbergL.WainerH. (1988). Use of item response theory in the study of group differences in trace lines. In WinterH.BraunH. (Eds.), Test validity (pp. 147-169). Hillsdale, NJ: Lawrence Erlbaum.
47.
ThissenD.SteinbergL.WainerH. (1993). Detection of differential item functioning using the parameters of item response models. In HollandP. W.WainerH. (Eds.), Differential item functioning (pp. 67-113). Hillsdale, NJ: Lawrence Erlbaum.
48.
TukeyJ. W. (1991). The philosophy of multiple comparisons. Statistical Science, 6, 100-116.
49.
Van den NoortgateW.De BoeckP.MeuldersM. (2003). Cross-classification multilevel logistic models in psychometrics. Journal of Educational and Behavioral Statistics, 28, 369-386.
50.
Van den NoortgateW.PaekI. (2004). Person regression models. In De BoeckP.WilsonM. (Eds.), Explanatory item response models: A generalized linear and nonlinear approach (pp. 167-187). New York, NY: Springer.
51.
WangW.-C. (2004). Effects of anchor item methods on the detection of differential item functioning within the family of Rasch models. Journal of Experimental Education, 72, 221-261.
52.
WangW.-C. (2008). Assessment of differential item functioning. Journal of Applied Measurement, 9, 387-408.
53.
WangW.-C.SuY.-H. (2004). Effects of average signed area between two item characteristic curves and test purification procedures on the DIF detection via the Mantel–Haenszel method. Applied Measurement in Education, 17, 113-144.
54.
WrightB. D.StoneM. H. (1979). Best test design. Chicago, IL: MESA.
ZwickR. (1990). When do item response function and Mantel–Haenszel definitions of differential item functioning coincide?Journal of Educational Statistics, 15, 185-197.
