The purpose of this simulation study was to establish general effect size guidelines for interpreting the results of differential bundle functioning (DBF) analyses using simultaneous item bias test (SIBTEST). Three factors were manipulated: number of items in a bundle, test length, and magnitude of uniform differential item functioning (DIF) against the focal group in each item in a bundle. A secondary purpose was to validate the current effect size guidelines for interpreting the results of single-item DIF analyses using SIBTEST. The results of this study clearly demonstrate that ability estimation bias can only be attributed to DIF or DBF when a large number of items in a bundle are functioning differentially against focal examinees in a small way or a small number of items are functioning differentially against focal examinees in a large way. In either of these situations, the presence of DIF or DBF should be a cause for concern because it would lead one to erroneously believe that distinct groups differ in ability when in fact they do not.
AbbottM. L. (2006). ESL reading strategies: Differences in Arabic and Mandarin speakers test performance. Language Learning, 56, 633-670.
2.
AbbottM. L. (2007). A confirmatory approach to differential item functioning on an ESL reading assessment. Language Testing, 24, 7-36.
3.
AckermanT. A.EvansJ. A. (1994). The influence of conditioning scores in performing DIF analyses. Applied Psychological Measurement, 18, 329-342.
4.
AngoffW. H. (1993). Perspectives on differential item functioning methodology. In HollandP. W.WainerJ. (Eds.), Differential item functioning (pp. 3-24). Mahwah, NJ: Lawrence Erlbaum.
5.
AugustD.CarloM.DresslerC.SnowC. (2005). The critical role of vocabulary development for English language learners. Learning Disabilities Research & Practice, 20, 50-57.
6.
BanksJ. A. (1995). The historical reconstruction of knowledge about race: Implications for transformative teaching. Educational Researcher, 24(2), 15-25.
7.
BanksJ. A. (2006). Multicultural education: Characteristics and goals. In BanksJ. A.BanksC. A. (Eds.), Multicultural education: Issues and perspectives (6th ed., pp. 3-26). Hoboken, NJ: Wiley.
8.
BanksK. (2006). A comprehensive framework for evaluating hypotheses about cultural bias in educational testing. Applied Measurement in Education, 19, 115-132.
9.
ClauserB. E.MazorK. M. (1998). Using statistical procedures to identify differentially functioning test items. Educational Measurement: Issues and Practice, 17(1), 31-44.
10.
Cochran-SmithM. (2003). Multicultural teacher education: Research, practice and policy. In BanksJ. A.BanksC. M. (Eds.), Handbook of research in multicultural education (2nd ed., pp. 931-975). San Francisco, CA: Jossey-Bass.
11.
DouglasJ. A.RoussosL. A.StoutW. (1996). Item-bundle DIF hypothesis testing: Identifying suspect bundles and assessing their differential functioning. Journal of Educational Measurement, 33, 465-484.
12.
FinchH. (2005). The MIMIC model as a method for detecting DIF: Comparison with Mantel-Haenszel, SIBTEST and the IRT likelihood ratio. Applied Psychological Measurement, 29, 278-295.
13.
FinchH.BartonK.MeyerP. (2009). Differential item functioning analyses for accommodated versus non-accommodated students. Educational Assessment, 14, 38-56.
14.
FurlowC. F.RossT. R.GagneP. (2009). The impact of multidimensionality on the detection of differential bundle functioning using simultaneous item bias test. Applied Psychological Measurement, 33, 441-464.
15.
GayG. (2003). Curriculum theory and multicultural education. In BanksJ. A.BanksC. M. (Eds.), Handbook of research in multicultural education (2nd ed., pp. 30-49). San Francisco, CA: Jossey-Bass.
16.
GierlM. J. (2005). Using dimensionality-based DIF analyses to identify and interpret constructs that elicit group differences. Educational Measurement: Issues and Practice, 24(1), 3-14.
17.
GierlM. J.BisanzJ.BisanzG. L.BoughtonK. A. (2003). Identifying content and cognitive skills that produce gender differences in mathematics: A demonstration of the multidimensionality-based DIF analysis paradigm. Journal of Educational Measurement, 40, 281-307.
18.
GierlM. J.BisanzJ.BisanzG. L.BoughtonK. A.KhaliqS. N. (2001). Illustrating the utility of differential bundle functioning analyses to identify and interpret group differences on achievement tests. Educational Measurement: Issues and Practice, 20(2), 26-36.
19.
GierlM. J.GotzmannA.BoughtonK. A. (2004). Performance of SIBTEST when the percentage of DIF items is large. Applied Measurement in Education, 17, 241-264.
20.
GierlM. J.KhaliqS. N. (2001). Identifying sources of differential item and bundle functioning on translated achievement tests: A confirmatory approach. Journal of Educational Measurement, 38, 164-187.
21.
IrvineJ. J. (2002). In search of wholeness: African American teachers and their culturally specific classroom practices. New York, NY: Palgrave.
22.
IrvineJ. J. (2003). Educating teachers for diversity: Seeing with a cultural eye. New York, NY: Teachers College Press.
23.
IrvineJ. J.CollisonM. (1999). Preparing teachers for urban classrooms. Black Issues in Higher Education, 16(20), 30-32.
24.
JordanA. H.LovettB. J. (2007). Stereotype threat and test performance: A primer for school psychologists. Journal of School Psychology, 45, 45-59.
25.
JiangH.StoutW. F. (1998). Improved Type I error control and reduced estimation bias for DIF detection using SIBTEST. Journal of Educational and Behavioral Statistics, 23, 291-322.
26.
Ladson-BillingsG. (1994). The dreamkeepers: Successful teachers of African American children. San Francisco, CA: Jossey-Bass.
27.
Ladson-BillingsG. (2000). Fighting for our lives: Preparing teachers to teach African American students. Journal of Teacher Education, 51, 206-214.
28.
LauferB. (1989). What percentage of text-lexis is essential for comprehension? In LaurenC.NordmanM. (Eds.), Special language: From humans thinking to thinking machines (pp. 316-323). Clevedon, England: Multilingual Matters.
29.
LeeC. D.MajorsY. J. (2003). Heading up the street: Localized opportunities for shared constructions of knowledge. Pedagogy, Culture & Society, 11, 49-67.
30.
LiuN.NationI. S. P. (1985). Factors affecting guessing vocabulary in context. RELC Journal, 16(1), 33-47.
31.
MahoneyK. (2008). Linguistic influences on differential item functioning for second language learners on the National Assessment of Educational Progress. International Journal of Testing, 8, 14-33.
32.
MartinielloM. (2008). Language and the performance of English language learners on math word problems. Harvard Educational Review, 78, 333-368.
33.
MartinielloM. (2009). Linguistic complexity, schematic representations, and differential item functioning for English language learners in math tests. Educational Assessment, 14, 160-179.
34.
Mendes-BarnettS.ErcikanK. (2006). Examining sources of gender DIF in mathematics using a confirmatory multidimensional model approach. Applied Measurement in Education, 19, 289-304.
35.
NandakumarR. (1993). Simultaneous DIF amplification and cancellation: Shealy-Stout’s measure test for DIF. Journal of Educational Measurement, 30, 293-311.
36.
OckeyG. J. (2007). Investigating the validity of math word problems for English language learners with DIF. Language Assessment Quarterly, 4, 149-164.
37.
RoussosL. A.StoutW. F. (1996). Simulation studies of the effects of small sample size and studied item parameters on SIBTEST and Mantel-Haenszel Type I error performance. Journal of Educational Measurement, 33, 215-230.
38.
RyanA. M. (2006). The role of social foundations in preparing teachers for culturally relevant practice. Multicultural Education, 13(3), 10-13.
39.
RyanK. E.ChiuS. (2001). An examination of item context effects, DIF, and gender DIF. Applied Measurement in Education, 14, 73-90.
40.
RyanK. E.FanM. (1996). Examining gender DIF on a multiple-choice test of mathematics: A confirmatory approach. Educational Measurement: Issues and Practice, 15(4), 15-20.
41.
RyanK. E.RyanA. M. (2005). Psychological processes underlying stereotype threat and standardized math test performance. Educational Psychologist, 40, 53-63.
42.
SackettP.HardisonC.CullenM. (2004). On interpreting stereotype threat as accounting for African American-White differences on cognitive tests. American Psychologist, 59, 7-13.
43.
ShealyR.StoutW. (1993). A model-based standardization approach that separates true bias/DIF from group ability differences and detects test bias/DTF as well as item bias/DIF. Psychometrika, 58, 159-194.
44.
SleeterC. (2001). Preparing teachers for culturally diverse schools: Research and the overwhelming presence of whiteness. Journal of Teacher Education, 52, 94-106.
45.
SnetzlerS.QuallsA. L. (2000). Examination of differential item functioning on a standardized achievement battery with limited English proficient students. Educational and Psychological Measurement, 60, 564-577.
46.
SteeleC. (1997). A threat in the air: How stereotypes shape intellectual identity and performance. American Psychologist, 52, 613-629.
47.
WalkerC. M.BeretvasS. N. (2001). An empirical investigation demonstrating the multidimensional DIF paradigm: A cognitive explanation for DIF. Journal of Educational Measurement, 38, 147-163.
48.
WalkerC. M.ZhangB.SurberJ. (2008). Using a multidimensional differential item functioning framework to determine if reading ability affects student performance in mathematics. Applied Measurement in Education, 21, 162-181.
49.
YoungJ. W.ChoY.LingG.ClineF.SteinbergJ.StoneE. (2008). Validity and fairness of state standards-based assessments for English language learners. Educational Assessment, 13, 170-192.
50.
ZwickR.ErcikanK. (1989). Analysis of differential item functioning in the NAEP history assessment. Journal of Educational Measurement, 26, 55-66.
