This study tracked gender differential item functioning (DIF) on the English subtest of the Korean College Scholastic Aptitude Test (KCSAT) over a nine-year period across three data points, using both the Mantel-Haenszel (MH) and item response theory likelihood ratio (IRT-LR) procedures. Further, the study identified two factors (i.e. reading strategy and perceived interest) that explained a portion of the variance in the magnitude of gender DIF via a series of multiple linear regression analyses. The results indicated (1) an interaction between item type and gender DIF and (2) a significant relationship between gender differences in the examinee’s perceived interest in test items and the magnitude of gender DIF. The study discusses the results based on previous DIF research and presents pedagogical implications and some avenues for further research.
AbbottM. L. (2006). ESL reading strategies: Differences in Arabic and Mandarin speaker test performance. Language Learning, 56(4), 633–670.
2.
AbbottM. L. (2007). A confirmatory approach to differential item functioning on an ESL reading assessment. Language Testing, 24(1), 7–36.
3.
AckermanT.GierlM.WalkerC. (2003). Using multidimensional item response theory to evaluate educational and psychological tests. Educational Measurement: Issues and Practices, 22, 37–53.
4.
AngoffW. H. (1993). Perspectives on differential item functioning methodology. In HollandP. W.WainerH. (Eds.), Differential item functioning (pp. 3–23). Hillsdale, NJ: Lawrence Erlbaum.
5.
BaconS. M.FinemannM. D. (1990). A study of the attitudes, motives, and strategies of university foreign language students and their disposition to authentic oral and written input. Modern Language Journal, 74(4), 459–473.
6.
BalowI. H.FarrR. C.HoganT. P. (1993). Compendium of instructional objectives: Metropolitan Achievement Tests (7th ed.). San Antonio, TX: Psychological Corporation.
7.
BloomB.MesiaB.KrathwohlD. (1964). Taxonomy of educational objectives. New York: David McKay.
8.
BoltD. M. (2002). A Monte Carlo comparison of parametric and nonparametric polytomous DIF detection methods. Applied Measurement in Education, 15, 113–141.
9.
BrantmeierC. (2001). Second language reading research on passage content and gender: Challenges for the intermediate-level curriculum. Foreign Language Annals, 34(4), 325–333.
10.
BuckG.KostinL.MorganR. (2002). Examining the relationship of content to gender-based performance differences in advanced placement exams (College Board Research Report RR-02-25). New York: College Entrance Examination Board.
11.
BügelK.BuunkB. P. (1996). Sex differences in foreign language text comprehension: The role of interests and prior knowledge. The Modern Language Journal, 80, 15–31.
12.
CamilliG.ShepardL. (1994). Methods for identifying biased test items. Thousand Oaks, CA: SAGE Publications.
13.
CarltonS. T.HarrisA. M. (1992). Characteristics associated with differential item functioning on the Scholastic Aptitude Test: Gender and majority/minority group comparisons. ETS Research Report, 92–64. Princeton, NJ: ETS.
14.
ChenZ.HenningG. (1985). Linguistic and cultural bias in language proficiency tests. Language Testing, 2(2), 155–163.
15.
CohenJ. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum.
16.
CohenA.BoltD. (2002). A mixture model analysis of differential item functioning. Paper presented at the annual meeting of the American Educational Research Association, New Orleans.
17.
CurleyW.SchmittA. P. (1993). Revising SAT-Verbal items to eliminate differential item functioning. College Board Report No. 93-2.
18.
DouglasJ. A.RoussosL. A.StoutW. (1996). Item bundle DIF hypothesis testing: Identifying suspect bundles and assessing their differential functioning. Journal of Educational Measurement, 33, 465–484.
19.
EhrmanM.OxfordR. (1988). Effects of sex differences, career choice, and psychological type on adult language learning strategies. Modern Language Journal, 72, 253–265.
20.
ElderC. (1996). The effect of language background on foreign language test performance: The case of Chinese, Italian, and modern Greek. Language Learning, 46(2), 233–282.
21.
EngelhardG. (1990). Gender differences in performance on mathematics items: Evidence from the United States and Thailand. Contemporary Educational Psychology, 15, 13–26.
22.
EngelhardG.HanscheL.RutledgeK. E. (1990). Accuracy of bias review judges in identifying differential item functioning on Teacher Certification Tests. Applied Measurement in Education, 3, 347–360.
23.
FerneT.RuppA. (2007). A synthesis of 15 years of research on DIF in language testing: Methodological advances, challenges, and recommendations. Language Assessment Quarterly, 4(2), 113–148.
24.
GafniN. (1991). Differential item functioning: Performance by sex on reading comprehension tests. ERIC Document ED 331844. Rockville, MD: Educational Resources Information Center.
25.
GierlMBisanzJ.BisanzG.BoughtonK. (2003). Identifying content and cognitive skills that produce gender differences in mathematics: A demonstration of the multidimensionality-based DIF analysis. Journal of Educational Measurement, 40(4), 281–306.
26.
HambletonR. K.SwaminathanH.RogersH. J. (1991). Fundamentals of item response theory. Thousand Oakes, CA: Sage Publications.
27.
HollandP. W.ThayerD. T. (1988). Differential item performance and Mantel-Haenszel procedure. In WainerH.BraunH. (Eds.), Test validity (pp. 129–145). Hillsdale, NJ: Lawrence Erlbaum.
28.
JinK.ParkC. (2005). Factors related to the difficulty of College Scholastic Ability Test English items. English Teaching, 60(3), 17–25.
29.
KICE [Korea Institute of Curriculum & Evaluation]. (2004). Test manual for the Korean College Scholastic Aptitude Test: English Subtest. Seoul: KICE.
30.
KimM. (2001). Detecting DIF across the different language groups in a speaking test. Language Testing, 18(1), 89–114.
31.
LawrenceI. M.CurleyW. E. (1989). Differential item functioning for males and females on SAT-Verbal Reading subscore items: Follow-up study. Educational Testing Service Research Report 89-22. Princeton, NJ: ETS.
32.
LawrenceI. M.CurleyW. E.McHaleF. J. (1988). Differential item functioning for males and females on SAT verbal reading subscore items. (Report No. 88-4). New York: College Entrance Examination Board.
33.
LinJ.WuF. (2003). Differential performance by gender in foreign language testing. Poster presented at the annual meeting of the National Council on Measurement in Education, Chicago.
34.
LinnR. L. (1993). The use of differential item functioning statistics: A discussion of current practice and future implications. In HollandP. W.WainerH. (Eds.), Differential item functioning (pp. 349–364). Hillsdale, NJ: Lawrence Erlbaum.
35.
LordF. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: LEA.
36.
MallerS. J. (1997). Deafness and WISC-III item difficulty: Invariance and fit. Journal of School Psychology, 35, 299–314.
37.
MallerS. J. (2001). Differential item functioning in the WISC-III: Item parameters for boys and girls in the national standardization sample. Educational and Psychological Measurement, 61, 793–817.
38.
McPeekW. M.WildC. L. (1992). Identifying differentially functioning items in the NTE Core Battery. ETS Research Report 92-62. Princeton, NJ: ETS.
39.
Mendes-BarnettS.ErcikanK. (2006). Examining sources of gender DIF in mathematics assessments using a confirmatory multidimensional model approach. Applied Measurement in Education, 19(4), 289–304.
40.
O’NeillK. A.McPeekW. M. (1993). Item and test characteristics that are associated with differential item functioning. In HollandP. W.WainerH. (Eds.), Differential item functioning (pp. 35–66). Hillsdale, NJ: Lawrence Erlbaum.
41.
O’NeillK. A.McPeekW. M.WildC. L. (1993). Differential item functioning on the graduate management admission test. ETS Research Report 93-35. Princeton, NJ: ETS.
42.
OshimaT. C.RajuN. S.FlowersC. P.SlindeJ. A. (1998). Differential bundle functioning using the DFIT framework: Procedures for identifying possible sources of differential functioning. Applied Measurement in Education, 11(4), 353–369.
43.
PaeT.-I. (2004a). DIF for examinees with different academic backgrounds. Language Testing, 21, 53–73.
44.
PaeT.-I. (2004b). Gender effect on reading comprehension with Korean EFL learners. System, 32, 265–281.
45.
PaeT.-I.ParkG.-P. (2006). Examining the relationship between differential item functioning and differential test functioning. Language Testing, 23(4), 475–496.
46.
ParkG.-P. (2008). Differential item functioning on an English listening test across gender. TESOL Quarterly, 42(1), 115–122.
47.
PlakeB. S. (1980). A comparison of a statistical and subjective procedure to ascertain item validity: One step in the test validation process. Educational and Psychological Measurement, 40, 397–404.
RyanK.BachmanL.F. (1992). Differential item functioning on two tests of EFL proficiency. Language Testing, 9(1), 12–29.
50.
SasakiM. (1991). A comparison of two methods for detecting differential item functioning in an ESL placement test. Language Testing, 8(2), 95–111.
51.
ScheunemanJ. D.GerritzK. (1990). Using differential item functioning procedures to explore sources of item difficulty and group performance characteristics. Journal of Educational Measurement, 27, 109–131.
52.
SchmittA. P.DoransN. J.CroneC. R.ManeckshanaB. T. (1991). Differential speededness and item omit patterns on the SAT. ETS Research Report 91-50. Princeton, NJ: ETS.
53.
ShealyR.T.StoutW. (1993). A model-based standardization approach that separates true bias/DIF from group ability differences and detects test bias/DTF as well as item bias/DIF. Psychometrika, 58, 159–194.
54.
StrickerL. J.EmmerichW. (1999). Possible determinants of differential item functioning: Familiarity, interest, and emotional reaction. Journal of Educational Measurement, 36(4), 347–366.
55.
TakalaS.KaftandjievaF. (2000). Test fairness: A DIF analysis of an L2 vocabulary test. Language Testing, 17, 323–40.
56.
ThissenD.SteinbergL.WainerH. (1988). Use of item response theory in the study of group differences in trace lines. In WainerH.BraunH. (Eds.), Test validity (pp. 147–169). Hillsdale, NJ: Lawrence Erlbaum.
57.
WendlerC. L.CarltonS. T. (1987, April). An examination of SAT verbal items for differential performance by women and men: Exploratory study. Paper presented at the annual meeting of the American Educational Research Association, Washington DC.
58.
YoungD. J.OxfordR. (1997). A gender-related analysis of strategies used to process written input in the native language and a foreign language. Applied Language Learning, 8(1), 43–73.
59.
ZiekyM. (1993). Practical questions in the use of DIF statistics in test development. In HollandP. W.WainerH. (Eds.), Differential item functioning (pp. 337–347). Hillsdale, NJ: Lawrence Erlbaum.
