This article reports the first results of a long-term research project focusing on
the detection and possible linguistic causes of differential item functioning (DIF)
for second generation immigrant students in the Final Test of Primary Education in
the Netherlands. The main aim of the project is to provide test constructors with
information which is as detailed as possible about sources of DIF in order to help
them avoid item bias in future forms of the test. The project was carried out in
three steps. First, DIF items were identified using two statistical procedures: IRT
and Mantel-Haenszel. The second stage was an investigation of sources of DIF,
beginning with an inventory of subject matter elements which can cause difficulty
for immigrant students, according to previous educational research. This was
followed by a careful evaluation of DIF items by the researchers and by external
experts, and a think-aloud experiment with students. At the third stage, judges
decided whether each DIF item was biased and thus reduced the construct validity of
the test. The article concentrates on the classification of linguistic sources of
DIF and a consideration of whether they work to the advantage or disadvantage of
immigrant students.
Get full access to this article
View all access options for this article.
References
1.
Allalouf, A.
,
Hambleton, R. K.
and
Sireci, S. G.1999:
Identifying the causes of DIF in translated verbal items
. Journal of Educational Measurement, 36,
185-198
.
2.
APA
,
AERA
and
NCME
(American Psychological Association, American Education Research Association
and National Council on Measurement in Education). 1999: Standards
for educational and psychological testing.
Washington, DC: American Psychological Association
.
3.
Boonman, J.
and
Kok, W.1986: Kennisverwerven uit teksten: onderzoek naar factoren die van
invloed zijn op het verwerven van kennis uit teksten door leerlingen van de
achtste groep van de basisschool [Acquiring knowledge from texts: study of
factors affecting the acquisition of knowledge from texts by Grade 8 students of
primary schools].
Utrecht: Rijksuniversiteit Utrecht
.
4.
Bügel, K.
and
Glas, C.1991:
Item-specifieke verschillen in prestaties van jongens en meisjes bij
tekstbegripexamens moderne vreemde talen [Item-specific differences in
achievement of boys and girls in reading comprehension tests for modern
foreign languages]
. Tijdschrift voor Onderwijsresearch, 16,
337-351
.
5.
Cacciari, C.
and
Glucksberg, S.1991: Understanding idiomatic expressions: the contribution of
word meanings. In
Simpson, G. B.
, editor, Understanding word and sentence.
Amsterdam: North-Holland/Elsevier Science
.
6.
Camilli, G.
and
Shepard, L. A.1994: Methods for identifying biased test items.
Thousand Oaks, CA: Sage
.
7.
Camilli, G.
and
Smith, J. K.1990:
Comparison of the Mantel-Haenszel test with randomized and jackknife test
for detecting biased items
. Journal of Educational Statistics15,
53-67
.
8.
Chen, H.
and
Henning, G.1985.
Linguistic and cultural biases in language proficiency tests
. Language Testing2,
155-164
.
9.
Coenen, M.
and
Vallen, T.1991:
Item bias in de Eindtoets Basisonderwijs [Item bias in the Final Test of
Primary Education]
. Pedagogische Studiën68,
15-26
.
10.
de Jong, M.
and
Vallen, T.1989:
Linguïstische en culturele bronnen van item bias in de Eindtoets
Basisonderwijs voor leerlingen uit etnische minderheidsgroepen [Linguistic
and cultural DIF sources in the Final Test of Primary Education for
nonnative students]
. Pedagogische Studiën66,
390-402
.
11.
Dorans, N. J.
and
Holland, P. W.1993: DIF detection and description: Mantel-Haenszel and
standardization. In
Holand, P. W.
and
Wainer, H. W.
, editors, Differential item functioning.
Hillsdale, NJ: Lawrence Erlbaum
, 35-66.
12.
Glas, C. A. W.
and
Ouborg, M. J.1993: Vraagonzuiverheid [Item bias]. In
Eggen, T. J. H. M.
and
Sanders, P. F.
, editors, Psychometrie in de praktijk [Psychometrics in practice].
Arnhem: Cito, Instituut voor toetsontwikkeling
, 349-370.
13.
Hacquebord, H.1989: Tekstbegrip van Turkse en Nederlandse leerlingen in het
voortgezet onderwijs [Reading comprehension of Turkish and Dutch students in
secondary education].
Dordrecht: Foris
.
14.
Hambleton, R. K.
and
Jones, R. W.1992: Comparison of empirical and judgmental methods for detecting
differential item functioning.
Princeton, NJ: Educational Testing Service
.
15.
Hambleton, R. K.
and
Rogers, H. J.1989:
Detecting potentially biased test items: comparison of IRT area and
Mantel-Haenszel methods
. Applied Measurement in Education2,
313-334
.
16.
Hambleton, R. K.
,
Clauser, B. E.
,
Mazor, K. M.
and
Jones, R. W.1993:
Advances in the detection of differentially functioning test items
. European Journal of Psychological Assessment9,
1-18
.
17.
Holland, W. P.
and
Thayer, D. T.1988: Differential item performance and the Mantel-Haenszel
procedure. In
Wainer, H.
and
Braun, H. I.
, editors, Test validity.
Hillsdale, NJ: Lawrence Erlbaum
, 129-145.
18.
Kerkhoff, A.1988: Taalvaardigheid en schoolsucces [Language skills and
educational success].
Lisse: Swets and Zeitlinger
.
19.
Kim, M.2001:
Detecting DIF across the different language groups in a speaking test
. Language Testing18,
89-114
.
20.
Maller, S. J.2001:
Differential item functioning in the WISC-III: item parameters for boys and
girls in the national standardization sample
. Educational and Psychological Measurement61,
793-817
.
21.
Mellenbergh, G. J.1989:
Item bias and item response theory
. International Journal of Educational Research13,
127-143
. Special issue entitled Applications of item response theory.
22.
Pander Maat, H.1994: Tekstanalyse: Een pragmatische benadering [Text analysis: a
pragmatic approach].
Groningen: Martinus Nijhoff Uitgevers
.
23.
Poortinga, Y. H.2001: Het gebruik van psychologische tests bij allochtonen: een
probleembeschrijving [The use of psychological tests for ethnic minorities: a
problem description]. In
Bleichrodt, N.
and
van de Vijver, F. J. R.
, editors, Diagnostiek bij allochtonen, mogelijkheden en beperkingen van
psychologische tests [Diagnostics in nonnatives, possibilities and restrictions
of psychological tests].
Lisse: Swets and Zeitlinger
, 15-42.
24.
Renkema, J.1994: Taal mag geen belasting zijn [Language should not be a
burden].
The Hague: Sdu Uitgeverij
.
25.
Schmitt, A. P.
,
Holland, P. W.
and
Dorans, N. J.1993: Evaluating hypotheses about differential item
functioning. In
Holland, P. W.
and
Wainer, H. W.
, editors, Differential item functioning.
Hillsdale, NJ: Lawrence Erlbaum
, 281-315.
26.
Stricker, L. J.
and
Emmerich, W.1999:
Possible determinants of differential item functioning: familiarity,
interest, and emotional reaction
. Journal of Educational Measurement36,
347-366
.
27.
Takala, S.
and
Kaftandjieva, F.2000:
Test fairness: a DIF analyses of an L2 vocabulary test
. Language Testing17,
323-340
.
28.
Thissen, D.
,
Steinberg, L.
and
Wainer, H.1993: Detection of differential item functioning using the
parameters of item response models. In
Holland, P. W.
and
Wainer, H. W.
, editors, Differential item functioning.
Hillsdale, NJ: Lawrence Erlbaum
, 67-113.
29.
Uiterwijk, H.1994: De bruikbaarheid van de Eindtoets Basisonderwijs voor
allochtone leerlingen [The usability of the Primary Education Final Test for
nonnative students].
Arnhem: Cito, Instituut voor Toetsontwikkeling
.
30.
Uiterwijk, H.
and
Vallen, T.1997:
Onderzoek naar bias voor allochtone leer-lingen in de Cito-Eindtoets
Basisonderwijs [Research into bias for non-native students in the Cito
Primary Education Final Test]
. Pedagogische Studiën74,
21-32
.
31.
van de Waal, M.1992: Expert-oordelen over potentiële bronnen van item bias
in de Eindtoets Basisonderwijs [Expert assessments of potential sources of item
bias in the Primary Education Final Test].
Tilburg: Katholieke Universiteit Brabant
.
32.
Verhelst, N. D.1988: De Mantel-Haenszel-toetsen [The Mantel-Haenszel tests].
Arnhem: Cito, Instituut voor Toetsontwikkeling
.
33.
Verhelst, N. D.
,
Glas, C. A. W.
and
Verstralen, H. H. F. M.1995: One-parameter logistic model: OPLM.
Arnhem: Cito, Instituut voor toetsontwikkeling
.
34.
Wijnstra, J. M.
, editor 1988: Balans van het rekenonderwijs in de basisschool
[An investigation of maths education in primary schools].
Arnhem: Cito, Instituut voor toetsontwikkeling
.
