Activity in the area of language testing is expanding beyond second language
acquisition. In many contexts, tests that measure language skills are being
translated into several different languages so that parallel versions exist for use
in multilingual contexts. To ensure that translated items are equivalent to their
original versions, both statistical and qualitative analyses are necessary. In this
article, we describe a statistical method for evaluating the translation equivalence
of test items that are scored dichotomously. We provide an illustration of the
method to a portion of the verbal subtest of the Psychometric Entrance Test, which
is a large-scale postsecondary admissions test used in Israel. By evaluating
translated items statistically, language test-developers can ensure the
comparability of tests across languages and they can identify the types of problems
that should be avoided in future translation efforts.
Get full access to this article
View all access options for this article.
References
1.
Allalouf, A.
,
Hambleton, R. K.
and
Sireci, S. G.1999:
Identifying the causes of differential item functioning in translated
verbal items
. Journal of Educational Measurement36,
185-198
.
2.
Allalouf, A.
and
Sireci, S. G.
1998:
Detecting sources of differential item functioning in translated verbal items
. Paper presented at the annual meeting of the American
Educational Research Association, San Diego, CA.
3.
American Educational Research Association
,
American Psychological Association and National Council on Measurement in Education1999: Standards for educational and psychological testing.
Washington, DC: American Psychological Association
.
4.
Angoff, W. H.1982: Use of difficulty and discrimination indices for detecting
item bias. In
Berk, R. A.
, editor, Handbook of methods for detecting test bias.
Baltimore, MD: Johns Hopkins University Press
, 96-116.
5.
Angoff, W. H.
and
Cook, L. L.1988: Equating the scores of the Prueba de Aptitud Academica and the
Scholastic Aptitude Test. Report No. 88-2.
New York: College Entrance Examination Board
.
6.
Angoff, W. H.
and
Ford, S. F.1973:
Item-race interaction on a test of scholastic aptitude
. Journal of Educational Measurement10,
95-106
.
7.
Angoff, W. H.
and
Modu, C. C.1973: Equating the scores of the Prueba de Aptitud Academica and the
Scholastic Aptitude Test (Research Report 3).
New York: College Entrance Examination Board
.
8.
Beller, M.1994:
Psychometric and social issues in admissions to Israeli universities
. Educational Measurement: Issues and Practice13,
12-21
.
9.
Beller, M.1995: Translated versions of Israel’s inter-university
Psychometric Entrance Test (PET). In
Oakland, T.
and
Hambleton, R. K.
, editors, International perspective of academic assessment.
Boston, MA: Kluwer Academic
, 207-217.
10.
Budgell, G.
,
Raju, N.
and
Quartetti, D.1995:
Analysis of differential item functioning in translated assessment instruments
. Applied Psychological Measurement19,
309-321
.
11.
Camilli, G.
and
Shepard, L. A.1994: Methods for identifying biased test items.
Thousand Oaks, CA: Sage
.
12.
Clauser, B. E.
and
Mazor, K. M.1998:
Using statistical procedures to identify differentially functioning test items
. Educational Measurement: Issues and Practice17,
31-44
.
13.
Clauser, B. E.
,
Nungester, R. J.
,
Mazor, K.
and
Ripkey, D.1996:
A comparison of alternative matching strategies for DIF detection in tests
that are multidimensional
. Journal of Educational Measurement33,
202-214
.
14.
Cook, L. L.
1996:
Establishing score comparability for tests given in different languages
. Paper presented at the annual meeting of the American
Psychological Association, Toronto, Canada.
15.
Dorans, N. J.
and
Holland. P. W.1993: DIF detection and description: Mantel-Haenszel and
standardization. In
Holland, P. W.
and
Wainer, H.
, editors, Differential item functioning.
Hillsdale, NJ: Lawrence Erlbaum
, 35-66.
16.
Dorans, N. J.
and
Kulick, E.1986:
Demonstrating the utility of the standardization approach to assessing
unexpected differential item performance on the scholastic aptitude test
. Journal of Educational Measurement23,
355-368
.
17.
Elley, W. B.1994: The IEA Study of Reading literacy: achievement and instruction
in thirty-two school systems.
Tarrytown, NY: Pergamon Press
.
18.
Gafni, N.
and
Canaan-Yehoshafat, Z.
1993:
An examination of differential item functioning for Hebrew- and
Russian-speaking examinees in Israel
. Paper presented at the Conference of the Israeli Psychological
Association, Ramat-Gan.
19.
Geisinger, K. F.1994:
Cross-cultural normative assessment: translation and adaptation issues
influencing normative interpretation of assessment instruments
. Psychological Assessment6304-312
.
20.
Gierl, M. J.
and
Khaliq, S. N.
2000:
Identifying the sources of translation item and bundle functioning on
translated achievement tests: a confirmatory analysis
. Paper presented at the annual meeting of the American
Educational Research Association, New Orleans, LA.
21.
Gierl, M. J.
,
Rogers, W. T.
and
Klinger, D.
1999:
Using statistical and judgmental reviews to identify and interpret
translation DIF
. Paper presented at the annual meeting of the National Council on
Measurement in Education, Montréal, Quebec, Canada.
22.
Hambleton, R. K.1994:
Guidelines for adapting educational and psychological tests: a progress report
. European Journal of Psychological Assessment10,
229-244
.
23.
Hambleton, R. K.
in press: Issues, designs, and technical guidelines for adapting
tests in multiple languages and cultures. In
Hambleton, R. K
,
Merenda, P.
and
Spielberger, C.
, editors, Adapting educational and psychological tests for
cross-cultural assessment.
Hillsdale, NJ: Lawrence Erlbaum
.
24.
Holland, P. W.
and
Thayer, D. T.1988: Differential item functioning and the Mantel-Haenszel
procedure. In
Wainer, H.
and
Braun, H. I.
, editors, Test validity.
Hillsdale, NJ: Lawrence Erlbaum
,129-145.
25.
Holland, P. W.
and
Wainer, H.
, editors, 1993: Differential item functioning.
Hillsdale, NJ: Lawrence Erlbaum
.
26.
Lord, F. M.1980: Applications of item response theory to practical testing
problems.
Hillsdale, NJ: Lawrence Erlbaum
.
27.
Mantel, N.
and
Haenszel, W.1959:
Statistical aspects of the analysis of data from retrospective studies of disease
. Journal of the National Cancer Institute22,
19-48
.
28.
Millsap, R. E.
and
Everson, H. T.1993:
Methodology review: statistical approaches for assessing measurement bias
. Applied Psychological Measurement17,
297-334
.
29.
Muñ iz, J.
,
Hambleton, R. K.
and
Xing, D.2001:
Small sample studies to detect flaws in test translation
. International Journal of Testing1,
115-135
.
30.
Potenza, M. T.
and
Dorans, N. J.1995:
DIF assessment for polytomously scored items: a framework for
classification and evaluation
. Applied Psychological Measurement19,
23-37
.
31.
Raju, N. S.1988:
The area between two item characteristic curves
. Psychometrika54,
495-502
.
32.
Rogers, H. J.
,
Swaminathan, H.
and
Hambleton, R. K.1993: DICHODIF: a FORTRAN program for DIF analysis of dichotomously
scored item response data [A computer program].
Amherst, MA: University of Massachusetts
.
33.
Shealy, R.
and
Stout, W.1993:
A model-based standardization differences and detects test bias/DTF as well
as item bias/DIF
. Psychometrika58,
159-194
.
34.
Sireci, S. G.1989: Districtwide testing results.
Newark, NJ: New Jersey Board of Education
.
35.
Sireci, S. G.1997:
Problems and issues in linking tests across languages
. Educational Measurement: Issues and Practice16,
12-19
.
36.
Sireci, S. G.
,
Bastari, B.
and
Allalouf, A.
1998:
Evaluating construct equivalence across adapted tests
. Invited paper presented at the annual meeting of the American
Psychological Association (Division 5), San Francisco, CA.
37.
Sireci, S. G.
and
Berberoglu, G.2000:
Evaluating translation DIF using bilinguals
. Applied Measurement in Education13,
229-248
.
38.
Sireci, S. G.
,
Fitzgerald, C.
and
Xing, D.
1998:
Adapting credentialing examinations for international uses
. Paper presented at the annual meeting of the American
Educational Research Association, San Diego, CA.
39.
Sireci, S. G.
,
Foster, D.
,
Olsen, J. B.
and
Robin, F.
1997:
Comparing dual-language versions of international computerized
certification exams
. Paper presented at the annual meeting of the National Council on
Measurement in Education, Chicago, IL.
40.
Sireci, S. G.
,
Hambleton, R. K.
and
Patsula, L.
in press: Statistical methods for identifying flawed items in the
test adaptations process. In
Hambleton, R. K.
,
Merenda, P.
and
Spielberger, C.
, editors, Adapting educational and psychological tests for
cross-cultural assessment.
Hillsdale, NJ: Lawrence Erlbaum
.
41.
Swaminathan, H.
and
Rogers, H. J.1990:
Detecting differential item functioning using logistic regression procedures
. Journal of Educational Measurement27,
361-370
.
42.
Thissen, D
,
Steinberg, L.
and
Wainer, H.1988: Use of item response theory in the study of group
differences in trace lines. In
Wainer, H.
and
Braun, H. I.
, editors, Test validity.
Hillsdale, NJ: Lawrence Erlbaum
, 147-169.
43.
Thissen, D
,
Steinberg, L.
and
Wainer, H.1993: Detection of differential item functioning using the
parameters of item response models. In
Holland, P. W.
and
Wainer, H.
, editors, Differential item functioning.
Hillsdale, NJ: Lawrence Erlbaum
, 67-113.
44.
Van de Vijver, F. J.
and
Tanzer, N. K.1997:
Bias and equivalence in cross-cultural assessment: an overview
. European Review of Applied Psychology47,
263-279
.
