Translation as a language testing procedure: does it work?

Abstract

This article looks at translation from L2 to L1 as a language testing procedure. Two studies are presented. The first examines the reliability and validity of a translation reading test under conditions where minimal, or no effort is made to ensure rater reliability. Seven raters, with no instructions or training, rated translations by 121 Japanese junior college students. All estimates of reliability proved acceptable. Criterion validity was examined by correlating the translations with other measures of passage comprehension, and were found to be satisfactory. The second study reports the inclusion of two tests of translation in a multitrait-multimethod validation study. They showed satisfactory reliability as estimated by four different methods, and examination of the correlation matrix indicated that the translation tests had acceptable construct validity with almost no method effect. It is argued that these results are generalizable to many other translation tests in other situations, and hence the widespread rejection of translation as a language testing procedure by teachers and testers is probably not warranted on psychometric grounds. However, it is further argued that translation would often have an undesirable washback effect on classroom practice, and thus ought to be used with extreme care.