This paper reports the findings of an empirical study on ESL/EFL teachers’ evaluation and interpretation of oral English proficiency as elicited by the national College English Test-Spoken English Test (CET-SET) of China. Informed by debates on the issue of native speaker (NS) norms which have become the focus of attention in recent years, this study addresses the question of whether judgments of language proficiency by non-native English speaking (NNES) teachers, who are currently used to assess performance on the CET-SET, correspond to those of native English speaking (NES) teachers or whether the two groups draw on different constructs of oral proficiency. Data for the study were derived from two sources: unguided holistic ratings given by a group of 19 NES and 20 NNES teachers to CET-SET speech samples from 30 test-takers, and written comments to justify the ratings assigned. Results yielded by both quantitative (MFRM) and qualitative analyses of teacher data, revealed no significant difference in raters’ holistic judgments of the speech samples and a broad level of agreement between groups on the construct components of oral English proficiency. However, the analysis of raters’ comments revealed both quantitative and qualitative differences in the way NES and NNES teachers weighed various features of the oral proficiency construct in justifying the decisions made. The paper concludes by considering the implications of the study’s findings for debates about the native speaker norm as the target for language learners and test-takers.
Alderson JC ( 1991). Bands and scores. In J. C. Alderson and B. North (Eds.), Language testing in the 1990s: The communicative legacy (pp. 71-86). London : Macmillan.
2.
Bachman LF, Lynch B., and Mason M. ( 1995). Investigating variability in tasks and rater judgments in a performance test of foreign language speaking. Language Testing, 12(2), 238-256.
3.
Barnwell D. ( 1989). ‘Naive’ native speakers and judgements of oral proficiency in Spanish. Language Testing, 6(2), 152-163.
4.
Beneke J. ( 1991). English als lingua franca oder als Medium interkultureller Kommunikation. In R. Grebing (Ed.), Grenzenloses Sprachenlernen (pp. 54-66). Berlin: Cornelsen.
5.
Berns M. ( 2005). Expanding on the Expanding Circle: Where do WE go from here? World Englishes, 24(1), 85-93.
6.
Bolton K. ( 2002). Chinese Englishes: From Canton jargon to global English . World Englishes, 21(2), 181-199.
7.
Bonk WJ, Ockey GJ ( 2003). A many-facet Rasch analysis of the second language group oral discussion task. Language Testing, 20(1), 89-110.
8.
Brown A. ( 1995). The effect of rater variables in the development of an occupation-specific language performance test. Language Testing , 12(1), 1-15.
9.
Brown A. ( 2000). An investigation of the rating process in the IELTS oral interview. In R. Tullon (Ed.), IELTS Research Reports 2000 (Vol. 3, pp. 49-84). Canberra : IELTS Australia, 2000.
10.
Brown A. ( 2004). Interviewer variability in oral proficiency. Unpublished PhD thesis. University of Melbourne.
11.
Brown A., Iwashita N., and McNamara T. ( 2005). An examination of rater orientations and test-taker performance on English-for-Academic-Purposes speaking tasks (No. RR-05-05).
12.
Bruthiaux P. ( 2003). Squaring the circles: Issues in modeling English worldwide . International Journal of Applied Linguistics , 13(2), 159-178.
13.
Brutt-Griffler J., Samimy KK ( 2001). Transcending the nativeness paradigm. World Englishes, 20(1), 99-106.
14.
Chalhoub-Deville M. ( 1995a). A contextualized approach to describing oral language proficiency. Language Learning, 45(2), 251-281.
15.
Chalhoub-Deville M. ( 1995b). Deriving oral assessment scales across different tests and rater groups. Language Testing, 12, 16-33.
16.
Cook V. ( 1999). Going beyond the native speaker in language teaching . TESOL Quarterly, 33(2), 185-209.
17.
Crewe W. ( 1977). Singapore English as a non-native dialect. In W. Crewe (Ed.), The English language in Singapore (pp. 96-119). Singapore: Eastern Universities Press.
18.
Crystal D. ( 1997). English as a global language. Cambridge : Cambridge University Press.
19.
Davies A. ( 1991). The native speaker in applied linguistics. Edinburgh: Edinburgh University Press .
20.
Davies A. ( 1996). Proficiency or the native speaker: What are we trying to achieve in ELT? In Guy Cook and B. Seidlhofer (Eds.), Principle and practice in applied linguistics (pp. 145-159). Oxford: Oxford University Press.
21.
Davies A. ( 1999). Standard English: Discordant voices. World Englishes, 18(2), 171-186.
22.
Davies A. ( 2003). The native speaker: Myth and reality (2nd ed.). Buffalo, NY: Multilingual Matters.
23.
Davies A. ( 2004). The native speaker in applied linguistics. In Alan Davies and C. Elder (Eds.), The handbook of applied linguistics (pp. 431-450). Malden, MA: Blackwell.
24.
Davies A., Hamp-Lyons L., and Kemp C. ( 2003). Whose norms? International proficiency tests in English . World Englishes, 22(4), 571-584.
25.
Elder C., Davies A. ( 2006). Assessing English as a lingua franca. Annual Review of Applied Linguistics, 26, 282-304.
26.
Elder C., Harding L. ( 2008). Language testing and English as an international language . Australian Review of Applied Linguistics, 31(3), 34.31-34.11.
27.
Fayer JM, Krasinski E. ( 1987). Native and non-native judgments of intelligibility and irritation. Language Learning, 37(3), 313-326.
28.
Fulcher G. ( 2003). Testing second language speaking. PearsonLongman.
29.
Galloway VB ( 1980). Perceptions of the communicative efforts of American students of Spanish. Modern Language Journal, 64, 428-433.
30.
Gnutzmann C. ( 2000). Lingua franca. In M. Byram (Ed.), The Routledge encyclopedia of language teaching and learning (pp. 356-359). London: Routledge .
31.
Graddol D. ( 1997). The future of English?: A guide to forecasting the popularity of the English language in the 21st century. London: British Council.
32.
Graddol D. ( 2001). The decline of the native speaker. In AILA Review 13: English in a changing world (pp. 57-68).
33.
Green A. ( 1998). Verbal protocol analysis in language testing research: A handbook. Cambridge, UK: University of Cambridge, Local Examinations Syndicate.
34.
Hadden BL ( 1991). Teacher and nonteacher perceptions of second-language communication. Language Learning, 41, 1-24.
35.
Hamilton J., Lopes M., McNamara T., and Sheridan E. ( 1993). Rating scales and native speaker performance on a communicatively oriented EAP test. Language Testing, 10, 337-353.
36.
Hamp-Lyons L., Zhang BW ( 2001). World Englishes: Issues in and from academic writing assessment. In J. Flowerdew and M. Peacock (Eds.), Research perspectives on English for academic purposes. Cambridge: Cambridge University Press.
37.
Hill K. ( 1996). Who should be the judge? The use of non-native speakers as raters on a test of English as an international language. Melbourne Papers in Language Testing, 5(2), 29-49.
38.
Hinofotis FB, Bailey KM, and Stern SL ( 1981). Assessing the oral proficiency of prospective foreign teaching assistants: Instrument development. In A. Palmer, P. J. Groot and G. H. Trosper (Eds.), The construct validation of tests of communicative competence (pp. 106-126). Washington, DC: TESOL.
39.
Iwashita N., McNamara T., and Elder C. ( 2001). Can we predict task difficulty in an oral proficiency test? Exploring the potential of an information-processing approach to task design. Language Learning, 51(3), 401-436.
40.
Jenkins J. ( 1998). Which pronunciation norms and models for English as an International Language? ELT J, 52, 119-126.
41.
Jenkins J. ( 2000). The phonology of English as an International Language: New models, new norms, new goals. Oxford: Oxford University Press.
42.
Jenkins J. ( 2002). A sociolinguistically based, empirically researched pronunciation syllabus for English as an International Language. Applied Linguistics, 23, 89-103.
43.
Jin Y. ( 2000). Washback study of CET Spoken English Test on teaching . Foreign Language World, 80(4), 56-61.
44.
Joseph JE ( 2004). Language and identity: National, ethnic, religious . Basingstoke, UK: Pal-grave Macmillan .
45.
Kachru BB ( 1985). Standards, codification and sociolinguistic realism: The English language in the outer circle. In R. Quirk and H. G. Widdowson (Eds.), English in the world: Teaching and learning the language and literatures (pp. 11-30). Cambridge: Cambridge University Press .
46.
Kim Y. ( 2009). An investigation into native and non-native teachers’ judgments of oral English performance: A mixed methods approach. Language Testing, 26(2), 187-217.
47.
Kirkpatrick A., Xu Z. ( 2002). Chinese pragmatic norms and ‘China English’ . World Englishes, 21(2), 269-279.
48.
Kramsch C. ( 2007). The uses of communicative competence in a global world . In J. Liu (Ed.), English language teaching in China: New approaches, perspectives and standards (pp. 55-74). London, New York : Continuum International Publishing.
49.
Lazaraton A. ( 2005). Non-native speakers as language assessors: Recent research and implications for assessment practice. Paper presented at the BAAL, Bristol.
Lowenberg PH ( 1993). Issues of validity in tests of English as a world language: Whose standards? World Englishes, 12(1), 95-106.
52.
Lowenberg PH ( 2000). Non-native varieties and issues of fairness in testing English as a world English. In A. J. Kunnan (Ed.), Fairness and validation in language assessment : Selected papers from the 19th Language Testing Research Colloquium. Orlando, Florida .
53.
Lowenberg PH ( 2002). Assessing English proficiency in the Expanding Circle . World Englishes, 21(3), 431-435.
54.
Lukmani Y. ( 2002). English in India: Assessment issues. Paper presented at the Hong Kong Seminar, Hong Kong.
55.
Lynch BK, McNamara TF ( 1998). Using G-theory and Many-facet Rasch Measurement in the development of performance assessments of the ESL speaking skills of immigrants . Language Testing, 15(2), 158-180.
56.
Malabonga V. ( 2004). Davies A 2003: The native speaker: myth and reality . Language Testing, 21(2), 245-248.
57.
McArthur T. ( 1998). The English language. Cambridge : Cambridge University.
58.
McArthur T. ( 2004). Is it world or international or global English, and does it matter?English Today, 20(3), 3-15.
59.
McKay S. ( 2002). Teaching English as an International Language . Oxford: Oxford University Press .
60.
McNamara TF ( 1996). Measuring second language performance. London and New York: Addison Wesley Longman.
61.
McNamara TF, Lumley T. ( 1997). The effect of interlocutor and assessment mode variables in overseas assessments of speaking skills in occupational settings . Language Testing, 14(2), 140-156.
62.
Medgyes P. ( 1999). Language training: A neglected area in teacher education . In G. Braine (Ed.), Non-native educators in English language teaching. Mahwah, NJ: Lawrence Erlbaum.
63.
Melchers G., Shaw P. ( 2003). World Englishes. London: Arnold.
64.
Mullis IVS ( 1980). Using the primary trait system for evaluating writing (Manuscript No. 10-W51). Princeton, NJ: Educational Testing Service.
65.
Myford CM, Wolfe EW ( 2003). Detecting and measuring rater effects using many-facet Rasch measurement: PartI. Journal of Applied Measurement , 4(4), 386-421.
66.
Myford CM, Wolfe EW ( 2004). Detecting and measuring rater effects using many-facet Rasch measurement: Part II. Journal of Applied Measurement , 5(2), 189-227.
67.
National College English Testing Committee ( 1999). CET-SET syllabus. Shanghai: Shanghai Foreign Language Education Press.
68.
North B., Schneider G. ( 1998). Scaling descriptors for language proficiency scales . Language Testing, 15(2), 217-262.
69.
Pennycook A. ( 2003). Global Englishes, rip slyme, and performativity . Journal of Sociolinguistics, 7(4), 513-533.
70.
Phillipson R. ( 1992). Linguistic imperialism. Oxford [England];New York: Oxford University Press.
71.
Pollitt A., Murray NL ( 1996). What raters really pay attention to. In M. Milanovic and N. Saville (Eds.), Performance testing, cognition and assessment (pp. 74-91). Cambridge: University of Cambridge Local Examinations Syndicated and Cambridge University Press.
72.
Quirk R. ( 1990). What is Standard English? In R. Quirk and G. Stein (Eds.), English in use (pp. 112-125). London: Longman .
73.
Rampton MBH ( 1990). Displacing the ‘native speaker’: Expertise, affiliation, and inheritance. ELT Journal, 44, 97-101.
74.
Ross JR ( 1979). Where’s English? In C. J. Fillmore, D. Kempler and W. S.-Y. Wang (Eds.), Individual differences in language ability and language behavior (pp. 127-163). New York: Academic Press.
75.
Santos TA ( 1988). Professors’ reactions to the academic writing of non-native speaking students. TESOL Quarterly, 22(1), 69-90.
76.
Schnitzer E. ( 1995). English as an international language: Implications for interculturalists and language educators. International Journal of Intercultural Relations, 19(2), 227-236.
77.
Seidlhofer B. ( 2001). Closing a conceptual gap: The case for a description of English as a lingua franca. International Journal of Applied Linguistics, 11(2), 133-158.
78.
Sheorey R. ( 1985). Goof gravity in ESL: Native vs. nonnative perceptions . Paper presented at the 19th annual TESOL convention , New York.
79.
Shi L. ( 2001). Native- and nonnative-speaking EFL teachers’ evaluation of Chinese students’ English writing. Language Testing , 18, 303-325.
80.
Stanlaw J. ( 2004). Japanese English: Language and culture contact. Hong KongHong Kong University Press.
81.
Timmis I. ( 2002). Native-speaker norms and International English: A classroom view. ELT J, 56(3), 240-249.
82.
Weigle SC ( 1998). Using FACETS to model rater training effects. Language Testing, 15(2), 263-267.
83.
Wigglesworth G. ( 1997). An investigation of planning time and proficiency level on oral test discourse. Language Testing, 14(1), 85-106.
84.
Wright BD, Linacre JM ( 1994). Reasonable mean-square fit values. Rasch Measurement: Transactions of the Rasch Measurement SIG, 8, 370.
85.
Yano Y. ( 2001). World Englishes in 2000 and beyond. World Englishes, 20(2), 119-132.
