Holistic reading is widely used to assess the proficiency of non-native-speaking (NNS) writers. However, ESL professionals, who have been profoundly influenced by the notion that attention to the NNS author's message is an integral part of teaching the writing process, have questioned how well native-speaking (NS) raters comprehend NNS texts, given that the task of decoding NNS prose is even further complicated by the time constraints of the holistic scoring process itself. This article describes a study that investigated the extent to which NS holistic raters comprehend NNS texts. After rating several practice compositions, subjects rated one of two qualitatively distinct essays, and then wrote recall protocols to test their comprehension. Data analysis revealed that readers of the better written text recalled significantly more than did readers of the less well written text, indicating that NS holistic raters attend to meaning when evaluating NNS writing proficiency.
Get full access to this article
View all access options for this article.
References
1.
Baker, L.
, & Anderson, R. I. (1982). Effects of inconsistent information on text processing: Evidence for comprehension monitoring. Reading Research Quarterly, 17(2), 281-293.
2.
Bernhardt, E. B.
(1983). Testing foreign language reading comprehension: The immediate recall protocol. Die Unterrichtspraxis, 16(1), 27-33.
3.
Charney, D.
(1984). The validity of using holistic scoring to evaluate writing: A critical overview. Research in the Teaching of English, 18(1), 65-81.
4.
Connor, U.
(1984). Recall of text: Differences between first and second language learners. TESOL Quarterly, 18(2), 239-256.
5.
Cooper, C. R.
(1977). Holistic evaluation in writing. In C. R. Cooper & L. Odell (Eds.), Evaluating writing: Describing, measuring, judging (pp. 3-31). Urbana, IL: NCTE.
6.
Davis, B.
, Scriven, M., & Thomas, S. (1981). The evaluation of composition instruction. Inverness, CA: Edgepress.
7.
de Beaugrande, R.
(1978). Theoretical issues in the process-product controversy. (ERIC Document Reproduction Service No. ED 161 034)
8.
Emig, J.
(1971). The composing processes of 12th graders. Urbana, IL: National Council of Teachers of English.
9.
Frederiksen, C. H.
(1977). Semantic processing units in understanding text. In R. O. Freedle (Ed.), Discourse production and comprehension. Hillsdale, NJ: Lawrence Erlbaum.
10.
Freedman, S. W.
(1979). How characteristics of student essays influence teachers' evaluations. Journal of Educational Psychology, 71, 328-338.
11.
Hayes, J. R.
, & Flower, L. S. (1978). Protocol analysis of writing processes. Paper presented at the annual meeting of the American Educational Research Association, Toronto, Canada.
12.
Hayes, J. R.
, & Flower, L. S. (1980). Identifying the organization of writing processes. In L. W. Gregg & E. R. Steinberg (Eds.), Cognitive processes in writing. Hillsdale, NJ: Lawrence Erlbaum.
Jacobs, S. E.
(1982). Composing and coherence. In Linguistics and literacy series 3. Washington, DC: Center for Applied Linguistics.
15.
Johnson, R. E.
(1970). Recall of prose as a function of the structural importance of the linguistic units. Journal of Verbal Learning and Verbal Behavior, 9, 12-20.
Mandler, J. M.
(1978). A code in the node: The use of a story schema retrieval. Discourse Processes, 1(1), 14-35.
18.
Meyer, B.J.F.
(1973). Identifying variables in prose. (ERIC Document Reproduction Service No. ED 082 218)
19.
Meyer, B.J.F.
(1982). Reading research and the composition teacher: The importance of plans. College Composition and Communication, 33(1), 37-49.
20.
Pearson, D. P.
, & Tierney, R. J. (1984). On becoming a thoughtful reader: Learning to read like a writer. In Becoming readers in a complex society: 83rd Yearbook of the National Society for the Study of Education. Chicago, IL: University of Chicago Press.
21.
Perkins, K.
(1983). On the use of composition scoring techniques, objective measures, and objective tests to evaluate ESL writing ability. TESOL Quarterly, 17(4), 651-671.
22.
Perl, S.
(1979). The composing processes of unskilled college writers. Research in the Teaching of English, 13(4), 317-339.
23.
Perl, S.
(1980). Understanding composing. College Composition and Communication, 31(4), 363-369.
24.
Raimes, A.
(1985a). What unskilled ESL writers do as they write: A classroom study of composing. TESOL Quarterly, 19(2), 229-258.
25.
Raimes, A.
(1985b). An investigation of how ESL writers write. Paper presented at the 19th annual meeting of TESOL, New York.
26.
Raimes, A.
(1986). Practical considerations in teaching writing as a process. Paper presented at the 20th annual meeting of TESOL, Anaheim, CA.
27.
Sommers, N.
(1980). Revision strategies of student writers and experienced writers. College Composition and Communication, 31(4), 378-388.
28.
White, E. M.
(1985). Teaching and assessing writing. San Francisco, CA: Jossey-Bass.
29.
Zamel, V.
(1983). In search of the key: Research and practice in composition. In J. Handscombeet al. (Eds.), On TESOL '83. Bloomington, IL: Pentagraph Printing.
30.
Zamel, V.
(1985). Responding to student writing. TESOL Quarterly, 19(1), 79-101.
31.
Zamel, V.
(1986). From process to product. Paper presented at the 20th annual meeting of TESOL, Anaheim, CA.
