The cognitive demands of summary writing are dependent upon the type of summary to be produced. This paper reports part of a larger study in which 157 Chinese undergraduates were asked to write summaries of extended English texts in both English and Chinese. It examines the differential effects of the use of the two languages on summarization as a measure of reading comprehension, drawing upon data from students' actual test performances as well as their perceptions of such differential effects, as evidenced in the post-summarization questionnaire and interviews. It was found that the use of the different languages had significant effects on both summarization processes and products. Students wrote significantly longer Chinese summaries but these were rated consistently of poorer quality than English summaries. However, Chinese summarization was found to be a better measure of students' English reading abilities. The implications of these findings are discussed with specific reference to summarization task design, in particular, how the use of the first and target languages may affect test takers' performance as well as rater behaviours and the potential of using first language in large-scale international language testing programmes incorporating integrated reading/writing tasks as a measure of reading comprehension.
Alderson J.C. (1996). The testing of reading. In C. Nuttall (Ed.), Teaching reading skills in a foreign language . London: Heinemann.
2.
Alderson, J.C., Clapham, C., & Wall, D. (1995). Language test construction and evaluation. Cambridge: Cambridge University Press.
3.
Axelrod, J. (1975). Getting the main idea is still the main idea. Journal of Reading, 18(5), 383-387.
4.
Bachman, L.F. (1990). Fundamental considerations in language testing . Oxford: Oxford University Press .
5.
Bensoussan, M., & Kreindler, I. (1990). Improving advanced reading comprehension in a foreign language: Summaries vs. short-answer questions. Journal of Research in Reading, 13(1), 55-68.
6.
Bernhardt, E.B. (1991). Reading development in a second language: Theoretical, empirical, and classroom perspectives. Norwood, NJ: Ablex.
7.
Brown, A.L., & Day, J.D. (1983). Macrorules for summarizing texts: The development of expertise. Journal of Verbal Learning and Verbal Behavior , 22(1), 1-14.
Cohen, A.D. (1993). The role of instructions in testing summarizing ability. In D. Douglas & C. Chapelle (Eds.), A new decade of language testing research (pp. 132-159). Washington, DC: TESOL, Inc.
10.
Cohen, A.D. (1994). English for academic purposes in Brazil: The use of summary tasks. In C. Hill & K. Parry (Eds.), From testing to assessment: English as an international language (pp. 174-204). London: Longman.
11.
Corbeil, G. (2000). Exploring the effects of first- and second-language proficiency on summarizing in French as a second language. Canadian Journal of Applied Linguistics, 3(1-2), 35-62.
12.
Cordero-Ponce, W.L. (2000). Summarization instruction: Effects on foreign language comprehension and summarization of expository texts. Reading Research & Instruction, 39(4), 329-350.
13.
Courchêne, R., & Bayliss, D. (1995). Summary cloze: What is it? What does it measure? In R. Courchêne, S. Burger, C. Cornaire, R. LeBlanc, S. Paribakht & H. Seguin (Eds.), Twenty-five years of second language teaching at the university of Ottawa (pp. 305-327). Ottawa: Second Language Institute, University of Ottawa .
14.
Cronbach, L.J., Linn, R.L., Brennan, R.L., & Haertel, E. (1995). Generalizability analysis for educational assessment . Los Angeles: UCLA's Center for the Study of Evaluation & National Center for Research on Evaluation, Standards, and Student Testing.
15.
Cumming, A. (2002). Assessing L2 writing: Alternative constructs and ethical dilemmas. Assessing Writing, 8(1), 73-83.
16.
Cumming, A., Grant, L., Mulcahy-Ernt, P., & Powers, D.E. (2004). A teacher-verification study of speaking and writing prototype tasks for a new TOEFL. Language Testing, 21(2), 107-145.
17.
Cumming, A., Kantor, R., Baba, K., Erdosy, U., Eouanzoui, K., & James, M. (2005). Differences in written discourse in independent and integrated prototype tasks for next generation TOEFL. Assessing Writing, 10(1), 5-43.
18.
Cumming, A., Kantor, R., & Powers, D.E. (2001). Scoring TOEFL essays and TOEFL 2000 prototype writing tasks: An investigation into raters' decision making and development of a preliminary analytic framework (TOEFL monograph series, report no. 22) . Princeton, NJ.: Educational Testing Service.
19.
Cumming, A., Kantor, R., & Powers, D.E. (2002). Decision making while rating ESL/EFL writing tasks: A descriptive framework. Modern Language Journal, 86(1), 67-96.
20.
Cumming, A., Kantor, R., Powers, D.E., Santos, T., & Taylor, C. (2000). TOEFL 2000 writing framework: A working paper . Princeton, NJ: Educational Testing Service.
21.
Cumming, A., Rebuffot, J., & Ledwell, M. (1989). Reading and summarizing challenging texts in first and second languages. Reading and Writing: An Interdisciplinary Journal, 2(3), 201-219.
22.
Dermody, M.M., & Speaker, R.B., Jr. (1999). Reciprocal strategy training in prediction, clarification, question generating and summarization to improve reading comprehension. Reading Improvement, 36(1),16-23.
23.
Deville, C., & Chalhoub-Deville, M. (1993). Modified scoring, traditional item analysis and Sato's caution index used to investigate the reading recall protocol. Language Testing, 10(2), 117-132.
24.
Enright, M.K., Grabe, W., Koda, K., Mosenthal, P., Mulcahy-Ernt, P., & Schedl, M.A. (2000). TOEFL 2000 reading framework: A working paper . Princeton, NJ: Educational Testing Service.
25.
Fløttum, K. (1985). Methodological problems in the analysis of student summaries. Text, 5(4), 291-307.
26.
Friend, R. (2002). Summing it up - teaching summary writing to enhance science learning. The Science Teacher, 69(4), 40-43.
27.
Godev, C.B., Martinez-Gibson, E.A., & Toris, C.M. (2002). Foreign language reading comprehension test: L1 versus L2 in open-ended questions. Foreign Language Annals , 35(2), 202-221.
28.
Hamp-Lyons, L., & Kroll, B. (1997). TOEFL 2000: Writing: Composition, community and assessment. Princeton, NJ: Educational Testing Service.
29.
Head, M.H., Readence, J.E., & Buss, R.R. (1989). An examination of summary writing as a measure of reading comprehension. Reading Research & Instruction , 28(4), 1-11.
30.
Hidi, S., & Anderson, V. (1986). Producing written summaries: Task demands, cognitive operations, and implications for instruction. Review of Educational Research, 56(4), 473-493.
31.
Holmes, J.L., & Ramos, R.G. (1993). Study summaries as an evaluation instrument: Questions of validity. English for Specific Purposes, 12(1), 83-94.
32.
Huhta, A., & Randell, E. (1996). Multiple-choice summary: A measure of text comprehension . In A. Cumming & R. Berwick (Eds.), Validation in language testing (pp. 94-110). Clevedon, England: Multilingual Matters.
33.
Jitendra, A.K., Cole, C.L., Hoppes, M.K., & Wilson, B. (1998). Effects of a direct instruction main idea summarization program and self-monitoring on reading comprehension of middle school students with learning disabilities. Reading and Writing Quarterly: Overcoming Learning Difficulties, 14(4), 379-396.
34.
Johns, A.M. (1985). Summary protocols of `underprepared' and `adept' university students: Replications and distortions of the original. Language Learning, 35(4), 495-517.
35.
Johns, A.M. (1988). Reading for summarizing: An approach to text orientation and processing. Reading in a Foreign Language, 4(2), 79-90.
36.
Johns, A.M., & Mayes, P. (1990). An analysis of summary protocols of university ESL students. Applied Linguistics, 11(3), 253-271.
37.
Johnson, N.S. (1983). What do you do if you can't tell the whole story? The development of summarization skills. In K. E. Nelson (Ed.), Children's language, volume 4 (pp. 315-383). Hillsdale, NJ: Lawrence Erlbaum.
38.
Kim, S.A. (2001). Characteristics of EFL readers' summary writing: A study with Korean university students. Foreign Language Annals, 34(6), 569-581.
39.
Kintsch, W., & Kozminsky, E. (1977). Summarizing stories after reading and listening . Journal of Educational Psychology, 69(5), 491-499.
40.
Kintsch, W., & van Dijk, T.A. (1978). Toward a model of text comprehension and production. Psychological Review, 85(5), 363-394.
Kobayashi, M. (2002). Method effects on reading comprehension test performance: Text organization and response format. Language Testing, 19(2), 193-220.
43.
Kozminsky, E., & Graetz, N. (1986). First vs second language comprehension: Some evidence from text summarizing. Journal of Research in Reading, 9(1), 3-21.
44.
Kuckartz, U. (1998). winMAX: Scientific text analysis for the social sciences (Version 98 Pro). Berlin: VERBI Software-Consult-Research GmbH (Germany).
45.
Lee, J.F. (1986). On the use of the recall task to measure L2 reading comprehension. Studies in Second Language Acquisition, 8(2), 201-211.
46.
Long, J., & Harding-Esch, E. (1978). Summary and recall of text in first and second languages: Some factors contributing to performance differences. In D. Gerver & H. W. Sinaiko (Eds.), Language interpretation and communication (pp. 273-287). New York: Plenum.
47.
Nuttall, C. (1996). Teaching reading skills in a foreign language . Oxford: Macmillan Heinemann.
48.
Penny, J., Johnson, R.L., & Gordon, B. (2000). The effect of rating augmentation on inter-rater reliability: An empirical study of a holistic rubric. Assessing Writing, 7(2), 143-164.
49.
Radmacher, S.A., & Latosi-Sawin, E. (1995). Summary writing: A tool to improve student comprehension and writing in psychology. Teaching of Psychology, 22(2), 113-115.
50.
Riley, G.L., & Lee, J.F. (1996). A comparison of recall and summary protocols as measures of second language reading comprehension. Language Testing, 13(2), 173-189.
51.
Rivard, L.P. (2001). Summary writing: A multi-grade study of French-immersion and Francophone secondary students. Language, Culture and Curriculum, 14(2), 171-186.
52.
Sawaki, Y. (2003). A comparison of summarization and free recall as reading comprehension tasks in web-based assessment of Japanese as a foreign language. Unpublished PhD thesis, University of California, Los Angeles (UCLA), Los Angeles.
53.
Scott, M.L., Stansfield, C.W., & Kenyon, D.M. (1996). Examining validity in a performance test: The listening summary translation exam (LSTE) - Spanish version. Language Testing, 13(1), 83-109.
54.
Seidlhofer, B. (1995). Approaches to summarization: Discourse analysis and language education. Tübingen: Gunter Narr.
55.
Selinger, B.M. (1995). Summarizing text: Developmental students demonstrate a successful method. Journal of Developmental Education, 19(2), 14-16, 18, 20.
56.
Shi, L. (2004). Textual borrowing in second-language writing. Written Communication, 21(2), 171-200.
57.
Shohamy, E. (1984). Does the testing method make a difference? The case of reading comprehension. Language Testing, 1(2), 147-170.
58.
Smith, C.B. (1988). Does it help to write about your reading? Journal of Reading, 32(3), 276-285.
59.
Stansfield, C.W., Wu, W.M., & van der Heide, M. (2000). A job-relevant listening summary translation exam in Minnan. In A. J. Kunnan (Ed.), Fairness and validation in language assessment: Selected papers from the 19th LTRC (pp. 177-200). Cambridge: CambridgeUniversity Press with University of Cambridge Local Examinations Syndicate.
60.
Taylor, K.K. (1986). Summary writing by young children. Reading Research Quarterly, 21(2), 193-208.
61.
Thomas, S., & Bridge, C.A. (1980). A comparison of subjects' cloze scores and their ability to employ macrostructure operations in the generations of summaries . In M. L. Kamil & A. J. Moe (Eds.), Perspectives on reading research and instruction. Twenty-ninth yearbook of the national reading conference (pp. 69-77). Washington, DC: National Reading Conference.
62.
Trites, L., & McGroarty, M. (2005). Reading to learn and reading to integrate: New tasks for reading comprehension tests? Language Testing, 22(2), 174-210.
63.
Urquhart, A.H., & Weir, C.J. (1998). Reading in a second language: Process, product and practice. London: Longman.
64.
van Dijk, T.A., & Kintsch, W. (1983). Strategies of discourse comprehension. New York: Academic Press.
65.
Weigle, S.C. (2004). Integrating reading and writing in a competency test for non-native speakers of English. Assessing Writing , 9(1), 27-55.
66.
Weir, C.J. (1993). Understanding and developing language tests. Hemel Hempstead: Prentice Hall International (UK) Ltd.
67.
Weir, C.J. (2005). Language testing and validation: An evidence-based approach. Basingstoke: Palgrave Macmillan.
68.
Wu, W.M., & Stansfield, C.W. (2001). Towards authenticity of task in test development. Language Testing, 18(2), 187-206.
69.
Yang, L., & Shi, L. (2003). Exploring six MBA students' summary writing by introspection. Journal of English for Academic Purposes, 2(3), 165-192.
70.
Yu, G. (2005). Towards a model of using summarization tasks as a measure of reading comprehension. Unpublished PhD thesis, University of Bristol, Bristol.
71.
Yu, G. (2007a). Effects of source texts on summarization performance . Paper presented at the Language Testing Research Colloquium (29th).
72.
Yu, G. (2007b). Students' voices in the evaluation of their written summaries: Empowerment and democracy for test takers? Language Testing, 24(4), 539-572.
