Patterns of variation in the interplay of language ability and general reading comprehension ability in L2 reading

Abstract

This study aims to investigate patterns of variation in the interplay of L2 language ability and general reading comprehension skills in L2 reading, by comparing item-level effects of test-takers’ results on L1 and L2 reading comprehension tests. The material comes from more than 500,000 people tested on L1 (Swedish) and L2 (English) in the Swedish Scholastic Assessment Test (2010–16). The statistical analysis found no significant correlations between degree of L2 language focus and text length or sentence length, nor could any significant differences be found with regard to the language focus in main point, inference or retrieval items. However, a statistically significant difference was found between the rational deletion gap test and the two sections based on content questions on long and short texts, in the sense that the former to a greater degree measures language ability. Moreover, even though the gap test consists of semantic deletion only, specifically aiming to test higher-order skills, the effect size by different measures was consistently large (Cohen’s d around 0.8). Finally, an additional, more exploratory, qualitative study supported the connection between the distribution of the operationalized language focus value and an item’s relative focus on higher- or lower-level reading skills. Furthermore, it gave some indication that strongly language-focused content question items may be characterized by intra-sentential complexity relating to syntactic parsing and semantic-proposition encoding more than to unfamiliar vocabulary. However, with regard to the results of the qualitative section, more research is clearly needed before it is possible to draw any general conclusions.

Keywords

L2 reading comprehension language ability logistic regression mixed-methods approach verbal protocol analysis

Get full access to this article

View all access options for this article.

References

Alderson

(1984). Reading in a foreign language: A reading problem or a language problem? In Alderson

Urquhart

A. H.

(Eds.), Reading in a foreign language (pp. 1–24). London: Longman.

Alderson

(2000). Assessing reading. Cambridge: Cambridge University Press.

Bachman

(1982). The trait structure of cloze test scores. TESOL Quarterly, 16(1), 61–70.

Bachman

(1985). Performance on cloze tests with fixed ratio and rational deletions. TESOL Quarterly, 19(3), 535–556.

Bachman

(1990). Fundamental considerations in language testing. Oxford: Oxford University Press.

Buck

Tatsuoka

Kostin

(1997). The subskills of reading: Rule-space analysis of a multiple-choice test of second language reading comprehension. Language Learning, 47(3), 423–466.

Carr

Levy

(Eds.). (1990). Reading and its development: Component skills approaches. San Diego, CA: Academic Press.

Carrell

(1991). Second language reading: Reading ability or language proficiency? Applied Linguistics, 12(2), 159–179.

Droop

Verhoeven

(2003). Language proficiency and reading ability in first- and second-language learners. Reading Research Quarterly, 38(1), 78–103.

10.

Ericsson

Simon

(1993). Protocol analysis: Verbal reports as data. Cambridge, MA: The MIT Press.

11.

European Commission/SurveyLang. (2012). First European survey on language competences: Final report. Luxembourg: Publications Office of the European Union.

12.

Grabe

(2009). Reading in a second language: Moving from theory to practice. Cambridge: Cambridge University Press.

13.

Green

(1998). Verbal protocol analysis in language testing research. Cambridge: Cambridge University Press.

14.

Greene

(2001). Testing reading comprehension of theoretical discourse with cloze. Journal of Research in Reading, 24(1), 82–98.

15.

Hellekant

(1997). Från utprövning till färdigt prov. Konstruktion och utfall av ELF 97A. September 1997. Skrifter från Avdelningen för språkpedagogik, Göteborgs universitet.

16.

Hosmer

Lemeshow

(1989). Applied logistic regression. New York: Wiley.

17.

Jeon

E. H.

Yamashita

(2014). L2 reading comprehension and its correlates: A meta-analysis. Language Learning, 64(1), 160–212.

18.

Khalifa

Weir

(2009). Examining reading: Research and practice in assessing second language reading. Cambridge: Cambridge University Press.

19.

Kintsch

(1998). Comprehension: A paradigm for cognition. Cambridge: Cambridge University Press.

20.

Koda

(2005). Insights into second language reading. Cambridge: Cambridge University Press.

21.

Koo

Becker

Kim

Y-S.

(2014). Examining differential item functioning trends for English language learners in a reading test: A meta-analytical approach. Language Testing, 31(1), 89–109.

22.

Markham

(1985). The rational deletion cloze and global comprehension in German. Language Learning, 35(3), 423–430.

23.

Mauranen

(1989). Can gaps measure comprehension? Modifications of cloze as tests of reading. In Laurén

Nordman

(Eds.), Special language: From humans thinking to thinking machines (pp. 337–346). Philadelphia, PA: Multilingual Matters.

24.

McKamey

(2006). Getting closure on cloze: A validation study of the “rational deletion” method. Second Language Studies, 24(2), 114–164.

25.

Pae

I.-P.

(2018). Effects of task type and L2 proficiency on the relationship between L1 and L2 in reading and writing: An SEM approach. Studies in Second Language Acquisition, 40(1), 63–90.

26.

Rupp

Ferne

Choi

(2006). How assessing reading comprehension with multiple choice questions shapes the construct: A cognitive processing perspective. Language Testing, 23(4), 441–474.

27.

Sadeghi

(2014). Phrase cloze: A better measure of reading? The Reading Matrix, 14(1), 76–94.

28.

Schoonen

Hulstijn

Bossers

(1998). Metacognitive and language-specific knowledge in native and foreign language reading comprehension: An empirical study among Dutch students in grades 6, 8 and 10. Language Learning, 48(1), 71–106.

29.

Shiotsu

Weir

(2007). The relative significance of syntactic knowledge and vocabulary breadth in the prediction of reading comprehension test performance. Language Testing, 24(1), 99–128.

30.

Song

(2008). Do divisible subskills exist in second language (L2) comprehension? A structural equation modeling approach. Language Testing, 25(4), 435–464.

31.

Stage

Ögren

(2004). The Swedish Scholastic Assessment Test (SweSAT): Developments, results and experiences. Umeå: Umeå universitet.

32.

Storey

(1997). Examining the test-taking process: A cognitive perspective on the discourse cloze test. Language Testing, 14(2), 214–231.

33.

SweSAT: The Swedish Scholastic Assessment Test. (2010A–2016B). English Reading Comprehension, Swedish Reading Comprehension, Sentence Completion. Universitets- och högskolerådet.

34.

Van Steensel

Oostdam

Van Gelderen

. (2012). Assessing reading comprehension in adolescent low achievers: Subskills identification and task specificity. Language Testing, 30(1), 3–21.

35.

Weir

Porter

(1994). The multi-divisible or unitary nature of reading: The language tester between Scylla and Charybdis. Reading in a Foreign Language, 10(2), 1–19.

36.

Wurr

(2003). Reading in a second language: A reading problem or a language problem? Journal of College Reading and Learning, 33(2), 157–169.

37.

Yamashita

(2003). Processes of taking a gap-filling test: Comparison of skilled and less skilled EFL readers. Language Testing, 20(3), 267–293.