Abstract
This study examined the role of science domain knowledge, reading motivation, and decoding skills in reading comprehension achievement in third-grade students who are English learners (ELs) and students who are monolingual, using a nationally representative data set. Multigroup probit regression analyses showed that third-grade science domain knowledge and motivation for reading, decoding skills, and early attainment of decoding skills were significantly associated with third-grade reading comprehension in both language groups. Also, using Wald chi-square tests, the study showed that the association between third-grade science domain knowledge and reading comprehension was stronger in students who were ELs than in students who were monolingual. These findings suggest that cultivating science domain knowledge is very important to supporting reading comprehension development in third grade, particularly for students who are ELs.
Keywords
Strong reading comprehension is essential for academic and professional achievement, as well as for civic engagement and social life (Snow, 2002). Students who are English learners (ELs) have the asset of developing knowledge of more than one language (Goldenberg, 2008), but they are also more likely than monolingual students to have difficulties with reading comprehension in English in the United States (Mancilla-Martinez & Lesaux, 2010; National Center for Education Statistics, 2015). Understanding second-language (L2) reading comprehension development in students who are ELs is an important step in effectively supporting their reading comprehension in L2. Currently, relatively little is known about L2 reading comprehension, as the body of research investigating components contributing to L2 reading comprehension is still emerging (August & Shanahan, 2006; Samson & Lesaux, 2015).
Although there is a need for further study of reading comprehension development at all grade levels, third grade is of particular interest in the current U.S. policy context. As of 2019, 19 states have passed laws mandating retention of third-grade students who do not reach a certain level of reading proficiency and are not exempted (e.g., were already retained in a previous grade; National Conference of State Legislatures, 2019). Students who are ELs are more likely to be retained than students who are monolingual (Winke & Zhang, 2019), which suggests that ELs may need additional supports to meet third-grade reading comprehension standards, amplifying the need for further research in the current policy content on reading comprehension in third-grade students who are ELs.
Decoding skills have been considered one of the major contributors to L2 reading comprehension, particularly during the first few years of formal schooling (e.g., Gottardo & Mueller, 2009; Yaghoub Zadeh et al., 2012). Although decoding has been, appropriately, an emphasis in early schooling, building content knowledge has not traditionally been. There is an assumption that children learn to read up to third grade, then start to read to learn content in fourth grade (Chall, 1983; Duncan & Murnane, 2011). However, evidence has indicated that even before fourth grade, children can learn content from reading (Davis & Guthrie, 2015; Strachan, 2015) and that content knowledge is indeed associated with reading comprehension development (e.g., Best et al., 2008; Droop & Verhoeven, 1998). Therefore, research on how the two components—decoding skills and prior knowledge—simultaneously predict third-grade reading comprehension can extend our understanding of L2 reading comprehension in third grade. Additionally, most research on prior knowledge has examined it in relation to first-language (L1) reading comprehension, whereas more research is needed in relation to L2 reading comprehension.
Another important factor that merits further investigation is reading motivation, an affective component of reading comprehension, particularly in light of the rigor of current reading standards. Reading curricula in third grade are likely to include more complicated texts than before (Hiebert & Mesmer, 2013). Reading motivation is crucial for students to persevere through challenging reading tasks (Schiefele, 1999). Therefore, research on the role of third-grade reading motivation in L2 reading comprehension, while accounting for the aforementioned cognitive components, can reveal important information about how we can support students who are ELs. Additionally, previous studies have shown that reading motivation predicts L1 reading comprehension (e.g., Guthrie et al., 2004). What is still not clear, however, is that the relationship between reading motivation and L2 reading comprehension in the elementary years.
The current study aims to fill the gaps in L2 research by investigating the role of decoding skills, a component traditionally emphasized in early grades, as well as prior knowledge and reading motivation, important but less-studied components, in L1 and L2 reading comprehension in third grade, by using a nationally representative data set, the Early Childhood Longitudinal Study–Kindergarten Class of 1998–1999 (Tourangeau et al., 2009). Importantly, we included third-grade students who were monolingual (whose primary home language was not English) as well as students who were ELs (whose primary home language was English) in order to compare the role of each component in L1 versus L2 reading comprehension because the process of L2 reading comprehension might not be the same with that of L1 reading comprehension (Hedgcock & Atkinson, 1993). Understanding reading comprehension development in third-grade students who are ELs and students who are monolingual, including similarities and differences, can enable educators to better support both groups (Duke & Carlisle, 2011).
Conceptual Framework
Reading Comprehension
Reading comprehension is a process of constructing a mental representation of text (Kintsch, 1998), which can be influenced by multiple components, including cognitive (decoding skills, prior knowledge), psychological (reading motivation), and ecological (language status) components (Aaron et al., 2008).
Among components in the cognitive domain, decoding skills and prior knowledge can influence the process of deriving different levels of text representation (Kintsch, 2013). Efficient decoding skills are crucial to form the first level of text representation, surface-level memory, representing exact wording of text. The semantic level, representing what text means (textbase), is constructed when readers extract and connect ideas, the process of which can be facilitated by prior knowledge (Kintsch, 1998). A further level of semantic representation of the text (situation model) is constructed when readers integrate the textbase with their prior knowledge. The quality of semantic representation of the text is, therefore, likely to depend on readers’ prior knowledge.
Beyond the cognitive components, reading motivation, which Aaronet al. (2008) characterize as a component in the psychological domain, can uniquely influence reading comprehension (e.g., Cartwright et al., 2015). Motivated readers are more likely to be engaged in reading and persist to comprehend text even when they encounter challenges (Cain & Barnes, 2017; Schiefele, 1999); spending more time and putting more effort in reading can support readers to efficiently comprehend text (Wang & Guthrie, 2004).
Moreover, the role of the cognitive and psychological components in reading comprehension might differ by language status, an important component of Aaron et al. (2008)’s ecological domain for reading comprehension. For example, in a study with college students, Carrell (1983) found that prior knowledge predicted reading comprehension in college students who were ELs, but not in students who were monolingual. Although differences in reading development between students who are ELs and students who are monolingual have been postulated (Hedgcock & Atkinson, 1993; Riddle Buly & Valencia, 2002), little comparative analysis has been conducted to understand whether and how the role of the predictors in reading comprehension differ by language status (Gámez & Lesaux, 2015). Comparative analyses of reading comprehension by language status can advance our understanding in this area and may offer suggestions to improve reading instruction in ways that meet the needs of students who are ELs (Lesaux & Kieffer, 2010).
Review of Literature
This section discusses three key contributors to reading comprehension—prior knowledge, reading motivation, and decoding—and the degree to which they have been studied in students who are ELs and students who are monolingual.
Prior Knowledge
Knowledge includes any information and the structure of the information stored in long-term memory (Cook & Gueraud, 2005), and in reading research, the term prior knowledge (used interchangeably with background knowledge) has been used to account for knowledge that readers have (e.g., Coiro, 2011; Priebe et al., 2012; Taft & Leslie, 1985). There are three dimensions of prior knowledge that can affect the process of reading comprehension (Cook & Gueraud, 2005): lexical knowledge (information about meanings of words), featural knowledge (information about characteristics of objects and ideas), and script/scenario knowledge (episodic information about different situations). For example, in reading a text about evergreens, prior knowledge about the meaning of word evergreen (trees that have green leaves for all seasons), prior knowledge about characteristics of evergreens (e.g., needle-like leaves, seeds inside cones), and prior experience of planting a pine tree can influence the quality and characteristics of situation models constructed with a text about evergreens. In this regard, vocabulary knowledge is a part of prior knowledge that enables readers to label objects or ideas, but prior knowledge also encompasses understanding of attributes and previous experiences (see also Townsend et al., 2012, for the finding that vocabulary knowledge accounted for only partial variance in a prior knowledge measure).
Prior knowledge has been operationalized with different degrees of breadth of knowledge assessed (Cervetti & Wright, 2019). Topic knowledge is a narrowly operationalized type of prior knowledge, representing knowledge specifically related to a topic of a text, often subsumed under a domain of study, whereas domain knowledge is broader, capturing knowledge of many topics related to a field or domain of study. With elementary-age learners, domain knowledge is often operationalized as knowledge in science or social studies. The relationship between topic and domain knowledge has rarely been investigated, but Anmarkrud and Bråten (2009) found a positive correlation (r = .37, p < .001) between ninth-grade topic knowledge about socialization and students’ self-reported grade in the domain of social studies. Finally, domain knowledge of more than one field of study is also referred to as general knowledge.
A positive relationship between topic knowledge and L1 reading comprehension of texts on that topic has been consistently observed. For example, having knowledge related to a topic of a text can facilitate inference generation during reading (Fincher-Kiefer, 1992) and compensate for low reading ability (Recht & Leslie, 1988) in L1. However, the relationship between topic knowledge and L2 reading comprehension of texts on that topic has remained unclear. For example, Rydland et al. (2012) found that topic knowledge predicted L2 reading comprehension of that topic even after decoding skills and vocabulary knowledge in L2 were accounted for in fifth-grade students who learned Norwegian as their second language (see also Burgoyne et al., 2013). However, Ridgway (1997) and Carrell (1983) did not find a significant relationship between topic knowledge and L2 reading comprehension of texts on that topic in college students who were learning English as their L2. The discrepancy in findings in this area may be due to differences in the age of the study participants, underscoring the need to study the topic further with third graders, currently an especially-reading-policy-relevant age-group.
Few studies have examined domain knowledge in relation to L1 reading comprehension, and almost no research was conducted to investigate the relationship between domain knowledge and L2 reading comprehension. Anmarkrud and Bråten (2009) and Tarchi (2010) demonstrated that secondary students reading in their L1 who had more domain knowledge of social studies were likely to comprehend a text related to the domain better than students with less domain knowledge. Reed et al. (2017) found a strong correlation between science achievement and L1 reading development in fifth-, eighth-, and ninth-grade students (approximately 0.8 standardized coefficients). These studies are typical of research on domain knowledge and reading comprehension to date in that they have included only students who are monolingual or have not considered the language status of children one way or the other. Most of L2 studies have been focused on the relationship between topic knowledge with reading development in secondary or postsecondary students (e.g., Al-Shumaimeri, 2006; Barry & Lazarte, 1995, 1998; Carrell, 1983; Ridgway, 1997). Thus, we need research that investigates the broader types of knowledge (domain or general knowledge), rather than topic knowledge, in students who are L2 learners, including ELs, particularly in the elementary years. Including both L1s and L2s in a study would be most informative because only that can inform us whether L2 status, at least in English, makes a difference.
Reading Motivation
Reading motivation refers to willingness or intention to initiate, engage in, and persist in reading behavior. Previous L1 studies have revealed that students who are more intrinsically motivated to read or have more positive reading attitudes are likely to read better than those who are less intrinsically motivated to read or have less positive reading attitudes in the elementary years (Guthrie et al., 2006; McKenna et al., 1995; Park, 2011) and in the secondary grades (Mucherah & Yoder, 2008; Retelsdorf et al., 2011). In addition, it has been observed that students’ beliefs that they can be successful at reading (i.e., reading self-efficacy) are positively related to reading development in the elementary years (Cartwright et al., 2015; De Naeghel et al., 2012; Katzir et al., 2009).
Previous research has rarely investigated reading motivation in students who are ELs (Ivey & Broaddus, 2007; Taboada et al., 2010). Most L2 studies have investigated the role of reading motivation in post/secondary students’ reading development (e.g., Proctor et al., 2014). Dhanapala and Hirakawa (2016) reported that intrinsic motivation for reading positively predicted L2 reading comprehension in college students whose L1 is Sinhala and L2 is English. Sani and Zain (2011) investigated the role of reading attitudes and reading self-efficacy in L2 reading comprehension in middle school students whose L1 is Bahasa Malaysia and L2 is English. They found that both reading attitudes and reading self-efficacy were associated with L2 reading comprehension. In addition, a study by Taboada Barber et al. (2015) offers a reason to think that the role of reading motivation might differ by language status. They included both middle school students who were ELs and middle school students who were monolingual. The results showed that reading self-efficacy was related to L2 reading comprehension even after an earlier measure of L2 reading comprehension was controlled for, whereas it was not a significant predictor for L1 reading comprehension.
To better understand whether the role of reading motivation differs by language status, further research that compares its role in L1 and L2 reading comprehension is needed, particularly in the elementary years. With research that compares literacy development in students who were ELs and that in students who were monolingual, we can extend our knowledge on the complicated nature of L2 literacy development (Hedgcock & Atkinson, 1993).
Decoding Skills
According to the Kintsch’s Construction-Integration Model (Kintsch, 1998, 2013), described in the Conceptual Framework section, the process of constructing a textbase is initiated with decoding of words; thus having fluent decoding skills is necessary for successful reading comprehension. Empirical studies also have established the important role of decoding skills in L1 reading comprehension (Garcia & Cain, 2014; Keenan et al., 2008). A meta-analysis of 110 studies by Garcia and Cain (2014) showed a sizeable corrected correlation (.74) between decoding skills and L1 reading comprehension. As well, early decoding skills have been observed to predict L1 reading comprehension (Carlson et al., 2013; Sénéchal et al., 2006). For example, Carlson et al. (2013) found that decoding skills at the age of 6 years predict later reading comprehension at ages 8 through 10 years.
Similarly, L2 reading studies have shown that a positive association between decoding skills and L2 reading comprehension (Francis et al., 2006; Geva & Yaghoub Zadeh, 2006). Proctor et al. (2012) demonstrated that English decoding predicted reading comprehension in Spanish-English bilingual students in the second, third, and fourth grades along with vocabulary knowledge. In addition, early L2 decoding skills have been reported to predict L2 reading comprehension ability in the later grades (Mancilla-Martinez & Lesaux, 2010). In a longitudinal study, Nakamoto et al. (2012) found that first-grade English language proficiency and English word reading predicted third-grade English reading comprehension in students whose L2 is English.
Decoding ability in the early grades might have a differential impact on reading comprehension development in the later grades in students who are ELs as compared with students who are monolingual. For example, Relyea and Fitzgerald (2018) demonstrated that the rate of reading comprehension growth from first through eighth grade was lower in students who were ELs with low decoding skills in first grade, compared with students who were monolingual with a similar first-grade decoding skills. In students who are monolingual, Best et al. (2008) showed that domain knowledge in science and social studies was a stronger predictor than decoding skills for informational reading comprehension, but previous studies have not looked into the relative contribution of decoding skills and domain knowledge for L2 readers specifically.
Present Study
The present study aimed to explore reading comprehension in third-grade students who are ELs and students who are monolingual in the United States, including the contributions, if any, of prior knowledge (operationalized as science domain knowledge), reading motivation, and decoding skills, to deepen our understanding of reading comprehension development among third graders. To meet these aims, this study used the 1998–1999 Early Childhood Longitudinal Study–Kindergarten cohort (ECLS-K; Tourangeau et al., 2009). This data set was used, although it is not as current as we might like, because there is not, to our knowledge, a more recent nationally representative data set that allows researchers to examine a reading comprehension measure separately from other literacy-related measures such as letter knowledge and decoding skills (cf. ECLS-K with the Kindergarten cohort 2010–2011; Tourangeau et al., 2015).
This study extended previous research on reading comprehension by examining reading comprehension in students who are ELs and students who are monolingual separately rather than simply controlling for language status in one analysis. The study addressed the following research questions: How do science domain knowledge, reading motivation, decoding skills, and early decoding skills jointly predict reading comprehension in U.S. students who are ELs and students who are monolingual at the end of third grade? Are there any differences in the magnitude of the associations of reading comprehension with these measured components by language status?
Method
Analytic Sample
Of the 21,357 third-grade students originally recruited for the ECLS-K study, 15,305 students responded to questionnaires and/or participated in direct assessments (unweighted frequency; Tourangeau et al., 2009). The analytic sample included 13,292 students who had a third-grade reading score. Of those students, information about 12,101 students’ home language is present in the data set. Approximately 14% of students (n = 1,705) reported they spoke a non-English language at home, whereas the rest of the students (n = 10,396) spoke English at home. On average, third-grade students were 9 years 3 months old. Approximately 80% of them went to public schools. Among students who were ELs, roughly half of the students spoke Spanish as their primary home language.
Little’s missing completely at random test showed that the data used in the current study were not missing completely at random (χ2 = 882.36, df = 65; p < .001; Little & Rubin, 1987). More missing data were found for socioeconomic status (SES; 16% for ELs and 8% for monolinguals), school poverty level (25% for ELs and 21% for monolinguals), and early decoding skills (44% for ELs and 15% for monolinguals), whereas no missing data or less than 0.3% missing data were observed on the other variables. Larger percentages of missingness on early decoding skills (kindergarten and first grade) in students who were ELs appear to be related to the language screening test in kindergarten and first grade. The direct cognitive measures were not administered when a child did not pass the language screening test in kindergarten and first grade, and disproportionately more students who were ELs did not pass the language screening test compared with students who were monolingual, resulting in more missing on early decoding skills.
Thus, missing data were imputed for this study using a Markov chain Monte Carlo multiple imputation method with 150 iterations and 40 imputations, as recommended by Graham et al. (2007), which results in power falloffs less than 1% in comparison with a full information maximum likelihood approach. The 40 imputed data sets were not merged into one data set. Instead, using the Mplus command TYPE=IMPUTATION, an analysis was conducted for each individual data set, and parameters and standard errors were averaged over the set of analyses (Muthén & Muthén, 2015). Sampling weight was used to account for effects of nonresponse and to compensate for different probabilities of selection. It is worth noting that the children included in the analysis for the current study were born in the United States or arrived in the country no later than 1998 (the spring of kindergarten). Thus, regardless of the language status, the children in the current study likely started formal schooling in the United States.
Measures
The data set for this study included six main categories of variables: language status, motivation for reading, science domain knowledge, reading comprehension, decoding, and demographics. The ECLS-K did not reveal items of the direct cognitive measures due to copyright regulations and interest in potentially reusing the items in future. Assessors/interviewers recruited by the ECLS-K were trained to administer tests/surveys in a 5-day workshop. Only those who passed the certification criteria (e.g., following standardized procedures) administered tests/surveys (Tourangeau et al., 2009).
Language status
A composite variable for the primary language spoken at home, originally present in the data set, was used to determine language status. Children were considered ELs when their primary reported home language was not English and monolingual when it was. Identifying language status solely based on home language is less than ideal, but third-grade L2 proficiency was not available in the data set. As well, home language avoids the complication of the distinction between limited versus proficient English proficiency being poorly defined and varying across school districts and states (Lesaux & Harris, 2015).
Motivation for reading
The Self-Description Questionnaire–I (SDQ-I; Marsh, 1990) was adapted and used to measure how third graders (spring semester) feel about themselves both academically and socially. Among 42 items, eight items asked about their perceived competence in reading and interest in and enjoyment of reading (Marsh, 1990; Marsh et al., 1984; Tourangeau et al., 2009): “I get good grades in reading,” “Work in reading is easy for me,” “I am good at reading,” “I like reading,” “I am interested in reading,” “I cannot wait to read each day,” “I like reading long chapter books,” and “I enjoy doing work in reading” (National Center for Education Statistics, 2010). The SDQ has been used to assess reading motivation (e.g., Froiland & Oros, 2014; Karasinski & Anderson, 2017). Specifically, the first three items appear to be closely related to reading self-efficacy (i.e., the belief that one can be successful at reading), whereas the remaining five items measure intrinsic motivation for reading (i.e., willingness to read for the sake of internal purposes of reading), which are two important dimensions of reading motivation recurring in reading research (Schiefele et al., 2012; Wigfield & Guthrie, 1997).
Assessors read the SDQ questions to each child in order to prevent children’s responses from being influenced by their reading ability. The variable for motivation for reading was created by the ECLS-K by computing the mean of the eight items (Tourangeau et al., 2009). The values ranged from 1 to 4. The alpha coefficient for motivation for reading items was .87.
Science domain knowledge
The ECLS-K measure for science domain knowledge assessed knowledge of concepts and science investigations in the science subdomains of earth and space science, physical science, and life science with equal emphasis placed on each. The items were developed by the ECLS-K researchers to reflect typical elementary school science curricula, and elementary-grade curriculum specialists and teachers reviewed the items in terms of content, relevance, and difficulty. The measure was a two-stage test to minimize possible ceiling and floor effects, and individually administered by a trained assessor. The assessor read all questions and available response alternatives to the child (Tourangeau et al., 2009).
The science domain knowledge test was administered for 15 minutes only in English. The majority of third-grade students (96%) were considered sufficiently proficient in English to understand the cognitive measures, based on the English-language screening test in kindergarten (Tourangeau et al., 2009). The short-answer format was likely to make the test linguistically less demanding for both language groups than a test requiring lengthier responses, but presenting answers orally could have been more challenging to students who were ELs as expressive L2 typically develops later than receptive L2 (Lesaux et al., 2010).
In the current study, the third-grade science knowledge item response theory score was used, which ranged from 17.68 to 96.1. The alpha coefficient of the first test was .75, and for the second test, the alpha coefficients ranged from .60 to .70. Somewhat lower reliability was observed because the science measure included more diverse content and fewer number of items in the second-stage measure (Pollack et al., 2005).
Reading assessment
The reading assessment included items written by the ECLS-K researchers and items from published standardized tests, specifically from the National Assessment of Educational Progress (Najarian et al., 2009). The reading measure was a two-stage test and individually administered by a trained assessor (for 30 minutes). The assessment was administered in booklet and consisted of informational, narrative, poetic, and some other genres. The alpha coefficient for the first test was .75, and for the second test, it ranged from .79 to .84. In the current study, the variable for the highest reading level mastered in the spring of kindergarten and first grade (Pollack et al., 2005) was used for early attainment of decoding skills; the variable for the highest reading level mastered in the spring of third grade was used for reading comprehension achievement. There are 10 levels of reading achievement as described by Najarian et al. (2009), summarized in Table 1.
Descriptions of Each Level of Reading Comprehension Proficiency by the ECLS-K
Note. ECLS-K = Early Childhood Longitudinal Study–Kindergarten cohort. Sample items are not available to the public because the ECLS-K did not release them due to copyright regulations and confidentiality. The ECLS-K assumed a Guttman model, indicating that there is a sequential order in the attainment of proficient levels. The hierarchical pattern of literacy development warrants more investigation on whether the same pattern holds true for different groups of students such as race and language status. However, the ECLS-K reported that less than 7% of reading scores did not follow the Guttman model from kindergarten through seventh grade and argued that the unusual pattern is likely to be due to chances of students’ correct guesses on multiple-choice items.
Early attainment of decoding skills
Reading Level 4 was the highest proficiency for achieving decoding skill, and levels beyond Level 4 do not address decoding specifically (Najarian et al., 2009). Thus, the attainment of Level 4 was used as an index for early attainment of decoding skills. The third graders were categorized into the three groups to make two binary variables: (1) the children who attained the fourth level or above in the spring semester of kindergarten, (2) the children who attained the fourth level or above in the spring semester of first grade, and (3) the children who did not attain the fourth level by the end of first grade. The third group of students was treated as the reference group.
Reading comprehension proficiency in third grade
Among the 10 levels previously mentioned, Levels 1 to 4 were recoded as nonattainment of reading comprehension proficiency level and recoded as a zero level of reading comprehension proficiency because achieving these levels indicated that the students attained basic literacy skills but not reading comprehension skills. Levels 5 to 10 were recoded as 1 to 4 (Level 5 recoded as Level 1, 6 as 2, 7 as 3, and 8 through 10 as 4). Thus, the dependent variable, level of reading comprehension in third grade, ranged from 0 to 4. Levels from 9 to 10 were coded as 4 in this study because the ECLS-K researchers did not administer evaluating nonfiction and syntax to third graders, but they estimated the score for evaluating nonfiction and syntax with item response theory by using reading scores from later grades.
Decoding skills in third grade
There was a set of four relatively difficult items for measuring decoding skills in third graders in the spring semester. According to the ECLS-K, words in the decoding items for third grade, in contrast to those for first grade, were unlikely to be in most children’s daily vocabulary (Pollack et al., 2005). The split-half reliability was .67. The psychometric report for the third-grade noted that the relatively low reliability might have been due to the small number of items (Pollack et al., 2005). The score reflects a count of the number correct, and the range of decoding scores is from 0 to 4.
Demographic information
Information about third graders’ school poverty level, SES, gender, and race/ethnicity were collected with a survey for parents or for a school administrator.
School poverty level
School poverty level was estimated with the percentage of third graders eligible for free school lunch, reported by school administrators. A composite variable for the percentage of third graders eligible for free school lunch in the ECLS-K data was used. The composite variable was treated as a child-level variable as the participating children in the ECLS-K have moved to many different schools. As a result, few of them were clustered in the same school by third grade.
Socioeconomic status
A composite variable in the ECLS-K data set based on a parent survey was used. It indicates SES of third graders’ household at the time of data collection (Tourangeau et al., 2009). Five variables were used by the ECLS-K researchers to create the composite SES variable: father/male guardian’s education, mother/female guardian’s education, father/male guardian’s occupation, mother/female guardian’s occupation (recoded based on the 1989 General Social Survey prestige score), and household income. Each of five variables was standardized, and the average of the five standardized scores was coded as the SES composite variable, which ranged from −2.49 to 2.58.
Gender and minoritized status
The two composite variables for third graders’ gender and race/ethnicity were based on the parent survey in the ECLS-K data. Gender had two values—female and male. In this study, the variable for gender was recoded to make male the reference group (50% of the sample was female). Race/ethnicity originally had eight values in the ECLS-K data as termed by the ECLS-K: “White, non-Hispanic,” “Black or African American, non-Hispanic,” “Hispanic, race specified,” “Hispanic, race not specified,” “Asians,” “Native Hawaiian, other Pacific Islander,” “American Indian or Alaska Native,” and “More than one race, non-Hispanic” (Tourangeau et al., 2009, pp. 7–10). In this study, in which race/ethnicity was not a central focus, it was recoded into a two-value variable to specify minoritized status of children. Students whose race/ethnicity was reported as White were recoded as nonminoritized and the other races were recoded as minoritized. Nonminoritized was used as the reference group (43% of the sample was minoritized).
Data Analytic Plan
Before addressing the research questions, correlations among the components and demographic covariates were analyzed (see Table 2), and reading comprehension proficiency and science domain knowledge were compared between students who were ELs and students who were monolingual (see Table 3). Then, for the first research question, a probit regression model was examined, rather than ordinary linear regression, because a dependent variable, reading comprehension proficiency, is more likely to be an ordinal dependent variable than an interval variable. A probit regression model assumes that a latent continuum underlies an observed ordinal or dichotomous dependent variable (Muthén & Muthén, 2009), thus the ordinal variable in the current study, reading comprehension proficiency, was treated as a latent continuous variable. In particular, for the second research question, a multigroup probit regression model was used to test the null hypotheses regarding coefficients between students who are ELs and students who are monolingual through the Model Test option in Mplus (i.e., Wald chi-square test). The coefficients of the predictors on reading comprehension could not be directly compared between the two language groups in this study due to the substantial difference in the sample size. However, a Wald chi-square test made the comparison possible by comparing chi-square statistics of a model in which the coefficients were constrained to be the same between the groups with chi-square statistics of the other model in which the coefficients were freely estimated. In addition, following Kieffer (2011), early L2 proficiency from direct child assessments was used as an additional covariate for a supplementary analysis to examine whether there were any noticeable differences in the predictor coefficients when early L2 proficiency was included in the probit regression model (see online Supplemental Appendix A).
Coefficients of Correlations Among Components and Demographic Covariates
Note. G3RC = third-grade reading comprehension; G3SDK = third-grade science domain knowledge; G3DCD = third-grade decoding skills; G3RM = third-grade reading motivation; KDS = decoding skills attained by kindergarten (vs. not attained by Grade 1); G1DS = decoding skills attained by Grade 1 (vs. not attained by Grade 1); SES = socioeconomic status; MinS = minoritized status; LangS = language status.
Reading Comprehension Proficiency Levels and Science Domain Knowledge by Language Status
Note. RC = reading comprehension; ELs = students who were English learners; ML = students who were monolingual.
p < .05. **p < .01. ***p < .001.
Results
The chi-square test indicated that students who were monolinguals outperformed students who were ELs on the reading comprehension measure (see Table 3). The largest percentage of students who are ELs showed Proficiency Level 1, whereas the largest percentage of students who are monolingual exhibited Proficiency Level 3. Also, students who were monolingual outperformed students who were ELs on science domain knowledge.
Using multigroup probit regression analyses, the associations of different predictors with reading comprehension were investigated separately for each group. Then, using a Wald chi-square test, each coefficient was compared between the two groups in two steps. First, a constraint was imposed to make coefficients of each variable equal between the two groups (df = 1). As shown in Table 4, a significant difference of the χ2 was found only for science domain knowledge between the two groups, χ2(1) = 8.481, p < .01, indicating that all coefficients except science domain knowledge did not differ at a level of statistical significance between the two groups. Second, constraints to make coefficients of one variable zero and the same between the two groups were imposed (df = 2). As seen in Table 4, the coefficients of early decoding, third-grade decoding, third-grade science domain knowledge, and third-grade motivation for reading were significantly different from zero in both groups, which means all direct measures had a significant association with reading comprehension in third grade regardless of language status. Regarding demographic covariates, coefficients of SES and gender were significantly different from zero. However, coefficients of school poverty and minoritized status were not significantly different from zero. Based on the result of the χ2 analyses, constraints were imposed in the final model on coefficients of all variables except science domain knowledge to be equal between the two groups.
Comparison of Chi-Square Statistics Between Unconstrained and Constrained Models
Note. KDS = decoding skills attained by kindergarten (vs. not attained by Grade 1); G1DS = decoding skills attained by Grade 1 (vs. not attained by grade 1); SES = socioeconomic status.
p < .05. **p < .01. ***p < .001.
In Table 5, the unstandardized and standardized probit regression coefficients are presented. Among the continuous independent variables—namely, third-grade science domain knowledge, motivation for reading, decoding skills, SES, and school poverty—the coefficient of science domain knowledge in third grade was larger than the others. A standard deviation increase in the science domain knowledge measure resulted in an increase of 0.49 standard deviations for students who are ELs and 0.44 for students who are monolingual in the predicted z score of the underlying continuous dependent variable, reading comprehension proficiency in third grade, while controlling for the rest of the variables. The coefficient of science domain knowledge was significantly larger in students who are ELs than in students who are monolingual. Third-grade decoding was the second largest coefficient, which was followed by third-grade motivation for reading and SES. The coefficients of third-grade motivation for reading and SES were similar in magnitude.
Probit Regression Analyses With Constraints Predicting Reading Comprehension Proficiency Level in Third Grade
Note. ELs = students who were English learners; ML = students who were monolingual; KDS = decoding skills attained by kindergarten (vs. not attained by Grade 1); G1DS = decoding skills attained by Grade 1 (vs. not attained by Grade 1); SES = socioeconomic status.
An unstandardized probit coefficient means that a unit increase in an independent variable is associated with the extent of the coefficient increase or decrease in a predicted z score of an underlying continuous dependent variable, while holding the rest of indicators constant.
A standardized probit coefficient indicates that the extent of standard deviation change in a predicted z score of underlying continuous dependent variable is attributable to a 1 standard deviation increase in an independent variable, while the other indicators are controlled for. The standardized probit coefficients of the same variable that were constrained to be the same were slightly different from each other because the raw unstandardized coefficients were standardized by using group-specific standard deviations. Both standard deviation of independent variable and the model estimated standard deviation of dependent variables were used to standardize the coefficients of continuous independent variables. However, for binary independent variables such as early decoding skills, gender, and minoritized status, only the standard deviation of the dependent variables was used to standardize coefficients because a standard deviation change of the binary variables was not meaningful. Thus, comparing the extent of association between each indicator and reading comprehension in third grade makes sense across the continuous independent variables, and separately, across binary variables.
p < .05. **p < .01. ***p < .001.
Among the dichotomous independent variables, the coefficients of early decoding skills were the largest among the binary variables. Children who obtained decoding skills by spring of kindergarten or first grade were more likely to achieve a higher level of reading comprehension than those who did not obtain decoding skills by spring of first grade. The coefficients of obtaining decoding skills by spring of kindergarten and first grade were almost identical. For gender, female third graders were more likely to achieve a higher reading comprehension level than male third graders, when partialing out the other variables. In addition, minoritized status was not statistically significant.
Coefficients of ordered probit models can be interpreted more meaningfully and easily when coefficients are converted to predicted probability of attaining a particular value of a dependent variable as a change of an independent variable, controlling for the other independent variables (e.g., Williams & Umberson, 2004). In the current study, four thresholds (five levels − 1) and probit coefficients were used to calculate predicted probability of attaining each reading comprehension level. The probability reading scores for students who are ELs and for students who are monolingual were calculated only as a function of science domain knowledge, while the rest of factors are controlled for (see Figure 1). Average values were used for continuous independent variables and modes were used for categorical variables (e.g., nonminoritized, males, and attainment of decoding skills by spring of first grade) to hold the other factors constant.

Predicted probabilities of attaining each reading proficiency level by standard deviation of third-grade science knowledge measure. ELs and ML indicate students who were ELs and students who were monolingual, respectively. SD = standard deviation; RC = reading comprehension.
When the science domain knowledge score was 3 standard deviations lower, the probability of obtaining beyond a word comprehension level was approximately 12% in both language groups, when controlling for the other variables. That is, even when children obtained decoding by first grade and their third-grade decoding skills were average, reading comprehension of children would be greatly compromised if they did not have enough science domain knowledge to support their reading comprehension. As science domain knowledge scores became higher, the probability of obtaining lower reading comprehension proficiency decreased. With science domain knowledge scores 3 standard deviations lower than average, the probability of obtaining the first level (comprehending words) or below was approximately 86% for both language groups. With an average science domain knowledge score, the probability was 8% for students who are ELs and 19% for students who are monolingual, approximately.
Similarly, the probability of obtaining a higher reading comprehension proficiency level increased sharply as the science score became higher. With an average science score, the probability of obtaining Level 3 (extrapolation) or Level 4 (evaluation) was 56% for students who are ELs and 43% for students who are monolingual. The probability was increased to more than 99% for ELs and approximately 97% for monolingual when science domain knowledge was 3 standard deviations higher. That is, children are highly likely to obtain the highest reading comprehension level in third grade if their science domain knowledge score was 3 standard deviations above average, even though their third-grade decoding skills are average, and they acquired decoding skills at the average rate. In sum, a higher science domain knowledge score was associated with a higher probability of obtaining more advanced reading comprehension proficiency for students who are ELs than for students who are monolingual.
Discussion
Components Related to Reading Comprehension Development in ELs and Monolingual in Third Grade
Science domain knowledge
Consistent with the previous studies that have revealed the positive role of prior topic knowledge in reading comprehension in L1 (Fincher-Kiefer, 1992; Garner & Gillingham, 1991; Lipson, 1983; McNamara & Kintsch, 1996; Priebe et al., 2012; Recht & Leslie, 1988) and in L2 (Barry & Lazarte, 1995, 1998; Burgoyne et al., 2013; Droop & Verhoeven, 1998; Rydland et al., 2012), in this study science domain knowledge was found to be associated with reading comprehension achievement for both students who are ELs and students who are monolingual. Science domain knowledge appears to play an essential role in making inferences (Anderson & Pearson, 1984) and in building a situation model (Kintsch, 1998, 2013): The probabilities of achieving Level 3 (using prior knowledge to generate inferences) and Level 4 (making connections between story and life problems) were substantially lower when science domain knowledge in third grade was below average, whereas the probability increased considerably when science domain knowledge was at or above average.
The coefficient of science domain knowledge is significantly larger in students who are ELs than in students who are monolingual. Science domain knowledge might have compensated for ELs’ still-developing language proficiency. Arguably, the reading test was more likely to be syntactically challenging to students who are ELs than students who are monolingual. Thus, students who are ELs might have had to make more inferences based on their prior knowledge, and having prior knowledge is likely to be helpful in generating accurate inferences (e.g., Barry & Lazarte, 1995, 1998; Fincher-Kiefer, 1992). Similarly, students who are ELs may have more difficulty in extracting meaning through bottom-up processes and thus benefit more from strengths related to top-down processes (e.g., Barry & Lazarte, 1995, 1998). The finding of this study is not consistent with Carrell (1983) and Ridgway (1997) who found nonsignificant association between topic familiarity and text comprehension on that topic in college students who were ELs. The inconsistent results between the current and previous studies might have been observed partially because the current study focused on science domain knowledge rather than on knowledge related to the topic of a specific text, and the previous studies by Carrell (1983) and Ridgway (1997) had not used a direct measure for topic knowledge.
The correlational nature of analyses warrants the consideration of alternative explanations of the findings. The positive coefficient of science domain knowledge on reading comprehension also indicates the potential of a mutually enhancing relationship between the two (Cervetti & Hiebert, 2015). That is, the finding suggests not only the positive role of science domain knowledge in reading comprehension but also the possibility that third-grade students with better reading comprehension are likely to have gained more science domain knowledge. In addition, a lurking variable might have played a role in explaining the significant relationship between science domain knowledge and reading comprehension. For example, science domain knowledge might be a proxy for the access to higher quality instruction in school. Third-grade students who have more and/or better opportunities to accumulate science domain knowledge in school may be likely to experience more and/or better reading comprehension instruction as well, and thus are likely to perform well on both science domain and reading comprehension measures. In fact, students who are ELs are overrepresented in special education programs as compared with students who are monolingual (Samson & Lesaux, 2009) and are often pulled out of mainstream science education, ultimately learning only simplified content (Minicucci & Olsen, 1992). Many teachers believe that students who are ELs are not ready to learn science due to their still-developing language proficiency (Callahan, 2005). As well, the analysis of the current study demonstrated that students who were ELs had lower score on the science and reading comprehension measures than students who were monolingual, on average.
Decoding skills and reading motivation
Third-grade decoding skills displayed the second largest coefficient, following science domain knowledge. The words for assessing decoding ability used by the ECLS-K for third grade were unlikely to be in most children’s daily vocabulary (Tourangeau et al., 2009), and previous research has shown a positive association between pseudo-word reading and reading comprehension (Garcia & Cain, 2014; Nakamoto et al., 2012). In terms of grade level when decoding skills were achieved (binary variables), those third-grade students who did not attain decoding skills by the spring of first grade displayed lower reading comprehension scores on average, compared with those who attained decoding skills by the spring of kindergarten or by the spring of first grade. This is consistent with the findings of several studies suggested that supporting students to develop fundamental decoding skills by the end of first grade is beneficial for the development of reading comprehension. As well, the coefficients of achieving decoding skills by the spring of kindergarten and by the spring of first grade were similar. This suggests that accelerating attainment of decoding skills by kindergarten might not benefit reading development above the attainment by first grade.
The coefficient of reading motivation was smaller than that of third-grade science domain knowledge and decoding skills; however, it was comparable with that of SES. Its coefficient was also similar between students who were ELs and students who were monolingual. That is, an increase in reading motivation would be related to a similar extent of increase in reading comprehension achievement as would occur as when SES increases for both groups of students who are ELs and students who are monolingual. In contrast to improving SES, supporting reading motivation is something schools can readily do to foster reading comprehension development (e.g., Guthrie et al., 2005; Taboada Barber, 2016).
Limitations
There are several limitations in the present study. First, the ECLS-K data set is not recent. The assessments were administered to third graders in 2002. Thus, there might be differences between the results of measured for third graders in 2002 and today. Second, some important contributors to reading comprehension identified in previous research were not included in this study, such as executive skills, different dimensions of reading motivation (e.g., value of reading, autonomy), and academic vocabulary as they are not available in the ECLS-K data set. Third, some variables in this study had limitations. The reliability of third-grade decoding skills was low because few decoding items were administered to children. The science domain knowledge measure was only administered in English. The ECLS-K appears to have addressed linguistic complexity in the science domain knowledge measure by the fact that 96% of third-grade students’ language proficiency was considered proficient in English enough to understand the cognitive measures (Tourangeau et al., 2009) and providing short-answer format questions, but in order to more accurately measure science domain knowledge in ELs, large-scale testing practice should provide accommodations to students who are ELs such as giving extra time and/or translations in L1 (National Academies of Sciences, Engineering, and Medicine, 2018). Fourth, the study does not eliminate the possibility that there is a lurking variable that caused the significant relationship between science domain knowledge and reading comprehension, such as that children who experience more opportunities to develop science knowledge are more likely to experience more opportunities to develop knowledge and skills related to reading comprehension. Future research is recommended to consider these contextual components to better understand the relationship between science domain knowledge and reading comprehension.
Implications for Practice
Acknowledging that further research that uses a strong causal design and that considers additional predictors, such as academic vocabulary, is needed to confirm, the results of the current study lend support to the facilitative role of science domain knowledge and reading motivation as well as decoding skills in L1 and L2 reading comprehension in third grade.
Activating and building students’ science domain knowledge might support third-grade reading comprehension (e.g., Cervetti & Hiebert, 2015; National Research Council, 2011). Reading instruction can be effective when it is situated in knowledge-building goals than in a generic context (e.g., Guthrie et al., 2004; Halvorsen et al., 2012). In this study, science domain knowledge played a more important role in reading comprehension development in students who are ELs than in students who are monolingual. The results support recommendations of Lesaux and Harris (2015) to situate much of the instruction provided to students who are ELs within a content area context.
Results of this study also call into question the practice of pulling students who are ELs out of content area instruction in order to teach them basic reading and language skills at the expense of content knowledge development. Systems providing reading intervention to students out of concern that they may be retained should consider carefully when that reading intervention occurs, particularly for students who are ELs.
The findings regarding the predictive value of reading motivation in reading comprehension suggest the need for deliberate motivational practices in reading instruction. Such practices include establishing the relevance of reading tasks, building on or inspiring students’ interest in topics, providing opportunities for choice and collaboration, targeting an appropriate level of challenge, and connecting reading with content-rich activities can sustain motivation, including in reading (Guthrie et al., 2007; Lazowski & Hulleman, 2016). Moreover, supporting students in understanding when and how to use reading strategies on their own can bolster students’ confidence in reading, as well as their reading comprehension (Taboada Barber, 2016; Wigfield et al., 2004).
Finally, given the strength of the relationship between decoding and reading comprehension for both students who are ELs and students who are monolingual, instructional support should be provided to children in the primary grades to enable them to recognize words effortlessly. Explicit instruction in simple and complex letter-sound relationships and in word recognition strategies, as well as extensive opportunities to apply learning in actual reading and writing are essential elements of effective reading instruction (Foorman et al., 2016). However, the findings of the current study do not suggest focusing on basic word reading skills at the expense of developing content knowledge. For example, the average level of decoding skills and attainment of decoding in the first year of schooling did not enable most third graders to successfully comprehend texts when they did not have sufficient science domain knowledge. In the face of mandatory retention, based on research available to date, thoughtful allocation of instructional opportunities across domains is the most advisable course.
Supplemental Material
AERA-18-0284.R2_SupplementaryFile – Supplemental material for Content Counts and Motivation Matters: Reading Comprehension in Third-Grade Students Who Are English Learners
Supplemental material, AERA-18-0284.R2_SupplementaryFile for Content Counts and Motivation Matters: Reading Comprehension in Third-Grade Students Who Are English Learners by HyeJin Hwang and Nell K. Duke in AERA Open
Footnotes
Authors
HYEJIN HWANG is a postdoctoral researcher at the Florida Center for Reading Research at Florida State University. Hwang’s research explores literacy development in the elementary years, particularly among students who are second language learners.
NELL K. DUKE is a professor at the University of Michigan. Duke’s work focuses on early literacy development, particularly among children living in poverty.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
