Sage Journals: Discover world-class research

Abstract

This paper presents evidence of the effects of dual language immersion (DLI) programs on the academic outcomes of students in elementary grades. Leveraging enrollment lotteries from four cohorts of DLI applicants across 10 oversubscribed programs, analyses estimate the intent-to-treat effect of access to bilingual education on reading and math test scores. On average, native English-speaking students in Grades 1 through 4 who win access to a DLI program score higher in reading and math by 0.12 and 0.14 SDs, respectively. The achievement gains in test scores are realized as early as first grade.

Keywords

dual language immersion bilingual education language education school choice lotteries

Introduction

Dual language immersion (DLI) offers general education in two languages—English and a target language—aiming to promote bilingualism and biliteracy (Boyle et al., 2015). The number of DLI programs has increased rapidly in recent years with estimates accounting for more than 3,600 programs across 45 states (Roberts, 2021). Much of this expansion has come as a response to increased demand from parents in English-dominant households who anticipate positive returns in their children’s education and future labor market outcomes (Ee, 2018; Parkes, 2008). Indeed, studies demonstrate that access to dual language education in kindergarten results in higher reading test scores by upper elementary grades, relative to students in monolingual classrooms (A. Bibler, 2021; A. J. Bibler, 2022; Steele et al., 2017). Participation in DLI is often promoted as a long-term commitment, and therefore, students are encouraged to remain in a given program for multiple years to experience the full potential benefits. Yet, it remains unclear whether the test score gains already documented in the literature also materialize in the shorter term.

Proponents of bilingual education often point to evidence documenting a strong relationship between second language acquisition and improvements in executive function (Barac et al., 2014), working memory (Grundy & Timmer, 2017; Morales et al., 2013), attention control (Adesope et al., 2010; Nicolay & Poncelet, 2015), and task switching (Wiseheart et al., 2016). Relatedly, studies assessing the causal effect of dual language programs on academic achievement show improvements in reading and English Language Arts test scores by third grade and above. For example, in research examining the causal impacts of access to DLI in Oregon and North Carolina, Steele et al. (2017), A. Bibler (2021), and A. J. Bibler (2022) find meaningful academic gains for native English speakers and English Learners. Likewise, studies employing quasi-experimental methods find a comparable positive relationship between participation in DLI education and student achievement in reading and math (Shen et al., 2022; Watzinger-Tharp et al., 2018).

This paper contributes to the thin literature examining the impacts of dual language education using large samples of students subject to enrollment lotteries (A. Bibler, 2021; A. J. Bibler, 2022; Steele et al., 2017). It primarily expands the scope of prior research by exploring the effects of access to DLI programs on students’ reading and math test scores with a focus on early elementary grades. Therefore, results from this analysis begin to shed light on whether the academic benefits of DLI require a long-term commitment on behalf of students.

The analysis was conducted using individual-level data from four cohorts of kindergarten and first-grade students who applied to at least one DLI lottery across two public school districts in metro Atlanta and whose test scores were followed up to fourth grade. Capitalizing on randomized access to dual language education, this paper estimates intent-to-treat effects of DLI comparing reading and math test scores of students who “won” access to a DLI program relative to those who did not obtain access and therefore received monolingual education.

In line with prior research, students who were randomly assigned to DLI in kindergarten or first grade have higher reading and math test scores in elementary grades. The estimated effects are significant and large, ranging from 16% to 18% of the average expected growth in reading and math test scores from first to fifth grade (Lipsey et al., 2012). Moreover, results from this study show that the positive impact of dual language education is observed in early elementary grades and tends to increase gradually over time. These results complement prior findings that focus on the medium-term effects of DLI (e.g., A. Bibler, 2021; Steele et al., 2017) and suggest that the test score gains associated with dual language education are also meaningful in the shorter term.

Institutional Background

This study leverages data from two public school districts in Georgia, a state where access to dual language education has grown substantially in recent years (Owens, 2020). Across both districts, dual language is a school choice alternative available to all kindergarten and first-grade students who live within a given district’s service boundary. Applications to dual language typically open during the spring semester of the academic year prior to enrollment, and students may apply to multiple programs. Whenever the number of applicants in a given DLI program exceeds the available seats, DLI-hosting schools determine access via decentralized lotteries implemented independently by each dual language program—one for students who live within the school attendance zone and a separate lottery for students who reside beyond the school boundary. Slots are typically assigned in three rounds. First, children of DLI teachers and siblings of incumbent DLI students are offered a seat. Then, the DLI-hosting school runs the in-zone lottery, with the remaining seats assigned via the out-of-school-zone lottery. Therefore, for oversubscribed programs, randomized access to DLI takes place within lottery strata defined at the intersection of a given DLI-hosting school, academic year, and students’ in-zone status. Therefore, each student within a lottery stratum has the same chance of winning access to a seat in a DLI program. Following the literature (e.g., A. Bibler, 2021; Steele et al., 2017), I account for lottery strata fixed effects to fix test score comparisons to students who faced the same probability of winning a seat in a dual language program but ultimately faced differential access as a result of varying lottery outcomes.

Data

This analysis leveraged two main data sources. First, records from the universe of four cohorts of kindergarten and first-grade students who applied to at least one DLI lottery from SY2016 to SY2019. For each applicant, these files contained the name of the DLI-hosting school to which students applied, their in-zone status, the grade of application, and the lottery outcome. These data were then merged with individual-level enrollment and test score files following students up to fourth grade. While the initial data set included all DLI lottery applicants, the final analysis sample was restricted to those who enrolled in either one of the study districts and therefore was matched to at least 1 year of enrollment and outcome records. That is, those who opted for private school or homeschool were, due to data limitations, not included in the analysis sample. See Supplementary Appendix A in the online version of the journal for a detailed description of the data and sample used in this analysis, including an assessment on the extent of student attrition.

The main outcome variables measure student achievement in reading and math captured by test scores in the Iowa Tests of Basic Skills (ITBS) and Measure of Academic Progress (MAP) formative assessments. Student outcomes were obtained for every winter quarter from SY2017 up to the winter quarter of SY2020. The ITBS and MAP tests are nationally norm-referenced assessments measuring student growth and are typically administered three times during the school year (Dunbar & Welch, 2015). For consistency across test types, MAP scores were normalized using the 2015 means and standard deviations by grade and subject, and the ITBS scores correspond to the standardized Normal Curve Equivalent scores.

Access to administrative records also included additional student-level information, such as demographic characteristics (sex, race/ethnicity), eligibility for free or reduced-price lunch (FRPL), and participation in gifted and special education programs. These records were obtained from district enrollment forms, often collected after the DLI lotteries were conducted. This is particularly the case for variables identifying eligibility for gifted and special education given that they are often the result of screenings and evaluations that occur during the academic year. As such, unlike student demographic information, program participation covariates are best construed as capturing individual-level information during the first year of enrollment in a given district rather than pre-lottery characteristics in the strict sense.

Overall, the sample consists of DLI programs that were oversubscribed for at least 1 year during the sample period to ensure randomized access to dual language education determined by lotteries. Moreover, due to the structure of the programs and overwhelming enrollment of native English-speaking students (88%), the sample excludes DLI applicants who were identified as English Learners. Furthermore, applicants who were not subject to the enrollment lotteries, for example, siblings of DLI students and children of DLI teachers, were also excluded. In total, the analysis sample covers 10 DLI programs—6 in Spanish, 3 in French, and 1 in German—and 1,423 applications subject to a lottery with 812 (57%) being randomly awarded access to DLI.

Methodology

Empirical Strategy

I estimate the impact of access to dual language education on student achievement in elementary grades following an intent-to-treat specification comparing the outcomes of students with randomized access to a DLI in kindergarten or first-grade relative to those who did not win a seat in a DLI program. The estimation equation is defined by the following:

A_{i j g} = β_{0} + β_{1}^{I T T} L o t t_{i j} + {X^{'}}_{i g_{0}} γ + ω_{j} + λ_{g} + υ_{i j g},

(1)

where $A_{i j g}$ denotes reading or math test scores for student i in grade g who participated in lottery strata j; $L o t t_{i j}$ is an indicator variable for whether student i was ever offered enrollment in an oversubscribed DLI program in kindergarten or first grade resulting from the outcome in lottery strata j; $ω_{j}$ indicates lottery strata fixed effects—a set of indicator variables for all combinations of a given DLI-hosting school, application year, and in-zone status; $λ_{g}$ is a set of grade fixed effects; and $X_{i g_{0}}$ is a vector of individual-level covariates at baseline, including demographic characteristics, FRPL, gifted and special education eligibility, and whether students applied to multiple DLI lotteries. The parameter of interest, $β_{1}^{I T T}$ , measures the impact of randomized access to a DLI program, relative to students who were not offered DLI enrollment. First, I evaluate Equation 1 using the pooled sample of students in Grades 1 through 4, as well as separately by grade level to assess the evolution of impacts across grades. Finally, I estimate heterogeneous analyses by target language grouping Spanish versus non-Spanish programs.

Recent studies highlight empirical issues of noncomparability between treatment and comparison units when using measures of “ever access” versus “first access” in the context of oversubscribed lotteries—a consideration that may be relevant to this paper (de Chaisemartin & Behaghel, 2020). While data limitations prevent a more in-depth examination of this issue, previous studies indicate that addressing these potential biases tends to yield either trivial differences or larger estimates compared with results obtained through more conventional approaches, as utilized in this analysis (Cohodes et al., 2021; de Chaisemartin & Behaghel, 2020).

Student-Level Covariates by DLI Lottery Outcomes

The empirical strategy relies on the assumption of randomized access to DLI programs to recover intent-to-treat parameters capturing the causal impact on test scores. To provide suggestive evidence in support of this assertion, I examine whether there are significant differences in baseline demographic characteristics and other program eligibility measures across students who won and did not win a DLI lottery. Failure to find significant differences across these student groups would suggest that, indeed, access to dual language education was independent of observed student characteristics, and therefore, one can credibly support the DLI lottery outcomes as randomized.

Table 1 shows the regression-adjusted means for baseline student demographic and program participation variables controlling for lottery strata fixed effects, which must be accounted for to fix the choice set of applications within which a lottery was administered. The first set of columns shows the adjusted average and standard errors for the group of students who were randomized into access to dual language education. The second set of columns shows the same statistics for those who did not win a DLI lottery. Finally, the last column shows the p-values associated with the difference in means between lottery winners and losers. In short, there is evidence to indicate that, within lottery strata, access to a DLI program was largely independent of student characteristics particularly those observed prior to lottery implementation.

Table 1

Test for Covariate Balance

	Won DLI lottery		Did not win DLI lottery		p-value of difference in means
Variables	M	SE	M	SE	p-value of difference in means
Panel A: Demographic variables
Female	0.49	0.02	0.50	0.02	.870
Hispanic/Latino(a)	0.17	0.01	0.18	0.01	.880
White	0.37	0.01	0.34	0.02	.180
Black	0.50	0.01	0.52	0.02	.346
Asian	0.04	0.01	0.04	0.01	.579
Other	0.09	0.01	0.09	0.01	.927
Panel B: Program participation variables
FRPL eligible	0.46	0.02	0.50	0.02	.078
ESOL	0.11	0.01	0.13	0.01	.239
Special education	0.04	0.01	0.07	0.01	.030
Gifted	0.04	0.01	0.06	0.01	.097
Observations	812		611

Note. The sample includes all applications submitted for a DLI program that were subject to a lottery. Demographic characteristics and program participation variables correspond to the first year of school enrollment in a given district. That is, variables shown in Panel B are measured after lottery outcomes are determined. Observations are measured at the application, not the applicant level. Excludes students from preference groups (siblings of DLI students and children of DLI teachers). Adjusted means are obtained as the predicted values of a series of regressions estimating the relationship between each of the demographic and program participation variables and an indicator for the lottery outcome while controlling for lottery strata. The third column shows the p-value of the difference in adjusted means between lottery winners and those who did not win access to a DLI seat. DLI = dual language immersion; SE = standard error; FRPL = free or reduced-price lunch; ESOL = English as a Second Language.

As seen in Table 1, girls made up 49% of students with access to a DLI program compared with 50% among those who did not win a seat. There is a comparably small difference in the remaining demographic characteristics as well. For example, 50% of those who won a DLI lottery identified as Black compared with 52% of students without access to dual language. Overall, for each of the demographic variables (race/ethnicity and sex), the differences are small and statistically insignificant at conventional levels as denoted by the p-values shown in the last column. Differences in the proportion of students who received English as a Second Language support are also indistinguishable across groups.

There are, however, small differences in the proportion of students eligible for FRPL, gifted programs, and special education services. In most of these cases, the differences are small and just marginally significant. Moreover, as underscored in the description of the data, records on these program participation variables do not strictly predate the implementation of the lotteries. As such, it is not surprising to observe small differences in these variables across lottery outcomes. That said, the regression results presented below include a host of demographic and program participation indicators as control variables to account for differences between groups and increase the precision of the point estimates. Refer to Supplementary Appendix A, and Tables A1 and A4 in the online version of the journal for further validity checks.

Results

Table 2 reports the intent-to-treat effects of access to DLI on reading and math test scores. Panel A shows the results from the sample of students in Grades 1 through 4 across all DLI programs. To assess heterogeneity by target language, Panel B presents findings for the subset of students who participated in Spanish DLI program lotteries, while Panel C shows parallel results for those who participated in non-Spanish DLI lotteries. Each column shows the coefficients from a separate regression. Columns (1) and (3) report estimates from models including only lottery strata and grade fixed effects. Columns (2) and (4) show results from fully specified models controlling for baseline student characteristics.

Table 2

Intent-to-Treat Effects of Access to Dual Language Immersion on Reading and Math Test Scores, All Grades

	(1)	(2)	(3)	(4)
Variables	Outcome: Normalized reading score		Outcome: Normalized math score
Panel A: All DLI programs
Won DLI lottery	0.13*** (0.04)	0.12** (0.05)	0.15** (0.06)	0.14* (0.07)
Observations	1,882	1,882	1,882	1,882
Panel B: Spanish DLI programs
Won DLI lottery	0.13** (0.05)	0.08(0.06)	0.16(0.12)	0.13(0.12)
Observations	899	899	899	899
Panel C: Non-Spanish DLI programs
Won DLI Lottery	0.17** (0.08)	0.22** (0.07)	0.21*** (0.05)	0.21** (0.08)
Observations	983	983	983	983
Lottery strata fixed effects	X	X	X	X
Grade fixed effects	X	X	X	X
Baseline covariates		X		X

Note. The sample is limited to students who submitted at least one application for a DLI program and who were subject to a lottery. Columns (1) and (2) report point estimates where the outcome variable is normalized reading scores. Columns (3) and (4) report point estimates where the outcome variable is normalized math scores. Each column corresponds to a separate regression varying by the set of control variables. Columns (1) and (3) include lottery strata and grade fixed effects only. Columns (2) and (4) build on the baseline specification by including demographic and program participation variables defined at the first year of school enrollment in a given district. Baseline covariates include indicators for sex, race/ethnicity, FRPL eligibility, participation in gifted or special education programs, and whether the student applied to multiple DLI lotteries. Standard errors clustered at the lottery level are shown in parentheses. There are 25 lottery strata clusters. DLI = dual language immersion; FRPL = free or reduced-price lunch.

p < .10. **p < .05. ***p < .01.

Overall, estimates from the pooled sample of first through fourth graders show an increase in reading and math test scores for those with access to a dual language compared with students who lost DLI lotteries. For example, students who won a dual language lottery scored on average 0.12 SDs higher in reading. These findings are robust across specifications with the point estimate remaining stable after accounting for demographic and program characteristics at baseline. Results from the fully specified model compare favorably to prior results reported by Steele et al. (2017) who find a 0.13 SD increase in reading test scores by fifth grade among DLI lottery winners. Likewise, students with access to DLI education scored on average 0.14 SDs higher in math.

Heterogeneous analyses by target language reveal that the positive impacts of access to DLI on test scores are observed across Spanish and non-Spanish programs. While the point estimates are nearly twice as large among those who participated in non-Spanish DLI lotteries, the magnitude of the coefficients for the subsample of Spanish DLI programs is comparable and statistically indistinguishable from that reported in Panel A. Explorations behind the differential effects by target language are beyond the scope of this study. However, selection into program type is likely a plausible factor driving these differences. See Supplementary Appendix B and Table B3 in the online version of the journal for supplemental analyses.

Finally, Figure 1 reports the point estimates from fully specified regressions evaluating the impact of access to dual language on reading and math scores separately by grade. Overall, students in Grades 1 through 4 experience gains in achievement that generally increase over time. While most of the coefficients reported in the figure are imprecisely estimated, largely due to small samples, all the magnitudes are positive and follow an upward trend. For example, the effect size for reading achievement increases from 0.07 SDs in first grade to over 0.3 SDs by fourth grade. A similar pattern is also observed for math tests. Interpreted with caution, these results are indicative that the positive effects of dual language documented in the literature are likely realized gradually over time and as early as first grade.

Figure 1.

Intent-to-treat effect of access to dual language immersion on reading and math test scores, effects by grade level.

Conclusion

This paper estimates the impact of dual language education on student achievement by leveraging lottery data from four cohorts of kindergarten and first-grade students who applied to oversubscribed DLI programs in two metro Atlanta school districts. Students with access to dual language education increased their reading and math test scores in elementary school, with estimated positive impacts as early as first grade.

Findings from these analyses suggest that the test score gains associated with dual language education are large and significant even in the shorter term, thereby complementing prior studies using large samples of students with randomized access to DLI which often focus on the academic outcomes of those in upper elementary grades (e.g., A. Bibler, 2021; Steele et al., 2017). Furthermore, it broadens the context in which DLI programs are evaluated by examining the impact of schools in Georgia—a state that is in an expansion phase of dual language education and where the student population is more diverse than that of other contexts in which DLI studies have been conducted. In contrast, much of the prior literature presents evidence limited to states with a long and established history of language immersion such as Oregon (e.g., Steele et al., 2017) and North Carolina (e.g., A. Bibler, 2021; Collier & Thomas, 2004) where bilingual education began as early as the late 1980s. Thus, results from this study provide policy-relevant findings applicable to new and ongoing expansions of DLI programs across states serving a diverse student population.

Supplemental Material

sj-pdf-1-epa-10.3102_01623737241228829 – Supplemental material for Dual Language Immersion Programs and Student Achievement in Early Elementary Grades

Supplemental material, sj-pdf-1-epa-10.3102_01623737241228829 for Dual Language Immersion Programs and Student Achievement in Early Elementary Grades by Camila Morales in Educational Evaluation and Policy Analysis

Footnotes

Acknowledgements

This research uses data from two school districts in metro Atlanta that chose to remain anonymous. The data were structured and maintained by the Georgia Policy Labs through its Metro Atlanta Policy Lab for Education (MAPLE). All findings and opinions are those of the authors and do not represent the opinions of any partner agencies or school districts. I am grateful to Tim Sass, Daniel Kreisman, Tom Mroz, Ross Rubenstein, three anonymous referees, and district staff for their generous feedback.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Camila Morales

Author

CAMILA MORALES, PhD, is an assistant professor at The University of Texas at Dallas. Her research focuses on the economics of education and immigration.

References

Adesope

O. O.

Lavin

Thompson

Ungerleider

(2010). A systematic review and meta-analysis of the cognitive correlates of bilingualism. Review of Educational Research, 80(2), 207–245. https://doi.org/10.3102/0034654310368803

Barac

Bialystok

Castro

D. C.

Sanchez

(2014). The cognitive development of young dual language learners: A critical review. Early Childhood Research Quarterly, 29(4), 699–714. https://doi.org/10.1016/j.ecresq.2014.02.003

Bibler

(2021). Dual language education and student achievement. Education Finance and Policy, 16, 634–658. https://doi.org/10.1162/edfp_a_00320

Bibler

A. J.

(2022). Language immersion and student achievement. Education Economics, 30(5), 451–464. https://doi.org/10.1080/09645292.2021.2001788

Boyle

August

Tabaku

Cole

Simpson-Baird

(2015). Dual language education programs: Current state policies and practices. American Institutes for Research.

Cohodes

S. R.

Setren

E. M.

Walters

C. R.

(2021). Can successful schools replicate? Scaling up Boston’s Charter School Sector. American Economic Journal: Economic Policy, 13(1), 138–167. https://doi.org/10.1257/pol.20190259

Collier

V. P.

Thomas

W. P.

(2004). The astounding effectiveness of dual language education for all. NABE Journal of Research and Practice, 2(1), 1–20.

de Chaisemartin

Behaghel

(2020). Estimating the effect of treatments allocated by randomized waiting lists. Econometrica, 88(4), 1453–1477. https://doi.org/10.3982/ECTA16032

Dunbar

Welch

(2015). Iowa assessments research and development guide. University of Iowa.

10.

(2018). Exploring Korean dual language immersion programs in the United States: Parents’ reasons for enrolling their children. International Journal of Bilingual Education and Bilingualism, 21(6), 690–709. https://doi.org/10.1080/13670050.2016.1208144

11.

Grundy

J. G.

Timmer

(2017). Bilingualism and working memory capacity: A comprehensive meta-analysis. Second Language Research, 33(3), 325–340. https://doi.org/10.1177/0267658316678286

12.

Lipsey

M. W.

Puzio

Yun

Hebert

M. A.

Steinka-Fry

Cole

M. W.

Roberts

Anthony

K. S.

Busick

M. D.

(2012). Translating the statistical representation of the effects of education interventions into more readily interpretable forms (NCSER 2013-3000). Institute of Education Sciences, National Center for Special Education Research. https://ies.ed.gov/ncser/pubs/20133000/pdf/20133000.pdf

13.

Morales

Calvo

Bialystok

(2013). Working memory development in monolingual and bilingual children. Journal of Experimental Child Psychology, 114(2), 187–202. https://doi.org/10.1016/j.jecp.2012.09.002

14.

Nicolay

A.-C.

Poncelet

(2015). Cognitive benefits in children enrolled in an early bilingual immersion school: A follow up study. Bilingualism: Language and Cognition, 18(4), 789–795. https://doi.org/10.1017/S1366728914000868

15.

Owens

(2020). English learners deserve more: An analysis of Georgia’s education for speakers of other languages. Georgia Budget and Policy Institute.

16.

Parkes

(2008). Who chooses dual language education for their children and why. International Journal of Bilingual Education and Bilingualism, 11(6), 635–660. https://doi.org/10.1080/13670050802149267

17.

Roberts

(2021). 2021 Canvass of dual Language and immersion (DLI) programs in U.S. public schools. American Councils Research Center. https://www.americancouncils.org/research-assessment/studies-and-reports

18.

Shen

Wang

Zhang

Barbieri

C. A.

Pasquarella

(2022). The effect of early enrollment in dual-language immersion programs on children’s English reading development: Findings from a 5-year longitudinal study. International Journal of Bilingual Education and Bilingualism, 25(10), 3773–3790. https://doi.org/10.1080/13670050.2022.2079373

19.

Steele

J. L.

Slater

R. O.

Zamarro

Miller

Burkhauser

Bacon

(2017). Effects of dual-language immersion programs on student achievement: Evidence from lottery data. American Educational Research Journal, 54(1), 282S–306S. https://doi.org/10.3102/0002831216634463

20.

Watzinger-Tharp

Swenson

Mayne

(2018). Academic achievement of students in dual language immersion. International Journal of Bilingual Education and Bilingualism, 21(8), 913–928. https://doi.org/10.1080/13670050.2016.1214675

21.

Wiseheart

Viswanathan

Bialystok

(2016). Flexibility in task switching by monolinguals and bilinguals. Bilingualism: Language and Cognition, 19(1), 141–146. https://doi.org/10.1017/S1366728914000273

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.27 MB