Sage Journals: Discover world-class research

Abstract

Under Florida’s developmental education (DE) reform since 2014, recent public high school graduates and active-duty military personnel became exempt from DE and traditional placement tests. The legislation also required colleges to provide accelerated instruction strategies for students remaining in DE and offer enhanced advising and support services for all incoming students. Focusing on the differential policy impacts on exempt and nonexempt students, we used statewide administrative data to examine changes in first-year completion rates for gateway math and English courses before and after the reform. Overall, and for those deemed to be college-ready, nonexempt students benefited more from the reform; however, the opposite is true for students who were not college-ready, with exempt students benefiting more. This study confirms that each component of the reform has played a role in contributing to its success.

Keywords

community college developmental education exemption status placement policy

Introduction

Developmental education (DE) courses are intended to improve the reading, writing, and/or math skills of students who are deemed underprepared for college-level classes. Among students who began their postsecondary education during 2013–14, nearly 60 percent of community-college students and one-third of 4-year college students took at least one developmental course (Chen et al., 2020). However, the vast majority of students assigned to DE never finish their DE sequence, nor do they take and complete introductory college-level (gateway) courses in English and mathematics (Bailey et al., 2010). Given the low completion rates in gateway courses for students required to take DE as well as the expenditures of students’ time and money, colleges and states across the country have been experimenting with new practices to improve the effectiveness and efficiency of the DE system over the past two decades.

Overall, these DE reforms focus on three aspects: reforming placement processes, reducing lengthy course sequences, and improving instructional approaches (Jaggars & Bickerstaff, 2018). These three factors have been linked with how the traditional DE system fails to fulfill its promise to help students succeed in college. For example, research suggests that the traditional placement system that relies solely on standardized tests often results in high levels of placement inaccuracy. One study estimates that a quarter of students assigned to DE could have succeeded in the relevant college-level course if they had been allowed to enroll in it directly (Scott-Clayton et al., 2014). The time-consuming and costly sequence of multiple developmental courses is also problematic because only a small percentage of students can complete it (Bailey et al., 2010).

Although many of the reforms in other contexts may include only isolated practices and tend to be small in scope, in 2014, Florida implemented one of the most comprehensive DE reform efforts under Senate Bill (SB) 1720 of 2013, which covers three of the reform factors mentioned above. First, the reform made placement tests and DE optional for exempt students—those who entered a Florida high school during or after the school year 2003–2004 and earned a standard high school diploma as well as students who are active-duty military members. Second, colleges were required to offer DE courses in one of four specific instructional strategies consisting of compressed, contextualized, corequisite, and modularized courses that were intended to reduce the length of time required to complete the DE sequence. These courses could be taken voluntarily by exempt students and were required for nonexempt students who scored below college-ready. Finally, colleges were required to provide additional advising and academic support services to all students, regardless of their exemption status.

The reform categorized the students at Florida College System (FCS) institutions into two groups: exempt and nonexempt students. Under SB 1720 of 2013, two-thirds of first-time-in-college (FTIC) students became exempt from placement tests and had the option to bypass developmental courses regardless of their level of college readiness, whereas nonexempt students were still required to take the placement tests and were assigned to different levels of courses according to their performances on the placement tests. The purpose of this study is to explore how the reform influences exempt and nonexempt students in different ways. Specifically, we ask: How have the first-year completion rates in introductory college-level math and English courses changed before and after their implementation of SB 1720 for exempt and nonexempt students? By comparing the policy effects for exempt and nonexempt students, this study can provide insights into the possible effects of the different components of SB 1720 and offer policy implications for institutions and states that are reforming their DE systems.

In the following sections, we first review relevant literature and provide a richer context around the DE reform in Florida. We then describe our analytic approach, which employs a comparative interrupted time series (CITS) design to examine differences in changes of first-year gateway-course completion before and after the reform for exempt and nonexempt students. We use student-level administrative data for students at all 28 FCS institutions from Cohort 2011 to Cohort 2016. After reporting our findings, we conclude with a discussion of the implications for policy and future research.

Literature Review

It is very difficult to assess the impact of DE because placement into developmental courses is inherently endogenous. Students who are assigned to DE are not directly comparable to those who initially enroll in college-level courses, particularly in terms of prior academic preparation. Rigorous research usually adopts a regression discontinuity (RD) design to address this methodological challenge by comparing students scoring just above the college-ready cut score on the placement test to those scoring just below college-ready. A meta-analysis of multiple RD studies finds that in comparison to their peers who are also on the margin of college-readiness but who were placed into college-level courses, students placed into DE earned fewer college credits after about 3 years, were about 8 percentage points less likely to pass the college-level course, and were about 1.5 percentage points less likely to earn a certificate or degree (Valentine et al., 2017). However, because RD studies focus on students who score within a few points of the college-ready cutoff score on the placement exam, it remains unclear whether these results are generalizable to the larger developmental population. Studies that have examined students with very low placement test scores usually yield null or negative effects (e.g., Melguizo et al., 2016; Xu, 2016). A noticeable exception comes from Boatman and Long (2018), who find that the lowest scoring students might experience positive effects from taking an additional developmental reading or writing course.

The past two decades have witnessed significant reforms in the DE system across the country. One important aspect of these reforms is to experiment with new strategies to track students into different courses that are aligned with their prior academic preparation. Traditionally, many institutions have administered a multiple-choice test in mathematics, reading, and writing to determine whether incoming students should be placed into developmental or college-level courses. However, evidence suggests that this test-placement policy tends to frequently “under place” students—assigning them to developmental courses when they could have succeeded in college-level courses (Scott-Clayton et al., 2014). In response, a growing number of public colleges has begun using additional measures to assess college readiness, such as students’ high school performance and noncognitive factors (e.g., motivation or commitment; Rutschow et al., 2019).

Although existing research generally indicates that incorporating multiple measures of high school achievement can improve placement accuracy (Bahr et al., 2019; Scott-Clayton et al., 2014), it is not always feasible to implement this multiple-measure approach, given the substantial administrative burdens for students and college personnel. In a recent study focusing on community-college students in Florida, Leeds and Mokher (2020) find that adjusting existing placement-score cutoffs tends to be more effective in minimizing misplacement than using new metrics, such as high school transcript data. In addition, Kosiewicz and Ngo (2020) have examined the impacts of self-placement—allowing students to determine on their own the extent to which they are ready for college-level coursework. Their results demonstrate that in comparison to test-based placement, self-placement can lead to positive outcomes, although these benefits are mostly seen in White, Asian, and male students (Kosiewicz & Ngo, 2020).

In addition to placement-policy changes, another important aspect of recent DE reforms focuses on how DE is delivered. Traditionally, students who were assigned to DE might need to take multiple semester-long developmental courses before they could take college-level credits. Many colleges are now experimenting with accelerated developmental sequences, which are designed to allow students to move more quickly to college-level courses. The most well-known acceleration model may be the corequisite DE, which pairs a college-level course with either a developmental course or another formally required learning support, such as tutoring (Jaggars & Bickerstaff, 2018). The Education Commission of the States notes that 24 states or systems allow or require the use of corequisite support (Whinnery & Odekar, 2021). For example, Texas House Bill 2223 (2018) requires all public colleges to implement corequisite DE. Given the extensive evidence of the effectiveness of corequisite courses (e.g., Boatman, 2021; Logue et al., 2019), corequisite DE has become an increasingly popular policy option across the United States.

Study Context and Conceptual Framework

Following the work of Brower et al. (2018), we use a conceptual framework based on Vygotsky’s theory of scaffolding and apply it to the context of Florida’s DE reform to better understand the different needs of college-ready and underprepared students. According to Vygotsky (1978), there are three levels of learning: (a) skills that students can master on their own, (b) more complex skills that students may be able to master with help from others, and (c) skills that are too advanced for students’ current level of development (regardless of the level of support received). Each student has their own “zone of proximal development” that contains the mid-level skills that they may be able to achieve by working with someone more competent, such as a teacher or more advanced peers, to scaffold their learning. As students begin to develop these skills, the scaffolding can gradually be removed until students have fully mastered them and can use them independently. Research by developmental psychologists, including Bruner (1984), has found that students tend to have the greatest learning gains when they are supported by others in mastering the most complex skills in their zone of proximal development. Work by such scholars as Bemprechat (1992) suggests that the use of scaffolding may be particularly important for at-risk students, who tend to experience weaker academic socialization from their parents.

Citing poor degree-completion rates and high costs for DE students, Florida passed SB 1720 in 2013, which would significantly transform the DE system at FCS institutions. The major intents of the DE reform were to reduce the cost of DE and accelerate students’ progression from developmental instruction to college-level coursework, which would eventually contribute to improving degree-completion rates. Policymakers also made special provisions for nonexempt students who did not attend a Florida public school following the implementation of Governor Jeb Bush’s 1999 “A+ Plan.” Although the rationale for the exemption criteria is not provided in the legislation, it is likely due to concerns that these students may not have received adequate preparation in high school under other standards to enroll directly into college-level courses. This relates back to the concept of scaffolding in that students enter college with different levels of competencies, which may require different levels of support from others to ensure the achievement of college-readiness.

In this study, we focus on some intermediate outcomes—completion of college-level courses within the first year—that are considered to be effective predictors of longer-term student success (Belfield et al., 2019). We establish a theory of change based on Florida’s context, as shown in Figure 1. We illustrate how each component of the reform is supposed to improve student success and how the policy effects vary depending on student background characteristics, such as exemption status and level of college readiness.

Figure 1.

Theory of change for Florida’s SB 1720.

Thanks to the changes in placement policy, the first component of Florida’s DE reform, about two-thirds of FTIC students became exempt from placement testing and DE courses. The remaining one-third are nonexempt students, such as students who graduated from a high school out of state, homeschooled students, private-school students, students who earned a general equivalency diploma, and private-charter-school students. Although exempt students still had the option to take developmental courses if they believed that they needed additional support, nearly half of underprepared exempt students—those who would have been assigned to developmental courses without a policy change—directly enrolled in a gateway math or English class (Woods et al., 2019; Zhao et al., 2020). Additionally, many exempt students did not take any math or English class in the first year of college enrollment. Only a small percentage of exempt students opted into DE after the reform (see Table A1 in the Appendix). As a result, the students who remained in DE were mostly nonexempt students who were still required to take placement tests and placed into different levels of classes according to their performance on these tests.

The second component of Florida’s DE reform was the implementation of new instructional strategies, including modularized, compressed, contextualized, and corequisite DE courses. Modularized courses break course material into smaller units and allow students to complete customized modules. Compressed courses combine two or more developmental courses into a single one-semester experience. Contextualized courses have content that is more tailored to a student’s intended major. Corequisite courses allow students to take a DE course and a gateway course in the same subject area concurrently in the same semester. Prior studies on these accelerated DE strategies have suggested positive results (Boatman, 2021; Edgecombe et al., 2014; Okimoto & Heck, 2015; Skuratowicz et al., 2020). As mentioned above, because the students who enrolled in DE courses after Florida’s reform were mostly nonexempt students, the changes to the DE instructional strategies would affect a larger percentage of nonexempt students than exempt students.

The final component of Florida’s DE reform required all 28 FCS institutions to offer enhanced advising and academic-support services for all incoming students, regardless of their exempt or nonexempt status. Many institutional leaders reported that they had increased the types and amount of advising offered since implementing SB 1720, such as extending advising office hours or expanding programming (Woods et al., 2017). The majority of institutional administrators surveyed in Woods et al. (2017) also reported that their advising systems were effective overall, particularly in relation to students’ exemption status under SB 1720. However, the survey results also indicated that many colleges struggled to handle the advising caseload, partly due to limited staff or other resource constraints. Only two out of 19 institutional representatives reported that their institutions offered separate orientations according to whether students were exempt or nonexempt (Woods et al., 2017).

Prior research has suggested the positive effects of Florida’s DE reform on student success. Many more students enrolled and completed introductory college-level English and math courses within the first year of enrollment after the reform (Park-Gaghan et al., 2020). Students also attempted and earned more college credits following the DE reform (Mokher et al., 2020). Existing studies also suggest that students of color benefited more from the DE reform than did White students, closing the racial gap in student outcomes (Mokher et al., 2020; Park-Gaghan et al., 2020). Moreover, one study explored the differential effects of Florida’s DE reform by students’ academic preparation and found that students with the lowest level of preparation benefited the most (Park-Gaghan et al., 2021). Prior studies usually focused on the overall students, including exempt and nonexempt, and investigated the impacts of the reform as a whole. An exception is Mokher et al. (2021), who conducted a RD study specifically focusing on nonexempt students. Their results suggest that first-year math-course outcomes tended to be worse for nonexempt students assigned to DE relative to similar students scoring just above college-ready.

Our study advances previous studies by comparing the policy effects on two groups that are defined by the policy itself: exempt and nonexempt students. We further disaggregate these two groups into four subgroups, based on their level of college readiness. Table 1 summarizes our hypotheses for how the four subgroups may be influenced by each component of the reform. We use the terms none, small, medium, and large to indicate the extent of the expected influence. The first group of college-ready exempt students is hypothesized to be only minimally influenced by the reform, as most of these students would enroll in college-level courses regardless of the reform and may not have much need for enhanced advising and support services because they are already college-ready. The second group of college-ready nonexempt students would also likely enroll in college-level courses regardless of the reform, although they may experience medium-sized effects from the enhanced advising and support services because this group includes older returning adults who may benefit more from this additional support as they transition back into being students again. In contrast to the college-ready students, the third group of underprepared exempt students is expected to be greatly influenced by all three components of the reform. Due to the changes in placement policy, underprepared exempt students are allowed to bypass DE and enroll directly in college-level courses. The remaining students who opt into DE should benefit from the new DE instructional strategies. Moreover, underprepared exempt students are also the primary target of institutions’ new advising and support services (Hu et al., 2021) and may need these services the most if they opt out of DE. Nonexempt students are not affected by the changes in placement policy, as they are still subject to test-based placement. The last group of underprepared nonexempt students who are assigned into DE courses may largely benefit from the new DE instructional strategies that are intended to help them move into college-level courses more quickly. Nonexempt students are also part of the target focus of the advising and support services, although they might have received less attention than underprepared exempt students (Hu et al., 2021). Based on these discussions, we posit that underprepared exempt students will benefit the most from the reform, while college-ready exempt students will benefit the least.

Table 1

Hypothesized differential policy effects, by exemption and college-ready status

	Changes in placement policy	New DE instruction strategies	Enhanced advising and support services
College-ready exempt students (received enhanced advising and support services only)	None	None	Small
College-ready nonexempt students (received enhanced advising and support services only)	None	None	Medium
Underprepared exempt students (received the option for self-placement or new DE instructional strategies, plus enhanced advising and support services)	Large	Medium	Large
Underprepared nonexempt students (received new DE instructional strategies, plus enhanced advising and support services)	None	Large	Medium

Note. DE = developmental education.

Methods

Data and Variables

We use data from the Florida PK-20 Education Data Warehouse, which tracks all Florida public-school students remaining in-state from kindergarten to postsecondary education. The sample includes all FTIC students at Florida’s 28 state and community colleges who enrolled 3 years prior to SB 1720 (2011–13 cohorts) and 3 years after SB 1720 (2014–16 cohorts). Our main outcome variable is dichotomous, indicating whether a student passed the college-level course within the first year (0 = no, 1 = yes). Because we treat students who did not take a college-level course in the first year as not passing rather than missing, the aggregated results can be viewed as cohort-based passing rates, which indicate the overall effectiveness of the reform with regard to increasing the percentage of students passing the college-level courses within the first year. We also conduct additional analyses on the likelihood of passing a college-level course, conditional on taking college-level courses during the first year. We refer to these outcomes as course-based passing rates and report the results in Section C of the Online Appendix.

For English, the statewide introductory college-level course is English composition (ENC 1101). Giving that some highly qualified students may skip ENC 1101 and enroll in more advanced courses, our outcome measure includes any advanced English course in addition to ENC 1101 (we refer to this as gateway English). For math, four gateway courses can fulfill the associate degree math requirement: MAC1105 (College Algebra), MGF1106 (Mathematics for Liberal Arts I), MGF1107 (Mathematics for Liberal Arts II), and STA2023 (Introductory Statistics). Additionally, certain highly qualified students may enroll in more advanced math courses. We include all of these college-level courses in constructing our measures for math passing rates. We also capture another course: MAT 1033 (Intermediate Algebra), the most common prerequisite math course taken by many students prior to enrollment into gateway courses. MAT 1033 is not a developmental course from which exempt students are able to opt out, and it counts for elective credit, although it does not fulfill the mathematics general education requirement for an associate degree. Therefore, we construct our indictor for math passing rates in two ways: with MAT 1033 (we refer to this as pre-gateway math) and without MAT 1033 (we refer to this as gateway math).

For the post-policy cohorts, the data include an indicator for whether the student was exempt from DE under SB 1720 as defined by the state. To create a comparable indicator for pre-policy cohorts, we treated students as “likely exempt” if they entered a Florida public school in 2003–04 or later and then obtained a standard high school diploma. Due to data limitations, we are unable to include active-duty military personnel in the pre-policy exemption variable. However, this should have little impact on the results, given that less than 1% of students enrolled in FCS institutions are classified as active duty (Florida College System, 2018).

We include a set of student background variables to control for the effects of individual student characteristics. Specifically, we include a categorical variable for race/ethnicity (including four categories: White, Black, Hispanic, and other races/ethnicities), a dichotomous indicator for gender, a dichotomous indicator for students age 25 or above, and a dichotomous indicator for free-/reduced-lunch eligibility. In addition, we include variables for students’ college readiness as measured by their postsecondary education readiness test (PERT) scores—the placement test required for nonexempt students. Students are allowed to take the PERT multiple times, and we have used their highest PERT scores. Some students have missing values for PERT math, reading, or writing scores. In these cases, we use the dummy variable adjustment method, where the value of the missing variables is set to a constant value of zero, and an additional dummy variable is added to the model to indicate whether the actual value is missing (Cohen & Cohen, 1983). Thus, college-readiness variables include three categories: (a) missing, (b) underprepared (those who scored below college-ready), and (c) college-ready. Descriptive characteristics of the sample, by exempt status and policy, are presented in Table 2. As shown in Table 2, exempt and nonexempt students experienced increases in gateway completion rates in the post-reform period compared with the pre-reform period. In all three outcome measures, nonexempt students experienced greater increases than did exempt students (15.86 versus 5.66 percentage points for pre-gateway math, 11.72 versus 2.90 percentage points for gateway math, and 14.16 versus 5.08 percentage points for gateway English).

Table 2

Descriptive statistics, by exemption status and policy

	Exempt			Nonexempt
	Pre-reform	Post-reform	Diff.	Pre-reform	Post-reform	Diff.
Student outcome
Completion rates: pre-gateway math	36.79	42.45	5.66	19.06	34.92	15.86
Completion rates: gateway math	21.14	24.04	2.90	9.83	21.55	11.72
Completion rates: gateway English	54.50	59.58	5.08	32.91	47.07	14.16
Student background
% White	40.76	37.60	−3.16	40.84	36.34	−4.50
% Black	21.46	21.17	−0.29	21.80	17.42	−4.38
% Hispanic	32.00	33.95	1.95	30.56	37.99	7.43
% Other race/ethnicity	5.78	7.28	1.50	6.80	8.25	1.45
% Female	51.89	51.98	0.09	52.92	53.07	0.15
% Age 25 or older	0.28	5.11	4.83	40.32	24.05	−16.27
% Free-/reduced-lunch eligible	42.53	48.20	5.67	17.31	26.13	8.82
Math readiness: % college-ready	20.03	20.72	0.69	10.92	18.04	7.12
Math readiness: % underprepared	50.15	46.77	−3.38	57.01	52.09	−4.92
Math readiness: % missing	29.82	32.52	2.70	32.07	29.87	−2.20
English readiness: % college-ready	27.16	28.75	1.59	28.36	33.53	5.17
English readiness: % underprepared	37.94	37.62	−0.32	38.38	34.79	−3.59
English readiness: % missing	34.90	33.62	−1.28	33.26	31.68	−1.58
N	130,018	136,030		75,787	66,881

Analytical Strategy

We used a CITS design to look for any differential changes in students’ completion of college-level courses before and after SB 1720 between exempt and nonexempt students. We estimated the model below for student i at college j in year (cohort) t:

\begin{matrix} Logit (y_{i j t}) = β_{0} + β_{1} (P o s t) + β_{2} {(E x e m p t)}_{i j t} \\ + β_{3} {(P o s t * E x e m p t)}_{i j t} + β_{4} {(S)}_{i j t} + ξ_{j} + λ_{t} . \end{matrix}

Under this specification, Post is a dichotomous indicator for post-policy cohorts. Exempt is a binary variable indicating students’ exempt status. We added an interaction between this indicator and exempt status to test the differential changes for exempt and nonexempt students. β ₄ is a vector of coefficients representing student background characteristics (S; including math and English readiness, based on placement test scores). In the model, we use math readiness for math outcomes and use English readiness when the outcome is gateway English. ξ_j is a college fixed effect to account for unobserved heterogeneity across institutions. λ_t is a cohort fixed effect to control for variations across cohorts.

Because exempt and nonexempt students are affected by the DE reform, a potential concern of our CITS design is that there is no control group under business-as-usual conditions, and any changes in outcomes over time might be attributed to changes in student demographics over time. However, this doesn’t seem to be much of an issue in our data. To examine the identification assumption of the CITS design of whether the pre-post changes of student characteristics for the treatment group were similar to changes in the control group, we ran a series of regression models using the equation from our impact estimates, where we replaced the outcome variable with each of the student characteristics. We examined the interaction term post * exempt to assess whether changes in student composition were similar for exempt and nonexempt students before and after the reform. Results from regression analyses (see Table A1 of the Online Appendix) indicate that changes in student background characteristics between exempt and nonexempt students are either nonsignificant or statistically significant with odds ratios close to 1, with the exception of the age variable. To further assess the magnitude of the differences in demographics, we calculated the standardized mean difference between the exempt and nonexempt students for pre- and post-reform cohorts separately. According to the What Works Clearinghouse standards for baseline equivalence (2020, p. 13), any difference with an absolute value of 0.25 standard deviation or smaller demonstrates baseline equivalence as long as there is statistical adjustment (e.g., regression adjustment). All standardized mean differences of the specified demographic characteristics are well below 0.25 except for the age variable (see Table A2 of the Online Appendix). As a robustness check, we conducted subgroup analyses for students below age 25 and those age 25 or above separately. We find that the pattern and significance of our results, for the most part, hold for students under age 25. However, for students age 25 or above, only accounting for a small share (13%) of the sample, we find that exempt and nonexempt students experienced similar increases in the completion rates of college-level math and English courses (see Tables A3–A8 of the Online Appendix). Therefore, we remind readers that although the results we report below are based on analyses of the overall students, these results largely reflect the experiences of traditionally aged college students (i.e., students under age 25), who account for 87% of the total students.

Results

Differential Impacts for Overall Exempt and Nonexempt Students

We begin by looking at the descriptive changes of completion rates for gateway courses over time. As shown in Figure 2, nonexempt students experienced sharp increases in 2014 immediately after the policy was implemented in all three outcome measures, whereas the completion rates for exempt students remained relatively stable from 2013 to 2014. The gaps between nonexempt and exempt students narrowed in the post-reform period, particularly for pre-gateway and gateway math.

Figure 2.

Completion rates for gateway courses over time for all students, by exemption status.

We now turn to our regression-adjusted results under the CITS design. To address the difficulties in the interpretation of interaction effects in nonlinear models, we report our results as predicted probabilities and calculate the average marginal effects (AMEs) of the policy (i.e., the pre-post differences in the predicted probabilities) for exempt and nonexempt students separately. As shown in Table 3, among nonexempt students, the probabilities of completion in pre-gateway and gateway math increased after the reform by 14.6 and 11.5 percentage points, respectively. Both increases were significantly greater than those of exempt students (9.7 and 6.0 percentage points, respectively). In other words, nonexempt students benefited more from the reform than did exempt students in completion rates for pre-gateway and gateway math. However, there were no significant differences in the policy’s effect on gateway English completion by exempt status. The probabilities of completing gateway English increased by 12.9 percentage points after the reform for nonexempt students, compared to 9.9 percentage points for exempt students.

Table 3

Predicted probabilities of first-year cohort-based passing rates in gateway courses before and after the reform among all students, by exemption status

	Pre-reform	Post-reform	AMEs	Contrasts
Panel A: Pre-gateway math
Exempt	0.321	0.418	0.097***
Nonexempt	0.239	0.385	0.146***	0.049***
Panel B: Gateway math
Exempt	0.178	0.238	0.060***
Nonexempt	0.129	0.243	0.115***	0.055***
Panel C: Gateway English
Exempt	0.500	0.599	0.099***
Nonexempt	0.368	0.497	0.129***	0.030

Note. Predicted probabilities are based on CITS analyses. Asterisks in the “AMEs” column show statistical significance of pre-post differences in course completion for each group. Asterisks in the “Contrasts” column show statistical significance of nonexempt group difference compared to exempt group difference. AME = average marginal effect; CITS = comparative interrupted time series.

p < 0.05; **p < 0.01; ***p < 0.001.

Differential Impacts for Exempt and Nonexempt Underprepared Students

Given the evidence of differential impacts of Florida’s DE reform by different levels of academic preparation (Park-Gaghan et al., 2021), we disaggregated our data into different subgroups based on students’ PERT scores and did subgroup analyses to check whether the results we reported above hold for different subgroups. In this section, we restricted our analytical sample to underprepared students based on their PERT performance. Figure 3 suggests that all underprepared students, regardless of exempt status, experienced some increases in the completion rates for gateway courses after the reform, and these increases appeared greater for exempt students than for nonexempt students. These results are confirmed through CITS analysis.

Figure 3.

Completion rates for gateway courses over time for underprepared students, by exemption status.

Predicted probabilities for completion of pre-gateway math (Panel A, Table 4) showed statistically significant increases for all underprepared students following the reform. The predicted probabilities of completing pre-gateway math increased by 21.3 percentage points for exempt underprepared students, significantly larger than the increase experienced by nonexempt underprepared students (17.4 percentage points). For completion of gateway math, exempt underprepared students also experienced greater increases after the reform relative to their nonexempt counterparts (17.1 versus 13.1 percentage points; p < .01). Additionally, exempt underprepared students experienced post-reform gains in the completion of gateway English that were 5.9 percentage points larger than nonexempt underprepared students (a total change of 27.0 percentage points). To sum, in all three outcome measures, exempt underprepared students benefited more from the policy than did nonexempt underprepared students.

Table 4

Predicted probabilities of first-year cohort-based passing rates in gateway courses before and after the reform among underprepared students, by exemption status

	Pre-reform	Post-reform	AMEs	Contrasts
Panel A: Pre-gateway math
Exempt	0.152	0.365	0.213***
Nonexempt	0.099	0.273	0.174***	−0.039**
Panel B: Gateway math
Exempt	0.029	0.200	0.171***
Nonexempt	0.009	0.140	0.131***	−0.040***
Panel C: Gateway English
Exempt	0.329	0.599	0.270***
Nonexempt	0.181	0.392	0.212***	−0.059*

Note. Predicted probabilities are based on CITS analyses. Asterisks in the “AMEs” column show statistical significance of pre-post differences in course completion for each group. Asterisks in the “Contrasts” column show statistical significance of nonexempt group difference compared to the exempt group difference. AME = average marginal effect; CITS = comparative interrupted time series.

p < 0.05; **p < 0.01; ***p < 0.001.

Differential Impacts for Exempt and Nonexempt College-Ready Students

We then restricted the analytical sample to college-ready students. As shown in Figure 4, among college-ready students, nonexempt students experienced greater increases in the completion of pre-gateway and gateway math after the reform and outperformed their exempt peers. For the completion of gateway English, there appeared an increasing trend before the reform among exempt students that stalled after the reform, whereas nonexempt students experienced moderate increases from 2011 to 2016.

Figure 4.

Completion rates for gateway courses over time for college-ready students, by exemption status.

According to the CITS analysis shown in Table 5, the predicted probability of completion in pre-gateway math increased after the reform by about 15.8 percentage points for nonexempt college-ready students, compared to an increase of 8.5 percentage points for exempt college-ready students. The patterns for gateway math were similar. Nonexempt college-ready students experienced significantly greater increases after the reform than did exempt college-ready students (18.0 versus 11.7 percentage points; p < .001). Completion rates in gateway English increased by 9 to 10 percentage points, with no statistically significant differences by exempt status.

Table 5

Predicted probabilities of first-year cohort-based passing rates in gateway courses before and after the reform among college-ready students, by exemption status

	Pre-reform	Post-reform	AMEs	Contrasts
Panel A: Pre-gateway math
Exempt	0.556	0.642	0.085***
Nonexempt	0.511	0.669	0.158***	0.072***
Panel B: Gateway math
Exempt	0.332	0.450	0.117***
Nonexempt	0.319	0.499	0.180***	0.063***
Panel C: Gateway English
Exempt	0.592	0.682	0.090***
Nonexempt	0.525	0.623	0.099***	0.009

p < 0.05; **p < 0.01; ***p < 0.001.

Discussion

This study examined the differential impacts of Florida’s DE reform on exempt and nonexempt students. It sought to advance prior research on the effect of Florida’s DE reform on student success by assessing how the impacts of the reform differ, depending on the types of changes experienced by exempt and nonexempt students. The results show that exempt and nonexempt students experienced increases in the completion rates for college-level math and English courses following the implementation of SB 1720. This demonstrates progress toward the reform’s goals of accelerating progress into college-level coursework for underprepared students, although it’s too early to tell whether these early gains will translate into longer-term impacts on college completion. The reform also intended to address different levels of high school preparation of incoming students by providing an exemption to DE requirements for students who attended a Florida public school under reformed standards, which are intended to be aligned with the state’s postsecondary readiness competencies. We believe that this demonstrates that policymakers were equally concerned with outcomes for nonexempt and exempt students, but that they perceived different means as being needed to achieve these goals for each group.

Overall, nonexempt students experienced greater increases than did exempt students, particularly in math. This pattern also holds for college-ready students. The changes in placement policy and new DE instructional strategies had little relevance for college-ready students, so any benefits of the reform may be attributed to the enhanced advising and support services. The findings for college-ready students consistent with our original hypothesis—that enhanced support services may only have a small effect for students who completed high school standards—more closely align with college courses in the FCS; nonexempt students (particularly older returning adults) may experience more medium-sized effects from the additional support as they make the transition back to the educational system.

Underprepared exempt and nonexempt students also experienced gains, but exempt students experienced greater increases in completion rates for college-level math and English courses. These results also correspond with our original hypothesis—that underprepared nonexempt students have the potential to benefit from all three components of the reform—and the support services may be particularly important for those who opt out of DE. In contrast, all underprepared nonexempt students were required to take DE under the new instructional strategies, even though some of these students may have benefitted more if given the option to bypass DE altogether. Below, we discuss various implications for policy and practice.

The first implication for practice is the need for policymakers to consider which students should be given the option of self-placement. In thinking about the context of Florida’s reform, we need to step back and look at the changes in students’ course enrollment patterns to fully understand the changes in completion rates. Following the reform, due to the changes in placement policy, many underprepared exempt students who would have been assigned to DE were able to skip it and enroll directly in college-level courses. As shown in Table B2 of the Online Appendix, for underprepared exempt students, gateway English enrollment rates increased by 25.80 percentage points, while pre-gateway and gateway math enrollment rates increased by 28.37 and 16.67 percentage points, respectively. Although nonexempt underprepared students also experienced some increases in first-year college-level math and English enrollment rates, partly because of the accelerated DE strategies, these increases are substantially smaller than those of exempt underprepared students, although there may be negative consequences on course-based passing rates in college-level courses for some students, given the curricular and pedagogical challenges associated with educating more academically underprepared students (see Table C3 of the Online Appendix). However, our results indicate that exempt underprepared students experienced greater increases in cohort-based passing rates for gateway math and English courses than did their nonexempt peers. These results confirm that exempt students, particularly exempt underprepared students, benefit from the changes in placement policy. Although it’s impossible to know what would have happened if exempt and unexempt students had been given the choice of opting out of DE, policymakers should consider that some underprepared students may benefit more from the option of enrolling directly in college-level courses.

A study that specifically focuses on nonexempt students scoring around the college-ready cutoff suggests that many nonexempt students should have benefited from the option to enroll directly in college-ready courses like their exempt peers (Mokher et al., 2021). Given that nonexempt students are still subject to a test-based placement policy following the reform, it could have limited the reform’s effects for this group. Instead of having two categories of students on campus and treating them differently, policymakers may consider extending the application of the self-placement policy to include nonexempt students as well (Mokher et al., 2021). California’s DE reform under Assembly Bill (AB) 705 and 1805 has made DE optional for all students, and early assessment of the reform has documented promising results (Mejia et al., 2020).

A second implication for practice is that advisors should provide guidance to address concerns about potential changes in students’ course-enrollment patterns under the self-placement policy. Although exempt students experienced increases in the enrollment rates for college-level courses thanks to the self-placement policy, the percentage of exempt students who did not enroll in any English or math course in the first year increased by 4.07 and 8.38 percentage points, respectively (see Table B3 of the Online Appendix). On the contrary, the percentage of nonexempt students who did not enroll in any English or math decreased by 6.19 and 5.18 percentage points, respectively (see Table B3 of the Online Appendix). It seems that when given the freedom to decide their initial course placement, exempt students tend to either enroll directly in college-level courses or not enroll in any math or English course at all. It is possible that they may postpone the enrollment into the second year. However, a recent study that explored the relationship between initial math enrollment and student outcomes by the third year found that students who did not enroll in any math achieved the worst 3-year outcomes compared to students who either enrolled in DE math or college-level math (Zhao et al., 2020). This has implications for practice, as advisors may need to more strongly emphasize the importance of not delaying core requirements as they guide students in developing their programs of study.

A third recommendation for practice is that policymakers in other states implementing similar reforms should consider that many students, regardless of their level of academic preparation, may benefit from enhanced support services. Our results demonstrate that although college-ready students were only minimally influenced by the changes in placement policy and DE instruction strategy, they also experienced increases in completion rates in gateway math and English courses. These results suggest that college-ready students may have benefited from the enhanced advising and academic-support services that are available to all incoming students. Before the reform, around 22% of nonexempt college-ready students did not enroll in any English course within the first year, and nearly 28% did not enroll in any math course (see Table B3 of the Online Appendix). These numbers decreased by 5.06 and 9.43 percentage points after the reform. Consequently, higher percentages of nonexempt college-ready students enrolled in gateway courses after the reform, whereas enrollment rates remain relatively stable among exempt college-ready students (see Table B2 of the Online Appendix). These differences in the changes in enrollment rates can partly explain why nonexempt college-ready students experienced greater gains in completion rates for pre-gateway and gateway math than did exempt college-ready students. Our additional analyses also demonstrate that nonexempt college-ready students experienced greater gains in course-based passing rates for pre-gateway math and gateway English than did exempt college-ready students (see Table C2 of the Online Appendix). The differential changes experienced by exempt students and nonexempt students among the college-ready student population might be associated with the take-up rates of the enhanced advising and support services. Nonexempt students tend to be older (see Table 2) and won’t have taken a math course in a while, so they may have a greater need for additional supports, such as tutoring, when they are in college, which could have contributed to greater gains.

In addition, FCS administrators reported that their new academic support services were targeted primarily at academically underprepared exempt students in the initial stage of policy implementation and gradually expanded their focus to include nonexempt students (Hu et al., 2021). These arrangements make sense, given that underprepared exempt students are the subgroup that is most directly affected by the changes in placement policy and have received the most attention in the beginning. On the contrary, college-ready exempt students have received the least attention in the implementation process. These differences may also be associated with our results suggesting that underprepared exempt students benefited the most from the reform, while college-ready exempt students benefited the least.

In conclusion, results from this study confirm that each component of Florida’s DE reform has played a role in contributing to its overall success. However, we are not able to determine which is more important than the others. We also acknowledge that as a comprehensive reform, its impacts may be greater than if each component were implemented as an isolated practice. Moreover, the reform also has some unexpected consequences. For example, although SB 1720 of 2013 has no mandate on changes in college-level courses, many institutional leaders reported that they made adjustments to the college-level courses, with more students of different levels of academic preparation now enrolling in them (Mokher et al., 2020). SB 1720 of 2013 also led to increased campus-wide coordination and the culture of fostering student access (Brower et al., 2021). Future research can focus on how the reform is implemented in practice and explore the variation in success across colleges. Case studies that combine quantitative and qualitative data would be helpful to advance our understanding of what is working and what is not.

Supplemental Material

sj-docx-1-ero-10.1177_23328584221126474 – Supplemental material for Who Benefited More From the Developmental Education Reform in Florida? The Role of Exemption Status

Supplemental material, sj-docx-1-ero-10.1177_23328584221126474 for Who Benefited More From the Developmental Education Reform in Florida? The Role of Exemption Status by Kai Zhao, Christine G. Mokher, Toby J. Park-Gaghan and Shouping Hu in AERA Open

Footnotes

Authors’ Note

Although coauthor Toby Park-Gaghan is currently an associate editor of AERA Open, this submission was under consideration before he became the associate editor.

The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R305A160166 to Florida State University, and in part by a grant from the Bill & Melinda Gates Foundation. The opinions expressed are those of the authors and do not represent views of the Institute, the U.S. Department of Education, or the Gates Foundation.

ORCID iDs

Kai Zhao

Christine G. Mokher

Open Practices

The data and analysis files for this article can be found at

Authors

KAI ZHAO is a research assistant professor at Lingnan University, Hong Kong. His research focuses on internationalization of higher education and higher education policy.

CHRISTINE G. MOKHER is an associate professor of higher education at Florida State University. Her research examines state and local policies focused on college- and career-readiness and success, with a particular emphasis on student transitions from secondary to postsecondary education.

TOBY J. PARK-GAGHAN is an associate professor of education policy at Florida State University. His research focuses on education policy and student success in postsecondary education, with a particular focus on community colleges.

SHOUPING HU is the Louis W. and Elizabeth N. Bender Endowed Professor of Higher Education and the founding director of the Center for Postsecondary Success at Florida State University. His research interests examine issues related to postsecondary readiness, outcomes, and success.

References

Bahr

P. R.

Fagioli

L. P.

Hetts

Hayward

Willett

Lamoree

Baker

R. B.

(2019). Improving placement accuracy in California’s community colleges using multiple measures of high school achievement. Community College Review, 47(2), 178–211.

Bailey

Jeong

D. W.

Cho

S. W.

(2010). Referral, enrollment, and completion in developmental education sequences in community colleges. Economics of Education Review, 29(2), 255–270.

Belfield

Jenkins

Fink

(2019). Early momentum metrics: Leading indicators for community college improvement. Columbia University, Teachers College, Community College Research Center. https://ccrc.tc.columbia.edu/publications/early-momentum-metrics-leading-indicators.html

Bemprechat

(1992). The role of parent involvement in children’s academic achievement. School Community Journal, 2(2), 31–41.

Boatman

(2021). Accelerating college remediation: Examining the effects of math course redesign on student academic success. Journal of Higher Education, 92(6), 927–960.

Boatman

Long

B. T.

(2018). Does remediation work for all students? How the effects of postsecondary remedial and developmental courses vary by level of academic preparation. Educational Evaluation and Policy Analysis, 40(1), 29–58.

Brower

R. L.

Nix

A. N.

Daniels

Jones

T. B.

(2021). A pedagogy of preparation: Helping underprepared students succeed in college-level coursework in community colleges. Innovative Higher Education, 46(2), 153–170.

Brower

R. L.

Woods

C. S.

Jones

T. B.

Park

T. J.

Tandberg

D. A.

Nix

A. N.

Rahming

S. G.

Martindale

S. K.

(2018). Scaffolding mathematics remediation for academically at-risk students following developmental education reform in Florida. Community College Journal of Research and Practice, 42(2), 112–128.

Bruner

J. S.

(1984). Vygotsky’s zone of proximal development: The hidden agenda. New Directions for Child and Adolescent Development, 1984(23), 93–97. https://doi.org/10.1002/cd.23219842309

10.

Chen

Duprey

M. A.

Ritchie

N. S.

Caves

L. R.

Pratt

D. J.

Wilson

D. H.

Brown

F. S.

Leu

(2020). High school longitudinal study of 2009 (HSLS: 09): A first look at the postsecondary transcripts and student financial aid records of fall 2009 ninth-graders(NCES 2020-003). U.S. Department of Education. National Center for Education Statistics.

11.

Cohen

(1983). Applied multiple regression/correlational analysis for the behavioral sciences (2nd ed.). Erlbaum.

12.

Edgecombe

N. D.

Jaggars

Barragan

(2014). Accelerating the integrated instruction of developmental reading and writing at Chabot College. Community College Research Center.

13.

Florida College System. (2018). 2017-2018 Annual report. https://indd.adobe.com/view/c715b209-0a10-4426-8f28-9e6b50c67026

14.

Bertrand Jones

Nix

You

Daniels

Brower

(2021). Understanding the implementation of developmental education reform in Florida. Center for Postsecondary Success.

15.

Jaggars

S. S.

Bickerstaff

(2018). Developmental education: The evolution of research and reform. In Paulsen

(Ed.), Higher education: Handbook of theory and research (Vol. 33, pp. 469–503). Springer.

16.

Leeds

D. M.

Mokher

C. G.

(2020). Improving indicators of college readiness: Methods for optimally placing students into multiple levels of postsecondary coursework. Educational Evaluation and Policy Analysis, 42(1), 87–109.

17.

Logue

A. W.

Douglas

Watanabe-Rose

(2019). Corequisite mathematics remediation: Results over time and in different contexts. Educational Evaluation and Policy Analysis, 41(3), 294–315.

18.

Kosiewicz

Ngo

(2020). Giving community college students choice: The impact of self-placement in math courses. American Educational Research Journal, 57(3), 1358–1391.

19.

Mejia

C. M.

Rodriguez

Johnson

(2020). A new era of student access at California’s community college. Public Policy Institute of California.

20.

Melguizo

Bos

J. M.

Ngo

Mills

Prather

(2016). Using a regression discontinuity design to estimate the impact of placement decisions in developmental math. Research in Higher Education, 57(2), 123–151.

21.

Mokher

C. G.

Park-Gaghan

(2021). Shining the spotlight on those outside Florida’s reform limelight: The impact of developmental education reform for nonexempt students. Journal of Higher Education, 92(1), 84–115.

22.

Mokher

C. G.

Park-Gaghan

T. J.

(2020). Accelerating to success: The impact of Florida’s developmental education reform on credit accumulation. Teachers College Record, 122(12), 1–37.

23.

Okimoto

Heck

(2015). Examining the impact of redesigned developmental math courses in community colleges. Community College Journal of Research and Practice, 39(7), 633–646.

24.

Park-Gaghan

T. J.

Mokher

C. G.

Spencer

(2020). What happened following comprehensive developmental education reform in the Sunshine State? The impact of Florida’s developmental education reform on introductory college-level course completion. Educational Researcher, 49(9), 656–666.

25.

Park-Gaghan

T. J.

Mokher

C. G.

Spencer

(2021). Do rising tides lift all boats? Exploring heterogenous effects of Florida’s developmental education reform by high school academic preparation. American Journal of Education, 127(3), 471–495.

26.

Rutschow

E. Z.

Cormier

M. S.

Dukes

Zamora

D. E. C.

(2019). The changing landscape of developmental education practices: Findings from a national survey and interviews with postsecondary institutions. Center for the Analysis of Postsecondary Readiness.

27.

Scott-Clayton

Crosta

P. M.

Belfield

C. R.

(2014). Improving the targeting of treatment: Evidence from college remediation. Educational Evaluation and Policy Analysis, 36(3), 371–393.

28.

Skuratowicz

Ota

St.

Clair

Pritzlaff

Anderson

Menefee

Miller-Loessi

(2020). The effectiveness of a contextualized developmental course in intermediate algebra for community college students. Community College Journal of Research and Practice, 44(5), 363–376.

29.

Valentine

J. C.

Konstantopoulos

Goldrick-Rab

(2017). What happens to students placed into developmental education? A meta-analysis of regression discontinuity studies. Review of Educational Research, 87(4), 806–833.

30.

Vygotsky

L. S.

(1978). Interaction between learning and development (M. Lopez-Morillas, Trans.). In Cole

John-Steiner

Scribner

Souberman

(Eds.), Mind in society: The development of higher psychological processes (pp. 79–91). Harvard University Press.

31.

What Works Clearinghouse. (2020). Standards handbook version 4.1. https://ies.ed.gov/ncee/wwc/handbooks

32.

Whinnery

Odekar

(2021). 50-state comparison: Developmental education policies. Education Commission of the States.

33.

Woods

C. S.

Park

Bertrand Jones

(2019). Reading, writing, and English course pathways when developmental education is optional: Course enrollment and success for underprepared first-time-in-college students. Community College Journal of Research and Practice, 43(1), 5–25.

34.

Woods

C. S.

Richard

Park

Tandberg

Jones

T. B.

(2017). Academic advising, remedial courses, and legislative mandates: An exploration of academic advising in Florida community colleges with optional developmental education. Innovative Higher Education, 42(4), 289–303.

35.

(2016). Assistance or obstacle? The impact of different levels of English developmental education on underprepared students in community colleges. Educational Researcher, 45(9), 496–507.

36.

Zhao

Park-Gaghan

T. J.

Mokher

C. G.

(2020, November). Self-placement in math courses: Differences by demographic characteristics and impacts on student success. Association for the Study of Higher Education (ASHE) in New Orleans, Louisiana.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.05 MB