Sage Journals: Discover world-class research

Abstract

Florida’s Senate Bill 1720 allowed many students to bypass developmental education and enroll directly in introductory college-level courses. We use an interrupted time series design to introductory college-level courses enrollment and passing rates in English and math for three cohorts of college students prereform and three cohorts postreform. Based on a cohort-by-cohort comparative analysis, we find that cohorts after the reform are more likely to enroll and pass introductory college-level courses in their 1st year of college, indicating that the reform may help to accelerate student success in college. Further, we find that Black and Hispanic students experience even greater gains in passing rates than White students, effectively narrowing the racial/ethnic achievement gap.

Keywords

community colleges econometric analysis higher education policy regression analyses

Students who arrive at college underprepared have historically faced an “obstacle course” of developmental education (DE) requirements upon college enrollment, which many never even begin or complete (Baily et al., 2010). Students are required to pay for these courses but do not receive any college credit. They also must complete these DE courses before enrolling in the gateway courses that fulfill degree requirements, thus slowing their academic momentum. In response to these challenges, many states and institutions have begun implementing a variety of different policies to reform their DE programs (e.g., Edgecombe, 2011). In this article, we look at policy impacts from the state of Florida, which implemented an extensive statewide reform that made DE optional for the majority of incoming students at the 28 state colleges in the Florida College System (FCS, the former community college system), and also required the colleges to change the instructional modalities for remaining DE courses to allow students to progress more quickly into college credit courses. Colleges were also required to develop a plan for providing enhanced advising and support services for students.

We begin by providing a brief overview of DE and recent reform measures across the country. Then, we detail the components of Florida’s DE reform efforts and the theory of action underlying how these efforts may improve students’ acceleration through introductory college-level courses (often termed gateway courses). Next, we describe our methods, which use an interrupted time series design to examine whether student success in introductory college-level courses during the 1st year of college has changed since the reform, and a comparative interrupted time series to study differential changes by race/ethnicity. After presenting our results, we offer a general conclusion regarding student success in introductory college-level courses following major DE reform in Florida.

Related Literature and Context

Historically, prior to enrolling in introductory college-level courses, many students across the country were first required to take DE courses in reading, writing, and/or math. Indeed, estimates suggest that over half of students enrolled in an associate degree program were required to take at least one developmental course (Bailey et al., 2010; Complete College America, 2012). In Florida alone, historical estimates indicate that up to 70% of first-time-in-college (FTIC) community college students were enrolled in at least one DE course (Underhill, 2013). And additional studies have shown that Black and Hispanic students tend to be enrolled in developmental courses at higher rates than White students (Attewell et al., 2006; Perry et al., 2010; Ross et al., 2012). Unfortunately, very few students enrolled in developmental courses ever attempt, let alone pass, an introductory college-level course (Bailey et al., 2010; Complete College America, 2012). Thus, for many students, DE courses were often the main obstacle in achieving the academic momentum necessary to be successful in postsecondary education. Still, some studies have shown that DE may benefit some students, particularly those scoring the lowest on traditional college placement tests (Boatman & Long, 2018).

In order to better serve all students, many states have turned to reform measures that alter the placement and/or instruction of DE, with the goal of accelerating students into introductory college-level courses (Rutschow & Meyer, 2018). Reform measures in other states have focused on policies that accelerate students through DE and/or limit students’ time in DE. Thus, the goal of DE reform is often to ensure student success in the introductory college-level courses, regardless of prior academic preparation. The theory of action underlying Florida’s Senate Bill (SB) 1720 is that the reform efforts will increase student success in college by changing state policy directives, institutional programs and practices, and student decision-making.

Prior to Florida’s DE reform, approximately 70% of FTIC students scored below college-ready on a statewide placement test and were required to take at least one DE course (Underhill, 2013). Under SB 1720, college students were exempt from college placement testing and developmental coursework if they entered a Florida public high school in 2003–2004 or later and went on to graduate with a standard high school diploma, or if they were active-duty members of the military. These students were assumed to be college-ready and could enroll directly into gateway courses in math and English. Prior research has shown that nearly one quarter of college students are misplaced into their first college course when placement decisions are based on test scores alone, most of these students being unnecessarily placed into DE courses (Scott-Clayton et al., 2014). Thus, making DE optional may reduce the financial and opportunity costs to students of being placed into courses they may not need, and may improve their academic progression.

The second component of the reform required colleges to change the way that DE courses are offered by implementing compressed, contextualized, modularized, or corequisite modalities. Compressed courses meet more frequently over fewer weeks, which potentially allows students to complete two sequential courses in the same semester. There is some evidence that compressed courses are associated with positive effects on short-term outcomes such as gateway course completion, as well as longer-term outcomes like credit accumulation and degree completion (Hodara & Jaggars, 2014). Contextualized courses present remedial material in an applied manner related to the student’s intended major. Although there is limited rigorous research on this modality, there is some evidence that these courses may improve students’ course performance by making the material more relevant to students’ interests and career goals (Perin, 2011). Modularized courses assess mastery of course standards and then allow students to complete customized modules for only the standards in which they did not demonstrate mastery. This often results in students completing fewer credits in DE courses than they would otherwise need (Kalamkarian et al., 2015), and experimental evidence suggests that students are just as successful in modularized courses as more traditional DE models (Weiss & Headlam, 2019). Lastly, corequisite courses allow students to take developmental and college-level courses concurrently. This approach has been shown to increase the percentage of students completing college-level courses in the first semester or 1st year of college in other states (Denley, 2015; Jones, 2015; Kalamakrian et al., 2015).

We note that some of the colleges had been experimenting/piloting the use of some of these instructional strategies in the years leading up to the reform (Park et al., 2016). As such, our identification strategy, which we discuss in the next section, accounts for any underlying upward trends that may be present in student outcomes prior to the reform that could be due to early implementation of this component of the reform.

The third component of the reform required colleges to provide enhanced advising services to increase student awareness of the developmental course options, as well as additional support services like tutoring for underprepared students. College students who receive enhanced advising tend to take and pass more courses (Scrivener & Weiss, 2009), and are also more likely to take advantage of tutoring and other types of academic support services (Visher et al., 2010). Support services typically tend to be underutilized by college students, even though they increase the likelihood that students will persist in college courses and tend to improve students’ college GPAs (Perin, 2004).

Specific to Florida and from a broad, overall policy impact perspective, our earlier work has shown some initial promising results from Florida’s DE reform. In the 1st year after SB 1720, the predicted probability of both taking and passing gateway courses increased by 5.3 percentage points in English and 3.7 percentage points in mathematics relative to the year prior to the reform. Further, our own work has shown that passing rates increase at a more rapid rate for Black and Hispanic students, compared to White students (Park et al., 2018). However, our work to date has been limited to one cohort of student outcomes following the reform. What is not yet known is whether these trends continue in the multiple years following the implementation of the reform. In other words, although the DE reform produced positive results for one cohort of students, did this continue to be the case for additional cohorts?

In this article, we employ a more rigorous approach and additional years by comparing three cohorts of students before and three cohorts of students after the reform. The balanced panel of 3 years of data prior to the reform and 3 years of data after the reform allows us to examine whether the reform has had an impact on student success for multiple cohorts of students following its implementation. Specifically, we examine enrollment rates as well as passing rates in two different ways: course-based passing rates and cohort-based passing rates. Whereas course-based passing rates are a more traditional notion of passing rates (the number of students passing the course expressed as a share of the number of students enrolled in the course), cohort-based passing rates (the number of students passing the course expressed as a share of the total number of students in a cohort of incoming students) provide a more global view of policy impacts. Specifically, cohort-based passing rates are a measure of the total impact of the policy and its ability to successfully increase student success. According to a recent study by Belfield et al. (2019), 1st-year measures of gateway course enrollment and completion rates are key predictors of longer-term student success and may be particularly important for explaining racial equity gaps in college completion rates. The following research questions guided this study:

(1) How have enrollment rates, course-based passing rates, and cohort-based passing rates in introductory college-level courses changed following the DE reform among FTIC students enrolled in the FCS institutions?

(2) How did the change differ for students of different racial/ethnic backgrounds?

Research Design

Data

We use data from Florida’s K-20 Education Data Warehouse (EDW), which tracks all Florida public school students remaining in-state from kindergarten to postsecondary education. For these analyses, we use data from college enrollment records, student background characteristics, and indicators of high school preparation. We include data from all 28 FCS institutions in Florida. We include six cohorts of FTIC students, three cohorts who were enrolled prior to SB 1720 (entering cohorts Fall 2011–Fall 2013), and three cohorts who were enrolled after (entering cohorts Fall 2014–Fall 2016).

For enrollment rates in college-level courses, we calculated the percentage of students enrolled in these courses, disaggregated by subject, as the share of the cohort of FTIC students who began their studies in FCS institutions in a given fall term. Then, for introductory college-level course passing rates, we used two different measures. The first is a course-based passing rate, calculated as the percentage of students passing the course as the share of the total number of students enrolled in the course, disaggregated by subject. The second measure is the cohort-based passing rate, which is calculated as the percentage of students passing the course as the share of the total number of students in the entire cohort. Thus, the cohort-based passing rate captures the overall effect of the policy and ascertains whether more students have been successful at taking and passing introductory college-level courses from the perspective of a cohort-by-cohort comparison. In addition, we also have measures of high school academic preparation, gender, race/ethnicity, and free/reduced-lunch eligibility. Descriptive statistics, by cohort, are presented in Table 1.

Table 1

Cohort Total Breakdown by Student Background

	Prepolicy			Postpolicy
	2011	2012	2013	2014	2015	2016
By race/ethnicity (students of other racial/ethnic background not reported here)
White	29,757	27,069	27,124	26,261	25,673	23,522
	41.03%	41.75%	39.63%	38.44%	38.19%	34.92%
Black	16,180	13,432	14,809	14,065	13,391	12,998
	22.31%	20.72%	21.64%	20.59%	19.92%	19.30%
Hispanic	22,241	20,333	22,197	23,514	23,516	24,556
	30.67%	31.36%	32.43%	34.42%	34.98%	36.45%
By gender
Male	34,396	30,947	32,894	32,374	32,123	32,212
	47.43%	47.73%	48.06%	47.39%	47.78%	47.82%
Female	38,131	33,891	35,546	35,941	35,109	35,152
	52.57%	52.27%	51.94%	52.61%	52.22%	52.18%
By free/reduced-lunch (FRL) status
FRL-eligible	22,392	20,641	25,385	27,128	27,973	27,945
	30.87%	31.83%	37.09%	39.71%	41.61%	41.48%
Not FRL-eligible	50,135	44,197	43,055	41,187	39,259	39,419
	69.13%	68.17%	62.91%	60.29%	58.39%	58.52%
Total	72,527	64,838	68,440	68,315	67,232	67,364

Analytic Approach

We use a time series design to examine whether there have been any changes in students’ enrollment and completion of introductory college-level courses before and after the reform. We use a balanced panel with 3 years of data prior to the reform and 3 years after the reform. We assess the likelihood of enrolling in and passing college-level courses in the 1st year of college. In English, the first introductory college-level course is English Composition 1 (ENC1101). In math, we examine results separately for Intermediate Algebra (MAT 1033), which counts for elective credit only, and gateway math courses that fulfill the transfer associate’s degree requirements in math (College Algebra, MAC1105; Liberal Arts Math 1, MGF1106; Liberal Arts Math 2, MGF1107; and Introductory Statistics, STA2023). For this first analysis, we use all of the students in each cohort as our full sample.

To empirically investigate the overall impact of the reform on enrollment and passing rates, we estimate the model below for student i at college j in year (cohort) t:

logit (y_{ijt}) = β_{0} + β_{1} {(2014)}_{t} + β_{2} {(S)}_{i j t} + β_{3} {(H S)}_{i j t} + ξ_{j} + λ_{t} .

In this specification, β₁ captures the change in the course enrollment/passing rate in the postreform period, β₂ is a vector of student background characteristics, β₃ is a vector of high school academic preparation indicators, ξ_j is a college fixed effect to account for unobserved heterogeneity across institutions, and λ_t is a continuous year (cohort) indicator to account for any underlying temporal trends.¹ By including the time trend, we are able to net out any impacts that may have resulted from changes other the 2014 implementation of the DE reform (such as early adoption of the new instructional modalities by some of the colleges) that could have impacted student outcomes (Jacob et al., 2017). In other words, by including a continuous year indicator, β₁ will only capture an increase in student outcomes that can be attributed directly to the policy and not to any prepolicy trend that could have continued after the policy was implemented. However, one limitation of this approach is that there could be other unobserved changes in the postpolicy years that may have affected the outcomes, even in the absence of the policy.

Further, in order to explore whether there were differential changes in enrollment/passing rates by race/ethnicity, in an additional set of analyses, we include indicators for Hispanic and Black students (White students form the comparison group; students of other race/ethnicity are excluded from these models) and interact these indicators with the postreform indicator.

We estimate the following model for student i at college j in year (cohort) t:

\begin{matrix} logit (y_{ijt}) = β_{0} + β_{1} (2014) + β_{2} {(B l a c k)}_{i j t} + β_{3} {(H i s p a n i c)}_{i j t} \\ + β_{4} {(2014 * B l a c k)}_{i j t} + β_{5} {(2014 * H i s p a n i c)}_{i j t} \\ + β_{6} {(S)}_{i j t} + β_{7} {(H S)}_{i j t} + ξ_{j} + λ_{t} \end{matrix}

Under this specification, β₁ captures the change in course enrollment/passing rate in the postreform period; β₂ and β₃ capture the overall difference in outcomes for Black and Hispanic students, respectively; β₄ and β₅ capture any differential changes in student outcomes for Black and Hispanic students, respectively, postreform; β₆ is a vector of student background characteristics; β₇ is a vector of high school academic preparation indicators; ξ_j is a college fixed effect to account for unobserved heterogeneity across institutions; and λ_t is a continuous year (cohort) indicator to account for any underlying temporal trends. This model captures the base changes in overall student outcomes between the prereform period and the postreform period, as well as whether there has been a differential change for Hispanic and Black students. In this second analysis, we only include White, Black, and Hispanic students.

Before presenting our findings, we note that although it is tempting (as well as potentially quite interesting) to investigate the impacts of individual components of Florida’s DE reform, we also see perhaps greater merit in examining the impacts of the policy as a whole. We also note that previous attempts at isolating impacts of components of DE reform have concluded that is difficult to disentangle these effects when states implement DE reform policies that contain several elements, such as those in North Carolina, Virginia, and Florida (Bickerstaff et al., 2016). Thus, we remind the reader that our results should be interpreted as overall policy effects for incoming cohorts of FTIC students in the FCS, including both exempt and nonexempt students. In addition, we note that the overall effect of the reform may be greater than the sum of its parts, further warranting a holistic analysis. Put differently, there may be a form of synergy that emerges from the three major components of the reform that adds to the impact in a way that is greater than the individual impacts of the three components added together. More to that point, our most recent findings from an annual survey to campus leaders across the FCS found that the DE reform brought about institutional transformation that extended far beyond the requirements of SB 1720 (Mokher, Spencer, et al., 2020). These findings suggest that Florida’s DE reform brought about comprehensive reform that has created an increased culture of student success. As such, we are interested in whether the reform had impacts for all incoming FTIC students as well as whether these impacts varied by race/ethnicity.

Findings

Descriptive Portrait

We begin with descriptive portrait highlighting enrollment and passing rates for introductory college-level courses. We present findings for all students, disaggregated by subject. Additional findings, disaggregated by race, can be found as an online supplement to this article at centerforpostsecondarysuccess.org.

Enrollment rates

As shown in Table 2, 1st-year introductory college-level course enrollment rates increased following the implementation of the DE reform. Immediately after implementation, introductory college-course enrollment rates increased by 6.49, 8.69, and 3.20 percentage points for English composition, intermediate algebra, and our pooled measure of gateway math courses, respectively. We note that we are modeling intermediate algebra and gateway math courses as different outcome measures due to the fact that some students, depending on their intended major, may be required to take intermediate algebra (a college-credit bearing course) as a prerequisite course prior to enrolling in a gateway math courses that will satisfy the math requirement for the associate’s degree. Further, this increase continued into the 2015 cohort before experiencing a slight plateau in the 2016 cohort.

Table 2

First-Year Introductory College-Level Course Enrollment Rates, All Students

	Prepolicy			Postpolicy
	2011	2012	2013	2014	2015	2016
English composition
Enrollment rate	56.14%	60.78%	63.64%	70.13%	73.04%	72.40%
Students enrolled	40,718	39,411	43,552	47,910	49,105	48,772
Number in cohort	72,527	64,838	68,440	68,315	67,232	67,364
Intermediate algebra
Enrollment rate	26.50%	28.23%	33.19%	41.88%	45.28%	43.36%
Students enrolled	19,219	18,306	22,712	28,613	30,446	29,208
Number in cohort	72,527	64,838	68,440	68,315	67,232	67,364
Gateway math courses
Enrollment rate	21.75%	22.93%	24.84%	28.04%	33.48%	33.66%
Students enrolled	15,772	14,866	17,000	19,153	22,508	22,672
Number in cohort	72,527	64,838	68,440	68,315	67,232	67,364

Course-based passing rates

As shown in Table 3, the 1st-year course-based passing rate for English composition remained relatively constant over time, although the number of students entering the course increased dramatically following the implementation of SB 1720. The course-based passing rate for intermediate algebra decreased by nearly 5 percentage points in 2014 but rebounded somewhat in the 2015 and 2016 cohorts. Just as with English, though, the number of students enrolling in intermediate algebra increased dramatically in the postimplementation cohorts. Course-based passing rates in gateway math courses declined slightly following the implementation of SB 1720.

Table 3

First-Year Course-Based Passing Rates in Introductory College-Level Courses, All Students

	Prepolicy			Postpolicy
	2011	2012	2013	2014	2015	2016
English Composition
Success rate	74.81%	75.26%	75.28%	74.51%	75.09%	74.41%
Number of successes	30,460	29,662	32,786	35,699	36,873	36,289
Number of students	40,718	39,411	43,552	47,910	49,105	48,772
Intermediate algebra
Success rate	65.28%	65.59%	62.65%	57.85%	60.29%	60.07%
Number of successes	12,547	12,006	14,229	16,554	18,357	17,545
Number of students	19,219	18,306	22,712	28,613	30,446	29,208
Gateway math courses
Success rate	68.50%	68.80%	67.24%	65.93%	66.40%	66.96%
Number of successes	10,804	10,228	11,431	12,615	14,839	15,055
Number of students	15,772	14,866	17,000	19,153	22,508	22,672

Cohort-based passing rates

As shown in Table 4, the number of students both taking and passing introductory college-level English and math courses in their 1st year increased following the reform. Cohort-based passing rates in both English composition and intermediate algebra saw immediate high jumps in the 1st year postreform, increasing by 4.36 and 3.44 percentage points, respectively. College-level math courses had a moderate passing rate increase of 1.77 percentage points in the 1st year postreform, but they experienced a higher increase in the 2nd year postreform, with a jump by 5.37 percentage points from 2013. All courses reached a plateau in the 3rd year postreform.

Table 4

First-Year Cohort-Based Passing Rates in Introductory College-Level Courses, All Students

	Prepolicy			Postpolicy
	2011	2012	2013	2014	2015	2016
English composition
Success rate	42.00%	45.75%	47.90%	52.26%	54.84%	53.87%
Number of successes	30,460	29,662	32,786	35,699	36,873	36,289
Number of students	72,527	64,838	68,440	68,315	67,232	67,364
Intermediate algebra
Success rate	17.30%	18.52%	20.79%	24.23%	27.30%	26.05%
Number of successes	12,547	12,006	14,229	16,554	18,357	17,545
Number of students	72,527	64,838	68,440	68,315	67,232	67,364
Gateway math courses
Success rate	14.90%	15.77%	16.70%	18.47%	22.07%	22.35%
Number of successes	10,804	10,228	11,431	12,615	14,839	15,055
Number of students	72,527	64,838	68,440	68,315	67,232	67,364

Regression-Adjusted Results

We move now to our regression-adjusted results under our interrupted time series/comparative interrupted time series designs. For ease of interpretation, we report our findings as predicted probabilities and marginal effects, calculated from the regression results. Full regression results are available from the authors.

Enrollment rates

All of the increases in the 1st-year introductory college-level course enrollment rates following the implementation of the DE reform were statistically significant. English enrollment rates increased by 4.79 percentage points, while intermediate algebra and gateway math courses increased by 8.19 and 4.89 percentage points, respectively (Table 5).

Table 5

First-Year Predicted Probabilities of Introductory College-Level Course Enrollment Rates Before/After Florida’s DE Reform

	Pred. Prob. Prereform	Pred. Prob. Postreform	Difference
English
ENC1101	65.44%	70.23%	4.79***
Math
MAT1033	31.02%	39.21%	8.19***
Gateway math	22.78%	27.67%	4.89***

Note. Significance reported for differences prereform/postreform.

***

p < .001.

Further, all of the differential effects by race/ethnicity were also statistically significant and positive (Table 6). Not only did all students enroll in introductory college-level courses at higher rates following the implementation of the reform, but Hispanic and Black students increased their enrollment at faster rates than White students, narrowing the enrollment gap for Black and White students and effectively closing the enrollment gap for Hispanic and White students. In English composition, for instance, while White students experienced an increase of 7.47 percent points in their enrollment rate, Hispanic and Black students experienced 12.75 and 22.64 percentage point gains, respectively. A similar pattern also occurred in intermediate algebra and gateway math courses.

Table 6

Predicted Probabilities and Marginal Effects for First-Year Introductory College-Level Course Enrollment Rates, by Race/Ethnicity

	English Composition
	2011–2013	2014–2016	Diff.
Predicted probabilities
Black	49.47%	72.11%	22.64***
Hispanic	61.70%	74.45%	12.75***
White	67.18%	74.65%	7.47***
Marginal effects
Black vs. White			15.17***
Hispanic vs. White			5.28***
	Intermediate Algebra
	2011–2013	2014–2016	Diff.
Predicted probabilities
Black	24.09%	44.51%	20.42***
Hispanic	31.04%	45.76%	14.72***
White	28.66%	39.32%	10.66***
Marginal effects
Black vs. White			9.76***
Hispanic vs. White			4.06***
	Gateway Math
	2011–2013	2014–2016	Diff.
Predicted probabilities
Black	11.86%	22.10%	10.24***
Hispanic	21.92%	31.97%	10.05***
White	24.60%	32.70%	8.10***
Marginal effects
Black vs. White			2.14***
Hispanic vs. White			1.95***

Note. Significance reported for differences prereform/postreform.

***

p < .001.

Course-based passing rates

Table 7 presents the results from our set of models predicting overall 1st-year course-based passing rates in introductory college-level courses before and after the implementation of Florida’s DE reform. Overall, course-based passing rates in English composition (ENC1101) and gateway math course passing rates (MAC1105, MGF1106, MGF1107, and STA2023) remained the same following the reform. In intermediate algebra, however, course-based passing rates declined by nearly 5 percentage points.

Table 7

Predicted Probabilities for Course-Based Passing Rates for 1st-Year Introductory College-Level Courses Before/After Florida’s DE Reform

	Pred. Prob. Prereform	Pred. Prob. Postreform	Difference
English
ENC1101	75.92%	76.09%	0.17
Math
MAT1033	64.17%	59.42%	−4.75***
Gateway Math	67.87%	68.32%	0.45

Note. Significance reported for differences prereform/postreform.

***

p < .001.

Table 8 presents the results from our set of models predicting differential course-based passing rates by race/ethnicity. In English composition, White and Hispanic students experienced a slight increase in course-based passing rates (1.25 and 0.54 percentage points, respectively), and Black students experienced a decline (0.67 percentage points). Further, in intermediate algebra, whereas all students experienced a decline, both Hispanic and White students had similar (non–statistically significant from each other) declines of around 4 percentage points, and Black students had declines of 6.07 percentage points, roughly 2 percentage points more than White and Hispanic students. Finally, White and Hispanic students experienced no significant change in course-based gateway math courses, and Black students experienced 4.3 percentage point decline in course-based passing rates.

Table 8

Predicted Probabilities and Marginal Effects for Course-Based 1st-Year Passing Rates for Introductory College-Level Courses, by Race/Ethnicity

	English Composition
	2011–2013	2014–2016	Diff.
Predicted probabilities
Black	69.32%	68.57%	−0.67***
Hispanic	77.50%	78.19%	0.54***
White	76.31%	77.60%	1.25***
Marginal effects
Black vs. White			−1.92***
Hispanic vs. White			−0.71
	Intermediate Algebra
	2011–2013	2014–2016	Diff.
Predicted probabilities
Black	56.08%	50.01%	−6.07***
Hispanic	65.39%	60.91%	−4.48***
White	64.13%	60.43%	−3.70***
Marginal effects
Black vs. White			−2.37*
Hispanic vs. White			−0.78
	Gateway Math
	2011–2013	2014–2016	Diff.
Predicted probabilities
Black	61.37%	57.07%	−4.30**
Hispanic	69.09%	68.95%	−0.14
White	69.47%	68.97%	−0.53
Marginal effects
Black vs. White			−4.16***
Hispanic vs. White			0.39

Note. Significance reported for differences prereform/postreform.

p < .05. **p < .01. ***p < .001.

Table 9 presents regression-adjusted 1st-year cohort-based passing rates before and after the implementation of Florida’s DE reform. Overall, we find that all three indicators for cohort-based passing rates increased significantly following the reform. Specifically, cohort based-basing rates increased by 3.38, 3.48, and 2.94 percentage points in English composition, intermediate algebra, and combined gateway math courses, respectively. Of note is that following the reform, over half of the incoming cohorts were successfully taking and passing gateway English courses within the 1st year, with nearly one in five students taking and passing gateway math courses.

Table 9

Predicted Probabilities for Cohort-Based Passing Rates for 1st-Year Introductory College-Level Courses Before/After Florida’s DE Reform

	Pred. Prob. Prereform	Pred. Prob. Postreform	Difference
English
ENC1101	47.85%	51.23%	3.38***
Math
MAT1033	19.71%	23.19%	3.48***
Gateway math	14.53%	17.47%	2.94***

Note. Significance reported for differences prereform/postreform.

***

p < .001.

Table 10 presents the results from our set of models predicting differential cohort-based passing rates by race/ethnicity. Across the three panels, students of all racial/ethnic backgrounds experienced increased cohort-based passing rates in their English and math courses, with Black and Hispanic students seeing even greater increases than White students. For instance, Black students’ passing rates in English composition increased by 14.18 percentage points, which is 7.89 percentage points higher than the increase for White students. Further, Black students experienced gains that were 3.27 percentage points higher than White students in intermediate algebra. We note that in terms of gateway math, the base gain for Black students (4.79 percentage points) appears smaller than that for White students (4.97 percentage points); however, we note that the marginal effect gauging the difference between these changes is not statistically significant. Put differently, White and Black students appear to have experienced similar prereform to postreform gains in terms of 1st-year, cohort-based passing rates for gateway math.

Table 10

Predicted Probabilities and Marginal Effects for 1st-Year Cohort-Based Passing Rates for Introductory College-Level Courses, by Race/Ethnicity

	English Composition
	2011–2013	2014–2016	Diff.
Predicted probabilities
Black	33.36%	47.54%	14.18***
Hispanic	46.44%	56.56%	10.12***
White	49.76%	56.05%	6.29***
Marginal effects
Black vs. White			7.89***
Hispanic vs. White			3.83***
	Intermediate Algebra
	2011–2013	2014–2016	Diff.
Predicted probabilities
Black	13.78%	22.52%	8.74***
Hispanic	20.44%	28.05%	7.61***
White	18.73%	24.20%	5.47***
Marginal effects
Black vs. White			3.27***
Hispanic vs. White			2.14***
	Gateway Math
	2011–2013	2014–2016	Diff.
Predicted probabilities
Black	6.97%	11.76%	4.79***
Hispanic	14.45%	20.76%	6.31***
White	16.20%	21.17%	4.97***
Marginal effects
Black vs. White			−0.18
Hispanic vs. White			1.34***

Note. Significance reported for differences prereform/postreform.

***

p < .001.

Discussion and Conclusion

Florida’s DE reform sought to provide learning opportunities more tailored to students’ needs by doing away with traditional placement models and making DE optional for many students while revamping the way DE courses are taught and enhancing advising and student support services to help students make good choices and continue on a path for success. The result is that many more incoming students are taking and passing introductory college-level courses in their 1st year of study, and preexisting achievement gaps in these courses for Black and Hispanic students narrowed. We argue that these outcomes are a result of the three components of the reform working in tandem and that, although not all students were directly affected by each component, all three are essential to achieve increased student momentum.

Making DE optional (the first component of the reform) resulted in a significant enrollment effect—many more students enrolled in introductory college-level English and math courses when DE was no longer required. Two groups of students were now directly enrolling in college-level courses: those who would have been able to do so previously based on the traditional placement model in effect and those who would have previously been required to first take DE courses. Thus, not only did enrollments increase, but introductory college-level classrooms also become increasingly heterogeneous in terms of student academic ability, since students could enroll directly in these courses regardless of prior academic preparation. Although there could have been unintended negative consequences given the curricular and pedagogical challenges associated with educating a more academically diverse group of students, we instead observed an overall increase in the total number of incoming students taking and passing introductory college-level courses once the reform was implemented.

This suggests that at least some of the students who would have previously been required to take DE under the previous placement model were indeed successful in passing the college-level course without first taking DE. This is not entirely surprising given that previous studies have demonstrated that traditional placement models often falsely assign students to DE courses when they would have been capable of passing the college-level course without first taking DE (Leeds & Mokher, 2020; Scott-Clayton et al., 2014). Thus, one sensible policy lever available to states seeking to increase student momentum is to do away with traditional placement models and make DE optional. However, Florida’s policy did more than this, and we stress caution in adopting any DE reform that fails to address the needs of a third group of students: those who may still benefit from DE courses.

Florida’s reform also contained two other major components: new instructional strategies for DE courses and enhanced advising and student support services. Importantly, we note that Florida’s comprehensive DE reform was intended to support students at all levels of preparation—those on the margins of college readiness who could directly enroll in gateway courses, as well as those further behind who could still take DE courses designed in new ways to help them progress more quickly to college-level courses. Thus, the new instructional strategies for DE courses are an important mechanism for increasing academic momentum for those students still required to take DE and those who elect to take the courses even though they are optional. Guiding much of this process is the other important mechanism by which students received benefits from the reform: enhanced advising and student support services. The reform also put additional advising supports into place to help ensure that students made good choices. We argue that adding these two additional components (new instructional strategies for DE courses and enhanced advising) made for a more sensible reform choice than just eliminating DE completely, which could have harmed the least-prepared students.

Although we argue that Florida’s reform functions as an interconnected set of three reform measures, and although we note that it may not be entirely possible to disentangle the impacts of specific components, we see a need for additional research that probes deeper into the components of the reform and how different students are impacted by the reform. For instance, although we found that gains in college-level passing rates were the greatest among nonexempt students who received the new DE instructional strategies and the enhanced support services (Mokher, Park-Gaghan, et al., 2020), we have not yet identified which instructional strategies are the most effective. And although we found that students who had the lowest levels of high school academic preparation tend to have the greatest gains in college-level course passing rates (Park-Gaghan et al., 2019), we have not yet studied the long-term educational trajectories of students who take, but do not pass, college-level courses. Future research is certainly warranted on Florida’s DE reform, particularly in looking at more distal outcomes, such as transfer and degree completion, particularly given that prior research has indicated that many DE reform efforts have had limited impacts on longer-term student outcomes (Jaggars & Bickerstaff, 2018). Finally, although not a direct focus of the legislation, another positive externality that warrants further investigation is the closing of the preexisting racial/ethnic achievement gap in introductory college-level course completion. Future research is certainly warranted that examines equity in other academic outcomes, particularly those more distal.

Taken collectively, the findings from this investigation suggest that DE reform can have a real and significant effect on increasing the academic momentum in introductory college-level courses, particularly for traditionally underrepresented groups. Although we do not yet know the long-term impacts of the reform, we are encouraged to see that the share of students completing introductory-level math and English courses continues to remain at all-time highs, even 3 years after the initial implementation of the policy. We encourage other states and contexts to consider the DE reform measures implemented in Florida

Footnotes

ORCID iD

Christine G. Mokher

Notes

Authors

TOBY J. PARK-GAGHAN, PhD, is an associate professor of education policy at Florida State University, P.O. Box 3064452, Tallahassee, FL 32306-4452; tjpark@fsu.edu . His research focuses on education policy and student success in postsecondary education, with a particular focus on community colleges.

CHRISTINE G. MOKHER, PhD, is an associate professor of higher education at Florida State University, P.O. Box 3064452, Tallahassee, FL 32306-4452; cmokher@fsu.edu . Her research examines state and local policies focused on college- and career-readiness and success, with a particular emphasis on student transitions from secondary to postsecondary education.

XINYE HU, MA, is a doctoral candidate in higher education at Florida State University, P.O. Box 3064452, Tallahassee, FL 32306-4452; xh14c@my.fsu.edu . Her research focuses on state educational policy implementation in postsecondary education and institutional impact on college student outcomes.

HAYLEY SPENCER, MEd, is a graduate research assistant at Florida State University, P.O. Box 3064452, Tallahassee, FL 32306-4452; has17@my.fsu.edu . Her research focuses on community colleges and student success, with a particular focus on postsecondary career and technical education.

SHOUPING HU, PhD, is the Louis W. and Elizabeth N. Bender Endowed Professor of Higher Education at Florida State University, P.O. Box 3064452, Tallahassee, FL 32306-4452; shu@fsu.edu . His research focuses on postsecondary readiness, outcomes, and success.

References

Attewell

Lavin

Domina

Levey

(2006). New evidence on college remediation. Journal of Higher Education, 77(5), 886–924.

Bailey

Jeong

D. W.

Cho

S. W.

(2010). Referral, enrollment, and completion in developmental education sequences in community colleges. Economics of Education Review, 29(2), 255–270.

Belfield

Jenkins

Fink

(2019). Early momentum metrics: Leading indicators for community college improvement. New York: Columbia University, Teachers College, Community College Research Center. Retrieved from https://ccrc.tc.columbia.edu/publications/early-momentum-metrics-leading-indicators.html

Bickerstaff

S. E.

Fay

Trimble

M. J.

(2016). Modularization in developmental mathematics in two states: Implementation and early outcomes. New York: Community College Research Center.

Boatman

Long

B. T.

(2018). Does remediation work for all students? How the effects of postsecondary remedial and developmental courses vary by level of academic preparation. Educational Evaluation and Policy Analysis, 40(1), 29–58. https://doi.org/10.3102/0162373717715708

Complete College America. (2012). Remediation: Higher education’s bridge to nowhere. Washington, DC. Retrieved from http://www.completecollege.org/docs/CCA-Remediation-final.pdf

Denley

(2015). Co-requisite remediation pilot study – Fall 2014 and Spring 2015. Office of the Vice Chancellor for Academic Affairs, Tennessee Board of Regents. Retrieved from http://www.ticua.org/meetings_resources/sm_files/TBR%20CoRequisite%20Remediation.pdf

Edgecombe

(2011). Accelerating the academic achievement of students referred to developmental education (CCRC Working Paper No. 30). New York: Community College Research Center, Columbia University. Retrieved from https://files.eric.ed.gov/fulltext/ED516782.pdf

Hodara

Jaggars

S. S.

(2014). An examination of the impact of accelerating community college students’ progression through developmental education. Journal of Higher Education, 85(2), 246–276.

10.

Jacob

Dynarski

Frank

Schneider

(2017). Are expectations alone enough? Estimating the effect of a mandatory college-prep curriculum in Michigan. Educational Evaluation and Policy Analysis, 39(2), 333–360.

11.

Jaggars

S. S.

Bickerstaff

(2018). Developmental education: The evolution of research and reform. In Higher education: Handbook of theory and research (pp. 469–503). Dordrecht, the Netherlands: Springer.

12.

Jones

(2015). The game changers: Strategies to boost college completion and close attainment gaps. Change: The Magazine of Higher Learning, 47(2), 24–29.

13.

Kalamkarian

H. S.

Raufman

Edgecombe

(2015, May). Statewide developmental education reform: Early implementation in Virginia and North Carolina. New York: Community College Research Center.

14.

Leeds

Mokher

C. G.

(2020). Improving indicators of college readiness: Methods for optimally placing students into multiple levels of postsecondary coursework. Educational Evaluation and Policy Analysis, 42(1), 87–109.

15.

Mokher

C. G.

Park-Gaghan

T. J.

(2020, March). Shining the spotlight on those outside Florida’s DE reform limelight: The impact of developmental education reform for non-exempt students. Paper presented at Annual Conference, Association for Education Finance and Policy, Fort Worth, TX.

16.

Mokher

C. G.

Spencer

Park

(2020). Exploring institutional change in the context of a statewide developmental education reform in Florida. Community College Journal of Research and Practice, 44(5), 377–390.

17.

Park

Tandberg

Hankerson

(2016). One policy, disparate reactions: Institutional responses in Florida’s developmental education reform. Community College Journal of Research and Practice, 40(1), 824–837.

18.

Park

Woods

Bertrand Jones

Cig

Tandberg

(2018). Gateway course accessibility and the racial/ethnic achievement gap: The case of student success following Florida’s developmental education reform. Teachers College Record, 120(12), 1–24.

19.

Park-Gaghan

T. J.

Mokher

C. G.

(2019). Do all students benefit from developmental education reform? Heterogeneous treatment effects by high school academic preparation. Paper presented at the meeting of Association for Public Policy Analysis and Management, Denver, CO.

20.

Perin

(2004). Remediation beyond developmental education: The use of learning assistance centers to increase academic preparedness in community colleges. Community College Journal of Research & Practice, 28(7), 559–582.

21.

Perin

(2011). Facilitating student learning through contextualization: A review of evidence. Community College Review, 39(3), 268–295.

22.

Perry

Bahr

P. R.

Rosin

Woodward

K. M.

(2010, June). Course-taking patterns, policies, and practices in developmental education in the California community colleges. Mountain View, CA: EdSource.

23.

Ross

Kena

Rathbun

KewalRamani

Zhang

Kristapovich

Manning

(2012, August). Higher education: Gaps in access and persistence study (NCES 2012-046). U.S. Department of Education, National Center for Education Statistics. Washington, DC: Government Printing Office.

24.

Rutschow

E. Z.

Mayer

A. K.

(2018). Early findings from a national survey of developmental education practices. New York: MRDC.

25.

Scott-Clayton

Crosta

P. M.

Belfield

C. R.

(2014). Improving the targeting of treatment: Evidence from college remediation. Educational Evaluation and Policy Analysis, 36(3), 371–393. doi:10.3102/0162373713517935

26.

Scrivener

Weiss

M. J.

(2009). More guidance, better results. New York: MDRC. Retrieved from http://dev.mdrc.org/sites/default/files/More%20Guidance%20ES_1.pdf

27.

Underhill

(2013, February). College remediation. Presentation at the Florida Senate, 2013 Regular Session, Appropriations Subcommittee on Education Meeting, Tallahassee, FL. Retrieved from http://www.flsenate.gov/PublishedContent/Committees/2012-2014/AED/MeetingRecords/MeetingPacket_2056.pdf

28.

Visher

M. G.

Butcher

K. F.

Cerna

O. S.

(2010). Guiding developmental math students to campus services: An impact evaluation of the Beacon program at South Texas College. New York: MDRC. Retrieved from https://www.mdrc.org/publication/guiding-developmental-math-students-campus-services

29.

Weiss

M. J.

Headlam

(2019). A randomized controlled trial of a modularized, computer-assisted, self-paced approach to developmental math. Journal of Research on Educational Effectiveness, 12(3), 484–513.

What Happened Following Comprehensive Developmental Education Reform in the Sunshine State? The Impact of Florida’s Developmental Education Reform on Introductory College-Level Course Completion

Abstract

Keywords

Related Literature and Context

Research Design

Data

Analytic Approach

Findings

Descriptive Portrait

Enrollment rates

Course-based passing rates

Cohort-based passing rates

Regression-Adjusted Results

Enrollment rates

Course-based passing rates

Discussion and Conclusion

Footnotes

ORCID iD

Notes

Authors

References