Abstract
Introduction
Many community college students arrive on campus academically under-prepared for success at the postsecondary level. Traditionally, community colleges have addressed this issue with developmental education (DE) programs designed to strengthen students’ skills in reading, writing, and/or math. In 2011/12, approximately 41% of students at public 2-year colleges reported enrolling in at least one developmental course (Chen, 2016). Based on placement test scores, many students were required to take a series of developmental courses that could take a year or more to complete before they were able to enroll in credit-bearing courses. Over the past decade, the traditional developmental education model has been under close scrutiny with increasing evidence from empirical studies showing a potential negative effect of developmental courses on college success. A meta-analysis of studies on the impacts of assignment to DE showed that relative to their peers who were on the margin of college readiness but who were placed into college-level courses, students scoring just below college-ready who were assigned to developmental courses earned fewer college credits within 3 years, and were about 1.5% points less likely to complete a degree (Valentine et al., 2017). One possible reason the authors speculated was that students who were assigned to developmental courses faced additional “roadblocks” due to the extra time and costs associated with remediation.
In response to growing concerns about the efficiency and effectiveness of developmental education, some states have been implementing reforms regarding who is required to take developmental courses and how the courses are taught. For example, many colleges have traditionally used students’ placement test scores as the sole measure of college readiness to determine whether students needed developmental education. However, evidence suggests that this placement approach can result in high misplacement into remediation. In other words, traditional placement models tend to assign into developmental courses a number of students who could have taken and passed college-level courses (Scott-Clayton et al., 2014). New placement models move away from the single-metric approach and instead consider different factors when placing students into introductory courses. For instance, California, Minnesota, Oregon, and Connecticut now require state colleges to use multiple measures for placement such as high school academic performance, and SAT/ACT scores (Hodges et al., 2020). Still, in other states reform measures have focused on the DE courses themselves and how they are taught. Colleges in Texas and Tennessee, for instance, now offer new instructional strategies intended to move students more quickly into college-level courses such as corequisite developmental education courses that are taken in the same semester with college-level courses (Scott-Clayton, 2018).
Florida took a drastic approach when Senate Bill 1720 (SB 1720) was implemented in fall 2014 across the 28 state and community colleges in the Florida College Systems (FCS). The reform required the colleges to change both who was required to take DE courses and how DE courses were taught, while also requiring colleges to provide enhanced advising and other student support services to students in DE education. In particular, placement tests were made optional for those who entered a Florida public high school in the 2003 to 2004 academic year or later, and earned a standard high school diploma, as well as students who were active-duty military members. These students also became exempt from developmental education and could enroll directly in introductory college-level English and math courses. The legislation also required colleges to offer developmental education courses using different instructional strategies (compressed, corequisite, contextualized, and modularized), and to develop enhanced advising and academic support services for students in developmental education.
The reform has produced substantial effects within the FCS. Prior research has documented significant increases in the enrollment and completion rates of college-level courses following the reform for all first-time-in-college students (FTIC) (Hu et al., 2019). The present study extends prior work by focusing on the subgroup of non-exempt students, which include out-of-state students, graduates of private high schools, and older students. Although non-exempt students are not yet allowed to bypass developmental education and are still required to take placement tests, the Florida legislative reform’s stipulations regarding different instructional approaches for DE courses and enhanced advising and supporting services still affect them. Existing research on the impact of the reform so far has considered only??? . As a result, this study’s focus on the way SB 1720 affects non-exempt students will not only deepen our understanding of the impacts of the reform, but also provide us with an opportunity to specifically focus on the role of instructional strategies and academic advising and support services. The two components of SB 1720 have received less attention in the growing research on the impacts of the reform relative to the changes in placement policy (e.g., Park et al., 2018; Park-Gaghan et al., 2020; Woods et al., 2019).
The purpose of this study is to examine the policy effects on the non-exempt students’ enrollment and success in college-level English and math courses during their first year of college. These outcomes serve as important early momentum measures that are predictive of longer-time student success (Belfield et al., 2019). Also, given the evidence suggesting the heterogeneous effects of developmental education for students from varying academic levels (Boatman & Long, 2018; Xu, 2016), we also explore differential impacts by college readiness on college course enrollment and completion rates following the reform. More specifically, two research questions guide this study: (1) how have enrollment and completion rates in college English and math courses changed before and after the developmental education reform for non-exempt students? and (2) do these changes in coursetaking patterns before and after the reform for non-exempt students differ depending on levels of college readiness?
In the following sections, we first discuss prior literature examining the impact of developmental education on student outcomes, with a focus on the possible effects of the recent DE reform measures required by SB 1720 in Florida. We then describe the data source, variables, and analytical methods used for the study. Next, we report the findings and conclude with a discussion of the implications for practice and future research.
Policy Context and Literature Review
Three related bodies of literature inform our analysis. First, we review literature that examined the effectiveness of developmental education, and we discuss various placement policies, including the one adopted in Florida’s reform. Second, we describe the new instructional strategies that have been implemented in recent developmental education reforms in different contexts, and review studies that examine how these new instructional strategies relate to student success. Third, we discuss the role of academic advising and support service in Florida’s developmental education reform. Together the literature provides additional insights into the ways Florida’s statewide developmental education reform may influence non-exempt students. Finally, the section considers this study’s contribution to prior research on the effectiveness of Florida’s DE reform.
Effectiveness of Developmental Education and Placement Policies
Traditionally, placement test scores have played a significant role in determining whether a student was assigned to developmental education courses or college-level courses. Many empirical studies have used a regression discontinuity (RD) design to investigate the effects of assignment into developmental education by comparing the outcomes of students whose scores are narrowly above or below the cut score required for placement into college-level courses (e.g., Melguizo et al., 2016; Moss et al., 2014). In a meta-analysis of RD studies, Valentine et al. (2017) found that students scoring just below college-ready cut score who are assigned to developmental education perform significantly worse than their counterparts scoring slightly above the college-ready cut score and assigned to college-level courses on all three outcomes, including the likelihood of passing college-level courses, credits earned, and degree completion. We note that due to the nature of the research design, RD studies only compare students who are just below the cut score with those just above. As a result, the RD approach has been helpful in capturing the causal effects of assignment to developmental course for students on the margins of college readiness, but it has excluded students with lower levels of academic preparation. Thus, the findings from studies using a RD approach have limitations in their generalizability to broader student populations. Studies that have examined students with very low placement test scores usually yield null or negative effects (e.g., Melguizo et al., 2016; Xu, 2016). A noticeable exception comes from Boatman and Long (2018) in which they found that the lowest scoring students might experience positive effects from taking an additional developmental reading or writing course.
Given the largely adverse consequence of assignment into developmental education and the heterogeneous impacts by level of academic preparation (Boatman & Long, 2018; Park-Gaghan et al., 2021), it is important to ensure the accuracy of the placement process. However, research has documented severe misplacement using the test score cutoff-based placement policies (Scott-Clayton et al., 2014). There are two kinds of misplacements that can occur with course assignments. The first type is overplacement, which occurs when students are assigned to college-level courses that are too difficult for them. The second type, underplacement (which is more common), occurs when students who likely could have succeeded in college level courses are assigned to remediation. Analyzing data from two large community college systems, Scott-Clayton et al. (2014) estimated that one-fifth to one-third of students are likely to have been severely misplaced.
Many states and colleges have started to use multiple measures to determine students’ initial course placements. Although empirical evidence generally suggests that adding high school data into placement consideration would increase placement accuracy (Leeds & Mokher, 2020; Scott-Clayton et al., 2014), placement policies using multiple measures are not always feasible to implement given the substantial administrative burdens for both students and college personnel. In a recent study focusing on community colleges in Florida, Leeds and Mokher (2020) found that adjusting placement tests cutoffs to minimize misplacement might be preferable to switching to the use of additional metrics.
The type of placement tests used also matters for the accuracy in predicting course success rates. Many community colleges use standardized tests produced by ACT, Inc. and the College Board, such as COMPASS and Accuplacer, in their placement process. There have been concerns that these national tests may not be well aligned with what students have learned in high school and thus cannot effectively predict college success (Belfield & Crosta, 2012). In Florida, all public state colleges used the Postsecondary Education Readiness Test (PERT) to determine initial course placement (Leeds & Mokher, 2020). PERT is a computer-adaptive test of math, reading and writing. Unlike the national tests used in many other states, the PERT is administered by the Florida Department of Education and is a state-specific test that is aligned with both the state K-12 standards and Postsecondary Readiness Competencies (Leeds & Mokher, 2020).
The course placement policies in Florida are different from other states. The reform measures under SB 1720 made placement testing (i.e., the PERT) and developmental education optional for a large proportion of students, regardless of their academic preparation. Exempt students are those who entered a Florida public high school in 2003 to 2004 or later and then graduated with a standard high school diploma, or active military duty personnel status. Exempt students can opt into developmental education if they think they need it, or they are advised to do so. On the contrary, non-exempt students are still required to take the PERT and follow the existing placement guidelines. Many non-exempt students are adult students (usually above 25) who have been out of school for many years (Mokher et al., 2021). There are also students who are non-exempt because they attended private high schools or public schools in other states. The non-exempt students are not affected with regard to opting out of placement testing and developmental education course requirements. They are, however, still affected by the reform policy because of the changes in DE instructional strategies and enhanced academic advising and support services required by the legislation. Thus, it is instructive to examine whether Florida’s developmental education reform affects non-exempt students.
Instructional Strategies and Student Success
Another major component of SB 1720 required colleges to offer multiple instructional strategies for students in DE courses, including modularized, contextualized, compressed, and corequisite strategies. Modularized instruction involves breaking the content into small units from which students can choose based on their individual needs. Contextualized instruction requires the content of the DE course to be aligned with the student’s major or program of study. Compressed courses are designed to shorten the length of time in the developmental course and in some cases, allow students to enroll in college-level courses in the same semester. Corequisite courses provide supplemental instruction that students take at the same time as credit-bearing college-level courses (Park et al., 2018). Institutions can choose which of these strategies to offer according to their abilities, but have to offer at least one of the four new instructional strategies in their developmental reading, writing, and math courses (Park et al., 2018). All non-exempt students who were required to enroll in developmental courses would be taught with these new strategies after the reform.
There has been some evidence from other states that the new DE instructional strategies are associated with positive student outcomes. For example, analyzing student data from a large urban community college, Guy et al. (2015) showed that students who took a compressed remedial course tended to pass the exit exam at significantly higher rates relative to those who took a traditional semester-long course. Students on compressed pathways were also more likely to complete the relevant college-level courses within 3 years than their equivalent peers (Jaggars et al., 2015). In addition, Jenkins et al. (2010) found that participation in the Community College of Baltimore’s Accelerated Learning Program (ALP) for DE English was associated with higher completion rates in college-level courses. Based on over-a-decade of data on English curriculum at Chabot College, Hern (2011) reported that students from accelerated courses complete college English at significantly higher rates than students enrolling in a two-semester developmental sequence. Moreover, even students with the lowest placement scores do at least as well as in the accelerated course as they do in the longer sequence.
An example of contextualized course strategies is Washington’s Integrated Basic Education and Skills Training (I-BEST) program that pairs academic instructors with career and technical education (CTE) instructors. By doing so, I-BEST aims to connect academic skills within the context of students’ career pathways. In a study of this contextualized strategy, Zeidenberg et al. (2010) found that students in the I-BEST program were more likely to persist to the next year and earn more college and CTE credits than their similarly matched peers.
Academic Advising and Support
Overall, studies have documented positive effects of academic advising and support services on student success (e.g., Bettinger et al., 2013; Hatch & Garcia, 2017). SB 1720 also requires colleges to increase advising services and enhance academic support. A survey of college administrators at 19 FCS institutions (Woods et al., 2017) showed that most colleges adopted a variety of advising tools such as early warning system, consideration of high school and college transcripts in course recommendations, checklists of degree requirements, individual education plans, and degree maps that list all of the courses offered in each semester. More than half of the administrators reported that they have increased the types of advising services offered after implementation of the reform (Woods et al., 2017). Moreover, the majority of administrators perceived their advising practices as effective regarding students’ exemption status (Woods et al., 2017). However, the survey results also indicated that colleges struggled to offer adequate advising services to their students partly due to limited staff or other resource constraints. Only a few administrators reported that they offered separate orientations for students who were exempt or non-exempt (Woods et al., 2017).
Effectiveness of Florida’s DE Reform
Prior studies suggest that the developmental education reform in Florida has contributed to an overall increase in student success in terms of introductory college-level course completion and college-level credit accumulation within the first year of enrollment among all FTIC students (Mokher, Park-Gaghan et al., 2020; Park-Gaghan et al., 2020). These positive outcomes are the result of a combination of the different components of the reform. Focusing on non-exempt students who were not affected by the placement policy changes, this study extends prior research on the effectiveness of Florida’s DE reform as we can specifically assess the effects of new instructional strategies and enhanced academic support on student success.
Conceptual Framework
Adelman (1999, 2006) conceptualization of academic momentum guides the analysis of this study. The academic momentum framework suggests that the rate at which undergraduate students progress during their early college experience significantly influences the likelihood of them completing a degree. Students who experience a sense of accomplishment at the beginning of their college careers can gain self-efficacy and academic self-concept, resulting in greater commitment to degree completion (Attewell et al., 2012). Anchored in the construct of academic momentum, Wang (2017) advanced a new holistic framework of momentum for community college student success. Her model incorporates three domains of momentum: (1) curricular momentum (e.g., proper course and program pathway); (2) teaching/learning momentum, which are “best cultivated through a set of instructional practices placing students at the front and center of classroom teaching and learning” (p. 284); and (3) motivational attributes and beliefs such as educational aspirations, attitudes, beliefs, and habits of mind.
The new DE instruction strategies required by SB 1720 may help build the curriculum and teaching/learning momentums in Wang’s (2017) model. In addition, Wang (2017) identified several “counter-momentum friction” (p. 287) factors in the model that may prevent students from gaining momentum, including financial barriers, lack of clear pathways aligned with student intent, and inadequate or lack of advising. As SB 1720 requires institutions to provide additional academic advising and support services, this reform could then reduce the “counter-momentum friction” for non-exempt students.
In summary, we posit that the developmental education reform in Florida could foster academic momentum that may set non-exempt students on a more successful trajectory on subsequent postsecondary outcomes. In this study, we specifically focus on enrollment and completion rates in college-level math and English courses within the first year. These indicators have been identified as effective predictors of longer-term student success (Belfield et al., 2019).
Methods
Data and Sample
The data in this study came from state administrative records for all FTIC students who began their studies at one of Florida’s 28 state colleges. We included two cohorts of students who were enrolled prior to SB 1720 (fall cohorts 2012 and 2013) and two cohorts who were enrolled after SB 1720 came into effect (fall cohorts 2014 and 2015). For the purpose of this study, we excluded students in the fall 2014 and fall 2015 cohorts who were exempt from DE under the new reform. Students from the fall 2012 and fall 2013 cohorts who would have likely been exempt had the reform been implemented earlier (based on high school attendance dates and graduation records) were also excluded from the analysis. Across the four cohorts in our analysis, the non-exempt students predominantly included out-of-state students, graduates of private high schools, and older students. About 35% of the total FTIC students among the pre-reform cohorts (2012–2013) and 30% in the post-reform cohorts (2014–2015) were non-exempt or likely non-exempt students. Each cohort consisted of approximately 22,000 non-exempt students, resulting in a total analytic sample size of 91,017 students.
In our sample, differences in the racial composition between exempt and non-exempt students include a higher percentage of White students, and lower percentages of Black and Hispanic students among non-exempt students, compared to exempt students (see Table 1). Non-exempt students also have a higher percentage of female students relative to exempt students (53.03% versus 51.84%). The biggest difference lies in age, with 34.14% of non-exempt students aged 25 or above, compared to only 2.17% of exempt students in the same age group.
Demographic Characteristics of Exempt and Non-exempt Students, Cohorts 2012 to 2015.
Variables
We are interested in students’ English and mathematics course enrollment and completion within their first year of enrollment. Prior research has indicated that first-year success in English and math was critical to persistence after 1 year (Callahan & Belcheir, 2017). Specifically, there are four outcome variables of interest: (1) highest English course enrolled; (2) highest English course completed; (3) highest math course enrolled; and (4) highest math course completed. The variable for the highest English course includes four levels of courses: (1) no English course at any time in the first year; (2) developmental reading and/or writing course; (3) the first college-level English course (ENC 1101: English Composition); and (4) an advanced English course beyond ENC 1101. Similarly, highest math course also includes four levels: (1) no math course at any time in the first year; (2) developmental math course; (3) college-level math courses (e.g., intermediate algebra, college algebra, liberal arts math 1 and 2, introductory statistics); and (4) an advanced math course.
Table 2 shows the distribution of students across the four levels for English and math courses, by cohort. Generally, in both English and math, more non-exempt students were able to take and complete higher level courses after the reform than they were before the reform. For example, in English, the percentage of non-exempt students enrolling in no English course within the first year decreased from 32.22% in 2013 (pre-reform) to 28.28% in 2014 (post-reform), while the percentage of non-exempt student enrolling in advanced English courses increased from 13.91% to 19.97%. In math, the percentage of non-exempt students who did not enroll in any math course within the first year decreased from 55.91% in 2013 to 52.15% in 2014, whereas the percentage of non-exempt students who enrolled in college-level math increased from 18.04% to 26.57%.
Descriptive Statistics for Variables Used in the Analysis, by Cohort.
Our control variables include student background characteristics for race/ethnicity (White, Black, Hispanic, other), gender (male vs. female), and age. Considering that many non-exempt students are older students who entered college many years after high school graduation, we constructed a dichotomous variable indicating whether the student was 25 years old or above. Additionally, we include a binary post-reform indictor (1 = post-reform, 0 = pre-reform), a continuous cohort variable to control for any underlying temporal trends, and institutional fixed effects. We also include variables for students’ level of preparedness defined by PERT scores. Based on students’ PERT math scores and FCS placement polices, students were categorized into three groups for math: students who were assigned to a lower level DE math course, students who were placed into an upper-level DE course, and students who were placed into college-level courses. In English, we first categorized students into different groups based on their PERT reading and writing scores respectively. We then combined the groups based on PERT reading and writing to define three groups of students’ levels of preparedness in English: (1) students who were assigned to a lower-level DE English course (either DE reading or DE writing), (2) students who were assigned to an upper-level DE English course (including students who tested college-ready in either reading or writing, but not both subjects; and students who were assigned to upper-level DE reading and writing courses), and (3) college-ready students (who tested college-ready in both reading and writing).
Table 2 also presents summary statistics by cohort on student background characteristics and level of preparedness in English and math. There was some variation in student background characteristics over time. The percentage of Hispanic students increased from 30.24% in 2012 to 34.86% in 2015, while the share of Black students declined from 21.12% to 17.22%. After the reform, there was a sharp decline in the share of students aged 25 or above, decreasing from nearly 40% in 2013 to 30.54% in 2014. In addition, the share of students who were placed into lower DE English courses and students scored ready for college-level English decreased after the reform, with corresponding increases in the percentage of students who were placed into upper-level DE. Similarly, the percentage of students who were placed into lower DE math decreased from 39.48% in 2012 to 21.63% in 2015, with corresponding increases in the share of students who were placed into upper DE and students who scored college-ready for college-level math courses. Despite these variations, we argue, there are no systematic differences in the pre/post-policy nonexempt students that would preclude us from proceeding with longitudinal analyses.
Analytic Strategy
To address the first research question about changes in coursetaking outcomes for all non-exempt students under the reform, we estimated multinomial logit models to predict students’ highest level of English and math courses taken and passed in the first year. While ordered logit models are useful when the outcome variable has a meaningful sequential order, it assumes that the coefficients of predictor variables are parallel across levels, a strong assumption that is frequently violated in practice (Long & Freese, 2014). A test of the proportionality assumption for the ordered logit approach suggested that the regressions were not parallel (we used the brant, detail command in Stata to conduct the test). We therefore adopted multinomial logit models, which estimated different coefficients for every possible outcome comparison. For each outcome, we estimated the following model for student i at institution j in year (cohort) t:
Mlogit (No, DE, College-level, Advanced) ijt = β0 + β1(S) ijt + β2(CR) ijt + β3(Post) t + β4(Post × CR) ijt + ξ j + λ t + ε ij ,
Under this specification, S is a vector for student background characteristics, including gender, race/ethnicity, and age; CR is a vector for students’ college readiness in English or Math based on the three different levels of placement test scores. Post is an indicator for post-policy cohorts (1 = yes, 0 = no). To capture the heterogeneous policy effects by college readiness, we included an interaction term between Post and CR. We also controlled for a continuous cohort indicator (λt) to account for any underlying temporal trends. The model also includes institutional fixed effects.
To address our second research question about whether the changes experienced by non-exempt students after the reform differ by the level of college readiness, we recoded the outcome variable into a dichotomous indicator of students’ taking/passing college-level courses (including advanced courses) and conducted logistic regressions models to determine how the policy effects might differ by college readiness. Variables used in these analyses were the same as those used in the multinomial models. To facilitate interpretation, we report predicted probabilities based on multinomial logit regression models. In particular, we provide pre- and post-policy average predicted probabilities of taking/passing certain level of English/math courses. Then we calculated the pre/post difference in the predicted probabilities and the associated statistical significance. Further, in order to better understand the interaction effects between the policy and college readiness, we graphed predictive probabilities of taking and passing college-level English and math courses by level of college readiness.
Limitations
This research evaluated the impacts of the SB 1720 on non-exempt students at 28 FCS institutions. Our model includes institutional fixed effects, but does not consider how treatment effects varied by institution. Future research can further explore institutional variations in policy effects. Next, we can only examine overall impacts of offering new instructional strategies and not whether some strategies may be more effective than others. Finally, although we try to identify the effects of different components of the reform, it is important to consider that the policy as a whole may have impacts that are greater than the sum of its parts. Many institutional leaders in FCS colleges reported that they had made significant changes that were beyond the requirements of the legislation to improve student success (Mokher et al., 2020). Such considerations stress the importance of the implementation of the policy at the institutional level, especially the efforts made by institutional leaders, faculty, and staff.
Findings
English Enrollment and Completion Patterns
We begin by presenting results for changes in English enrollment and completion patterns in the first year for the two pre-policy cohorts (2012–2013) and the two post-policy cohorts (2014–2015). Overall, there were significant decreases in the share of students who took no English course and students who only enrolled in DE English courses following the reform, with a corresponding increase in the enrollment rates in advanced English courses (see Table 3). For example, prior to the reform, 30.86% of non-exempt students took no English course in their first year; this group decreased to 28.70% after the reform. In contrast, enrollment rates in advanced English courses increased from 15.56% prior to the reform to 19.41% following the reform. We note, however, that the percentage of non-exempt students who enrolled in college-level English course (but not advanced English course) has remained relatively stable over time. The findings for English completion patterns are very similar to those for English enrollment. The share of non-exempt students who did not complete any English courses and non-exempt students who only completed a DE English course significantly decreased following the reform, while the completion rates in both college-level English courses and advanced English courses significantly increased at the same time.
Predicted Probabilities of the Highest Level of English Courses Took and Completed Before and After Florida’s DE Reform.
Note. The predicted probabilities are calculated based on the model including such covariates as gender, race, age, college readiness, post policy indicator, as well as an interaction term between post policy indicator and college readiness. Institutional fixed effects were controlled. Variables other than post policy indicator are set at their observed values.
p < .001.
In order to test the extent to which the policy may have affected non-exempt students with varying levels of college readiness differently, we fit logistic regression models predicting the probabilities of taking/completing college-level courses (including advanced courses) in the first year, and compared the pre/post differences across varying levels of college readiness. Although all non-exempt students experienced significant gains in enrollment rates in college-level English after the reform, students in lower DE experienced the biggest increases (see Figure 1). The pattern for English completion rates is similar. Non-exempt students who placed into lower DE experienced the largest increases in completion rates of college-level math.
Probability of taking (left) and passing (right) college-level English before and after the policy, by college-readiness.
Math Enrollment and Completion Patterns
Table 4 reports results of changes in math enrollment and completion patterns in the first year of college for the two pre-policy cohorts (2012–2013) and the two post-policy cohorts (2014–2015). Predicted probabilities of only enrolling in a DE math course in the first year show statistically significant decreases following the reform (from 35.30% to 30.34%), with corresponding significant increases in the predicted probabilities of enrolling in college-level or advanced math courses. The predicted probabilities of enrolling in no math courses were 31% in the pre-reform cohorts and did not change significantly in the post-reform cohorts.
Predicted Probabilities of the Highest Level of Math Courses Took and Completed Before and After Florida’s DE Reform.
Note. The predicted probabilities are calculated based on the model including such covariates as gender, race, age, college readiness, post policy indicator, as well as an interaction term between post policy indicator and college readiness. Institutional fixed effects were controlled. Variables other than post policy indicator are set at their observed values.
p < .001; **p < .01.
The results for math completion patterns are slightly different. The predicted probabilities of completing no math in the first year decreased from 54.36% to 52.17%. In fact, more than half of non-exempt students did not complete any math course within the first year of enrollment in both pre- and post-cohorts. Similar to the results for math enrollment, the predicted probabilities of only enrolling in DE math courses in the first year decreased from 22.59% to 19.33%, whereas the predicted probabilities of completing college-level and advanced math courses significantly increased.
Figure 2 shows math enrollment and completion patterns, disaggregated by level of college readiness based on PERT scores. Overall, the predicted probabilities for enrollment in college-level math courses significantly increased for all college readiness groups. Student numbers in the lower DE group increased from 5.91% to 17.41% while college ready student numbers increased from 65.72% to 78.05%. However, the difference between the gains for the two groups were not statistically significant. For math completion, non-exempt students in all college readiness groups experienced significant increases in the likelihood of completing college-level math courses after the reform. Moreover, college-ready students had the largest gains in completion rates following the reform compared to students with PERT scores for assignment to lower and upper DE.
Probability of taking (left) and passing (right) college-level English before and after the policy, by college-readiness.
Discussion
This study examines the policy effects of SB 1720 on non-exempt students who are still required to take placement tests and subsequently enroll in DE courses if needed. Although non-exempt students were not the primary target of the legislation, they constitute one-third of the student population. Understanding how non-exempt students are influenced by the reform can contribute to a more complete picture of the impacts of SB 1720 and ensure no students were left behind. We investigate unintended policy consequences on non-exempt students by focusing on changes in early momentum metrics (i.e., taking and passing college-level math/English courses within the first year). These metrics are key predictors of longer-term postsecondary success and thus relevant indictors of the effectiveness of the development education reform in Florida (Belfield et al., 2019).
Our findings demonstrate that non-exempt students benefited from the Florida DE reform, regardless of their level of college readiness. A higher percentage of non-exempt students took and completed college-level English and math courses within the first year of enrollment after the reform, compared to their equivalent peers in the pre-reform cohorts. These findings are consistent with the larger study that examined the policy effects for the overall student population (including both exempt and nonexempt students) (Hu et al., 2019). As the majority of students were exempt from the mandatory placement tests under the reform, it is tempting to assume that the placement policy changes play a large role in leading to the overall increase in student success. By focusing on the non-exempt students that were not affected by the placement policy changes, finding from this study highlight the importance of the other components in the reform: the new instructional strategies for DE courses and enhanced academic advising and support services.
New instructional strategies, such as corequisite and compressed instruction, allow non-exempt students to enroll in developmental courses and college-level courses at the same time. Although we cannot know to what extent the positive changes experienced by the non-exempt students can be attributed to the new instructional strategies relative to other components of the reform (e.g., additional academic advising and support services), there is experimental evidence from other states suggesting the effectiveness of these new strategies. Results from a randomized controlled trial indicated that students randomly assigned to corequisite remediation earned more credits in the year following the intervention and had higher pass rates in college-level courses and higher graduation rates than did the students randomly assigned to traditional remediation (Logue et al., 2016, 2019).
The enhanced academic advising and support services may also play a role in improving student success for non-exempt students. In a survey of FCS leaders that included questions about the extent to which various components of SB 1720 were perceived to have affected student success, institutional leaders reported that they believed that the enhanced student advising and support services had the most positive effects on students’ success (Hu et al., 2018). Academic advising and support services are important channels through which institutions meet the needs of students from diverse backgrounds. As the DE reforms in many other states have been focused on the placement policies, it is important to emphasize that simply removing or changing the requirement for DE will likely fail to increase student success without the commitment to meet individual student needs through academic advising and student support services.
The share of non-exempt students who did not complete any English or math classes and those who only finish developmental courses in the first year significantly decreased following the reform. We note that even with the significant decreases, more than 40% and 50% of non-exempt students did not complete any English or math courses in the post-reform periods, respectively. Given that our study only focuses on student outcomes in the first year of college enrollment, it is possible that some students just delay the enrollment in English or math courses. However, evidence shows that first-year successes in English and math are critical for retention after the first year (Callahan & Belcheir, 2017). Putting off math and English gateway courses will likely reduce the likelihood that students will ever earn a degree. Considering a significant proportion of non-exempt students are adult returning students who may face additional family and work obligations, qualitative research will be useful in uncovering the reasons or obstacles that prevent these students from enrollment in English or math courses and offering suggestions to better serve these students. We also note that the completion rates in both college-level English and math courses are much lower than the corresponding enrollment rates, indicating that a lot students who enrolled in college-level courses did not succeed. More research is needed on factors associated with success in college-level English and math courses for non-exempt students.
Students with different levels of college readiness experienced differential gains in the reform. In English, non-exempt students who were placed into the lower level of developmental education experienced the greatest increases in the completion rates in college-level English courses, whereas college-ready students experienced no significant change in college-level English course completion following the reform. It is likely that students who were placed into lower-level DE benefited from the new instructional strategies, in addition to the advising and student support services after the reform, while the college-ready students were largely only affected by the enhanced student support services.
However, the story for enrollment and completion rates in college-level math courses is different. College-ready students experienced the greatest increases in the completion rates of college-level math courses following the reform. The differences in the results for math and English may be partly associated with the nature of math and English learning. In their study of first-time college students in Virginia Community College System, Roksa et al. (2009) found that the probability of passing college-level English courses was unrelated to placement test scores. However, placement tests scores in math had a stronger association with passing college-level math (Roksa et al., 2009). This may partly explain why college-ready students with highest PERT scores experienced the greatest gains in completion rates for college-level math when additional academic support was provided as a result of SB 1720. The differences in baseline passing rates in English and math may also play a role. Many more students passed college-level English in the pre-reform period (36% for English vs. 23% for math, see Tables 3 and 4), suggesting the course is easier to pass than college-level math. Therefore, even students with lower levels of readiness may experience gains under the reform that result in passing college-level English.
These results hold important implications for practice. Helping non-exempt students to progress though developmental education more quickly and allowing more students to enroll in college-level courses may prove to be effective. When additional academic support is provided, many students, including those who would have been assigned to lower DE, can succeed in college-level courses. While SB 1720 did not require institutions to make changes to college-level courses, changes in instructional practices and curriculum of these courses are likely needed to improve the success rates. As a result of the legislation, many students who would not have been able to take college-level courses now have the opportunity to enroll, creating challenges for instructors to effectively address students’ diverse needs. Indeed, many institutional leaders reported that instructors adapted pedagogical behaviors and redesigned curriculum in college-level courses following the reform (Mokher, Spencer et al., 2020). This may be particularly important for college-level math courses as success in these courses requires more rigorous academic preparation.
As other states such as California are moving toward optional DE in public colleges and universities, a critical question to think about is who may benefit or be harmed from being non-exempt to inform those policy decisions. For example, Kurlaender (2018) expressed concerns that California’s approach would harm students who have less access to opportunities that determine college readiness. She argued that colleges and universities should make additional efforts to support students who were shortchanged by their prior educational experiences. Also, although there is some evidence suggesting the benefits of placement to multiple levels of DE for the lowest-performing students (Boatman & Long, 2018), it is unclear whether certain students might benefit more from optional DE or eliminating DE completely. Because of the diversity in student backgrounds and heterogenous policy effects for different populations, it is important to ensure that all students, whether or not part of the primary target group of a policy, can benefit from the reform.
Footnotes
Author Note
Kai Zhao is now affiliated to Lingnan University, Tuen Mun, Hong Kong.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R305A160166 to Florida State University, and in part by a grant from the Bill & Melinda Gates Foundation. The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education, or the Gates Foundation.
