Abstract
Research has shown upwards of 40% of adult learners entering community college may need developmental education intervention in reading and writing skills. Of those who enroll in developmental education, a large portion will never receive college-level credit and will leave the community college. This research investigated a compressed reading and writing developmental education course for adult learners in a community college setting. The newly designed course integrated skill development, contextualized the content, and connected students and teachers in a meaningful way. The developmental education reading and writing course was compressed to eight weeks and attached to a compressed (eight weeks) college-level course. The course’s efficacy was analyzed using multiple patterns of outcomes including for the developmental education coursework (pre and post new design), gateway courses’ pass rate, and program completion. Moreover, this research compares the outcomes to national, statewide, and institutional data. The review of the patterns of gateway course success before and after implementation (2009–2020) saw mixed results when examining career and technical programs and transfer-level programs. Results also showed that while college-level competency was enhanced, the outcome did not translate fully to the completion of programs of study.
Adult learners in developmental education: Programming for equity and access
Adult learners are as varied in their skills and backgrounds as in their career goals and aspirations. In response to the heterogeneity in learners’ preparation for engaging in programs aimed at future careers, postsecondary readiness initiatives have taken a prominent spot in state departments of education as well as the US Department of Education, with estimates of 65% of all jobs requiring some form of postsecondary education or training by 2020 (NCES, 2015).
Of those adult learners entering postsecondary programs, a startling 40% of all who enter community colleges are placed into some type of developmental education (DE) class (Shapiro et al., 2016). Additional research indicated that approximately 66% of community college students who receive DE placements do not achieve college-level competency (Bahr, 2012). Bailey et al. (2010) analyzed placement scores for 250,000 students from 57 community colleges; they found that 59% tested into DE mathematics (n = 147,500) and 33% tested into DE reading and writing (n = 82,500). If 66% of these 250,000 students do not achieve college-level competency, then more than 150,000 students with aspirations of higher education would not achieve college-level credits.
In 2016, a rural community college took aim at transforming their DE reading and writing program to attend to the findings of the aforementioned research as well as other studies that questioned the effectiveness of existing DE programs. The preparations for the new design focused on the adult learners who were beginning their educational journeys with different levels of preparedness. In 2016, Complete College America reported that only 20% of students who began college in DE completed the next-level college course within two years (Shapiro et al., 2016). This redesigned DE program set a goal of improving the outcomes for students. The purpose of this research was to document the redesigned DE program, analyze the outcome of the intervention, document the completion of the immediately following college-level course, and monitor the impact of the redesigned course compared to previous DE coursework in reading and writing.
When researching the initiatives that addressed deficits in DE programs, four major issues emerged: (1) DE policies related to placement into entry level or “gateway” coursework, (2) curriculum design that considers adult learners’ needs, (3) the differences in skills among adult learners, and (4) support for students from DE to gateway courses and program completion. These issues were prioritized as the program investigated students’ learning gains in DE courses, success rates for the gateway courses for those completing the redesigned DE reading and writing course and in following this cohort through the process to measure program completion. As a follow up to the analysis of the newly designed program’s impact, a wider view of gateway course completion from 2009 to 2022 looked at rates before and after implementation.
Redesigning timeframes, instructional delivery, curriculum, and student support in developmental education
The newly designed DE program changed both the timeframe for the coursework and the instructional delivery. The new DE program used a compressed model for instructional delivery meaning an eight-week course in DE combining reading and writing instruction was linked with an eight-week course for the gateway course. Previously, the DE program used multiple sixteen-week courses in separate reading and writing courses. After completion of the DE coursework, the students were then asked to enroll in a gateway course for another sixteen weeks.
Instructional delivery in the newly designed program
Lavonier (2015) explored two different approaches to delivering instruction in developmental reading and writing education at a community college. The study used a control group (n = 32) and a treatment group (n = 32) designed to investigate the outcomes of a textbook instructional delivery approach versus a “strategic reading” approach. The “strategic reading” approach included reading and writing assignments and instruction based on students’ needs and interests. The pre- and post-test scores were analyzed with a t test using the Nelson-Denny Reading Test (N-DRT), Form G; it was found that both approaches resulted in student gains in scores on the N-DRT. The redesigned approach to DE curriculum used a contextualized, strategic reading approach.
DE instruction effectiveness includes building a curriculum through contextualization, which allows adult learners to construct meaning by using texts and materials that connect to the adult learners’ needs and interests (Zimmerer et al., 2018). According to Grubb (2010), ineffective developmental education approaches are numerous and include decontextualized instruction. Adult learners utilized the newly designed DE curricula which were girded by three interconnected learning theories linked to contextualized learning: the simple view of reading (SVR), lexical quality hypothesis (LQH), and the self-teaching hypothesis (STH).
Learning theories supporting the curriculum
According to SVR, decoding skills and language comprehension are components of reading comprehension (Gough & Tunmer, 1986), while STH posits that decoding is essential for readers to transfer a written word into a spoken word, which they often recognize and understand the meaning (Share, 1995). Finally, LQH suggests that successful text comprehension is impacted by the learner’s background knowledge of the content (Perfetti & Hart, 2002; Verhoeven et al., 2002). In further intertwining these theories, Wang et al. (2019) suggested that instructors could impact the development of reading and writing skills by identifying and scaffolding the “insufficient decoding skills” that hinder self-teaching mechanisms among students. In using text with content familiar to the learner’s background knowledge, the familiar content helps to accelerate decoding skills and vocabulary development and further connects to comprehension of the language (i.e., construct meaning) within the text as the written word, when spoken then has meaning. Thus, Wang et al. (2019) posited that wide variations in reading comprehension can be explained by decoding and linguistic comprehension.
Student support
Bahr (2012) called attention to attrition rates within DE coursework, finding students who did not pass a DE course on their first attempt may simply leave the institution and forfeit their plans for a career. Woods et al. (2019) described prevention policies which seek to intervene with students prior to placement in DE (or that happen concurrently) to avoid the sequence of DE that can result in 80% leaving (Shapiro et al., 2016). The policies recommended in supporting students within DE coursework included using prevention measures including multiple metrics for placement into developmental programs by addressing preparedness for postsecondary coursework with secondary schools.
Attrition from DE and how to address the issues pointed out in research findings were contributing factors and became a motivating force for both the redesign of the DE program and a rationale for studying of learning outcomes, including any long-term implications. Addressing how students are placed into developmental programs is essential but just as critical as placement into DE is aligning programs to students’ needs. Meaning, those students being asked to participate in DE should be assured that the course design addresses the objectives and course outcomes in the course for which they are being prepared. Achieving a passing grade in a DE course is not the end goal of any DE course for students or instructors. Being successful in the course, the DE course is preparing the students to master, is the purpose of DE. Given the consequences of mediocre performance in DE (i.e., failing in the DE course or failure to achieve mastery in the subject area as measured by the gateway course), it is imperative to channel all available resources toward interventions (including but not limited to tutoring) to support students in an academic setting (Bahr, 2008).
In supporting students within the newly designed program, a staff member was assigned to individual students. The tutor remained with the student through the DE and gateway course. All tutors within the program were certified teachers and acted in the capacity of instructional tutors for the sequence of coursework.
Research design
Student demographic information for the college.
Participants
The cohorts in the developmental reading and writing courses were placed in developmental education as a result of policies at the institution, which are based on cut-off scores from placement testing. The cut-off scores used to place students into developmental education were identical for the pre-redesign and newly design placements. In total, there were 87 students in the pre-redesign program and 70 students in the newly designed program.
Assessment strategies
Assessment tool for pre- and post-student outcomes
Cohorts in both the original and newly designed programs were pre-tested and post-tested using the Nelson-Denny Reading Test, Form G assessment tool (2016). The Nelson-Denny Reading Test is a standardized test of reading comprehension (Binder et al., 2016; Ready et al., 2012) and has been used to measure vocabulary and reading comprehension (Binder et al., 2016; Harrison et al., 2019; Ready et al., 2012; Vermeiren et al., 2022). To measure participants’ “vocabulary breadth” and reading comprehension, the Nelson-Denny Reading Test includes both vocabulary and comprehension questions, which are administered in two sections.
The first section of the Nelson-Denny Reading Test measures the breadth of a participant’s vocabulary, which is defined as the number of words in their lexicon (Binder et al., 2016). There are a total of 80 questions; from five options, participants are asked to identify the correct definition of a vocabulary word. The participant’s score is the total number of correct responses out of 80 questions. The second section of the Nelson-Denny Reading Test is the reading comprehension test. Participants must read a total of seven passages and answer 38 factual and inferential questions (five to eight questions per passage; Binder et al., 2016).
Developmental program outcomes and program completion rates
Information about statewide outcomes in other programs that used a developmental education compressed model was obtained from the Illinois Board of Higher Education and Illinois Community College Board (2020). This data concerned (1) the number of enrolled students, (2) the number of students who completed the developmental education course and enrolled in a gateway college-level course, and (3) students completing their chosen program of study within 150% of the time designated for its completion. For example, students who begin a program that is designed to be completed in two semesters are considered successful if they complete it in three semesters. In the current study, participants in the newly designed program were tracked from enrollment to completion using internal data to document college-level course success and program completion.
Gateway course success rates for technical and transfer programs (2009–2020)
To obtain a wider view of the efficacy of the developmental courses, a 12-year investigation (6 years before the redesign and five years after the redesign; 2009–2020) on patterns of grades in two gateway courses sequenced after the developmental reading and writing courses (i.e., “Composition” and “Communication”) was conducted. This information was collected by examining internal data at the organization.
Data analysis
Paired sample T-tests
Using the SPSS software package, a paired sample t test was performed to analyze the gain scores using the pre- and post-tests. The analysis of the pre-redesign intervention examined pre- and post-vocabulary and comprehension test scores among 87 students. In addition, an analysis of the newly designed intervention was conducted to examine pre- and post-vocabulary and comprehension test scores for 70 students.
Results
Pre- and post-test scores
The analysis of the pre-redesign course model indicated significant differences between the pre-test scores (vocabulary and comprehension; M = 63.98, SD = 9.79) and post-test scores (M = 67.40, SD = 10.54, t(86) = 4.10, p = .000). However, the effect size was .165, which was determined to be small.
In the analysis of the newly designed model, significant differences were also found between pre-test scores (vocabulary and comprehension; M = 62.54, SD = 6.60) and post-test scores (M = 72.49, SD = 6.90, t(69) = 10.10, p = .000). The effect size was .59, which was determined to be large.
Analysis of the developmental reading and writing program and student outcomes
An analysis was conducted to determine how many of the 77 students who participated in the redesigned intervention (2016–2018) completed the developmental reading and writing course and enrolled in a gateway course. In total, this number was found to be 67 students (87%). In addition, 44 students (66%) successfully completed the developmental course, and then successfully completed a college-level gateway course. Finally, 20 students (26%) who were enrolled in the developmental reading and writing course (2016–2018) completed their program of study within 150% of the designated time.
Analysis of gateway course success 2009–2020
An analysis was conducted on gateway course success for programs categorized as career and technical programs and transfer programs for a 12-year timeframe that covered the years of both the pre-redesign and newly designed developmental reading and writing programs. The outcomes for students for the gateway courses during this timeframe represent one of the metrics used by statewide research to measure student DE program effectiveness.
Career and technical programs
Over the 12-year period under review (2009–2020), 637 students entered career and technical education programs and enrolled in the gateway Communication course. The course description refers to the “practical application […] particularly in the comprehension and expression of English as it applies […]” (Illinois Eastern Community Colleges, 2019). • From 2009 to 2015, 456 students received a grade in the Communication course. Of this number, 63% received a grade of “A,” “B,” or “C.” • From 2016 to 2020, 181 students enrolled in the Communication course. Of these, 69% received a grade of “A,” “B,” or “C.” • This was equivalent to a 6% increase in students who successfully completed the gateway course for career and technical education programs from 2016 to 2020 (the years after the redesign of the program).
Transfer-level programs
From 2009 to 2020, a total of 3,129 students enrolled in “Composition,” the gateway course for students entering “transfer-level” programs. According to the course description, “Composition” is “an introductory course in composition and rhetoric emphasizing expository prose” (Illinois Eastern Community Colleges, 2019). • From 2009 to 2015, 2,151 students enrolled in the Composition course, and 77% received a grade of “A,” “B,” or “C.” • From 2016 to 2020, 978 students enrolled in the Composition course and 71% received a grade of “A,” “B,” or “C.” • This was equivalent to a 6% decrease in students who successfully completed the gateway course for transfer-level credit (the years after the redesigned program).
Limitations
As with any assessment tool, reliability and validity are constant concerns for researchers using raw and derived scores to measure student outcomes. As a tool, the Nelson-Denny Reading Test has been used by many educators as a relatively rapid and easy method of measuring ability in reading, vocabulary, and comprehension and by those who assess disabilities in adults. However, a better methodology would be to use multiple ways to gauge reading and writing skills or detect symptoms of dyslexia (reading difficulties) and/or dysgraphia (writing difficulties).
This research was limited by the metrics used before the newly designed DE program to measure student outcomes. Replicating the pre- and post-test scores was mandatory to compare outcomes. However, the effectiveness of the intervention is not solely determined by post-test scores on the Nelson-Denny Reading Test. Rather, the effectiveness of a developmental education program is measured by student success in a gateway course and beyond. Therefore, another limitation was in using the outcome of a course beyond the DE course to measure the success of the intervention. Attrition rates among adult learners are affected by a myriad of factors, which confounded the measurement of this program’s effectiveness.
The rationale for taking the further step of examining patterns of success in gateway courses before and after the redesign as another measure of effectiveness was a “backstop” type measurement. If the outcomes of the gateway courses were enhanced after the redesign for students in both career and technical programs and transfer programs, then there would not be a warning issued, but this was not the finding. The mixed outcomes for the twelve years prior and after, the redesigned program might indicate that not only could the redesign be less than effective for the transfer-level Composition course (indicated by the course success rates) but might also be interfering with program completion rates. Program completion rates are intrinsically connected to success in gateway courses.
Results, discussion, and implications for practice
Fall 2017 Illinois community college and Olney central college outcomes fall 2017, 2016–2018, reading and writing developmental model: compressed (Final Report, 2020).
Effectiveness of the intervention: Implications for practice
The effects of the interventions, as measured by learning gains in vocabulary and comprehension, were found to be statistically significant, and the pre-redesign impact was determined to have a small effect size. The impact of the newly designed program from pre-test to post-test scores was not only found to be statistically significant but also demonstrated a large effect size. Effect size can be viewed as a measure of the effectiveness of an intervention (Hetrick, 1999) or a more direct way of examining the impact of the intervention. In this research, the redesign not only improved the effectiveness of the intervention, but it also required students to spend significantly less time in class (eight weeks vs. 16 weeks and two hours per week vs. 6 hours per week). Consequently, the second half of the semester was designed to allow students to spend time on college-level coursework.
In reviewing the finding of the contextualized, integrated, accelerated intervention being more effective than a traditional text-book approach, the ideas related to prior knowledge, retrieval practice, and Bloom’s taxonomy of learning (Agarwal, 2019) may help explain the outcome. Prior knowledge generally refers to all of the experiences and learning that readers have acquired throughout their lives. Retrieval practice is a learning strategy that is designed to bring information to mind, which enhances and boosts learning. The taxonomy of learning has been illustrated in many sources and includes knowledge, comprehension, and application. These learning strategies contribute to the operationalization of the reading theories (i.e., SVR, STH, and LQH) included in the curriculum redesign.
For example, by using text about automobile mechanics for students who are interested in the automobile mechanics program, the quality of the text was exponentially increased for readers. In other words, instructors’ ability to identify insufficient decoding skills was increased, which perhaps serves to hinder the self-teaching mechanisms among students and could impact the development of their reading and writing skills. By using text with content familiar to the learner’s background knowledge, this would help to accelerate skill development and connect to comprehension of the language within the text as the written word, when spoken would then have meaning.
According to research, older adults use context to create message-level meaning (Steen-Baker et al., 2017). More often than their younger reader counterparts, adults construct message-level semantics (i.e., utterances that form word recognition, then integration) through the context of sentences. Thus, questions for an intervention strategy revolve around how to build, retrieve, or scaffold strategies that adult students can use in reading comprehension to foster growth among those who need such assistance.
Within this study, students participated in reading and writing development at their ability level, and then rapidly progressed as the retrieval practice of skills accelerated their acquisition of new abilities. Agarwal (2019) discussed the retrieval of facts and skills as an initial step in the hierarchical structure of learning and described retrieval practice as beneficial for learning in a diverse student population. Retrieval practice can be defined as students remembering what they had forgotten but mastered (or at least practiced) at one point in their education. Examples include rules for writing (e.g., noun-verb agreement or a pronoun, participial phrase, or appositive having to clearly refer to the proper subject) or reading comprehension strategies (e.g., identifying main points and summarizing texts).
As the retrieval practice of skills and knowledge is particular to an individual, the intervention must be individualized to be continuous for each student. The individualization of content also helps to build the quality of the text for the individual. Students need individualized levels of support and differing amounts of practice. LQH relies on accessibility to word meanings for readers. Three learning theories (SVR, STH, and LQH) served as a foundation for explaining the superior results of a contextualized (leading to individualized), integrated approach; the operationalization of these theories resulted in time spent working with individuals in the retrieval practice of reading skills (e.g., decoding and word meanings) and scaffolding decoding skills to assist the transformation of written words into spoken words, which encouraged the recognition of content-specific vocabulary.
As students regain, renew, or achieve mastery, they need to move forward in mastering the skills necessary to be successful in the gateway course. Asking adult learners to pause for a length of time until the semester ends or the rest of the class all masters a skill, can frustrate them, which may result in attrition from the program’s sequence.
Comparing program outcomes to national and statewide data: Implications for practice
The compressed intervention described in this paper enabled students to surpass the national completion success rate in a college-level course; however, this success rate did not fully transfer to the success rate for the completion of a program of study at the same rate. In other words, 66% of the students enrolled in DE in this program completed a college-level course, compared to 34% enrolled in DE programs nationwide (Behr, 2012). Moreover, 26% of those enrolled in DE in this program achieved program completion within 150% of the designated time, compared to 12% enrolled in DE statewide (IBHE-ICCB, 2020). Although the students within this community college, who began their educational journey in DE, doubled those statewide finishing their program in 150% of the program’s timeframe (26% vs 12%), this finding also indicated that only approximately half of students in the program who completed a college-level course within a semester (n = 44) also completed a program of study within the 150% of the program’s timeframe (n = 20). This is not a satisfying statistic to those policymakers who seek to reshape developmental education curriculum and policies.
Twelve-year analysis of completion rates in the gateway courses and programs: Implications for practice
An analysis of grade patterns in two gateway courses before and after the introduction of the redesigned program of developmental reading and writing programs renews concerns about preparation and instruction in DE. In addition to the skills and background knowledge that adult learners bring to the classroom, the skills required to be successful in the gateway course of choice must also be considered. The practical nature of the communication course, as defined by the catalogue’s description of the course, may be positively impacted by the contextualization of the compressed developmental education course in reading and writing. However, the developmental course may also need to include additional work on the use of literacy skills for academic writing that are more aligned with composition courses. A further investigation is warranted to determine other factors that contribute to the decline of passing grades in composition courses. Nevertheless, developmental education proponents must take note of these possible patterns when designing coursework. It is possible that DE policies will need to be differentiated based on skills needed to be successful within various gateway courses and programs of study.
Saxon et al. (2016) examined practices in several projects that use integrated reading and writing instruction and noted a lack of efficacy studies to support claims regarding the effectiveness of integrated models. While some institutions have successfully prepared students for college-level coursework in accelerated programs that integrate reading and writing instruction, it is unclear whether all have found success with this model.
During the review of the results, there appeared to be encouraging evidence for the effectiveness of a compressed model for developmental reading and writing and for the gateway course in comparison to the longer sequence. However, the success of the DE course did not fully materialize when looking at students who completed their program of study within 150% of the designated time. It is clear that more work is needed.
Conclusion
Current research suggests that compressing DE programs have advantages for both students and colleges, integrate the content used to teach and practice skills, and meaningfully connect students and teachers. The goal of the newly designed DE program was primarily to provide a seamless transition from DE to college-level coursework. However, while college-level competency was enhanced, the outcome of gateway course success did not fully translate to the completion of programs of study. Furthermore, when examining grade patterns in gateway courses for composition, conflicting results were found for the years before and after the new design of the developmental reading and writing course which calls for a further examination of the curriculum and objectives in DE coursework. Students in the DE process are vulnerable; vulnerable students need and deserve integrated coursework and programs that are designed to meet their needs.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
