Abstract
This pilot study explores the extent to which frequent quizzes given in a developmental Academic Critical Reading (ACR) course in a community college affects student success on a standardized CUNY (The City University of New York) assessment test on reading skills. Participants included 41 students in two class sections of ACR class who received frequent quizzes and 47 students in two sections of ACR without frequent quizzes as the control group. Results showed that students who received frequent quizzes showed greater improvement on the standardized CUNY test compared with controls, with gender moderating this effect. In addition, students who received a Pell Grant improved their performance on the standardized CUNY test more than those who did not receive a Pell Grant. Lower levels of absences were also associated with improvement on the standardized CUNY test. Quiz attitude survey results indicate that students have a positive attitude toward completing frequent quizzes, and they felt much more confident when taking the CUNY test at the end of the semester. Receiving frequent quizzes also encouraged students to attend classes at a higher rate.
Keywords
Introduction: The Changing Nature of Student Demographics in Community Colleges
Globalization, shifts in immigration, and increasing access to higher education have changed the characteristics of students in colleges and universities. The profile of students has changed as universities and colleges became open access institutions to students from nontraditional lower socioeconomic backgrounds. First-generation college students, students who aspire to attain education beyond high school for the first time in the family, have become an important demographic to study (Terenzini, Springer, Yaeger, Pascarella, & Nora, 1996; Trow, 2001; Woods-Warrior, 2014). Community colleges in the United States provide education to nearly 50% of all undergraduates, many with diverse backgrounds and needs. Out of this group of students, 36% constitute first-generation college students (American Association of Community Colleges, Fact Sheet, 2018).
Community colleges offer flexible programs and various financial aid opportunities. Their affordability allows many students to achieve their higher education goals. Both administrators and professors in community colleges work hard to increase success rates for all students, especially for minority and economically disadvantaged students (Niu & Tienda, 2013; Shugart, 2013). Although many students enter community colleges with great aspiration, they are at a greater risk to take at least three years to graduate due to various reasons, such as depending on financial aid, working part time, and having less persistence getting a degree (Fike & Fike, 2008; Thayer, 2000). After making the transition from high school to community college, their academic progress is rather slow. Only one-third of community college students get a college degree within six years (Bailey, Calcagno, Jenkins, Leinbach, & Kienzl, 2006).
There are several economic and flexible curricular incentives for students to graduate in a timely manner. The Pathway Project, Pell Grants, and the Accelerated Study in Associate Programs (ASAP) provide the opportunity for students to graduate in three years through financial, academic, and support programs (Scrivener & Weiss, 2013; Scrivener, Weiss, Sommo, & Fresques, 2012). As the Pell Grant is offered by the Federal Government, students need to make satisfactory academic performance in a degree-oriented program and receive the grant after the completion of this course.
The Pathway Project at Borough of Manhattan Community College (BMCC) provides a structured academic curriculum, which helps students identify a major and possible career paths. It also eases the sometimes-difficult process of transferring credits from the community to four-year college. (The Pathway Project, n.d.; “Pathways Project Eyes Fundamental College Restructuring,” 2016). Pathway Projects are needed at community colleges because students who start their education at community colleges are less likely to graduate than those students taking their entire undergraduate degree at four-year institutions. Community college students suffer significant credit loss upon transfer, with some studies showing only 58% of transfer students able to transfer more than 90% of their community college credits to their new four-year college (Monaghan & Attewell, 2015). This loss of transfer credits leads to a significant drop in graduation rates. Pathways projects, such as ones offered at the City University of New York (CUNY), guarantee the transfer of all 60 community college credits taken upon graduation.
Challenges Facing Community College Students
Despite the assistance readily available, a significant number of community college students continue to struggle academically due to passivity, resistance to asking for help, financial constraints and barriers to trust and openness (Jenkins, 2013). Besides the challenges of establishing trust and openness in reading and writing classes, many students do not have regular reading habits and struggle with basic critical reading and writing skills. For example, finding the main and supporting ideas could be challenging to these remedial students.
Increasingly, students are joining community colleges with less college-level cognitive skills (Greene, Marti, & McClenney, 2008; Hoachlander, Sikora, & Horn, 2003). Recent graduation rates from high school and universities have indicated a widespread trend regarding the lack of college preparation, especially from students of lower socioeconomic status (Diehl, 2012; Niu & Tienda, 2013). College freshmen lack the essential knowledge to be successful in science, math, writing, and subject matter core courses (Goldrick-Rab, 2010; Zimmerman, 1990, 1998). Ethnic minority first-generation college students have also been shown to have higher risks of poor academic performance and higher dropout rates (Dennis, Phinney, & Chuateco, 2005).
In 2013 to 2014, when this study was conducted, there were 24,186 students enrolled at the college. Most students were first-time college students with responsibilities of taking care of family members at home. Forty-three percent of the students were male and 57% of them were female. According to the BMCC fact data, 40% of the students were identified as Hispanic, 32% were Black, 13% were White, while 15% of the students were identified as Asian. Approximately 80% of students were qualified for either federal or state grants (CUNY, n.d.). Thus, many students came from low-income families.
With the clear majority of students at BMCC being younger and ethnically diverse, academic support mechanisms are vital for student success. Couple these risk factors with the 84.9% of all incoming freshmen needing remediation, classes such as academic critical reading (ACR) and developmental writing are essential components of the community college curriculum at BMCC. Students tend to lack critical thinking and study skills to navigate the reading load in early freshman years of college. Improvements must be made at the remedial levels to better prepare these community college students for the large amount of reading that forms the core of a traditional college curriculum.
The Benefits of Frequent Quizzes
Increasing student achievement in many classes at colleges and universities is an ongoing mission for both professors and college administrators. Although formative assessment is acknowledged as a powerful tool, high stakes summative assessment is accepted in many college settings. Because summative testing is used widely in higher education through midterms and final exams, not enough time is left for correction at the end of the semester. Therefore, frequent effective feedback is provided through formative assessment. Frequent quizzing is used both before lecture and after class sessions to improve student retention and motivation (McKenzie, 1973). There are a wide range of benefits of using daily quizzes, such as enhancing learning, academic performance, and maintaining attendance (Kamuche, 2005; Rezaei, 2015; Rutan, 2002; Tropman, 2014). Administering quizzes prior to in-class discussion or lecture is another way of increasing the likelihood of students being prepared for class (Geiger & Bostow, 1976; Graham, 1999; Tuckman, 1998). Narloch, Garbin, and Turnage (2006) investigated the effect of prelecture quizzes and found that students ask high-level questions and perform better on essay exams. Similarly, Klionsky (2008) concluded that quizzes lead to learning and retaining information. Frequent quizzes improve test scores, and having students review graded quizzes yields positive learning instead of reading over the assigned material.
Rutan (2002) gave students in a community college reading across disciplines program, weekly quizzes on reading assignments, which motivated them to read assigned texts. On surveys, these students responded favorably when asked about these weekly quizzes. Tropman (2014) used an upper level philosophy class to get students to use collaborative or group quizzes, which provide another range of possibilities for learning and are perceived as beneficial and desirable by students.
Most quizzes include multiple choice, fill in the blanks, and true/false statements. However, they can also be used in a creative way by having students work collaboratively to find and discuss the answers to the questions together (Quinn & Eckerson, 2010). Quizzes also allow students to process information in a more meaningful way and motivate them to study much more intensely (Roediger, 2014). A similar study by Rezaei (2015) investigated the effect of collaborative frequent quizzes and conceptual learning. Weekly quizzes were given, and students could work with each other in an open-book format. The results showed that when students worked in pairs to discuss the answers, they achieved a much higher score on their final exam and produced better final projects for the course.
One of the first studies to examine the relationship between the achievement gap and frequent quizzes revealed that the frequent assessment model reduces the achievement gap. Students from lower socioeconomic background perform like middle-class students until the last 2 to 3 weeks of the semester (Pennebaker, Gosling, & Ferrell, 2013). It was concluded that using frequent quizzes can be an important mechanism for boosting performance in a large part of the student population.
Purpose of the Study
The purpose of this study is to explore how students in a remedial academic reading course perform when given frequent quizzes over a semester and their success in passing the final CUNY test. Students pass or fail developmental reading courses based on the score they received on a standardized test that measures whether students possess the basic skills required to successfully complete college-level coursework. The two possible scores are either satisfactory (S) if students get a score of 70 from the standardized CUNY test or a repeat (R). Student motivation in these courses tends to be low and the use of frequent quizzes was chosen to boost course participation and performance.
The use of frequent quizzes will be defined in this study as administering a quiz during every class session for the entire semester, a total of 32 quizzes administered. It is hypothesized that students who take frequent quizzes will perform better on the standardized CUNY test at the end of the semester. This effect will be moderated by those receiving financial aid (Pell Grants), which will serve as additional motivation for students to perform at their best.
Although frequent testing has been shown to be effective in a variety of contexts, little to no research has been conducted in remedial community college classes. As remedial students have shorter attention spans, low levels of attendance, and an inability to integrate large amounts of information (Hagedorn, Sagher, & Siadat, 2000), the use of frequent quizzes is a learning paradigm that may be effective with this population.
When students are given frequent quizzes, they are also expected to have a strong attendance record, which will help them not to miss any course content. Another benefit of frequent quizzes is that the students receive immediate feedback on their quiz performance, which helps them develop effective studying routines. DePaolo and Wilkinson (2014) found that quizzes motivate students to attend and participate in class. Moreover, frequent quizzes are considered as a well-established classroom routine, which helps students to develop good habits of learning.
Research Questions
Considering the above discussion on the nature of developmental reading courses and the justifications for frequent quizzing, we will be examining the following research questions:
Method
Participants and Procedure
Eighty eight (N = 88) students enrolled in ACR classes, designed to prepare students for college-level course material, participated in the current study. Forty six (n = 46) of these students were enrolled in two ACR classes where frequent quizzes were given, whereas the other forty two (n = 42) students were enrolled in two ACR classes where frequent quizzes were, not given. The cohort of participants exhibited great diversity, which is representative of the student body at BMCC. Fifty six (n = 56) of the participants were female, whereas thirty two (n = 32) were male. Thirty-one (n = 31) participants were Hispanic, twenty four (n = 24) were Asian, twenty one (n = 21) were African American, nine (n = 9) were White, and three participants failed to report their ethnic background. Finally, fifty nine (n = 59) participants received a Pell Grant, whereas the remaining twenty nine (n = 29) did not.
In the experimental condition, there were 32 quizzes given during the course of the semester. Each quiz consisted of 10-15 multiple choice or short answer questions on topics and skills covered in the previous class. The two class sections of ACR-95 were chosen randomly from full-time faculty members who had at least 3 years of experience teaching this remedial course. This was done to ensure that the faculty members were familiar with the content of the course and the frequent challenges that students face as they progress through the course. The same procedure was used to select the two classes composing the control condition. Students could change sections during the first week of class, but none of the students requested such a change.
Reading and Vocabulary Quizzes
The following is an account of how reading quizzes were used in the developmental ACR-95 courses. Students who get a score on the standardized CUNY test between 57 and 69 are placed into ACR courses. In these courses, frequent quizzes are administered that last for 10-15 minutes depending on the text and the questions. Quiz questions are generated using a CD-Rom of assessment items of Pathways—Reading, Writing, and Academic Thinking—Level Three (Vargo & Blass, 2013).
On the first day of class, students are instructed that they will be receiving frequent quizzes in each class session. The quizzes will be administered at the start of every class. If a student is late to class, then he or she is given the quiz as a homework assignment. The significance of taking quizzes is emphasized and students are told that it will help them get ready to take the final standardized CUNY test.
Quizzes typically consist of a part of the passage from the assigned chapter of the textbook followed by multiple choice and/or true false statements. Some questions focus on the main idea of the passage, others on supporting details, while still others focus on vocabulary. Vocabulary tests are generated by an online quiz maker (Wordsmyth) based on new vocabulary appearing in textbook chapters.
Pathways-Level Three includes a variety of interesting topics for students in higher education. While introducing each topic, the instructor uses a short video related to the topic of each chapter selected. The articles in each chapter are one-to-two-page long followed by reading exercises asking students to identify the main ideas, key details, or supporting ideas. There are also some vocabulary exercises where students either use a dictionary or use the context clues to get familiar with the new words in chapter readings. Some of the topics that have been chosen to be included in the quizzes are comparisons of social and family relationships in the animal world to human social relationships, recent technological developments, unlocking the mysteries of Egyptian kings, the future of big cities, and important medical developments.
As far as reading skills are concerned, identifying main and supporting ideas, identifying the implied main idea, understanding referencing and cohesion, making predictions, identifying sequence of events, cause and effect reasoning, and understanding the tone used in the passage are introduced in increments. Throughout the semester, basic reading skills, such as finding the main and supporting ideas are integrated in our frequent quizzes. Nevertheless, as the semester progressed, there was a progression of complexity in reading skills. Therefore, implied main idea, tone, making inferences, purpose and tone questions are added later to the quizzes. Students are asked the purpose of the article, whether it was to entertain, to inform, or to persuade readers. Similarly, the final standardized CUNY Assessment includes these reading skills, and the frequent quizzes administered mirrored the final assessment model so that students would establish self-efficacy and gain confidence in their performance.
The graded quizzes would be reviewed in groups the next class session. This class routine practice enables students to ask questions to each other to clarify the answers and make sure each student understands the correct response. Sometimes students would be called on to make sure everyone understands the most frequently selected distractor in the quiz questions. By taking a short quiz at the beginning of each class period, students do not only gain confidence on what they can remember and accomplish in a short test, they also get immediate feedback from their classmates and professors.
Results
Research Question 1: Do Frequent Quizzes Affect Students’ Improvement of Reading in the Community College Context?
A one way, repeated measures ANOVA was conducted to compare the effect of frequent quizzes on scores on the standardized CUNY competency exam. Students who received frequent quizzes showed a greater improvement on the standardized competency test than those students who did not receive weekly quizzes, Wilks’s Lambda = 0.834, F(1, 65) = 12.909, p = .001. Figure 1 shows the greater improvement from pretest to posttest for the students who received weekly quizzes. This finding supports the primary hypothesis that administering frequent quizzes is an effective pedagogical technique for assisting students in a developmental skills class.
The effect of frequent quizzes on CUNY standardized test.
Research Question 2: Do Gender, Pell Grants, and Attendance Have an Impact on Student Improvement on Reading in the Community College Setting?
Within the frequent quiz condition, women improved more than men, Wilks’s Lambda = 0.891, F(1, 35) = 4.282, p = .046 (Figure 2). It was also hypothesized that the ACR classes receiving frequent quizzes would be better attended than those not receiving frequent quizzes.
The effect of gender on CUNY standardized test scores among students taking frequent quizzes.
Figure 3 demonstrates that students enrolled in a class that gives frequent quizzes attend more regularly than those students who do not take frequent quizzes, t(86) = 3.079, p = .003.
Absences as a function of frequent quizzes.
The interaction effect between absences and frequent quizzes on the CUNY assessment test score was also explored. A median split was initially performed on the absences variable to create a dichotomous variable with values of low absences and high absences. This was carried out to allow absences to be used as a between-subjects variable in the repeated measures ANOVA. The interaction effect between absences and frequent quizzes was not statistically significant, Wilks’s Lambda = 0.958, F(1, 67) = 2.961, p = .09. However, there was a statistically significant effect of absences on improvement in the standardized CUNY test score, Wilks’s Lambda = 0.943, F(1, 72) = 4.319, p = .041. Students who had a lower number of absences showed more improvement than those with a higher number of absences (Figure 4).
CUNY standardized test score as a function of absences.
The hypothesis that receiving a Pell Grant would improve performance in this remedial population of ACR students was also tested. Figure 5 shows that across all ACR classes, students who had received a Pell Grant improved more than students who did not receive a Pell Grant, Wilks’s Lambda = 0.937, F(1, 73) = 4.933, p = .029.
The effect of receiving Pell Grants on CUNY standardized test scores.
Additional results show that receiving frequent quizzes did not moderate the effect of obtaining a Pell Grant on the CUNY assessment scores, Wilks’s Lambda = 0.570, F(1, 71) = .570, p = .453. There was also no interaction effect between receiving a Pell Grant and one’s gender in terms of CUNY standardized test score, F(1, 71) = .242, p = .624, Wilks’s Lambda = 0.997.
Research Question 3: What Are Students’ Perceptions of Frequent Quizzes in an ACR Course in a Community College Context?
To obtain a genuine understanding of students’ perceptions of frequent quizzing, an anonymous frequent quizzes attitude survey was filled out by 31 students in two sections of ACR-95 (21 students in one section, 10 students in the other). The survey results generally indicate a positive attitude toward taking these quizzes, with scores ranging from one to four and higher scores indicating a greater amount of agreement. As shown in Figure 6, most students acknowledged that quizzes were helpful reviewing for the final standardized CUNY assessment test, motivated the students to practice, and did not increase their anxiety levels. Overall, students thought they benefited from the quizzes and improved their reading scores.
Mean quiz attitude score by question.
Table 1 shows the pass rates in both the frequent quiz condition and the control condition. The table shows that pass rates were higher for both frequent quiz conditions than the two control conditions. The pass rate for the frequent quiz conditions was also much higher than the overall pass rates for all sections of the given course during that same semester.
Pass Rates of Experimental and Control Groups.
Note. ACR = academic critical reading.
Discussion
In the current study, students who were administered frequent quizzes score significantly higher on the standardized CUNY test than students who do not take frequent quizzes in their remedial reading classes. These findings are in line with previous studies showing the benefits of taking frequent quizzes (Klionsky, 2008; Narloch et al., 2006; Rezaei, 2015; Roediger, 2014; Tropman, 2014). Women also improved more than men when administered frequent quizzes, which is in line with prior findings that gender plays a role in academic achievement (Barrow, Reilly, & Woodfield, 2009).
Women have traditionally been shown to outperform men on measures of reading and writing during the past 30 years (Meece, Glienke, & Burg, 2006). Nekby, Thoursie, and Vahtrik (2008) have also shown similar gender differences in a microeconomics course. However, this gender effect has not been shown in a remedial setting until the current study.
Both absences and Pell Grants had a significant influence on improvement shown on the standardized CUNY test. In remedial classes, attendance is seen as a vital factor towards success (Moore, 2003). As a community college, BMCC has a large cohort of students entering each year requiring remediation. Therefore, a higher level of attendance needs to be encouraged to help students reach their maximum potential within the college. As Pell Grants are performance-based, students may be more motivated to attend, study, and participate in classes. This may allow students to make greater improvements from the pretest to the posttest.
Students also felt that frequent quizzes were helpful in preparing them for the standardized CUNY test. They reported that having frequent quizzes motivated them to study regularly and even improved their levels of communication. Interestingly, taking frequent quizzes was not associated with an increase in anxiety. In contrast, having frequent quizzes may lead to a decrease in test anxiety for the CUNY test through a process of systematic desensitization.
Limitations
The main limitation was that students were not randomly assigned to the frequent quiz or control conditions. Students were already assigned to the classroom before the class was designated a frequent quizzes class or a non-frequent quizzes class (control). However, the contents of these remedial classes are standardized, and the abilities of the students in each class should be around the same level. There was no effect of ethnicity on improvement seen throughout the semester, but with the current sample size, this cannot be fully evaluated. Three-way interaction effects could not be performed because of the small sample size and corresponding lack of statistical power. For further research purposes, follow-up studies regarding students’ academic performance in credit-bearing courses after exiting the remedial reading courses should be carried out to provide a more genuine understanding of study habits of incoming freshman students.
Conclusion
All three of our research hypotheses were either supported or partially supported. Utilizing frequent quizzes as a pedagogical tool allows community college students to accept accountability for their own learning. Students are actively involved in the learning process while taking the quiz and as they go over the questions with classmates. Involving students and establishing a routine of taking quizzes leads to better student retention in most classes.
The repeated testing certainly gives multiple opportunities to get ready to take the high stakes standardized CUNY test, and this has implications for both learning and class performance. Students who join colleges without academic reading skills can improve their test taking performance through the process of frequent quizzing. The principles of time on task, giving immediate feedback, and cooperation among students as they go through the answers of graded tests clearly help students achieve a better score on the final CUNY assessment test.
Although promising, this pilot study is based on limited number of students of two sections of ACR in two semesters. However, it provides directions for future research on remedial undergraduate education and assessment. The digital readings and class discussions might have contributed to the improvement of scores on the CUNY assessment test.
Implications for Research
Frequent quizzing creates a community of learners and positive class dynamics in which learners create a shared sense of collective efficacy by going over the quiz results in groups. This practice enables them to believe that they can get a passing score on the CUNY assessment test. The findings of this study suggest that frequent quizzing is an important component of college-level teaching at the developmental level.
Administering frequent quizzes in developmental classes can better prepare students to enroll in credit-bearing courses in which frequent quizzes are administered. If students feel more confident in their abilities, then they might take much greater advantage of opportunities provided to them and graduate from college in a timely manner.
Footnotes
Acknowledgements
Deniz Gokcora would like to thank Dr. Gay Brooks and Kate Walter for their support in doing this study.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.