Abstract
Two studies examined the effectiveness of a flashcard-based study strategy, Flashcards-Plus, in an ecologically valid context. The strategy requires students to create flashcards designed to increase their ability to retain, comprehend, and apply textbook material to exams. In Studies 1a (n = 73) and 1b (n = 62), we introduced all students to the Flashcards-Plus method and compared their exam scores. Students who used this strategy scored significantly higher than those who did not. In Study 2 (n = 434), we randomly assigned six introductory psychology courses to either receive a classroom lecture with the Flashcards-Plus strategy (i.e., three experimental courses) or no lecture (i.e., three control courses). Students in the experimental courses scored significantly higher than those in the control courses after the lecture. The results from all three studies demonstrate that students who were introduced to the Flashcards-Plus study strategy scored significantly higher on exams following the lecture than students who were not. These findings suggest that this easily implemented teaching strategy can help students achieve deeper levels of processing (i.e., comprehension and application) in a self-directed manner, which benefit students’ performance.
Introduction
The study techniques students use can affect their exam performance (Dunlosky, Rawson, Marsh, Nathan, & Willingham, 2013; Hattie, 2012, p. 265), but not all studying techniques are equally effective (Balch, 1998; Dunlosky et al., 2013; Gurung & McCann, 2012; Hackathorn et al., 2012). We focused on flashcards, a commonly used study method (Hartwig & Dunlosky, 2012) by studying the effectiveness of a new way to use flashcards, the Flashcards-Plus (FP) technique (Appleby, 2013), in two different naturalistic classroom environments. FP expands the use of flashcards to help students understand and prepare for three types of multiple-choice questions commonly found on psychology tests: those that measure retention, comprehension, and application.
In reviewing the efficacy of 10 commonly used studying techniques, Dunlosky and colleagues (2013) identified practice testing (or practice retrieval) and distributed practicing (or spaced practice) as the two most effective techniques. Students can engage in both of these high utility techniques through the use of flashcards (Wissman, Rawson, & Pyc, 2012). In the basic form of flashcards, a term or concept appears on one side of an index card and its definition appears on the other side. Students look at the term and test whether they know the answer, flipping the card to check. Surprisingly little classroom research on college student flashcard use exists and has yielded mixed results (Golding, Wasarhaley, & Fletcher, 2012; Hartwig & Dunlosky, 2012).
Although flashcards are relatively easy to create and commonly used by college students (Wissman et al., 2012), the effectiveness of student-created flashcards in a classroom is unclear. In an introductory psychology class, Golding et al. (2012) found that students who used flashcards for the first exam scored higher than students who did not. However, the use of flashcards was not beneficial on the second exam, and the effect was only marginal on the third exam. Furthermore, Hartwig and Dunlosky (2012) demonstrated that student-created flashcards use was unrelated to students’ grade point average (GPA). These results suggest that retrieval practice and distribution of practice increases retention of information, but the way in which flashcards are typically used may not be sufficient to be successful in college courses.
One possible reason for the ineffectiveness of the traditional use of flashcards is that students (70%) use self-testing techniques such as flashcards to determine how well they have learned the current material instead of to learn more (18%, Karpicke, Butler, & Roediger, 2009). Flashcards can facilitate repetitive learning, but repetition is only a superficial level of processing (Brown, Roediger, & McDaniel, 2014). While 82.9% of students in a study indicated they used flashcards to remember vocabulary, no students reported using flashcards to develop deeper understanding of or the application of concepts (Wissman et al., 2012).
When students use deep-level processing, they focus on substance and the underlying meaning of the information, make a personal commitment to understanding, and concentrate on relationships between different pieces of information they encounter (Entwistle, 2009). Students also strive to apply what they are learning to their everyday lives and integrate and synthesize information with their prior learning, essentially moving up to the next level of Bloom’s (revised) hierarchy of learning (Anderson & Krathwohl, 2001). Moreover, deep learning is associated with more enjoyable learning, whereas the surface approach tends to be less satisfying (Biggs, 2003; Tagg, 2003). The FP method goes beyond the traditional use of flashcards to facilitate deeper understanding and increase learning.
The FP strategy begins in the same way as traditional flashcards. Students identify bold-faced terms from the textbook and write them on one side of a notecard. Students write the textbook definition on the other side. Students memorize the definition from the textbook (or the instructor) to increase retention. While most students stop here with traditional flashcards, FP creates deeper levels of learning by having the student perform two more actions. In the next step, students write a definition for the same key term in their own words. Rephrasing the term helps students understand the material that will increase comprehension. Finally, students generate a realistic example of the key term from their own lives that will increase application.
Appleby (2013) originally designed the FP strategy as a study aid for multiple-choice question exams to promote active learning in students, particularly those enrolled in large introductory courses. The strategy employs a scaffolding technique (Vygotsky, 1978), during which students learn how to move beyond their current developmental skill set and master more difficult tasks. While most exam questions assess students’ retention, comprehension, and application (Gronlund, 1998; Palmoba & Banta, 1999), many students need support to move beyond simple retention to understand and apply the material (Appleby, 2008, 2013). The FP strategy offers a clear explanation of the importance of the higher-level understanding and a strategy to achieve this goal in a self-directed manner. Instructors can find full details on how to use the FP strategy online (with downloadable PowerPoint slides): http://teachpsych.org/Resources/Documents/otrp/resources/appleby13flashcard.pdf/. The introduction of the slides takes approximately 10 minutes in class.
The FP strategy involves one of the most well-known memory-facilitating processes. Remembering is enhanced when the meaning of material is fully processed at the time of encoding (e.g., Craik, 1979, 2002; Craik & Lockhart, 1972; Paller, Kutas, & Mayes, 1987). For example, students who are asked to create their own explanations of a concept recalled that information better than those who were given explanations passively (Pressley, McDaniel, Turner, Wood, & Ahmad, 1987). Furthermore, information processed in relation to one’s self results in particularly strong recall (Klein & Loftus, 1988; Rogers, Kuiper, & Kirker, 1977).
It is, however, also well known that engaging in practice retrieval significantly increases long-term retention of knowledge (Karpicke & Blunt, 2011; Karpicke et al., 2009; Karpicke & Roediger, 2010; Kornell, Hays, & Bjork, 2009). Interestingly, researchers suggest that an essential component of retrieval-based learning is remembering the context of learning; that is, the act of thinking back to what occurred in a particular place and time in association to the concept to be learned (Karpicke, Lehman, & Aue, 2014; Tulving, 1983). Taken together, retrieval effects can be even stronger if the information is related to students’ own experiences.
Finally, a large number of studies have shown that explicit instruction to provide basic tools for academic skills is a crucial characteristic of pedagogical scaffolding (Archer & Hughes, 2011). The FP strategy is an ideal application of these empirical studies to help students engage in meaningful learning that can be implemented in the classroom. With FP, students: (1) receive an explicit instruction of what is expected; (2) develop an example and apply the concepts they are required to learn; and (3) engage in practice retrieval of all three levels (i.e., retention, comprehension, and application) of the assigned material.
In the current two studies, we tested the efficacy of the FP strategy. Studies 1a and 1b used quasi-experimental designs comparing students who volunteered to use the method with those who did not. In Study 2, we randomly assigned students to either receive instruction regarding the FP method or not. In these two studies, we hypothesized that students who used the method (Study 1) and students who were exposed to the FP method (Study 2) would learn better, which we operationalized as higher scores on their course exams.
Study 1
Study 1 had three goals. We sought to obtain an initial assessment of the effectiveness of the FP strategy on the exam performance, whether this strategy was related to any other studying techniques, and whether students’ perceptions of using FP influenced their performance (Study 1a). In Study 1, students chose to use the FP or not as extra credits, and we expected students who used the FP method to improve their performance on the exam after implementing the studying strategy. In Study 1b, we aimed at replicating the results by controlling for the timing of the introduction of the FP studying method, and also gather qualitative data to further understand students’ experience with the FP method.
Study 1a
Method
Participants
Seventy-three undergraduate students from two sections of an upper level course in Human Development at a midsized Midwestern public university participated in the study. The first author taught both classes. Students were predominantly women (n = 62) and juniors (n = 40), followed by seniors (n = 20) and finally sophomores (n = 13). The mean age was 21.5 years (standard deviation (SD) = 3.25, range = 19–41) and the mean self-reported GPA was 3.0 (SD = 0.51).
Materials and Procedure
The study took place over the course of one semester, and we measured performance on each exam as the outcome. There were three non-cumulative exams, each consisting of approximately 30 multiple-choice questions and four short-answer questions. All students took the first exam before we delivered instructions for the FP strategy. After Exam 1, students in Section 1 were given access to Appleby’s original PowerPoint slides (Appleby, 2013) via an online course management system, Desire 2 Learn. After Exam 2 the instructor gave students in both sections detailed instruction on the FP strategy with the PowerPoint slides in class, to encourage active exposure. The instructor also emphasized the rationale behind the flashcards studying strategy and provided extra credit if they used the FP strategy to study for Exam 3. Students submitted their flashcards to the instructor to ensure they followed the three steps as described. The same instructor taught both sections, and the only difference between sections being when the FP method was introduced.
At the end of the semester students completed a studying strategy survey, assessing FP method use (e.g., hours per week and perceptions of the helpfulness and difficulty). The survey also asked about the use of twelve other studying techniques: summarizing material; highlighting/underlining; memorizing keywords using mnemonics; using imagery for text; explaining material to yourself; rereading; practice testing; spreading out study sessions (rather than cramming); studying material with a friend; making up examples to help understand; applying material to their lives; and taking notes on the book (modified from Bartolewski & Gurung, 2014). The reliability of this self-report scale was relatively high (Cronbach’s α = 0.79). Finally, we asked students to think back to the beginning of the semester and report how motivated they were to take the course at that time. Students provided a signed informed consent to complete the survey.
Results and Discussion
Means (and Standard Deviations (SDs)) of Exam Scores and Grade Point Average (GPA) Separated by Sections in Studies 1a and 1b.
We examined whether exam performance improved significantly as a result of FP use. A mixed-model analysis of variance (ANOVA) with Exam as the repeated-measure variable and FP (Users vs. Non-users) the between-subject variable yielded a main effect of Exam, F(1, 71) = 48.44, p < 0.001, Mean (and standard errors) exam scores on Exams 1 and 3 from Flashcards-Plus users and non-users in Study 1a.
Motivation Level and Studying Methods in Study 1a.
p < 0.05; **p = 0.079.
Students’ perception of the easiness of the FP strategy (M = 1.97, SD = 0.78) significantly correlated with their exam score, r(39) = 0.53, p < 0.001. Students who indicated that the use of FP method was easy scored higher than students who indicated that the method was difficult. Students’ perception of the helpfulness (M = 3.23, SD = 0.71) and the hours they spent on the method (M = 3.33, SD = 2.46; range = 0–10 hours) were not correlated with their exam scores.
Study 1b
In Study 1b we better controlled the timing of the introduction of this strategy to the students. We also asked whether students have used flashcards to study in the past and to further indicate their experience with the FP studying method in an open-ended format.
Method
Participants
Sixty-two students in two sections of an upper level course in Human Development at a midsized Midwestern public university participated in the study. Students were predominantly women (n = 52) and sophomores (n = 32), followed by juniors (n = 20) and seniors (n = 7). The mean age was 20.9 years (SD = 4.34, range = 18–43) and the mean self-reported GPA was 3.18 (SD = 0.55).
Materials and Procedure
Study 1b used the same material and followed the same procedure as Study 1a. In both sections, all students received active instruction about the FP method immediately after Exam 2, approximately 7 weeks before Exam 3. At the end of the semester, students completed a survey that asked whether they used the FP study method, the frequency of their use, their perception of its helpfulness, whether they have used flashcards to study before, and an open-ended question about their experience with the FP method. Students received extra credit for completing the survey regardless of their FP method use.
Results and Discussion
We found no significant differences between the exam scores of sections 1 and 2: Exam 1, t(60) = 1.51, p = 0.14, Exam 2, t(60) = 1.01, p = 0.28, and Exam 3, t(60) = 1.82, p = 0.06. Furthermore, the reported GPA did not differ significantly across two sections, t(60) = 1.84, p = 0.07; thus we combined the data from Sections 1 and 2 in the main analyses. After combining both sections, 36 students reported using the FP studying strategy to study for the third exam, and 26 students reported that they did not use this method. GPA between FP users and non-users did not differ, t(60) = 0.81, p = 0.42.
To test the effectiveness of the FP strategy, we conducted a mixed-design ANOVA
2
with Exam as the repeated-measure variable and FP as the between-subject variable. The results revealed a main effect of Exam, F(2, 120) = 10.80, p < 0.001, Mean (and standard errors) exam scores on Exams 1 and 3 from Flashcards-Plus users and non-users in Study 1b.
Students’ Perception of the Helpfulness of the Flashcards-Plus System (Study 1b)
Means (and Standard Deviations (SDs)) of Exam Scores in Percentage Separated by Sections in Study 2
The results of Studies 1a and 1b demonstrated that FP helped students increase exam scores. Although these two studies provide us with rich data in real classrooms, classroom interventions are often difficult to examine, and this intervention was no exception. All students were provided the instructions and, as a result of this, a limitation of the two studies presented here was that the students’ self-selected to use the FP. Moreover, students were offered extra credit, which may have inadvertently led to Hawthorne or novelty effects. To address these questions, Study 2 introduced randomization to examine the effects of exposure to the study strategy.
Study 2
In this study, we randomly selected class sections to receive the FP instruction. We hypothesized that students who received the FP instruction would perform better than students who did not receive the instruction and who studied as they normally did.
Method
Participants
Participants included students enrolled in six different introductory psychology courses at a Midwestern regional state university (n = 434). Different instructors (none of them authors of the current research) taught different sections of the same course. We collected student demographic information via a self-report measure given on the last day of the course. The population was predominantly women (62.4%), and first year (n = 244; 56.2%), but also contained sophomores (n = 87), juniors (n = 40), and seniors (n = 21). The majority of the students were Caucasian/White (74%), however other ethnicities were also represented: African-American/Black (9.7%); Asian (3.2%); Hispanic (1.8%); biracial (2.1%); Middle Eastern (1.2%); Native American (0.5); and others (1.2%).
Materials and Procedure
During the semester, the second author visited each classroom to provide a short lecture regarding improving studying skills (e.g., distributed rehearsal, avoiding distractions, and situation dependent memory). We randomly assigned three (n = 231 students) of the six sections to receive the introduction of the FP strategy. In the three experimental sections, the students received the same active instruction regarding the FP as in Study 1. The control sections did not receive any instructions regarding studying with flashcards. None of the students were offered extra credit for employing any study strategy; they were simply informed that using any of these strategies could result in positive outcomes. This was to control for potential Hawthorne or good participant biases.
Additionally, the experimental sections received the FP lecture at different times in the semester. One section received the instructions after Exam 1 (n = 79), one received the instructions near midterm immediately following Exam 3 (n = 81), and one section received the instructions prior to the Final Exam (n = 67). The way that we delivered the intervention to three classes at different times is based on the time-series experiments, particularly multiple baseline studies (Biglan, Ary, & Wagenaar, 2000). This design is useful in fields such as community intervention research and education (Kratochwill, 1978), because if the intervention is introduced at the same point of time for many groups, change in the dependent variable could be due to numerous other factors that co-occur with the change in the independent variable. For example, a particular exam that occurs before or after the intervention can be particularly easy or difficult. By introducing the intervention at a different point in time, we can avoid this limitation and be more confident in the results of change in exam performance.
At the end of the semester, the author who gave the study strategy lecture as well as a graduate student returned to each class. Students were given a hard copy of an informed consent form. It was explained that the effectiveness of multiple teaching strategies was being examined and that access to exam grades and the contents of a brief survey would be helpful in the investigation. Students were informed that completing the survey would indicate consent. Finally, students were told that the information would be entered by the graduate student, and the instructor would never see who consented and who did not.
Following the consent process, students completed a brief survey that asked whether they used the FP studying method and their frequency of studying. In total, 39 students did not complete the survey which indicated non-consent, absence, or potentially dropped enrollment. The procedure was approved by the institutional review board.
Results
Effects on Exam Scores
As each of the experimental sections received the FP lecture at different times and the number of total exams varied (either four or five) across sections, exam scores were recorded so we could compare how well students performed before and after the lecture that introduced the FP method. The mean exam scores (in percentage) and the SDs were listed in Table 4. Thus, exam scores prior to and following the lecture were averaged. Students in the control courses were subject to the same coding as the experimental course who received the lecture prior to Exam 2, so the scores of Exam 1 were used as pre-scores and the average of Exam 2 through the final exam were used as post-scores. As before, we used arcsine transformation for analyses. The exam scores were then submitted to a 2 (Condition: Experimental vs. Control) × 2 (Exposure: Pre- vs. Post- FP lecture) mixed-model ANOVA.
The results indicate there was a main effect of exposure, F(1, 431) = 38.53, p < 0.001, Mean (and standard errors) averaged exam scores in percentage pre- and post-exposure from control and experimental groups in Study 2.
Flashcard-Plus Use
Because students were introduced to the FP method randomly without any extra credit for using it, we also examined whether or not students used the new system. According to self-report, 48.3% of students reported they used flashcards of any format to study during the semester. Slightly over half (n = 207) reported that they did not use flashcards to study, and the remaining students used them sometimes (n = 107) or all the time (n = 86).
The experimental sections contained 231 students who were exposed to the FP studying strategy. A binary logistic regression using exposure to predict whether or not students will report using flashcards in general was significant, χ2(1, n = 438) = 18.19, p < 0.001, Nagelkerke R2 = 0.06. Specifically, individuals who were exposed to the new study strategy were twice as likely to use flashcards at some point in the semester (B = 0.87, Wald = 17.76, p < 0.001, odds ratio (OR) = 2.38); 37.3% of the students reported using flashcards in the control courses, whereas 58.3% of the students reported using flashcards in the experimental courses, χ2(1, n = 400) = 18.04, p < 0.001.
Additionally, students were asked to describe the type of flashcard system they used., Over one-third of the experimental students (n = 114) who responded to the question described using the FP strategy (38.6%; n = 44). For example, one student responded “concept on front, and book’s definition, my definition, and example on back.” Thus, a second binary logistic regression was conducted, using exposure condition to predict whether or not students will use the FP system. The results indicated the exposure was significant, χ2(1, n = 438) = 32.68, p < 0.001, Nagelkerke R2 = 0.18. Specifically, individuals who were exposed to the new study strategy were 10 times as likely to use the FP study strategy (B = 2.37, Wald = 19.45, p < 0.001, OR = 10.72); only 3.5% (n = 4) of the students reported using a FP technique in the control courses, whereas 28% (n = 46) of the students in the experimental courses reported using the FP system, χ2(1, n = 278) = 27.46, p < 0.001.
Discussion
The advantage of Study 2 was that it experimentally manipulated the instruction of the FP system by randomly assigning exposure over six introductory psychology courses. Our analysis indicates that individuals who are exposed to the FP system were more likely to use flashcards to study in general than those who were not exposed to it, and they were much more likely to use the new FP system than those who were not exposed to it. This duplicates past studies indicating many students may not know how to study for college level exams, but are eager to learn how to do so (Bjork, Dunlosky, & Kornell, 2013).
Our analyses also indicate that introducing students to the system has a positive influence on exam scores and study habits. Overall, students exposed to the FP strategy fared better on exams than those who were not, and their exam grades increased at a greater rate. This may hint that students are actively searching for ways to improve their grades and/or study habits.
It is possible that being exposed to the FP system draws students’ attention to the types of studying that is appropriate for college level questions. We conducted a brief exploratory analysis of the types of studying the students reported to determine if students mentioned higher levels of learning (e.g., application, and creating examples). We found a significant shift in student thinking regardless of whether they used the new flashcard strategy, χ2(n = 298) = 17.29, p < 0.001. That is, the control condition explicitly mentioned higher levels of learning approximately 19% of the time, whereas the experimental conditions mentioned higher levels 81% of the time. This may indicate that although the participants did not explicitly mention using the FP strategy in the survey, they may have inadvertently changed the level at which they study for exams.
General Discussion
The present studies report an initial attempt to examine the effectiveness of the enhanced flashcards studying strategy developed by Appleby (2013) in real classrooms. Study 1 demonstrated that students who used FP improved their exam performance more than students who did not use this strategy. Although we did not randomly assign FP method use to specific sections because we wanted to assess how many students would decide to use it when given the option, we demonstrated that FP users and non-users in the current study were not different in many demographic characteristics including their GPA and general motivation. Importantly, students who used the FP strategy were more likely to create examples of important concepts and engage in practice testing than students who did not use the strategy. FP users’ reports of higher rates of these two studying methods is consistent with the idea that this strategy promotes application of concepts through the creation of original examples and subsequent practice testing of these concepts.
In Study 2, we experimentally manipulated FP use by introducing the techniques to three of six classes, and found that exposed students scored higher than those in the control group. Although the effect size for actual users was relatively small, we believe the results of these studies can provide encouragement for classroom teachers who would like to foster higher-order thinking in their students. Additionally, we believe that because our study is high in ecological validity (i.e., it was conducted in the actual classroom and did not force students to engage in any technique), some of these effect sizes may have been hindered. Future studies that examine this strategy in a laboratory setting may find increased effect sizes due to the maximization of the independent variable. We cautiously infer that the use of the FP method can contribute to increases in exam scores, but urge additional pedagogical research on this topic.
Previous research indicates that when students create their flashcards in a traditional manner, they tend to focus on the materials that they have already learned by checking their knowledge (Karpicke et al., 2009). Our findings add to previous work on flashcards by encouraging students to go beyond simple repetition of a definition to deeper processing. By paraphrasing the textbook definition and then generating an application, FP enables students to ascend Bloom’s taxonomy for deeper understanding, increased application, and potentially longer-term retention.
While the results of the current studies provide possible benefits of the FP strategy in real classroom settings, there were limitations usually associated with research conducted in this environment. First, we did not directly test traditional flashcard use against FP in a detailed manner because previous research has demonstrated that overly focusing on key terms, as fostered by traditional flashcard use, may hurt student grades (Gurung, 2004). Consequently, we did not include a comparison group forced to engage in such an activity. Instead, we opted for a traditional control group (no-treatment) and allowed students to engage in their normal studying behaviors, believing this to be the more appropriate ethical option.
Another limitation is the possible effect of extra credit; in Studies 1a and 1b, students received extra credit to complete the FP system, which may have increased the motivation to try the new method. We attempted to avoid this issue in Study 2 by randomly assigning specific classes to be exposed with the FP system without any incentives to change their study habits, and, as a result, the number of students who completed the FP system (at least explicitly) was smaller than those in Studies 1a and 1b. Part of this problem may be because students are often naturally resistant to changes in their study habits (Langley, 1993). Students may hear about new study strategies, and may even be hopeful and excited to use them. However, as the exam gets closer, anxiety, fear, procrastination, or perhaps stubbornness may influence students to revert back to old habits (Dembo & Seli, 2004). This would have a deleterious effect on the number of students who were willing to try or continue the strategy. In fact, in the self-report analysis students explicitly described re-reading notes, textbooks, or PowerPoint slides; the proportions were higher in the control group (46%) than the experimental group (28%). Future studies could be designed to provide a means of giving both conditions an incentive to study—that would also simultaneously avoid Hawthorne effects—in order to increase the number of students willing to try the new system.
Additional work on establishing the efficacy of even basic flashcard use is important, and future studies should directly test the effect of the traditional, student-created flashcard and the FP method. It would also help us better understand the mechanisms of learning by closely examining the use of FP. Although our current data suggest that the number of hours spent using FP did not predict exam performance, it is plausible that retrospective questions asked at the end of the semester may not have captured the exact studying hours. Finally, future studies may also examine the differential effects of exposure to the FP strategy specifically as compared to a general study skills lecture.
In conclusion, the FP method presents a straightforward way for students to improve scores on multiple-choice exams. The results of the research reported here provide evidence for the use of flashcards, a finding that when shared with students may make it more likely they will use it, and it also provides a more active study strategy than traditional flashcard use. FP arms instructors faced with the repeated question of “How should I study?” with a clear-cut set of answers. Although this method can be improved, our data provide a starting point and strong foundation for the design of future similar interventions in the classroom.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.