Test frequency, stakes associated with educational tests, and feedback from test results have been identified in the research literature as relevant factors in student achievement.
Objectives:
Summarize the separate and joint contribution to student achievement of these three treatments and their interactions via multivariable meta-analytic techniques using a database of English-language studies spanning a century (1910–2010), comprising 149 studies and 509 effect size estimates.
Research design:
Analysis employed robust variance estimation. Considered as potential moderators were hundreds of study features comprising various test designs and test administration, demographic, and source document characteristics.
Subjects:
Subjects were students at all levels, from early childhood to adult, mostly from the United States but also eight other countries.
Results:
We find a summary effect size of 0.84 for the three treatments collectively. Further analysis suggests benefits accrue to the incremental addition of combinations of testing and feedback or stakes and feedback. Moderator analysis shows higher effect sizes associated with the following study characteristics: more recent year of publication, summative (rather than formative) testing, constructed (rather than selected) item response formats, alignment of subject matter between pre- and posttests, and recognition/recall (rather than core subjects, art, or physical education). Conversely, lower effect sizes are associated with postsecondary students (rather than early childhood–upper secondary), special education population, larger study population, random assignment (rather than another sampling method), use of shadow test as outcome measure, designation of individuals (rather than groups) as units of analysis, and academic (rather than corporate or government) research.
AckermanP. L.KanferR. (2009). Test length and cognitive fatigue: An empirical examination of effects on performance and test-taker reactions. Journal of Experimental Psychology: Applied, 15, 163–181.
2.
AgarwalP. K.D’AntonioL.RoedigerH. L.McDermottK. B.McDanielM. A. (2014). Classroom-based programs of retrieval practice reduce middle school and high school students’ test anxiety. Journal of Applied Research in Memory and Cognition, 3, 131–139.
3.
AgarwalP. K.RoedigerH. L.IIIMcDanielM. A.McDermottK. B. (2013). How to use retrieval practice to improve learning. St. Louis, MO: Washington University.
4.
American Educational Research Association (AERA), American Psychological Association (APA), and National Council on Measurement in Education (NCME). (1999). Standards for educational and psychological testing. Washington, DC: AERA.
5.
American Educational Research Association (AERA), American Psychological Association (APA), and National Council on Measurement in Education (NCME). (2014). Standards for educational and psychological testing. Washington, DC: AERA.
6.
Bangert-DrownsR. L.KulikJ. A.KulikC. L. C. (1991, November/December). Effects of frequent classroom testing. Journal of Educational Research, 85, 89–99.
7.
BasolG.JohansonG. (2009). Effectiveness of frequent testing over achievement: A meta-analysis study. International Journal of Human Sciences [Online], 6, 99–121.
8.
BertschS.PestaB. J.WiscottR.McDanielM. A. (2007). The generation effect: A meta-analytic review. Memory & Cognition, 35, 201–210.
9.
BishopJ. H. (2000). Curriculum-based external exit exams systems: Do students learn more?How? Psychology, Public Policy, & Law, 6, 199–215.
10.
BishopJ. H. (2004, July). High school diploma exams: Explaining high achievement levels in students of some Commonwealth countries. Fraser Forum, 15–17.
11.
BishopJ. H.ManeF.BishopM.MoriatryJ. (2001). The role of end-of-course exams and minimum competency exams in standards-based reforms. In RavitchD. (Ed.), Brookings papers in education policy (pp. 267–330). Washington, DC: Brookings Institution.
12.
BlackP.WiliamD. (2014). Inside the black box: Raising standards through classroom assessment. London, England: Learning Sciences International.
13.
BlockJ. H.BurnsR. B. (1976). Mastery learning. In ShulmanL. S. (Ed.), Review of research in education (pp. 3–49). Itasca, IL: F.E. Peacock.
14.
BorensteinM.HedgesL.HigginsJ.RothsteinH. (2009). Introduction to meta-analysis. New York, NY: Wiley.
15.
BoudD.MolloyE. (2012, May). Rethinking models of feedback for learning: The challenge of design. Assessment & Evaluation in Higher Education, 38, 698–712.
16.
BrownP. C.RoedigerH. L.McDanielM. A. (2014). Make it stick: The science of successful learning. Cambridge, MA: Belknap Press.
17.
BrownS. M.WalbergH. J. (1993, January/February). Motivational effects on test scores of elementary students. Journal of Educational Research, 86, 133–136.
18.
CameronJ.PierceW. D. (1994, Fall). Reinforcement, reward, and intrinsic and extrinsic motivation: A meta-analysis. Review of Educational Research, 64, 363–423.
19.
ColeJ. S.BerginD. A.WhittakerT. A. (2008, January). Predicting student achievement for low stakes tests with effort and task value. Contemporary Educational Psychology, 33, 609–624.
20.
CooperH. M. (1989). Integrating research: A guide for literature reviews. Newbury Park, CA: Sage.
21.
CrooksT. J. (1988, Winter). The impact of classroom evaluation practices on students. Review of Educational Research, 58, 438–481.
22.
Darling-HammondL. (2014). Beyond the bubble test: How performance assessments support 21st century learning. San Francisco, CA: Jossey-Bass.
23.
DraperS. (2009). What are learners actually regulating when they are given feedback?British Journal of Educational Technology, 40, 306–315.
24.
EklofH. (2007). Test-taking motivation and mathematics performance in TIMSS 2003. International Journal of Testing, 7, 311–326. doi:10.1080/15305050701438074d
25.
Evans. C. (2013). Making sense of assessment feedback in higher education. Review of Educational Research, 83, 70–120.
GocmanG. B. (2003). Effectiveness of frequent testing over academic achievement: A meta-analysis study. Dissertation presented to the faculty of the College of Education Ohio University in partial fulfillment of the requirements for the degree doctor of philosophy. UMI #3099579
28.
GrevingS.RichterT. (2018, December). Examining the testing effect in university teaching: Retrievability and question format matter. Frontiers in Psychology, 9, 1–10.
29.
GuskeyT. R.GatesS. L. (1986). Synthesis of research on the effects of mastery learning in elementary and secondary classrooms. Educational Leadership, 43, 73–80.
30.
GuskeyT. R.PigottT. D. (1988, March–April). Research on group-based mastery learning programs: A meta-analysis. Journal of Educational Research, 81, 197–216.
31.
HallJ.RosenthalR.Tickle-DegnenL.MostellerF. (1994). Hypotheses and problems in research synthesis. In CooperH.HedgesL. (Eds.), The handbook of research synthesis. New York, NY: Russell Sage Foundation.
32.
HarrisL.BrownG.HarnettJ. (2014). Understanding classroom feedback practices: A study of New Zealand student experiences, perceptions and emotional responses. Educational Assessment, Evaluation and Accountability, 26, 107–133.
33.
HattieJ.TimperleyH. (2007, March). The power of feedback. Review of Educational Research, 77, 81–112. doi:10.3102/003465430298487
34.
HawthorneK. A.BolL.PribeshS.SuhY. (2015, Summer). Effects of motivational prompts on motivation, effort, and performance on a low-stakes standardized test. Research and Practice in Assessment, 10, 30–38.
35.
HedgesL. V.TiptonE.JohnsonM. C. (2010). Robust variance estimation in meta-regression with dependent effect size estimates. Research Synthesis Methods, 1, 39–65.
36.
HembreeR. (1988). Correlates, Causes, Effects and Treatment of test anxiety. Review of Educational Research, 58, 47–77.
37.
HigginsJ. (2008). Commentary: Heterogeneity in meta-analysis should be expected and appropriately quantified. International Journal of Epidemiology, 37, 1158–1160.
38.
HigginsJ.ThompsonS.DeeksJ.AltmanD. (2003). Measuring inconsistency in meta-analyses. BMJ, 327, 557–560.
KirklandM. C. (1971). The effects of tests on students and schools. Review of Educational Research, 41, 303–350.
41.
KlugerA.de NisiA. (1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119, 254–284.
42.
KulikC. L.KulikJ. A. (1987). Mastery testing and student learning: A meta-analysis. Journal of Educational Technology Systems, 15, 325–345.
43.
KulikJ. A.KulikC. C. (1988). Timing of feedback and verbal learning. Review of Educational Research, 58, 79–97.
44.
LarsenD. P.ButlerA. C. (2013). Test-enhanced learning. In WalshK. (Ed.), Oxford textbook of medical education (pp. 443–452). Oxford, England: Oxford University Press.
45.
LeightonJ. P. (2009). Mistaken impressions of large-scale diagnostic testing. In PhelpsR. P. (Ed.), Correcting fallacies about educational and psychological testing (pp. 219–246). Washington, DC: American Psychological Association.
46.
LigonG. D. (2007). Why Eva Baker doesn’t seem to understand accountability: The politimetrics of accountability. Nonpartisan Education Review/Articles, 3, 1–34.
47.
LipnevichA.SmithJ. (2008). Response to assessment feedback: The effects of grades, praise, and source of information (ETS Research Report RR-08–30). Rutgers, The State University of New Jersey.
48.
LipseyM.WilsonB. (2001). Practical meta-analysis. Thousand Oaks, CA: Applied Social Research Methods Series, Sage.
49.
LockeE. A.LathamG. P. (2002, September). Building a practically useful theory of goal setting and task motivation: A 35-year odyssey. American Psychologist, 57, 705–717.
50.
MarshR. (1984, November/December). A comparison of take-home versus in-class exams. Journal of Educational Research, 78, 111–113.
51.
MarzanoR. (2006). Classroom assessment and grading that work. Alexandria, VA: ASCD.
52.
McDanielM.AndersonJ.DerbishM.MorrissettN. (2007). Testing the testing effect in the classroom. European Journal of Cognitive Psychology, 19, 494–513.
53.
NatrielloG.DornbuschS. M. (1984). Teacher evaluative standards and student effort. New York, NY: Longman.
54.
NicolD.Macfarlane-DickD. (2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 31, 199–218.
55.
OrwinR. G. (1983). A fail-safe N for effect size in meta-analysis. Journal of Educational Statistics, 8, 157–159. doi:10.2307/1164923
56.
PriceM.HandleyK.MillarJ.O’DonovanB. (2010). Feedback: All that effort, but what is the effect?Assessment & Evaluation in Higher Education, 35, 277–289.
57.
ProgerB. B.MannL. (1973). An historical review of theoretical and experimental literature on the teaching values of informal (non-standardized), teacher-made achievement tests: 1913-1968(ERIC Document Reproduction Service No. ED 084 292). Blue Bell, PA: Montgomery County Intermediate Unit 23.
58.
RamsheM. H. (2014, November). A review of the studies on the frequent administrations of English tests. Journal of Language Teaching and Research, 5, 1412–1416. doi:10.4304/jltr.5.6.1412–1416
59.
ResnickD. P.ResnickL. B.(1985). Standards, curriculum, and performance: A historical and comparative perspective. Educational Researcher, 14, 5–20.
60.
RhodesW. (2012). Meta-Analysis: An introduction using regression models. Evaluation Review, 36, 24–71.
RoedigerH. L.IIIPutnamA.SmithM. (2011). Ten benefits of testing and their application to educational practice. In MestreJ.RossB. (Eds.), The psychology of learning and motivation, Cognition in Education (Vol. 55). San Diego, CA: Elsevier.
63.
RosenthalR. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86, 638–641.
64.
RosenthalR. (1984). Meta-analytic procedures for social research, Applied Social Research Methods Series(Vol. 6). Newbury Park, CA: Sage.
65.
SegoolN.CarlsonJ.GoforthA.Von der EmbseN.BarterianJ. (2013). Heightened test anxiety among young children: Elementary school students´ anxious responses to high-stakes testing. Psychology in the Schools, 50, 489–499.
SteedleJ. T. (2018, November). Keeping your cool: Does test anxiety bias performance on the ACT? (ACT Research Report|R1713). Iowa City, Iowa: ACT.
68.
Tanner-SmithE. E.TiptonE. (2014). Robust variance estimation with dependent effect sizes: Practical consideration including a software tutorial in Stata and SPSS. Research Synthesis Methods, 5, 13–30.
69.
Tanner-SmithE. E.WilsonS. J.LipseyM. W. (2013). The comparative effectiveness of outpatient treatment for adolescent substance abuse: A meta-analysis. Journal of Substance Abuse Treatment, 44, 145–158.
70.
UttalD. H.MeadowN. G.TiptonE.HandL. L.AldenA. R.WarrenC.NewcombeN. S. (2013). The malleability of spatial skills: A meta-analysis of training studies. Psychological Bulletin, 139, 352–402.
71.
von der EmbseN.HassonR. (2012). Test anxiety and high-stakes test performance between school settings: Implications for educators. Preventing School Failure, 56, 180–187.
72.
WildemuthB. M. (1977). Minimal competency testing: Issues and procedures, an annotated bibliography. ERIC ED150188.
73.
WinstoneN. E.NashR. A.ParkerM.RowntreeJ. (2017). Supporting learners’ agentic engagement with feedback: A systematic review and a taxonomy of recipience processes. Educational Psychologist, 52, 17–37. doi:10.1080/00461520.2016.1207538
74.
WiseS. L. (2006). An investigation of the differential effort received by items on a low stakes computer-based test. Applied Measurement in Education, 19, 95–114.
75.
WiseS. L.DeMarsC. E. (2005). Low examinee effort in low-stakes assessment: Problems and potential solutions. Educational Assessment, 10, 1–17. doi:10.1207/s15326977ea1001_1
76.
WiseS. L.DeMarsC. E. (2010). Examinee noneffort and the validity of program assessment results. Educational Assessment, 15, 27–41. doi:10.1080/10627191003673216
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
