Restricted accessResearch articleFirst published online 2026-1
Practice Still Makes Proficient: A Replication of Kershaw et al. (2018,2023;Kershaw & Gordon,2024) Using an Alternate Form of the Journal Article Comprehension Assessment
Recent research suggests that empirical research comprehension can be improved through structured assignments that distribute deliberate practice opportunities over the semester.
Objective
This recent research used identical Journal Article Comprehension (JAC) assessments of this structured practice in a pretest/posttest design. The current study's aim was to develop an alternate form of the JAC assessment.
Method
Across four semesters, 125 students completed one form of the JAC at the pretest and another at the posttest. These different forms were matched on question type and answer options.
Results
All groups improved in empirical research comprehension between the pretest and posttest, as shown through the total JAC score. This improvement occurred regardless of which combination of JAC articles they received at the pretest or posttest.
Conclusion
The findings suggest that both JACs are equivalent measures of empirical research comprehension. The current results also replicate past research showing overall improvement, that improvement is not just a practice effect, and that improvement can occur with only a few practice opportunities.
Teaching Implications
The equivalence of the new form of the JAC gives educators an alternate way to assess empirical research comprehension within the context of a semester or to assess the transfer of skills across the curriculum.
ApplebyD. C. (2018). Preparing psychology majors to enter the workforce: Then, now, with whom, and how. Teaching of Psychology, 45(1), 14–23. https://doi.org/10.1177/0098628317744944
4.
BachiochiP.EvertonW.EvansM.FugereM.EscotoC.LettermanM.LeszczynskiJ. (2011). Using empirical article analysis to assess research methods courses. Teaching of Psychology, 38(1), 5–9. https://doi.org/10.1177/0098628310387787
5.
BeilockS. L.CarrT. H. (2005). When high-powered people fail: Working memory and “choking under pressure” in math. Psychological Science, 16(2), 101–105. https://doi.org/10.1111/j.0956-7976.2005.00789.x
6.
BreslinC. W.SaferM. A. (2011). Effects of event valence on long-term memory for two baseball championship games. Psychological Science, 22(11), 1408–1412. https://doi.org/10.1177/0956797611419171
7.
CalamiaM.MarkonK.TranelD. (2012). Scoring higher the second time around: Meta-analyses of practice effects in neuropsychological assessment. The Clinical Neuropsychologist, 26(4), 543–570. https://doi.org/10.1080/13854046.2012.680913
8.
ClauseC. S.MullinsM. E.NeeM. T.PulakosE.SchmittN. (1998). Parallel test form development: A procedure for alternate predictors and an example. Personnel Psychology, 51(1), 193–208. https://doi.org/10.1111/j.1744-6570.1998.tb00722.x
9.
CohenJ. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Erlbaum.
10.
CoilD.WenderothM. P.CunninghamM.DirksC. (2010). Teaching the process of science: Faculty perceptions and an effective methodology. CBE—Life Sciences Education, 9(4), 524–535. https://doi.org/10.1187/cbe.10-01-0005
11.
DunloskyJ.RawsonK. A.MarshE. J.NathanM. J.WillinghamD. T. (2013). Improving students’ learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the Public Interest, 14(1), 4–58. https://doi.org/10.1177/1529100612453266
12.
FaulF.ErdhelderE.LangA.-G.BuchnerA. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39(2), 175–191. https://doi.org/10.3758/BF03193146
13.
GoudsouzianL. K.HsuJ. L. (2023). Reading primary scientific literature: Approaches for teaching students in the undergraduate STEM classroom. CBE—Life Sciences Education, 22(3), 1–13. https://doi.org/10.1187/cbe.22-10-0211
14.
HalonenJ. S.DunnD. S. (2018). Embedding career issues in advanced psychology major courses. Teaching of Psychology, 45(1), 41–49. https://doi.org/10.1177/0098628317744967
15.
HausknechtJ. P.HalpertJ. A.Di PaoloN. T.Moriarty GerrardM. O. (2007). Retesting in selection: A meta-analysis of coaching and practice effects for tests of cognitive ability. Journal of Applied Psychology, 92(2), 373–385. https://doi.org/10.1037/0021-9010.92.2.373
KershawT. C.FugateJ. M. B.O’HareA. J. (2023). Teaching undergraduates to understand published research through structured practice in identifying key research concepts. Scholarship of Teaching and Learning in Psychology, 9(2), 216–233. https://doi.org/10.1037/stl0000239
18.
KershawT. C.GordonL. T. (2024). Practice assignments improve research comprehension skills and reinforce concept learning. Scholarship of Teaching and Learning in Psychology, 10(4), 536–551. https://doi.org/10.1037/stl0000337
19.
KershawT. C.LippmanJ. P.FugateJ. M. B. (2018). Practice makes proficient: Teaching undergraduate students to understand published research. Instructional Science, 46(6), 921–946. https://doi.org/10.1007/s11251-018-9456-2
20.
MillisK.HalpernD.WiemerK.WallaceP. (2021). Evaluating research summaries. In Society for the Teaching of Psychology, Resources for teachers of psychology. https://teachpsych.org/page-1603066.
SegoS. A.StuartA. E. (2016). Learning to read empirical articles in general psychology. Teaching of Psychology, 43(1), 38–42. https://doi.org/10.1177/0098628315620875
StoaR.ChuT. L.GurungR. A. R. (2022). Potential potholes: Predicting challenges and learning outcomes in research methods in psychology courses. Teaching of Psychology, 49(1), 21–29. https://doi.org/10.1177/0098628320979881
25.
Van LacumE. B.OssevoortM. A.GoedhartM. J. (2014). A teaching strategy with a focus on argumentation to improve undergraduate students’ ability to read research articles. CBE – Life Sciences Education, 13(2), 253–264. https://doi.org/10.1187/cbe.13-06-0110
