We focused on Park's class method, a writing-to-learn approach that integrates writing, peer assessment, and revision activities as preparatory activities, all of which reflect in the grading. However, it did not account for the extent of revisions in grading, which hindered student motivation to revise. Therefore, we developed a revision extent scoring algorithm.
Objective
We aimed to examine the reliability of our algorithm and its effect on students’ revision behavior in an undergraduate psychology class.
Method
We analyzed essays of 86 undergraduate students from two psychology classes: one with the algorithm reflecting grades and one without. We compared the revision extent of two classes and analyzed the correlation between algorithm scores and human raters’ scores.
Results
A strong positive correlation between the algorithm scores and human raters’ scores demonstrated high reliability. Students in the class where the algorithm score included in the grade made more substantial revisions.
Conclusion
Integrating the Park's class method with our proposed algorithm enhances revision behavior, which could contribute to improved writing skills.
Teaching Implications
Results suggest that educators should consider incorporating a revision extent scoring algorithm into the grading of writing-to-learn psychology classes to help students revise more effectively, thereby enhancing their writing skills.
AdamsA. M.SimmonsF.WillisC.PawlingR. (2010). Undergraduate students’ ability to revise text effectively: Relationships with topic knowledge and working memory. Journal of Research in Reading, 33(1), 54–76. https://doi.org/10.1111/j.1467-9817.2009.01432.x
2.
AlthauserR.DarnallK. (2001). Enhancing critical reading and writing through peer reviews: An exploration of assisted performance. Teaching Sociology, 29(1), 23–35. https://doi.org/10.2307/1318780
3.
BeanJ. C. (2011). Backward design: Towards an effective model of staff development in writing in the disciplines. In M. Deane & P. O'Neil (Eds.), Writing in the disciplines: Universities into the 21st century (pp. 215–236). Houndmills. https://doi.org/10.1007/978-0-230-34451-8_13
4.
BeatonA. E.FerrisJ.JohnsonE.JohnsonJ.MislevyR.ZwickR. (1988). The NAEP 1985-86 reading anomaly: A technical report. Educational Testing Service.
5.
BeinsB. C.SmithR. A.DunnD. S. (2010). 16 Writing for psychology majors as a developmental process. In DunnD. S.BeinsB. B.McCarthyM. A.HillI. V.WG. (Eds.), Best practices for teaching beginnings and endings in the psychology major: Research, cases, and recommendations (pp. 253–278). Oxford University Press.
6.
BerggrenJ. (2015). Learning from giving feedback: A study of secondary-level students. ELT Journal, 69(1), 58–70. https://doi.org/10.1093/elt/ccu036
7.
BerndtM.StrijbosJ. W.FischerF. (2018). Effects of written peer-feedback content and sender’s competence on perceptions, performance, and mindful cognitive processing. European Journal of Psychology of Education, 33(1), 31–49. https://doi.org/10.1007/s10212-017-0343-z
8.
ChanquoyL. (2001). How to make it easier for children to revise their writing: A study of text revision from 3rd to 5th grades. British Journal of Educational Psychology, 71(1), 15–41. https://doi.org/10.1348/000709901158370
9.
CheongC. M.LuoN.ZhuX.LuQ.WeiW. (2023). Self-assessment complements peer assessment for undergraduate students in an academic writing task. Assessment & Evaluation in Higher Education, 48(1), 135–148. https://doi.org/10.1080/02602938.2022.2069225
10.
ChoK.ChungT. R.KingW. R.SchunnC. (2008). Peer-based computer-supported knowledge refinement: An empirical investigation. Communications of the ACM, 51(3), 83–88. https://doi.org/10.1145/1325555.1325571
ChoK.MacArthurC. (2011). Learning by reviewing. Journal of Educational Psychology, 103(1), 73–84. https://doi.org/10.1037/a0021950
13.
CotosE. (2015). Automated writing analysis for writing pedagogy: From healthy tension to tangible prospects. Writing and Pedagogy, 7(2-3), 197–232. https://doi.org/10.1558/wap.v7i2-3.26381
14.
CovillA. E. (2010). Comparing peer review and self-review as ways to improve college students’ writing. Journal of Literacy Research, 42(2), 199–226. https://doi.org/10.1080/10862961003796207
DrabickD. A.WeisbergR.PaulL.BubierJ. L. (2007). Keeping it short and sweet: Brief, ungraded writing assignments facilitate learning. Teaching of Psychology, 34(3), 172–176. https://doi.org/10.1080/00986280701498558
Finkenstaedt-QuinnS. A.PolakowskiN.GundersonB.ShultzG. V.GereA. R. (2021). Utilizing peer review and revision in STEM to support the development of conceptual knowledge through writing. Written Communication, 38(3), 351–379. https://doi.org/10.1177/07410883211006038
19.
Finkenstaedt-QuinnS. A.WattsF. M.ShultzG. V. (2024). Reading, receiving, revising: A case study on the relationship between peer review and revision in writing-to-learn. Assessing Writing, 59, 100808. https://doi.org/10.1016/j.asw.2024.100808
20.
FisherM.OppenheimerD. M. (2021). Harder than you think: How outside assistance leads to overconfidence. Psychological Science, 32(4), 598–610. https://doi.org/10.1177/0956797620975779
21.
FlowerL.HayesJ. R. (1980). The cognition of discovery: Defining a rhetorical problem. College Composition & Communication, 31(1), 21–32. https://doi.org/10.58680/ccc198015963
22.
FreedmanA.PringleI. (1980). Writing in the college years: Some indices of growth. College Composition & Communication, 31(3), 311–324. https://doi.org/10.58680/ccc198015943
23.
FreedmanS. W.HullG. A.HiggsJ. M.BootenK. P. (2016). Teaching writing in a digital and global age: Toward access, learning, and development for all. In D. Gitomer & C. Bell (Eds.), Handbook of research on teaching (pp. 1389–1450). Casemate Group. https://doi.org/10.3102/978-0-935302-48-6_23
GambhirM.GuptaV. (2017). Recent automatic text summarization techniques: A survey. Artificial Intelligence Review, 47(1), 1–66. https://doi.org/10.1007/s10462-016-9475-9
26.
GaoY.SchunnC. D. D.YuQ. (2019). The alignment of written peer feedback with draft problems and its impact on revision in peer assessment. Assessment & Evaluation in Higher Education, 44(2), 294–308. https://doi.org/10.1080/02602938.2018.1499075
27.
GrahamS.HarrisK. R.AdkinsM.CampingA. (2021). Do content revising goals change the revising behavior and story writing of fourth grade students at-risk for writing difficulties?Reading and Writing, 34(7), 1915–1941. https://doi.org/10.1007/s11145-021-10142-9
28.
GrahamS.HarrisK. R.MacArthurC.SantangeloT. (2017). Self-regulation and writing. In D. H. Schunk, J. A. Greene, & D. H. Schunk (Eds.), Handbook of self-regulation of learning and performance (pp. 138–152). Routledge.
29.
GrahamS.PerinD. (2007). A meta-analysis of writing instruction for adolescent students. Journal of Educational Psychology, 99(3), 445–476. https://doi.org/10.1037/0022-0663.99.3.445
30.
GravesB.C. (2023). Specifications grading to promote student engagement, motivation and learning: Possibilities and cautions. Assessing Writing, 57, 100754. https://doi.org/10.1016/j.asw.2023.100754
GrimesD.WarschauerM. (2010). Utility in a fallible tool: A multi-site case study of automated writing evaluation. The Journal of Technology, Learning and Assessment, 8(6), 1–44.
GupteT.WattsF. M.Schmidt-McCormackJ. A.ZaimiI.GereA. R.ShultzG. V. (2021). Students’ meaningful learning experiences from participating in organic chemistry writing-to-learn activities. Chemistry Education Research and Practice, 22(2), 396–414. https://doi.org/10.1039/D0RP00266F
35.
HackerD. J.KiuharaS. A.LevinJ. R. (2019). A metacognitive intervention for teaching fractions to students with or at-risk for learning disabilities in mathematics. ZDM, 51(4), 601–612. https://doi.org/10.1007/s11858-019-01040-0
36.
HarrisS. G. K. R. (2023). The role and development of self-regulation in the writing process. In SchunkD. H.ZimmermanB. J. (Eds.), Self-regulation of learning and performance (pp. 203–228). Routledge.
HewettB. L. (2006). Synchronous online conference-based instruction: A study of whiteboard interactions and student writing. Computers and Composition, 23(1), 4–31. https://doi.org/10.1016/j.compcom.2005.12.004
39.
HodgesK. (1982). A history of revision: Theory versus practice. In SudolR. (Ed.), Revising: New perspectives for teachers of writing (pp. 24–42). National Council of Teachers of English.
40.
HsiehY.-C.HillC. (2022). Reconceptualizing the value of peer and instructor feedback using a sequential structure. Assessment & Evaluation in Higher Education, 47(7), 1043–1056. https://doi.org/10.1080/02602938.2021.1998341
41.
JensenW.FischerB. (2005). Teaching technical writing through student peer-evaluation. Journal of Technical Writing and Communication, 35(1), 95–100. https://doi.org/10.2190/MBYG-AK7L-5CT7-54DU
42.
KarimK.NassajiH. (2020). The revision and transfer effects of direct and indirect comprehensive corrective feedback on ESL students’ writing. Language Teaching Research, 24(4), 519–539. https://doi.org/10.1177/1362168818802469
43.
KaufmanJ. H.SchunnC. D. (2011). Students’ perceptions about peer assessment for writing: Their origin and impact on revision work. Instructional Science, 39(3), 387–406. https://doi.org/10.1007/s11251-010-9133-6
44.
KelloggR. T.WhitefordA. P. (2009). Training advanced writing skills: The case for deliberate practice. Educational Psychologist, 44(4), 250–266. https://doi.org/10.1080/00461520903213600
LatifiS.NorooziO.HatamiJ.BiemansH. J. (2021). How does online peer feedback improve argumentative essay writing and learning?Innovations in Education and Teaching International, 58(2), 195–206. https://doi.org/10.1080/14703297.2019.1687005
47.
LiW.ZhangF. (2021). Tracing the path toward self-regulated revision: An interplay of instructor feedback, peer feedback, and revision goals. Frontiers in Psychology, 11, Article 612088. https://doi.org/10.3389/fpsyg.2020.612088
48.
LimpoT.AlvesR. A.FidalgoR. (2014). Children's high-level writing skills: Development of planning and revising and their contribution to writing quality. British Journal of Educational Psychology, 84(2), 177–193. https://doi.org/10.1111/bjep.12020
49.
LinkS.MehrzadM.RahimiM. (2022). Impact of automated writing evaluation on teacher feedback, student revision, and writing improvement. Computer Assisted Language Learning, 35(4), 605–634. https://doi.org/10.1080/09588221.2020.1743323
50.
LiuN.-F.CarlessD. (2006). Peer feedback: The learning element of peer assessment. Teaching in Higher Education, 11(3), 279–290. https://doi.org/10.1080/13562510600680582
51.
LiuY.XiongW.XiongY.WuY.-f. B. (2024). Generating timely individualized feedback to support student learning of conceptual knowledge in writing-to-learn activities. Journal of Computers in Education, 11(2), 367–399. https://doi.org/10.1007/s40692-023-00261-3
52.
LópezP.RijlaarsdamG.TorranceM.FidalgoR. (2018). How to report writing interventions? A case study on the analytic description of two effective revision interventions. Journal of Writing Research, 10(2), 279–329. https://doi.org/10.17239/jowr-2018.10.02.05
53.
LuQ.ZhuX.CheongC. M. (2021). Understanding the difference between self-feedback and peer feedback: A comparative study of their effects on undergraduate students’ writing improvement. Frontiers in Psychology, 12, Article 739962. https://doi.org/10.3389/fpsyg.2021.739962
54.
MacArthurC. A.GrahamS. (2016). Writing research from a cognitive perspective. In MacArthurC. A.GrahamS.FitzgeraldJ. (Eds.), Handbook of writing research (2nd ed., pp. 24–40). The Guilford Press.
55.
MacArthurC. A.PhilippakosZ.GrahamS.HarrisK. (2012). Writing instruction. Learning about learning disabilities, 243–270.
56.
MinB. H.OhY.BrowerR. S. (2021). The effects of diverse feedback dynamics on performance improvement: A typology of performance feedback signals. Administration & Society, 53(1), 123–157. https://doi.org/10.1177/0095399720933822
57.
MooreN. S.MacArthurC. A. (2012). 21: Effects of being a reader and observing readers on argumentative writing. In TorranceM.MangenA.Tolchinsky-LandsmannL.Van WaesL. (Eds.), Learning to write effectively: Current trends in European research (pp. 157–159). Emerald.
58.
MridhaM. F.LimaA. A.NurK.DasS. C.HasanM.KabirM. M. (2021). A survey of automatic text summarization: Progress, process and challenges. IEEE Access, 9, 156043–156070. https://doi.org/10.1109/ACCESS.2021.3129786
59.
NevidJ. S.PastvaA.McClellandN. (2012). Writing-to-learn assignments in introductory psychology: Is there a learning benefit?Teaching of Psychology, 39(4), 272–275. https://doi.org/10.1177/0098628312456622
60.
NewellG. E. (2006). Writing to learn. In MacArthurC. A.GrahamS.FitzgeraldJ. (Eds.), Handbook of writing research (pp. 235–247). The Guilford Press.
61.
NicolD. (2021). The power of internal feedback: Exploiting natural comparison processes. Assessment & Evaluation in Higher Education, 46(5), 756–778. https://doi.org/10.1080/02602938.2020.1823314
62.
NücklesM.RoelleJ.Glogger-FreyI.WaldeyerJ.RenklA. (2020). The self-regulation regulation-view in writing-to-learn: Using journal writing to optimize cognitive load in self-regulated learning. Educational Psychology Review, 32(4), 1089–1126. https://doi.org/10.1007/s10648-020-09541-1
63.
OutlookJ. (2014). National Association of Colleges and Employers, 2013.
64.
ParkJ. (2017). Class P rep: A peer review system for class preparation. British Journal of Educational Technology, 48(2), 511–523. https://doi.org/10.1111/bjet.12390
65.
PatchanM. M.CharneyD.SchunnC. D. (2009). A validation study of students’ end comments: Comparing comments by students, a writing instructor, and a content instructor. Journal of Writing Research, 1(2), 124–152. https://doi.org/10.17239/jowr-2009.01.02.2
66.
PatchanM. M.SchunnC. D.CorrentiR. J. (2016). The nature of feedback: How peer feedback features affect students’ implementation rate and quality of revisions. Journal of Educational Psychology, 108(8), 1098–1120. https://doi.org/10.1037/edu0000103
67.
PelaezN. J. (2002). Problem-based writing with peer review improves academic performance in physiology. Advances in Physiology Education, 26(3), 174–184. https://doi.org/10.1152/advan.00041.2001
68.
PhilippakosZ. A. T. (2023). Writing instruction: Evidence-based practices and critical perspectives. In FisherD.LappD. (Eds.), Handbook of research on teaching the English language arts (pp. 71–95). Routledge.
69.
PonceH. R.MayerR. E.LoyolaM. S.LópezM. J. (2020). Study activities that foster generative learning: Notetaking, graphic organizer, and questioning. Journal of Educational Computing Research, 58(2), 275–296. https://doi.org/10.1177/0735633119865554
70.
PriorP. A.HengstJ. A. (Eds.). (2010). Exploring semiotic remediation as discourse practice. Palgrave Macmillan.
71.
RoothooftH.Lázaro-IbarrolaA.BultéB. (2022). Task repetition and corrective feedback via models and direct corrections among young EFL writers: Draft quality and task motivation. Language Teaching Research, 29(3), 1311–1344https://doi.org/10.1177/13621688221082041
72.
SchillingsM.RoebertsenH.SavelbergH.DolmansD. (2023). A review of educational dialogue strategies to improve academic writing skills. Active Learning in Higher Education, 24(2), 95–108. https://doi.org/10.1177/1469787418810663
73.
Scott-SimmonsD.BarkerJ.CherryN. (2003). Integrating research and story writing. The Reading Teacher, 56(8), 742–745.
74.
SongM. H.LimJ.ParkJ. (2022). The effect of weekly writing and peer reviewing on students’ writing competence. In ChinnC.TanE.ChanC.KaliY. (Eds.), Proceedings of the 16th International Conference of the Learning Sciences (ICLS 2022) (pp. 1029–1032). International Society of the Learning Sciences. https://doi.org/10.22318/icls2022.1029
75.
SoysaC. K.DunnD. S.DottoloA. L.Burns-GloverA. L.GurungR. A. (2013). Orchestrating authorship: Teaching writing across the psychology curriculum. Teaching of Psychology, 40(2), 88–97. https://doi.org/10.1177/0098628312475027
76.
StevensonM.PhakitiA. (2014). The effects of computer-generated feedback on the quality of writing. Assessing Writing, 19, 51–65. https://doi.org/10.1016/j.asw.2013.11.007
77.
StewartT. L.MyersA. C.CulleyM. R. (2010). Enhanced learning and retention through “writing to learn” in the psychology classroom. Teaching of Psychology, 37(1), 46–49. https://doi.org/10.1080/00986280903425813
78.
ThamrinH.VerdikhaN. A.TriyonoA. (2021, December). Text classification and similarity algorithms in essay grading. In 2021 4th International Seminar on Research of Information Technology and Intelligent Systems (ISRITI) (pp. 201–205). IEEE.
79.
ToppingK. J. (2010). Peers as a source of formative assessment. In AndradeH. L.CizekG. J. (Eds.), Handbook of formative assessment (pp. 61–74). Routledge.
80.
WallaceD. L.HayesJ. R.HatchJ. A.MillerW.MoserG.SilkC. M. (1996). Better revision in eight minutes? Prompting first-year college writers to revise globally. Journal of Educational Psychology, 88(4), 682–688. https://doi.org/10.1037/0022-0663.88.4.682
81.
WangE. L.MatsumuraL. C.CorrentiR.LitmanD.ZhangH.HoweE.MagoodaA.QuintanaR. (2020). Erevis (ing): Students’ revision of text evidence use in an automated writing evaluation system. Assessing Writing, 44, 100449. https://doi.org/10.1016/j.asw.2020.100449
82.
WuY.SchunnC. D. (2021). The effects of providing and receiving peer feedback on writing performance and learning of secondary school students. American Educational Research Journal, 58(3), 492–526. https://doi.org/10.3102/0002831220945266
83.
ZhangF.SchunnC. D.BaikadiA. (2017). Charting the routes to revision: An interplay of writing goals, peer comments, and self-reflections from peer reviews. Instructional Science, 45(5), 679–707. https://doi.org/10.1007/s11251-017-9420-6
84.
ZhaoX.EpleyN. (2022). Surprisingly happy to have helped: Underestimating prosociality creates a misplaced barrier to asking for help. Psychological Science, 33(10), 1708–1731. https://doi.org/10.1177/09567976221097615
85.
ZhuM.LiuO. L.LeeH.-S. (2020). The effect of automated feedback on revision behavior and learning gains in formative assessment of scientific argument writing. Computers & Education, 143, 103668. https://doi.org/10.1016/j.compedu.2019.103668
86.
ZinsserW. (2006). How to write a memoir: Be yourself, speak freely, and think small. The American Scholar, 75(2), 109–117.
