Sage Journals: Discover world-class research

Abstract

Freeform comments as a means of providing formative feedback on engineering problems, from the perspective of feedback providers (i.e. assessors), is examined. The aim of this research is to collect and analyze assessors’ ratings on the usability of this type of task. Two course topics with different error loads in the sample solutions were used as the basis for the work. Assessors were divided into two groups: Group 1 received an evaluation package containing first year mechanics students’ test solutions with a high error load (Error₁=32), while Group 2 received first year circuits students’ test solutions with low error load (Error₂=11). Assessment time was held constant (t_tot=20min). A standard instrument for usability was utilized. Analysis of the survey data from the assessors (n₁=11, n₂=19) revealed some significant differences between the two groups. In particular, Group 1 reported a lower degree of perceived consistency in marking relative to Group 2.

Get full access to this article

View all access options for this article.

References

Bransford

J. D.

Brown

A. L.

Cocking

R. R.

(2000). How people learn: Brain, mind, experience, and school: Expanded edition. Washington, DC: National Academy Press. https://doi.org/10.17226/6160

Clason

D. L.

Dormody

T. J.

(1994). Analyzing data measured by individual Likert-type items. Journal of Agricultural Education, 35(4). https://doi.org/10.5032/jae.1994.04031

Dochy

Gijbels

Segers

(2006). Learning and the emerging new assessment culture. In In Verschaffel

Dochy

Boekaerts

Vosniadou

(Eds.), Instructional psychology: Past, present and future trends. Oxford, Amsterdam: Elsevier.

Dumas

J. S.

Redish

(1999). A practical guide to usability testing. Intellect books.

Hamer

Purchase

Luxton-Reilly

Denny

(2015). A comparison of peer and tutor feedback. Assessment & Evaluation in Higher Education, 40(1), 151–164. https://doi.org/10.1080/02602938.2014.893418

Kluger

DeNisi

(1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. https://doi.org/10.1037/0033-2909.119.2.254

McKay

E. N.

(2013). UI is communication: How to design intuitive, user centered interfaces by focusing on effective communication. Amsterdam ; Boston: Elsevier, Morgan Kaufmann.

Memarian

McCahan

(2018). Exploring Challenges of Assessing Students’ Errors in Engineering Problem Solving; a Thematic Analysis. In Proceedings of Canadian Engineering Education Association Annual Conference (CEEA 2018). Vancouver (in press).

Mory

E. H.

(2004). Feedback research revisited. Handbook of Research on Educational Communications and Technology, 2, 745–783.

10.

Moskal

B. M.

Leydens

J. A.

(2000). Scoring rubric development: Validity and reliability. Practical Assessment, Research & Evaluation, 7(10), 71–81.

11.

Nicol

D. J.

Macfarlane-Dick

(2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199–218. https://doi.org/10.1080/03075070600572090

12.

Orsmond

Merry

Reiling

(1997). A study in self-assessment: tutor and students’ perceptions of performance criteria. Assessment & Evaluation in Higher Education, 22(4), 357–368. https://doi.org/10.1080/0260293970220401

13.

Pitts

S. E.

(2005). “Testing, testing…” How do students use written feedback? Active Learning in Higher Education, 6(3), 218–229. https://doi.org/10.1177/1469787405057663

14.

Rodgers

K. J.

Horvath

A. K.

Jung

Fry

A. S.

Diefes-Dux

Cardella

M. E.

(2015). Students’ perceptions of and responses to teaching assistant and peer feedback. Interdisciplinary Journal of Problem-Based Learning, 9(2), 2. https://doi.org/10.7771/1541-5015.1479

15.

Shute

V. J.

(2008). Focus on formative feedback. Review of Educational Research, 78(1), 153–189. https://doi.org/10.3102/0034654307313795

16.

Weaver

M. R.

(2006). Do students value feedback? Student perceptions of tutors’ written responses. Assessment & Evaluation in Higher Education, 31(3), 379–394. https://doi.org/10.1080/02602930500353061

Usability Analysis of Freeform Marking on Engineering Problem Solving

Abstract

Get full access to this article

References