Sage Journals: Discover world-class research

Abstract

Initiating effective feedback processes is a major goal in university teaching. However, systematic investigations of structural feedback elements making instructor feedback economic, concise, motivating and beneficial for learning are still scarce. In our study, we compare two feedback modes with respect to learning gains and changes in self-efficacy in a quasi-experimental pre-post design. Participants (N = 75 first-year students) received either scoresheet or textual instructor feedback on four individual assignments during a seminar. Outcome variables were knowledge gain, change in self-efficacy and changes in metacognitive monitoring. After the semester, we observed substantial knowledge gains for both feedback groups with only small advantages for scoresheet feedback. In contrast, self-efficacy was relatively stable across the semester and was not influenced by feedback mode. Achievement motivation measures normative ability and challenge-mastery goal orientation did not moderate the observed relationships but influenced knowledge gain and change in self-efficacy directly. Changes in metacognitive monitoring did not depend on feedback mode. Taken together, our data suggest that scoresheet and textual feedback conveying identical feedback content have comparable effects on achievement and self-evaluation measures. For university settings, scoresheets can be recommended as parsimonious feedback tools.

Keywords

achievement motivation feedback mode metacognitive monitoring self-efficacy university teaching written feedback

Feedback is one of the most effective tools to foster successful learning (Hattie and Timperley, 2007). Contemporary views conceptualize feedback as a process or a series of processes (Henderson et al., 2019) in which learners try to understand evaluations of their performance in a specific situation. It is a major goal in university teaching to initiate effective feedback processes. However, feedback processes can have no or even detrimental effects on student learning (Forsythe and Johnson, 2017; Fyfe and Brown, 2020).

It has been found that students’ engagement with feedback in university quickly deteriorates as students become increasingly dissatisfied with feedback processes, for example, when the feedback process does not include opportunities of dialogue (Ali et al., 2018) or when it is not personalized (Ali et al., 2015). Even if instructors value feedback very highly, they often miss state-of-the-art feedback standards (Knight et al., 2018) because high-quality feedback might result in higher workload; an unrealizable demand given the limited resources of academics (Nicol et al., 2014). That is, instructors in undergraduate courses with many students face the challenge to offer high-quality feedback within an acceptable time frame. From that perspective, we aim to investigate structural elements of feedback that make instructor feedback economic, concise, motivating and beneficial for learning.

Design factors of effective feedback: How should feedback be presented?

A critical question in the feedback process is the choice of feedback mode. Some modes of written feedback meet high quality standards and also take into account the limited resources of lecturers. For example, semi-individualized template feedback with pre-written statements might be perceived as detailed and personalized feedback and offers an interesting combination of parsimony and effectiveness (Crisostomo and Chauhan, 2019). Another approach to concise and efficient written feedback is to provide feedback using rubrics in scoresheets. Rubrics and rubric-like assessment tools seem to be a powerful instrument for learning (Brookhart, 2018). Different views have been expressed on what an appropriate rubric should look like (cf. Brookhart, 2018). As a minimal criterion, we define rubrics in the context of assessment situations as listed requirements that can be marked as fulfilled (yes/no) or rated on a scale as to the degree of their fulfilment (not fulfilled . . . completely fulfilled).

The benefits of scoresheets are often investigated in case studies, evaluated qualitatively or with control groups without any feedback (Brookhart, 2018). Experimental research relying on control groups receiving different modes of feedback is needed to assess the specific impact of scoresheets on feedback effects. Most studies investigate written feedback in naturalistic settings (e.g., Dirkx et al., 2021; Nordrum et al., 2013), in which different feedback effects could be a result of the mere combination of feedback modes. For example, teachers provide more feedback (information on learning progress; Hattie and Timperley, 2007) in comments and more feed forward (information of specific next steps in learning; Hattie and Timperley, 2007) in scoresheets, if they use them concurrently for one assignment (Dirkx et al. 2021). With regard to limited resources of university instructors, an important question is whether one feedback mode (e.g. only scoresheets) suffices for effective feedback. Until now, (quasi-)experimental research on this question is still scare (Evans, 2013).

Feedback effects: What are possible feedback outcomes?

Feedback impacts on a wide range of dimensions, including cognitive skills, motivation and self-assessment skills (e.g. Henderson et al., 2019). So far, research suggests that feedback outcomes are independent of feedback mode. For example, audio, video and text feedback seem to have comparable effects on performance (Espasa et al., 2022). Moreover, self-efficacy (i.e. one’s belief in one’s ability to succeed or accomplish specific tasks; Bandura, 1997) was not differentially affected by written versus verbal feedback (Agricola et al., 2020). However, up to now, written feedback is the predominant feedback form in higher education (Agricola et al., 2020). Therefore, we want to investigate whether different modes of written feedback also have comparable outcome effects. To this end, we focus on three outcome dimensions.

First, we consider cognitive skills by looking at learning gains. Many instructors want to positively influence student’s learning when providing feedback. Meta-analytic evidence confirms that feedback positively affects students’ learning (Wisniewski et al., 2020). Most original studies measured learning in terms of student achievement and, on average, found medium effects (d = 0.51). However, it is possible that feedback mode influences how students can capitalize on feedback. Specifically, the clarity of rubrics in a scoresheet (cf. Brookhart, 2018) might make it easy for students to identify areas in which they can improve. In contrast, in long text passages of textual feedback, important information could be overlooked. Therefore, in the current study, we inspect whether scoresheet feedback has higher effects on learning gain than textual feedback.

Second, we regard motivational factors by looking at self-efficacy. Meta-analytic evidence shows that feedback effects are somewhat lower for motivational outcomes (d = 0.33) than for cognitive outcomes. Importantly, feedback can have unintended as well as intended outcomes on motivation (Fyfe and Brown, 2020; van de Ridder et al., 2015). For example, negative feedback framing using only a subtle phrase already reduces self-efficacy (van de Ridder et al., 2015). Possibly the evaluation with percentages and (un-)ticked boxes in scoresheets highlight errors and incompletenesses. In contrast, longer text passages with formulated assessments could more likely convey that the instructor is invested in the student’s learning. This in turn might affect motivational outcomes such as self-efficacy. Therefore, the current study investigates whether textual feedback has more positive effects on self-efficacy than scoresheet feedback.

Third, regarding self-assessment skills we look at the metacognitive skills of students to assess what knowledge they have already acquired and where they perceive knowledge gaps. This monitoring skill is an essential component to improve performance within self-regulated learning (Kostons et al., 2012) because inaccurate self-evaluations can reduce long-term retention (Dunlosky and Rawson, 2012). Metacognitive training that includes both performance and metacognitive feedback improves not only students’ monitoring accuracy but also performance in a final exam (Händel et al., 2020). However, so far, studies have not disentangled effects of performance feedback and metacognitive feedback on metacognitive skills. Therefore, in the current study, we want to investigate whether performance feedback alone influences metacognitive skills. In addition, the clarity of scoresheets could also offer an advantage over text feedback in terms of identifying knowledge gaps. Thus, we additionally investigate whether the different modes of providing performance feedback are differentially effective.

Feedback moderators: What influences feedback utilization?

Because of the multidimensionality of the feedback process, the utilization of feedback depends on motivational factors (Narciss, 2008). For example, openness to feedback is directly connected to the feedback situation. Openness to external feedback regards how sensitive students are to feedback, how they pay attention to feedback, consider feedback as important and respond to threats implied by feedback (King, 2016). After immediate verbal feedback, students who are very sensitive to feedback exhibited lower performance scores than their peers due to a higher tendency to form negative attributions (King, 2016). This shows that openness to feedback can directly influence how feedback is perceived. Therefore, we investigate openness to feedback as a moderating variable in the feedback process.

Feedback utilization could also be influenced by a student’s achievement motivation. Achievement goals emerge as learning and performance orientation towards a task and are strong predictors of academic performance (Grant and Dweck, 2003; Huang, 2012). Performance orientation is identified by outcome goals (wanting to perform really well), ability goals (demonstrate high abilities), normative ability goals (confirm superiority) and normative outcome goals (outperform others; Grant and Dweck, 2003). In contrast, learning orientation is identified by learning goals (acquiring new skills) and challenge-mastery goals (seeking challenges; Grant and Dweck, 2003). Learning goals are associated with higher intrinsic motivation and greater academic improvement over time (Rawsthorne and Elliot, 1999; Utman, 1997). This effect is mediated by a tendency to a deeper processing of learning material (Grant and Dweck, 2003). Since learning goals are particularly important when facing highly challenging situations with different materials (Richardson et al., 2012), it is likely that they influence performance in first year students. In addition, learning orientation influences learning indirectly by relating to feedback seeking behaviour in two ways: Students with higher challenge-mastery goal orientation, first, generally prefer inferring feedback information from interactions with others (monitoring), and second, show stronger engagement in active feedback seeking by inquiry (Leenknecht et al., 2019). For these reasons, we investigate achievement motivation as a second potential moderator in the feedback process.

Research questions

Thus far, research about written feedback modes offers some promising strategies such as semi-individualized textual feedback (Crisostomo and Chauhan, 2019) and rubric-based scoresheet feedback (Brookhart, 2018) that combine parsimony with effectiveness. However, it remains unclear whether different feedback modes are comparable in their effectiveness. We want to explicitly test whether effects of scoresheet feedback on learning and self-efficacy significantly differ from those of standardized textual feedback. Therefore, we want to assess how learning gain depends on feedback mode (RQ1). Furthermore, we want to investigate whether change in self-efficacy depends on feedback mode (RQ2). With regard to the complexity of the feedback process, we also examine if openness to feedback and achievement motivation have moderating effects on both outcomes (RQ3). Exploratorily, we will look into metacognitive calibration and test to what extent our feedback manipulation influences monitoring accuracy (RQ4).

Method

Sample

The study was pre-registered via OSF (see https://osf.io/a56yh, anonymized dataset and data analysis available under https://osf.io/38rwv/).We conducted the study between October 2019 and January 2020 at a German university, at which first-year students choose either developmental psychology or general psychology as their major field of study. Students were assigned to an obligatory seminar (four parallel seminars per major field of study) with respect to their thematic and scheduling priorities. Students attending developmental psychology seminars (N = 80) worked on assignments for partial course credit and filled out our questionnaires as a monitoring tool. Students were asked on a voluntary basis to give consent for their data to be used in this study without further compensation. All 80 students gave their informed consent. Five participants were excluded from subsequent analyses due to missing posttest data (n = 4) or irregular answer patterns (n = 1). The final sample consisted of 75 students (M = 20.48 years, SD = 2.79, 61 women, 14 men, none non-binary).

Design

We applied a quasi-experimental pre-post design with feedback mode as an independent variable. Participants of two courses received a scoresheet feedback, and the participants of the other two courses received textual feedback with standardized formulations. As dependent variables, knowledge gain, self-efficacy and four metacognitive self-assessment variables were measured at the beginning and the end of the semester. In addition, in the pretest questionnaire, we measured potential moderator variables (openness to feedback and achievement motivation).

Procedure

In the first session of the seminar, students were informed about the study aim and were asked, for experimental reasons, not to share their feedback with each other but to use it as a private learning tool. After that, students signed an informed consent which was stored separately from the study material. Subsequently, students filled out the pretest questionnaire.

For the focus topics of the study, students received an individual assignment at four sessions on a scientific text (overall two textbook chapters and two empirical articles). For each assignment, students had to answer two questions on the corresponding text (one retrieval, one elaborative task; all tasks see Appendix A). In the seminar session before an upcoming assignment was due, the instructor explained the assignment tasks. After that, students could download tasks and texts from the Moodle eLearning system. Students had to hand in their assignment two workdays before the topic was covered in the seminar. They received an individual feedback via Moodle on the evening before the next session. Assignment tasks were discussed at the beginning of the session and students were encouraged to ask questions about their assignments.

At week 10, students filled out the post-test questionnaire. One week later, we debriefed students and showed them preliminary results. Students could request individual feedback on their test scores.

Manipulation of the independent variable: The feedback sheet

The authors rated pseudonymized versions of student’s assignments and prepared individual feedback for each student. To ensure a consistent rating across our two groups, raters filled out a digitalized scoresheet. For the textual feedback-group this scoresheet was automatically transferred into a text by using macros. See Appendices B and C for sample feedbacks. Feedbacks consisted of four parts including the same content modules in both groups.

Part A of the feedback contained information about the correctness of answers. For retrieval tasks, responses were rated with respect to correctness and completeness, for example, whether a definition was correct and complete. In elaborative tasks, such as the development of a timeline, it was assessed whether facts were present, arranged in the correct order and correctly assigned to the corresponding age. At the end of this section, students received a summarizing evaluation. The scoresheet group received the number of points reached (e.g. 8/10 aspects) and an evaluation on a percentage basis (e.g. 80% of the contents were correct). If applicable, potential pre-defined bonus points were denoted. The textual feedback group received a textual evaluation of the proportion of correct content (e.g. predominantly fulfilled). Textual evaluation was classified as not fulfilled (below 10%), predominantly not fulfilled (10–24%), fulfilled to a small proportion (25–49%), half fulfilled (50–74%), predominantly fulfilled (75–99%), completely fulfilled (100%) or fulfilled beyond expectation (100% plus additional bonus points).

Part B of the feedback contained an evaluation of basic formal elements (e.g. suitable typeface, punctuality of submission). All criteria were rated as very good, ok or expandable, respectively, and were designed as a contrast to the rather strict first section.

In part C, structure and readability was rated in terms of sticking to the task, readability and clearness of the produced text, use of paragraphs and academic language. For some tasks, unique task requirements were added (e.g. sequence in reporting results from descriptive to inferential statistics). Criteria were rated as very good, ok or expandable.

Part D on key skills contained feed-forward elements. This section comprised recommendations regarding formal features (e.g. plan time ahead for punctual hand-in) and tips relevant to the assignment task (e.g. search the text for verbal descriptions of tables or graphs). Both feedback modes ended with a general thank you statement and encouragement for the next assignments.

Measures

Dependent variables

Content knowledge was measured with a self-developed task consisting of 20 items covering the topics of the four assignments. Of these items, 16 (four per topic) were constructed in multiple-choice format (one single right answer out of four options), and 4 (one per topic) were constructed in a free answer format (maximum points per item: two points, scoring in half point steps). The test comprised a mixture of questions regarding reproduction and knowledge application. Due to differences in assignment demands, the ratio of application to reproduction questions varied from 2:3 to 1:4. All free-format questions were rated independently by the two authors. Inter-rater reliability for the free-format questions was acceptable to good (Kendall’s τ_b: 0.65 ⩽ τ_b ⩽ .83). Inconsistencies were resolved by joint judgment in all instances. For each person, a sum score for content knowledge and a change score for knowledge gain (post minus pre) was computed for further analyses.

To measure academic self-efficacy, we used an adaptation of the self-efficacy scale by Schwarzer and Jerusalem (1999). We changed some item wordings to fit the university context. As in the original test, academic self-efficacy was measured on a four-point Likert-type scale. Reliabilities are depicted in Table 1. For each person, a mean score and a change score (post minus pre) was computed for further analyses.

Table 1.

Sample items and scale reliabilities of self-efficacy and potential moderator variables.

Scale	Sample item	No. items	Measurement point	α
Self-efficacy	I am confident that I am able to master even difficult tasks in my course of study.	6	t₁ and t₂	.77/.79
Openness to feedback	I think that feedback on my personal learning aims is unnecessary. (reversed)	5	t ₁	.44
Learning and achievement motivation goals
Normative ability	In university, I focus on showing that I’m smarter than other students.	3	t ₁	.95
Normative outcome	One of my biggest goals is to perform better than other students.	3	t ₁	.90
Challenge-mastery	It is important to me to experience that my studies offer real challenges.	3	t ₁	.79
Learning	In courses I strive towards learning and continuous improvement of my skills.	3	t ₁	.70
Ability	In university, I focus on demonstrating my intellectual abilities.	3	t ₁	.75
Outcome	It is important to me to receive high grades in my courses.	3	t ₁	.86

t1: beginning of the semester; t2: end of the semester,

Sample sizes: nscoresheet = 39, ntextual = 36

Metacognitive self-assessment was measured by asking students for each multiple choice item in the knowledge test whether they thought their response was correct or not. From these responses, we calculated the metacognitive calibration variables bias, accuracy, sensitivity and specificity (Händel et al., 2020). Bias is the signed difference between actual performance and rating of expected correctness, averaged over the 16 test items. Scores can range from −1 to 1, with a score close to zero indicating a realistic judgment. A negative score indicates underconfidence and a positive score indicates overconfidence (Händel et al., 2020; Schraw, 2009). Accuracy is a measure of judgment precision, since it accumulates the number of wrong judgments (overconfident and underconfident judgements) to a theoretical maximum of 1. Sensitivity represents the relative frequency of accurately detected correct answers from all correct answers. Specificity represents the relative frequency of accurately detected incorrect answers from all incorrect answers. Both relative measures have the advantage of taking into account, that each student has solved a given number of items correctly or incorrectly. Therefore, they give additional diagnostic information about how good a student is in predicting their own correct and incorrect responses (Händel et al., 2020).

Moderating variables

As potential moderators Openness to feedback (Meijer et al., 2013) and Achievement motivation (German version of the Achievement Goal Inventory by Sudmann et al., 2014) were measured at the beginning of the semester (see Table 1 for sample items). Due to unreliability of the openness to feedback scale, we dropped this variable as a potential moderator. For the Achievement Goal Inventory, mean scores of each subscale were computed for further analyses.

Data analysis

To assess whether feedback mode had effects on knowledge gain (RQ1) and on self-efficacy changes (RQ2), separate univariate, repeated-measures ANOVAS were conducted with the pre-post scores¹. To check for potential moderators, we assessed whether the achievement motivation subscales correlated with knowledge gain and the self-efficacy change score. Normative ability substantially correlated with knowledge gain (r = 0.24), whereas challenge-mastery substantially correlated with self-efficacy changes (r = 0.29). Therefore, normative ability was entered as a covariate in the ANOVA for content knowledge and challenge-mastery goal was entered as a predictor in the ANOVA for self-efficacy (RQ3). We assessed main effects for time (meaning significant changes on our dependent variables during the semester) and interaction effects of time and experimental condition (meaning different changes of both experimental groups). Significant interactions were plotted using the simple slope analysis online-tool by Preacher et al. (2006). For the fourth research question, we assessed effects of feedback mode on changes in meta-cognitive calibration in univariate, repeated-measures ANOVAS.

Results

Descriptive statistics of pretest and posttest measures are displayed in Table 2. Correlations show that throughout the semester knowledge gain was higher if students reported higher normative ability goals (Table 3). In contrast, self-efficacy decreased if students elicited higher challenge-mastery goals.

Table 2.

Descriptive statistics of study variables.

Variable	t	Min–max	Scoresheet feedback			Textual feedback
Variable	t	Min–max	M	SD	Range	M	SD	Range
Dependent variables (single scores t₁ and t₂)
Content knowledge	t ₁	1–20	7.80	2.04	3.00–11.00	8.61	2.07	3.00–11.50
Content knowledge	t ₂	1–20	12.32	2.58	5.00–18.00	12.06	3.16	6.00–19.00
Self-efficacy	t ₁	1–4	3.21	0.42	2.33–4.00	3.20	0.43	2.16–4.00
Self-efficacy	t ₂	1–4	3.23	0.44	2.00–4.00	3.21	0.42	2.33–4.00
Meta-cognitive calibration variables (single scores t₁ and t₂)
Accuracy (%)	t ₁	0–100	42.6	13.4	18.8–75.0	46.3	11.5	18.8–75.0
Accuracy (%)	t ₂	0–100	34.8	11.4	12.5–68.8	34.4	12.6	12.5–56.3
Bias	t ₁	−100–100	13.2	17.9	−25.0–56.3	5.9	27.5	-50.0–56.3
Bias	t ₂	−100–100	7.6	18.0	−18.7–50.0	8.6	23.5	−56.3–43.8
Sensitivity (%)	t ₁	0–100	62.2	24.0	0.0–100	51.1	29.7	0.0–100
Sensitivity (%)	t ₂	0–100	79.8	12.9	54.5–100	80.1	20.3	18.2–100
Specificity (%)	t ₁	0–100	52.3	20.9	0.0–100	51.2	28.7	0.0–100
Specificity (%)	t ₂	0–100	38.8	23.0	0.0–87.5	40.9	29.0	0.0–100
Learning and achievement motivation goals
Normative ability	t ₁	1–7	1.86	1.16	1.00–5.00	1.61	0.95	1.00–4.67
Normative outcome	t ₁	1–7	2.85	1.51	1.00–6.33	2.32	1.18	1.00–6.00
Challenge-mastery	t ₁	1–7	4.09	1.21	1.33–6.67	3.97	1.06	2.00–6.00
Learning	t ₁	1–7	5.75	0.93	3.33–7.00	6.13	0.77	4.33–7.00
Ability	t ₁	1–7	3.83	1.09	1.33–6.33	3.43	1.25	1.33–6.67
Outcome	t ₁	1–7	5.31	1.03	2.33–7.00	5.35	1.18	2.00–7.00
Dependent variables (change scores from t₁ to t₂)
Knowledge gain		−20–20	4.53	2.62	−1.50–11.00	3.44	3.01	−3.50–8.00
Change in self-efficacy		−3–3	0.03	0.36	−0.67–0.83	0.01	0.38	−0.80–0.90
Meta-cognitive calibration variables (change from t₁ to t₂)
Change in accuracy		−100–100	−7.6	20.5	−62.5–43.8	−12.8	19.6	−62.5–18.8
Change in bias		−100–100	−4.1	21.5	−37.5–50.0	0.4	22.4	−56.3–37.5
Change in sensitivity		−100–100	17.7	26.3	−33.3–72.7	24.9	26.9	−55.6–87.5
Change in specificity		−100–100	−13.6	28.6	−75.0–44.4	−10.3	30.7	−77.8–75.0

t₁: beginning of the semester; t₂: end of the semester.

Sample sizes: n_scoresheet = 39, n_textual = 36.

Table 3.

Correlations of study variables across both feedback groups.

Variable	1	2	3	4	5	6	7	8
1 Knowledge gain	—
2 Change in self-efficacy	0.06	—
3 Normative ability	0.24*	0.01	—
4 Normative outcome	0.22	0.04	0.61***	—
5 Challenge-mastery goal	0.20	−0.29**	0.24*	0.20	—
6 Learning goal	0.04	−0.11	−0.03	−0.07	0.30**	—
7 Ability goal	0.16	−0.12	0.62***	0.63***	0.34**	−0.04	—
8 Outcome goal	−0.06	−0.18	0.11	0.48***	0.24*	0.19	0.42***	—

Sample sizes: n_scoresheet = 39, n_textual = 36.

p < 0.05. **p < 0.01. ***p < 0.001.

Results of our repeated-measures ANOVAS on the two main outcomes are depicted in Table 4. For content knowledge, we observed a significant main effect for time indicating that students answered on average four more questions correctly at the end of the semester than at the beginning of the semester. Knowledge gain of the scoresheet group (M_change = 4.53, SD = 2.62) was significantly higher than knowledge gain of the textual-feedback group (M_change = 3.44, SD = 3.01, see Figure 1). In addition, a significant normative ability x time interaction was found. The examination of the interaction plot revealed that students with higher normative ability goals had slightly higher than average knowledge gains (β = 0.94, p < 0.05) than students with lower normative ability goals, who had no significant knowledge gain (β = 0.10, p = 0.71, see Figure 2). The impact of normative ability goals did not differ for the two feedback conditions.

Table 4.

Repeated-measures ANOVAS for effects of feedback group (scoresheet vs. textual) and time (beginning vs. end of the semester) on knowledge gain and change in self-efficacy.

Variable	ANOVA
Variable	Effect	F	df_var,df_res	η²_p
Content knowledge	Group	0.08	1,71	0.00
	Time	21.07***	1,71	0.23
	Normative ability	0.14	1,71	0.00
	Group × Time	5.59*	1,71	0.07
	Normative Ability × Time	5.36*	1,71	0.07
	Group × Normative Ability	0.00	1,71	0.00
	Group × Normative Ability × Time	3.50	1,71	0.05
Self-efficacy	Group	0.05	1,68	0.00
	Time	7.85**	1,68	0.10
	Challenge-mastery goal	4.60*	1,68	0.06
	Group × Time	2.02	1,68	0.03
	Challenge-Mastery Goal × Time	7.85**	1,68	0.10
	Group × challenge-Mastery Goal	0.07	1,68	0.00
	Group × Challenge-Mastery Goal × Time	2.47	1,68	0.03

Sample sizes: n_scoresheet = 39, n_textual = 36.

p < 0.05. **p < 0.01. ***p < 0.001.

Figure 1.

Group (scoresheet vs. textual) × Time (beginning vs. end of the semester) interaction on knowledge.

Figure 2.

Normality Ability × Time (beginning vs. end of the semester) interaction on knowledge.

For self-efficacy, we observed a significant time effect representing a general small decline in academic self-efficacy, although raw scores did not indicate such a decline (Table 4). That is, our control for feedback mode and challenge-mastery goal orientation uncovered systematic changes of self-efficacy throughout the semester. The time effect was qualified by a significant interaction with challenge-mastery goal. Together with the observed negative correlation of challenge-mastery goal with change in self efficacy (r = −0.29), this indicates that the generally small decline is slightly more pronounced in students with higher challenge-mastery goals (β = 0.73, p < 0.05) than in students with lower challenge-mastery goal (β = 0.65, p < 0.05, see Figure 3). That is, challenge-mastery goal orientation moderates how self-efficacy changes in university first-year students throughout the semester.

Figure 3.

Challenge-Mastery × Time (beginning vs. end of the semester) interaction on self-efficacy.

Four explorative repeated-measures ANOVAS were conducted with the metacognitive calibration variables (Table 5). We observed neither group differences nor time changes in the degree of over- or underestimation (bias). Accuracy and sensitivity of correctness of one’s own responses in the knowledge test increased throughout the semester, while specificity decreased. Thus, at the end of the semester, students predicted correctly-solved items better but incorrectly-solved items worse than at the beginning of the semester.

Table 5.

Explorative repeated-measures ANOVAS for effects of feedback group (scoresheet vs. textual) and time (beginning vs. end of the semester) on meta-cognitive calibration variables.

Variable	ANOVA
Variable	Effect	F	df_var,df_res	η²_p
Accuracy	Group	0.87	1,63	0.01
	Time	16.80***	1,63	0.21
	Group × Time	1.13	1,63	0.02
Bias	Group	0.74	1,63	0.01
	Time	0.48	1,63	0.01
	Group × Time	0.70	1,63	0.01
Sensitivity	Group	0.67	1,73	0.01
	Time	48.06***	1,73	0.40
	Group × Time	1.42	1,73	0.02
Specificity	Group	0.01	1,73	0.00
	Time	12.15***	1,73	0.14
	Group × Time	0.22	1,73	0.00

Sample sizes: n_scoresheet = 39, n_textual = 36.

***

p < 0.001.

Discussion

Our study goal was to investigate structural elements of feedback that make instructor feedback economic, concise, motivating and beneficial for learning. Specifically, we aimed at disentangling influences of feedback mode and feedback content on learning and self-efficacy. The results extend previous findings in two important ways. First, when systematically manipulating feedback mode while excluding confounding influences of feedback content, feedback mode has only minimal influences on feedback effects. Second, achievement motivation variables do not moderate the relationship between feedback mode and feedback effects (i.e. students benefit from different feedback modes independently of their achievement motivation). To provide feedback that is not only motivating and beneficial for learning but also economic and timely, scoresheet feedback is a suitable feedback mode fulfilling these criteria.

Feedback effects on learning

As expected, both the scoresheet and the textual feedback group showed substantial knowledge gain throughout the semester. The results show medium to high effects exceeding typical effects of repeated testing (Rowland, 2014). Additionally, knowledge gain was somewhat higher in the scoresheet group compared to the textual feedback group. Presumably, students in the scoresheet group capitalized on the higher clarity and transparency of rubrics (cf. Brookhart, 2018)compared to the textual group that received longer text passages in their feedback. This implies that clarity and transparency of feedback as provided in the scoresheet group might enhance learning.

We assume that no matter which feedback mode is chosen, instructors need to think about the criteria they will use to evaluate coursework, ideally by formulating levels of expectations. This can serve as a direct basis for the creation of the scoresheet, so that hardly any additional effort is to be expected in the creation of the scoresheet compared to the textual feedback. In addition, score sheets can help instructors not to forget important points in their feedback. This contributes to the objectivity and reliability of the feedback process. Considering that actually giving feedback via a scoresheet is more economic compared to writing textual feedback, our result underlines our recommendation for scoresheet-based feedback in higher education.

Feedback effects on self-efficacy

We did not observe differential influences of feedback mode on changes in self-efficacy. That is, in the current study, neither feedback mode caused changes in self-efficacy any more than the other due to, for example, a stronger promotion of student’s sense of relatedness to the instructor (cf. Ajjawi et al., 2022). This result extends previous findings insofar as we now have evidence that neither textual feedback as compared to verbal feedback (Agricola et al., 2020) nor scoresheet feedback as compared to textual feedback are detrimental to self-efficacy. Overall, we found self-efficacy to be relatively stable throughout the semester, an observation in line with former studies (Agricola et al., 2020; Duijnhouwer et al., 2010). In the current study, the high stability of self-efficacy might be due to situational factors. Performance in the knowledge test was not used for grading and in addition, students did not receive formal grades on other occasions in the seminar. Thus, this study took place in a low-stakes environment, not threatening student’s self-efficacy by a graded knowledge test or an upcoming examination. Moreover, the posttest took place before students received any formal grading in the university context at all. The limited information students could have obtained about their own abilities until post-test apart from our positively-framed feedback might be a further reason for the high stability of self-efficacy throughout the semester.

Feedback moderators

We identified two achievement motivation variables influencing feedback effects in university first year students: the degree to which students want to show that they can outperform their peers (normative ability goals) and the degree to which students want to experience challenges in their studies (challenge-mastery goals). Both variables were independent predictors of feedback effects but not moderators in the association between feedback mode and feedback effects.

Students with higher normative ability goals showed more knowledge gain, regardless of the feedback condition. This might indicate that students who aimed at demonstrating their intellectual superiority to their peers interpreted the experimental context as a performance situation. The feeling of being challenged could have resulted in improved memorability for the pretest items, active search for solutions within the seminar context and/or more active usage of their respective feedback sheets (cf. Leenknecht et al., 2019).

In addition, more mastery-oriented students experienced a higher decline in self-efficacy throughout the semester independent of feedback mode. Possibly, more mastery-orientated students interpreted difficulties with answering knowledge questions as a failure in pursuing personal learning goals causing a decline in reported self-efficacy in the posttest. Moreover, more mastery-orientated students might have achieved a more realistic view about their own abilities throughout the semester, for example, by capitalizing more on feedback situations. Such an adaptation to the university context could protect mastery-oriented students from over-optimism, an adaptation strategy viewed as questionable (Haynes et al., 2006). However, these results have to be interpreted cautiously, since student ranking regarding their self-efficacy was comparably stable throughout the semester (r = 0.63).

Metacognitive calibration

Students improved on two of four metacognitive calibration variables throughout their first semester, indicating that their perception of whether they were able to give a correct answer to an item was more accurate at the end than at the beginning of the semester. Strikingly, students’ monitoring accuracy improved more strongly than improvements typically observed through metacognitive trainings (Händel et al., 2020). However, students’ ability to correctly detect wrong answers declined. In comparison to the other metacognitive calibration variables, specificity seems the variable which can be influenced least (Händel et al., 2020), a characteristic that might explain the observed result. Although other learning situations within the semester might have influenced this result, this finding signifies that both feedback modes could help students improve their self-assessment of their abilities.

Limitations and future research

One limitation of our study is, that by pseudonymizing our feedback we might have involuntarily eliminated the personal character of feedback, which is an important quality dimension (Ajjawi et al., 2022; Nicol, 2010). Informal comments from students at the end of the semester describing both feedback modes as ‘rather mechanical’ point in that direction. In addition, students rarely used the offered opportunity to personally ask questions about their feedback. To support students’ engagement with written feedback, in future studies it might be helpful to include more semi-personalized statements (cf. Crisostomo and Chauhan, 2019) or to address students using their names (Ajjawi et al., 2022). Nonetheless, our substantial effects on knowledge gain and metacognitive monitoring imply that these pitfalls of standardized feedback were not detrimental to students’ learning.

A second limitation of our study is that we compared only two feedback modes. It is possible that other feedback modes such as video feedback or face-to-face feedback influence learning gains and self-efficacy differently than the written modes we investigated. Yet, as we set out to quasi-experimentally test feedback that could be not only motivating and beneficial for learning but also economic and concise, we chose these two feedback modes because they are more feasible regarding technical demands and limited resources of university instructors. Nonetheless, an interesting direction for future research is to investigate whether other feedback modes influence feedback effects differently than the two written modes we used.

A third limitation of our study follows directly upon the previous point. To disentangle feedback effects from learning gain that would have occurred without feedback, we would have needed a no-feedback group. We deliberately decided against this control condition for two reasons. First, it was important for us to give all participants feedback that, based on previous research, would most likely be helpful to them. Even if our students did not write an exam at the end of the course, they should still have comparable learning opportunities in their first semester at university. Second, in a no-feedback group, we would have created a rather unnatural situation, as not getting any feedback on course work is not standard in higher education. Such a clear disadvantage of the control group could have had a negative impact on motivation. Thus, the usefulness of the control group and at the same time the validity of the study would have been questionable.

A fourth limitation of our study is that we tested in a low-stakes environment, meaning that students awaited no formal grading on course contents. This might have influenced the motivation with which students engaged in course topics. Therefore, it should be tested whether the findings of the current study can be replicated in a high-stakes situation, in which performance directly relates to grading.

Practical implications

The current study shows that different feedback modes are positively associated with knowledge gain in a university context, with a slight advantage for scoresheet feedback. This means, instructors relying on scoresheet feedback can justifiably expect that their feedback will be effective. However, the advantage of scoresheet feedback over textual feedback was only small indicating that form follows function. That is, content is more important than “superficial” feedback mode, at least for written feedback. Taking into account that teachers seem to prefer different feedback modes for different feedback content (cf. Dirkx et al., 2021), our findings imply that instructors can choose a feedback mode they find economic and suitable with regard to the feedback content they want to provide. Taken together, instructors and students can benefit from both scoresheet feedback and textual feedback on assignments. Our results support an elective choice of feedback mode based on economic requirements of university instructors.

Supplemental Material

sj-docx-1-alh-10.1177_14697874221131970 – Supplemental material for The impact of feedback mode on learning gain and self-efficacy: A quasi-experimental study

Supplemental material, sj-docx-1-alh-10.1177_14697874221131970 for The impact of feedback mode on learning gain and self-efficacy: A quasi-experimental study by Christine Johannes and Astrid Haase in Active Learning in Higher Education

Supplemental Material

sj-docx-2-alh-10.1177_14697874221131970 – Supplemental material for The impact of feedback mode on learning gain and self-efficacy: A quasi-experimental study

Supplemental material, sj-docx-2-alh-10.1177_14697874221131970 for The impact of feedback mode on learning gain and self-efficacy: A quasi-experimental study by Christine Johannes and Astrid Haase in Active Learning in Higher Education

Supplemental Material

sj-docx-3-alh-10.1177_14697874221131970 – Supplemental material for The impact of feedback mode on learning gain and self-efficacy: A quasi-experimental study

Supplemental material, sj-docx-3-alh-10.1177_14697874221131970 for The impact of feedback mode on learning gain and self-efficacy: A quasi-experimental study by Christine Johannes and Astrid Haase in Active Learning in Higher Education

Supplemental Material

sj-docx-4-alh-10.1177_14697874221131970 – Supplemental material for The impact of feedback mode on learning gain and self-efficacy: A quasi-experimental study

Supplemental material, sj-docx-4-alh-10.1177_14697874221131970 for The impact of feedback mode on learning gain and self-efficacy: A quasi-experimental study by Christine Johannes and Astrid Haase in Active Learning in Higher Education

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Christine Johannes

Astrid Haase

Supplemental material

Supplemental material for this article is available online.

Notes

Author biographies

Christine Johannes is lecturer at the University of Erfurt, Germany. Her research focus lays on teaching and learning processes in university teaching, e.g. instructional design, teacher qualification, measurement of teaching quality and self-regulated learning.

Astrid Haase is a postdoctoral researcher at the Department of Educational Psychology at the Georg-August-University Göttingen, Germany. Her current research includes cognitive predecessors of reading and spelling with a special focus on morphological awareness.

References

Agricola

Prins

Sluijsmans (2020) Impact of feedback request forms and verbal feedback on higher education students’ feedback perception, self-efficacy, and motivation. Assessment in Education: Principles, Policy & Practice 27(1): 6–25.

Ajjawi

Kent

Broadbent

, et al. (2022) Feedback that works: A realist review of feedback interventions for written tasks. Studies in Higher Education 47(7): 1343–1356.

Ali

Ahmed

Rose

(2018) Identifying predictors of students’ perception of and engagement with assessment feedback. Active Learning in Higher Education 19(3): 239–251.

Ali

Rose

Ahmed

(2015) Psychology students’ perception of and engagement with feedback as a function of year of study. Assessment & Evaluation in Higher Education 40(4): 574–86.

Bandura

(1997) Self-Efficacy: The Exercise of Control. New York: W. H. Freeman.

Brookhart

(2018) Appropriate criteria: Key to Effective Rubrics. Frontiers in Education 3: 22.

Crisostomo

Chauhan

(2019) Individualized student feedback: Are templates the solution? Management Teaching Review 4(4): 371–382.

Dirkx

Joosten-ten Brinke

Arts

, et al. (2021) In-text and rubric-referenced feedback: Differences in focus, level, and function. Active Learning in Higher Education 22(3): 189–201.

Duijnhouwer

Prins

Stokking

(2010) Progress feedback effects on students’ writing mastery goal, self-efficacy beliefs, and performance. Educational Research and Evaluation, 16(1): 53–74.

10.

Dunlosky

Rawson

(2012) Overconfidence produces underachievement: Inaccurate self evaluations undermine students’ learning and retention. Learning and Instruction 22(4): 271–280.

11.

Espasa

Mayordomo

Guasch

, et al. (2022) Does the type of feedback channel used in online learning environments matter? Students’ perceptions and impact on learning. Active Learning in Higher Education 23(1): 49–63.

12.

Evans

(2013) Making sense of assessment feedback in higher education. Review of Educational Research 83(1): 70–120.

13.

Forsythe

Johnson

(2017) Thanks, but no-thanks for the feedback. Assessment & Evaluation in Higher Education 42(6): 850–859.

14.

Fyfe

Brown

(2020) This is easy, you can do it! Feedback during mathematics problem solving is more beneficial when students expect to succeed. Instructional Science 48(1): 23–44.

15.

Grant

Dweck

(2003) Clarifying achievement goals and their impact. Journal of Personality and Social Psychology 85(3): 541–553.

16.

Händel

Harder

Dresel

(2020) Enhanced monitoring accuracy and test performance: Incremental effects of judgment training over and above repeated testing. Learning and Instruction 65: 101245–101249.

17.

Hattie

Timperley

(2007) The power of feedback. Review of Educational Research 77(1): 81–112.

18.

Haynes

Ruthig

Perry

, et al. (2006) Reducing the academic risks of over-optimism: The longitudinal effects of attributional retraining on cognition and achievement. Research in Higher Education 47(7): 755–779.

19.

Henderson

Ajjawi

Boud

, et al. (2019) Identifying feedback that has impact. In: Henderson

Ajjawi

Boud

, et al (eds) The Impact of Feedback in Higher Education. Cham: Springer International Publishing, pp.15–34.

20.

Huang

(2012) Discriminant and criterion-related validity of achievement goals in predicting academic achievement: A meta-analysis. Journal of Educational Psychology 104(1): 48–73.

21.

King

(2016) When do students benefit from performance feedback? A test of feedback intervention theory in speaking improvement. Communication Quarterly 64(1): 1–15.

22.

Knight

Greenberger

McNaughton

(2018) An interdisciplinary perspective: The value that instructors place on giving written feedback. Active Learning in Higher Education 22(2): 115–128. https://doi.org/10.1177/1469787418810127

23.

Kostons

van Gog

Paas

(2012) Training self-assessment and task-selection skills: A cognitive approach to improving self-regulated learning. Learning and Instruction 22(2): 121–132.

24.

Leenknecht

Hompus

van der Schaaf

(2019) Feedback seeking behaviour in higher education: The association with students’ goal orientation and deep learning approach. Assessment & Evaluation in Higher Education 44(7): 1069–1078.

25.

Maxwell

Delaney

Kelley

(2017) Designing Experiments and Analyzing Data: A Model Comparison Perspective, 3rd ed. New York: Routledge.

26.

Meijer

Sleegers

Elshout-Mohr

, et al. (2013) The development of a questionnaire on metacognition for students in higher education. Educational Research 55(1): 31–52.

27.

Narciss

(2008) Feedback strategies for interactive learning tasks. In: Spector

Merill

van Merriënboer

JJG

, et al. (eds) Handbook of Research on Educational Communications and Technology, 3rd edn. Mahaw, NJ: Lawrence Erlbaum Associates, pp.125–144.

28.

Nicol

(2010) From monologue to dialogue: Improving written feedback processes in mass higher education. Assessment & Evaluation in Higher Education 35(5): 501–517.

29.

Nicol

Thomson

Breslin

(2014) Rethinking feedback practices in higher education: A peer review perspective. Assessment & Evaluation in Higher Education 39(1): 102–122.

30.

Nordrum

Evans

Gustafsson

(2013) Comparing student learning experiences of in-text commentary and rubric-articulated feedback: Strategies for formative assessment. Assessment & Evaluation in Higher Education 38(8): 919–940.

31.

Preacher

Curran

Bauer

(2006) Computational tools for probing interaction effects in multiple linear regression, multilevel modeling, and latent curve analysis. Journal of Educational and Behavioral Statistics 31: 437–448.

32.

Rawsthorne

Elliot

(1999) Achievement goals and intrinsic motivation: A meta-analytic review. Personality and Social Psychology Review 3(4): 326–344.

33.

Richardson

Abraham

Bond

(2012) Psychological correlates of university students' academic performance: A systematic review and meta-analysis. Psychological Bulletin 138(2): 353–387.

34.

Rowland

(2014) The effect of testing versus restudy on retention: A meta-analytic review of the testing effect. Psychological Bulletin 140(6): 1432–1463.

35.

Schraw

(2009) A conceptual analysis of five measures of metacognitive monitoring. Metacognition and Learning 4(1): 33–45.

36.

Schwarzer

Jerusalem

(1999) Skalen zur Erfassung von Lehrer- und Schülermerkmalen. Dokumentation der psychometrischen Verfahren im Rahmen der Wissenschaftlichen Begleitung des Modellversuchs Selbstwirksame Schulen. Berlin: Freie Universität Berlin.

37.

Sudmann

Rath

Forkmann

, et al. (2014). Lern- und Leistungsmotivation im Studium. Aachen: Psychometrikon. https://doi.org/10.6099/1000138

38.

Utman

(1997) Performance effects of motivational state: A meta-analysis. Personality and Social Psychology Review 1(2): 170–182.

39.

Van Breukelen

(2006) ANCOVA versus change from baseline: More power in randomized studies, more bias in nonrandomized studies. Journal of Clinical Epidemiology 59(9): 920–925.

40.

van de Ridder

JMM

Peters

CMM

Stokking

, et al. (2015). Framing of feedback impacts student’s satisfaction, self-efficacy and performance. Advances in Health Sciences Education, 20(3): 803–816.

41.

Wisniewski

Zierer

Hattie

(2020) The power of feedback revisited: A meta-analysis of educational feedback research. Frontiers in Psychology 10: 3087.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.03 MB

0.04 MB

0.03 MB

0.04 MB