Sage Journals: Discover world-class research

Abstract

Formative assessment and feedback strategies play a core role in effective learning and instruction. Thus, teachers should be able to effectively apply the theoretical and empirical insights on formative assessment and feedback strategies in their classrooms. However, designing and implementing formative feedback strategies are complex tasks. Based on the Interactive Tutoring Feedback Model we developed a seminar concept combining case-based reflections of empirically based principles for the design of formative feedback strategies; planning and designing formative feedback strategies for a typical oral or written student assignment; and implementing, evaluating, and iteratively adapting this feedback strategy using on-campus micro-teaching sessions. The benefits and constraints of this seminar concept were explored in an evaluation study with teacher students (N = 87). Students participating in the treatment seminars had higher gains in knowledge and competencies compared to students of control groups that had either a short theoretical input or no input on feedback. Furthermore, they appreciated the mix of instructional and micro-teaching components, and rated the transferability of the knowledge and competencies they had acquired as high.

Keywords

Case-based learning competency-assessment formative feedback strategies micro-teaching teacher education

Introduction

Formative feedback strategies are considered a core factor for effective learning and instruction (e.g., Hattie, 2012; Shute, 2008). However, designing and implementing formative feedback strategies is challenging and requires combining content knowledge with psychological and pedagogical knowledge on the conditions and effects of formative feedback strategies (e.g., Schütze et al., 2017). To promote the acquisition of the knowledge and competencies necessary for accomplishing these challenging tasks, teacher education programs should include courses or training components explicitly addressing issues of how to combine content knowledge with psychological and pedagogical knowledge on formative assessment and feedback strategies. We used the Interactive Tutoring Feedback Model (ITF-Model; e.g., Narciss, 2017), and approaches to teacher training (e.g., Desimone, 2009), case-based learning (e.g., Mostert, 2007; Orr & Weekley, 2019), and on-campus micro-teaching (e.g., Amobi & Irwin, 2009; Ostrosky et al., 2013) as frameworks for developing such a course concept.

The purposes of the project presented in this paper were threefold. First, we aimed at developing a comprehensive syllabus that targets the acquisition of knowledge and competencies with regard to designing and implementing formative feedback strategies in instructional contexts. Second, we implemented this syllabus in several preservice teacher courses in order to explore its benefits and constraints, and to iteratively optimize it. Third, the effects of the training were evaluated in an exploratory field study in order to contribute findings to the following research questions:

What are the effects of the seminar on students’ knowledge gains regarding the conditions and effects of formative assessment and feedback strategies?

What are the effects of the seminar on students’ competencies in applying formative assessment and feedback strategies?

Which constraints have to be taken into account when investigating the seminar concept within a regular preservice teacher course program at the university?

Interactive Tutoring Feedback Strategies

The ITF-Model views feedback as a multi-dimensional instructional activity that aims at empowering students to become self-regulated learners by providing them with formative or tutoring information on their current state of learning. Its first version was developed in the late 1990s, inspired by seminal reviews and meta-analyses of feedback research (most importantly, Butler & Winne, 1995; Kluger & DeNisi, 1996), as well as insights from systems theory (for further details, see Narciss, 2013, 2017). Rooted in these lines of research, the ITF-Model suggests that the mixed findings on the effects and conditions of feedback in instructional contexts can be explained by: (a) situating feedback as a core component of two interacting feedback loops (see Figure 1); and (b) considering not only the factors and processes related to providing feedback from an external source (e.g., a teacher, parent, peer or computer-based learning environment), but also the factors and processes related to generating internal feedback (i.e., self-assessing one’s current state of learning; see also Butler & Winne, 1995), and uptaking the externally provided feedback.

Figure 1.

Interactive Tutoring Feedback-Model (cf. Narciss, 2017, p. 177).

These theoretical considerations led to the assumption that researchers and teachers have to take into account at least three sets of conditions for the design and evaluation of feedback strategies: firstly, the quality of the feedback strategy (e.g., scope, nature, and structure of the information provided and form of presentation); secondly, individual learning conditions (e.g., prior knowledge or level of competencies, meta-cognitive strategies, motivational dispositions and strategies), and thirdly, situational conditions of the instructional setting (e.g., instructional goals, educational standards and competencies, learning content and tasks).

According to the ITF-Model, good feedback strategies should empower students to become self-regulated and productive lifelong learners. In order to do so, they should be interactive rather than just transmitting some kind of evaluative external feedback. Furthermore, they need to be designed according to the principles described in Table 1.

Table 1.

Principles for Designing Feedback Strategies Derived from the ITF-Model (Narciss, 2017).

Principle	Description of optimal implementation
Transparency regarding Criteria and Standards
• Identify and communicate the criteria that indicate the current state of task completion	• The concrete criteria serving as indicators for various levels of task completion are communicated orally as well as in a written format, and discussed with the students
• Specify and communicate the standards for the criteria	• The optimal standard for each of the criteria is specified, communicated and explained to students
Provide opportunities to generate Internal Feedback before offering external feedback
• Prompt the generation of internal feedback before providing external feedback	• Provide students with learning tasks that include prompts to self-assess and reflect their task processing and outcomes
• Facilitate the generation of formative, competency-oriented internal feedback	• Provide students with self-assessment forms, they can use to rate their level of meeting the criteria and standards
Design the external Feedback-Content
• Address extent to which criteria / standards are met	• The external feedback provides concrete information on how well the standards for the relevant criteria have been met
• Address discrepancies between external and internal feedback	• The external feedback is provided in a way that elicits discrepancies between the internal and external feedback
Present external feedback in an interactive way
• Prompt the reflection on corrective actions for closing gaps between the current and the desired level of performance	• Students are provided with the opportunity to figure out corrective actions, and are not immediately provided with these actions or the correct answer
• Offer tutorial support for identifying corrective actions	• xamples etc. if they failed to figure out corrective actions on their own
Implement Feedback formatively respecting the Mastery Learning Principle
• Offer opportunities for engaging in corrective actions based on the feedback	• Provide students with opportunities of applying the feedback information (e.g., revision of the current draft of an assign- ment; working on further tasks with similar requirements)
• Elicit progress	• Highlight if and how students improved their achievements

Design of the Seminar Concept and Material

The instructional design of the treatment course was inspired by several lines of teacher education research including: (a) frameworks on effective professional development of (prospective) teachers (Desimone, 2009); (b) work on case-based learning with text or video material (e.g., Kramer et al., 2020); (c) work on case-based learning with own versus external (video) cases (e.g. Seidel et al., 2011); (d) work on using video for pre-service teacher education (Blomberg, Renkl, Gamoran, Borko, & Seidel, 2013); and (e) work on on-campus micro-teaching (e.g., Amobi & Irwin, 2009). Table 1 provides an overview of how we combined components targeting the acquisition and elaboration of pedagogical content knowledge with components targeting reflective competencies, and the integration and transfer of the pedagogical content knowledge.

Goals and Study Objectives

Developing professional expertise as a teacher requires, on the one hand, acquiring conceptual knowledge that is grounded in theoretical as well as empirical evidence. On the other hand, it requires gaining experiences in applying this knowledge to instructional contexts (Elvira et al., 2017). Thus, the main goals of the seminar were to foster the acquisition and elaboration of: (a) knowledge on formative assessment and feedback strategies; and (b) competencies in designing, implementing, and reflecting formative feedback strategies. More specifically, and in terms of the European Qualification Framework for the Higher Education Area, the seminar should qualify students: (a) to demonstrate knowledge and conceptual understanding on current theoretical frameworks of formative assessment and feedback; (b) to apply their knowledge and understanding on formative assessment and feedback to a reflected analysis of typical feedback scenarios in school contexts; (c) to communicate clearly the reflections and conclusions concerning the feedback scenarios, the knowledge and a scientifically grounded rationale underpinning these; and (d) to integrate and apply their knowledge on formative assessment and feedback when designing, implementing, and reflecting feedback strategies for instructional contexts.

Content, Instructional Material, and Structure

The seminar was conceptualized for a regular semester with 12–14 weeks. It included a knowledge acquisition and elaboration phase (5 units of 90 min.), and 2 knowledge integration and transfer phases (7 units of 90 min.). Table 2 provides an overview with further details.

Table 2.

Overview of Course Structure, Goals, Content, Material, and Activities.

Session(s)—phase	Goals – Content – Material – Activities	Data collection
Unit 1Opening Session	• Welcoming students• Introducing seminar goals and organizational methods and structure• Gathering data on prior knowledge and competencies	Pre-Test• Recall of Feedback Principles• Analysis of Case Vignette

Units 2 – 6Knowledge acquisition and elaboration phase	• Deepening and elaborating students’ knowledge and understanding of research on formative assessment and feedback, by studying and discussing scientific papers on○ ITF-Model and ITF-Principles (Narciss, 2012; 2017) ○ Formative Assessment (e.g., Schütze, et al. 2018) ○ Competency-based assessment and instruction (e.g., Ziener, 2008; Schott & Ghanbari, 2012) • Applying knowledge and understanding to the analysis of typical assessment and feedback instruments by comparing and contrasting○ examples of formative feedback instruments○ examples of formative assessment instruments○ examples of competency evaluation rating forms• Applying knowledge and understanding to the analysis of typical feedback scenarios in schools○ Case-vignettes => reflected analysis
Units 6 – 8Knowledge integration and transfer phase 1	• Integrating and applying elaborated knowledge to the design of formative assessment and feedback strategies for a selected school topic and context. ○ Planning a formative feedback strategy for a typical written or oral assessment situation○ Develop and/or select the assessment and feedback instruments○ Implement the strategy in a micro-teaching unit	Artefacts produced by students• Plans of feedback strategy• Material and instruments for feedback strategy• Video from micro-teaching
Units 9 – 11Knowledge integration and transfer phase 2	• Integrating and applying elaborated knowledge on the evaluation and revision of formative assessment and feedback strategies○ Reflecting – generating internal feedback○ Receiving external feedback from peers○ Discussing commonalities and differences between internal and external feedback○ Revising plans and materials○ Implementing revised strategy – micro-teaching	Artefacts produced by students• Internal Feedback• External Peer-Feedback• Revised Plans and Materials of feedback strategy• Video from micro-teaching
Unit 12Closing session	• Gathering data on knowledge and competencies• Share and discuss the lessons learned• Summarize core insights in terms of take home messages	Post-Test• Recall of Feedback Principles• Analysis of Case Vignette• Students’ evaluations

In the knowledge acquisition phase, students studied and discussed core scientific texts on formative assessment and feedback. Furthermore, they were asked to compare and contrast selected typical to best-practice examples of: (a) formative feedback and assessment instruments; and (b) competency evaluation rating forms. Additionally, to offer students opportunities to reflect on and elaborate their knowledge on the ITF principles, we developed vignettes describing typical instructional scenarios in which a teacher is providing feedback on (either) an oral or a written student assignment. The teachers’ behaviors in these vignettes met some of the feedback principles derived from the ITF-Model but not others. The vignettes served as material for case-based reflections and elaboration of empirically based principles for the design of formative feedback strategies.

The first knowledge integration and transfer phase (4 units of 90 min.) consisted of planning and designing a formative feedback strategy for a typical oral or written student assignment in a self-selected classroom scenario. Furthermore, students were required to implement the strategy within a micro-teaching section, in which their peer students played the roles of the learners. The second knowledge integration and transfer phase (3 units of 90 min.) comprised the reflection of the micro-teaching experience, based on internal (i.e., self-assessment) and external feedback (peer and teacher feedback), the revision of the feedback strategy, as well as the implementation of the revised strategy within a second micro-teaching section.

The opening and closing sessions started with a period for collecting the pre- and post-test data. Furthermore, they served for providing an organizational frame for sharing goals, and reflecting on the current level of knowledge and competencies concerning the core principles of formative assessment and feedback.

Evaluation of the Seminar

The benefits and constraints of this course concept have been piloted with several groups of teacher students (e.g., Höppner et al., 2019). The findings of these pilots have been used to iteratively optimize the seminar components and structure. The final version of the seminar is described in Table 2.

Participants

The effects of the optimized version of the seminar on students’ knowledge and competency gains have been explored with a pre-test–post-test design within several seminar groups of the teacher education program at a large German university. Over all groups, 87 teacher education students participated in the pre- and post-tests of the study. They were all participants of the advanced psychology module, which can only be attended after successful participation in two introductory lectures for the basic psychology module. Since the study was run in the field (that is, in regular seminar groups), we had to respect that students did not want to provide demographic data that would potentially allow identification of individual cases. Thus, we can provide only approximate demographic information. In each group there were more females (ca. 80%), and the students’ ages ranged from 22 to 26.

Design and Procedure

Data were collected in five seminar groups. The first group was provided with the program described in Table 2 over a semester with 12 sessions of 90 minutes each (formative assessment and feedback with micro-teaching—semester; n = 13; treatment time 15 h). The second group was provided with the program described in Table 1, but due to organizational constraints the 12 sessions were provided block-wise over an intensive course lasting 3 days with 6 hours per day (formative assessment and feedback with micro-teaching—block; n = 25; treatment time 15 h). These two groups are referred to as the ITF-treatment groups hereafter. Students of the third group were participants in a regular seminar on formative assessment and feedback that included most parts of the knowledge and acquisition phase described in Table 2, but not the parts of the two knowledge integration and transfer phases (seminar on formative assessment and feedback; n = 11; 9 h). Students of the fourth group were participants in a seminar on motivation that included one session on formative feedback (seminar on motivation; n = 18; 90 min.). In order to reduce potential teacher effects, this session on formative feedback was taught by the same teacher as in the third group. Finally, a course focusing on psychological issues related to students with special needs served as the control condition, because this course did not include any instructional input on formative feedback (control, n = 20). In all non-treatment groups, the pre-test data were gathered in the first sessions of the courses and the post-test data six weeks later. In the ITF-treatment groups, the pre-test data were gathered before the opening session, and the post-test data in the closing session. In the blocked treatment version this meant that the post-test data were collected at the end of the third day, while there was a week in between the last session and the post-test for the semester-wise treatment group.

Measures and Data Analyses

Students’ perceptions of the seminars including the micro-teaching-components were gathered through a questionnaire addressing: (a) satisfaction with the content, organization, and methods of the training (12 items; e.g., “the seminar was interesting for me”; “the course material supported my understanding”; Cronbach’s alpha .78); (b) the perceived utility of the acquired knowledge and competencies for teachers (3 items; e.g., “I will be able to apply the knowledge I have acquired in this seminar to my future work as a teacher”; Cronbach’s alpha .72); and (c) the perceived knowledge and competency gain (13 items; e.g., “I can describe core principles of designing formative feedback”; “I can plan a feedback strategy for an instructional context”; Cronbach’s alpha .81). Students had to provide their responses to the questionnaire items on a rating scale ranging from 1 (do not agree) to 5 (totally agree). Furthermore, they were asked to state briefly which of the seminar components they considered beneficial for their professional development.

Students’ knowledge and competencies on formative feedback strategies were assessed in the pre- and post-test. The knowledge test consisted of a free recall task (i.e., describe the six most important formative feedback principles that can be derived from the ITF-Model; note that these principles have been explained and illustrated in the educational psychology lecture that is a mandatory prerequisite for joining the seminars). The competency test assessed students’ reflective and argumentative competencies. It provided a vignette that was designed in a similar way to the vignettes that served in the knowledge elaboration phase as a basis for reflecting and discussing issues on how to apply formative assessment and feedback principles. The pre-test vignette included a feedback scenario in which a teacher is providing feedback on an oral presentation in a geography lesson. Her feedback strategy includes communicating and discussing the standards for the oral presentation, a self-assessment phase, and an external oral feedback phase, in which she provides external oral feedback in a bilateral conversation with a student. The post-test vignette included a feedback scenario in which a teacher provides feedback on a written language assignment (writing a fairy-tale). His feedback strategy includes an individual feedback phase (providing written feedback with the help of rubrics), and an open plenary feedback phase in which he provides oral feedback to selected students. In both vignettes, half of the ITF principles were respected and half of them were ignored. Students had to answer seven case-based reflection and argumentation items (e.g., which feedback principle is not respected by the teacher in this phase of the scenario? Explain why it is not respected and what it would mean for the present phase to respect it). To analyze students’ responses to the pre- and post-test items we developed a coding scheme based on the formative feedback principles derived from the ITF-Model (Kleineberg, 2017; see also Table 1). Two independent raters coded the student responses with this scheme (interrater reliability κ = .95).

Results

Students’ Perceptions of the Formative Feedback Seminars with Micro-Teaching Phases

The questionnaire data of the two ITF-treatment groups show that students of both groups were highly satisfied with the content, organization, and methods of the seminars (mean_block = 4.49; SD =.35; mean_weekly = 4.57; SD =.35). Furthermore, they considered the knowledge and competencies they had acquired through the seminar as highly relevant for their future professional life as a teacher (mean_block = 4.71; SD =.43; mean_weekly= 4.79; SD =.26). Moreover, they had very positive perceptions of their knowledge and competency gains (mean_block = 4.27; SD =.53; mean_weekly = 4.43; SD =.34).

The analyses of the open statements revealed that the students valued both the knowledge acquisition and elaboration components, as well as the knowledge integration and transfer phases. They particularly emphasized that they appreciated having the opportunity to apply the knowledge they have acquired in the first phase when planning, implementing, reflecting, and revising a concrete feedback strategy, as well as receiving and generating formative feedback on the micro-teaching units.

Students’ Gains in Knowledge and Competencies on Formative Assessment and Feedback

The analyses of the knowledge and case-based competency test responses revealed that students had a very low level of knowledge and competencies on formative feedback strategies in the pre-test even though they had attended an introductory lecture on educational psychology that included two sessions on concepts, models, and principles of formative feedback (see Table 3 for descriptive data).

Table 3.

Means and Standard Deviations of the Knowledge and Competence Scores from Pre-test (t1) and Post-test (t2) Obtained within the Five Study Conditions.

Condition	Knowledge t1		Knowledge t2		Competence t1		Competence t2
Condition	Mean	SD	Mean	SD	Mean	SD	Mean	SD
Control No input on formative feedback	.42	1.06	.90	1.17	1.65	1.43	1.70	1.87
Seminar—motivation 1 session on formative feedback	.25	.60	1.78	1.31	1.47	1.03	3.33	2.20
Seminar—feedback 6 weekly sessions on formative feedback	.94	1.00	3.18	.98	2.67	1.57	4.09	2.12
Treatment—blocked over 3 days 12 sessions on formative feedback	.12	.44	4.48	1.42	1.28	1.10	5.04	2.24
Treatment—semester 12 weekly sessions on formative feedback	.54	.88	3.23	1.01	1.54	.88	5.23	3.63

Yet the analyses also revealed a significant gain in knowledge and competencies from pre- to post-test (Wilks’ Lambda .25, F(2,81) = 123.94, p < .0001, eta² = .75), and a significant multivariate interaction among condition and measurement time (Wilks’ Lambda .38, F(8,162) = 12.34, p < .0001, eta² = .38).

Follow-up univariate tests and Dunnett’s post-hoc tests indicate that the increase in knowledge and competency vary significantly among the groups. Compared to the participants in the control group, the participants in the ITF-treatment groups, and the participants in the seminars including some instructional input on formative feedback, obtained significantly higher knowledge and competency scores in the post-test compared to the pre-test (Figure 2).

Figure 2.

Mean knowledge and competence gains from pre- to post-test by study conditions.

The knowledge gain is significantly highest in the ITF-treatment group with the blocked condition, followed by the ITF-treatment group with the semester-wise condition, and the group attending the regular seminar on formative assessment and feedback. The motivation group, which had only one session on formative feedback, had a smaller knowledge gain than the treatment groups with 12 sessions on feedback (see also Table 4).

Table 4.

Post-hoc Comparisons and Effects Sizes for the Mean Differences of Knowledge and Competence Gains.

					95% confidence interval
Post-hoc comparisons—Dunnett’s T3		Mean difference	SE	p	Lower bound	Upper bound	Hedges g
		Knowledge gains
Seminar—motivation 1 session on formative feedback	Control—no input—formative feedback	.828	.439	.197	−.2714	1.9270	0.67
Seminar—feedback 6 sessions on formative feedback	Control—no input—formative feedback	2.005^*	.507	.001	.7346	3.2745	1.81
Treatment—block12 sessions over 3 days	Control—no input—formative feedback	3.910^*	.406	.000	2.8950	4.9250	2.97
Treatment—weekly12 sessions over 12 weeks	Control—no input—formative feedback	2.242^*	.482	.000	1.0370	3.4476	2.02
Seminar—feedback 6 sessions on formative feedback	Seminar—motivation 1 session on formative feedback	1.177	.515	.267	−.4464	2.7999	0.98
Treatment—block12 sessions over 3 days	Seminar—motivation 1 session on formative feedback	3.082	.424	.000	1.8310	4.3334	2.24
Treatment—weekly12 sessions over 12 weeks	Seminar—motivation1 session on formative feedback	1.415	.472	.059	−.0336	2.8626	1.18
Treatment—block12 sessions over 3 days	Seminar—feedback 6 sessions on formative feedback	1.906	.539	.019	.2353	3.5756	1.46
Treatment—weekly12 sessions over 12 weeks	Seminar—feedback 6 sessions on formative feedback	.238	.578	1.000	−1.5538	2.0293	0.24
Treatment—weekly12 sessions over 12 weeks	Treatment—block12 sessions over 3 days	−1.668	.499	.023	−3.1747	−.1607	−1.29
		Competence gains
Seminar—motivation1 session on formative feedback	Control—no input—formative feedback	1.728	.813	.121	−.3073	3.7629	0.83
Seminar—feedback 6 sessions on formative feedback	Control—no input—formative feedback	1.677	.939	.238	−.6741	4.0286	0.83
Treatment—block12 sessions over 3 days	Control—no input—formative feedback	3.710^*	.751	.000	1.8308	5.5892	1.75
Treatment—weekly12 sessions over 12 weeks	Control—no input—formative feedback	3.642^*	.892	.000	1.4107	5.8739	1.32
Seminar—feedback 6 sessions on formative feedback	Seminar—motivation 1 session on formative feedback	−.051	.964	1.000	−3.0711	2.9701	−0.01
Treatment—block12 sessions over 3 days	Seminar—motivation 1 session on formative feedback	1.982	.697	.068	−.0871	4.0516	0.88
Treatment—weekly12 sessions over 12 weeks	Seminar—motivation1 session on formative feedback	1.915	1.172	.664	−1.7688	5.5978	0.65
Treatment—block12 sessions over 3 days	Seminar—feedback 6 sessions on formative feedback	2.033	.912	.299	−.8761	4.9416	0.89
Treatment—weekly12 sessions over 12 weeks	Seminar—feedback6 sessions on formative feedback	1.965	1.311	.755	−2.0903	6.0204	0.62
Treatment—weekly12 sessions over 12 weeks	Treatment—block12 sessions over 3 days	−.068	1.129	1.000	−3.6682	3.5328	−0.02

Regarding the differences in the competency gains, the post-hoc tests revealed that compared to the control group the two treatment groups achieved a significantly higher level of post-test competency, while the differences among the competency levels of the motivation as well as the feedback group and the control group was not statistically significant (Table 4).

Finally, it is worth noting that we found significantly positive correlations among the knowledge and competency scores of the pre- (r = .272) and post-test (r = .498), indicating that higher levels of declarative knowledge are associated with higher levels of (reflective, argumentative) competencies.

Discussion

Current insights on the role of formative assessment and feedback strategies for effective learning and instruction (e.g., Hattie, 2012; Narciss, 2017; Shute 2008) imply that future teachers should be able to effectively apply the theoretical and empirical insights on formative assessment and feedback strategies. To do so, they need to be provided with seminars or courses offering scientifically sound material to gain knowledge on these theoretical and empirical insights on the one hand and on the other hand opportunities to exercise the application of this knowledge. The major goals of the project presented in this paper were to develop, implement, evaluate, and iteratively optimize a seminar concept meeting the two requirements. This paper presents the final version of the seminar, and the findings of an exploratory evaluation study. As outlined in Table 2, the seminar included: (a) a knowledge acquisition and elaboration phase; and (b) two knowledge integration and application phases that required students to plan, design, and implement a formative feedback strategy for a typical oral or written student assignment, as well as to reflect and revise this feedback strategy using micro-teaching sessions. The evaluation study reveals the following findings concerning our research questions:

Knowledge: students who studied and discussed the core assumptions, insights and implications of the ITF-Model were significantly more able to recall and describe the ITF principles in the post-test than the students of the control group who were not provided with any input on formative assessment and feedback strategies. Yet the achieved level of knowledge differs among the groups: the motivation group, which had been working only one session with the ITF-model, had lower knowledge gains than the treatment groups and the regular formative assessment and feedback seminar group. The latter three groups had several sessions (5–6) to study and discuss in-depth scientific work on formative assessment and feedback, and elaborate on it with the help of case-based examples. However, the higher level of post-test knowledge of the treatment group which attended the block-wise seminar version might be explained by the timing of the post-test for this group: while all other groups had a week’s delay before the closing session with the post-test, there was no delay for the block-wise group (students had to complete the post-test in the last session of the third day).

Competency: the pattern of result for the competency gains elicits the benefits of providing students not only with a knowledge acquisition and elaboration phase, but also with knowledge integration and transfer phases which include instructional design tasks that require the reflected application of psychological knowledge to the analysis of external text-cases, and the creation, reflection and revision of own video-cases. The treatment groups achieved the highest scores in the competency test of the closing session.

Our findings are consistent with the findings of the training study of Schütze et al. (2017) who also found that participants of their formative feedback training did not only acquire declarative knowledge on formative feedback, but were also able to apply this knowledge in a feedback generation task, and in a case-based reflection task. Furthermore, they are in line with prior work that revealed that both text-based and video-based material are beneficial for fostering the acquisition of pedagogical content knowledge (e.g., Kramer et al., 2020). Moreover, our findings indicate that there can be an added value of developing and using own cases generated through micro-teaching experiences which confirms partly findings from studies on case-based learning with own versus external (video) cases (e.g., Seidel et al., 2011).

Limitations and Implications

The evaluation study was conducted in regular seminars in a preservice teacher education program at a large German university. This field context imposed several challenging constraints that led to several limitations which qualify the findings of the study. First, the treatment seminars were provided within a non-mandatory module, while all other seminar groups were part of a mandatory module within the psychology program for the teacher students. Despite the mandatory nature of the non-treatment seminars, the attendance policy that has been implemented after the Bologna Reforms allows students to be absent from sessions. This happens in particular at the end of the semester, and partly explains the low number of participants in each condition. Thus, we do not know to what extent selection biases and/or selective drop-out biases have contributed to the findings. The latter issue is particularly critical in the groups that ended up with small numbers of participants. Second, students of the control group were participants in seminars that had their focus on different psychological concepts and topics. Thus, they struggled with being asked to work on the pre-test-, and questioned in particular their participation in the post-test assignments. This may have influenced their motivation in responding mindfully to the post-test assignments. To disentangle potential motivation effects from the treatment effects, further studies should collect control variables on students’ motivation and commitment in responding mindfully. Third, the study included two ITF-treatment and three comparison groups, and the treatments of these groups varied in various aspects (e.g., blocked vs. distributed scheduling for the treatment groups; number of sessions related to formative feedback (0 – 1 – 6 – 12); timing of post-test for the blocked ITF-group). The various treatment differences and the small sample size make it very difficult to disentangle treatment effects from confounding effects resulting from the treatment variations, and/or inconsistent mortality effects. They have thus to be kept in mind when interpreting the data. Fourth, for organizational reasons, we used on-campus micro-teaching with peer students instead of micro-teaching with school students. Thus, it is an open issue for further research if and how actively participating in a seminar like the treatment seminars contributes to the acquisition and application of feedback competencies for more authentic school contexts.

Despite these limitations we consider that the findings of our study are promising enough to encourage further work on seminar concepts including theoretical and empirical insights on formative assessment and feedback, as well as training tasks requiring students to apply the knowledge to analyzing and discussing typical external text-cases, and/or creating, reflecting and revising own video-cases.

Footnotes

Acknowledgement

We would like to express our sincere gratitude to the editor Professor Joerg Zumbach as well as the anonymous reviewers for their critical reading of former versions of the manuscript and for providing us with valuable critiques and suggestions.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The work in this paper has been partly supported by the “Qualitätsoffensive Lehrerbildung”, a joint initiative of the Federal Government and the Länder which aims to improve the quality of teacher training. The program is funded by the Federal Ministry of Education and Research (BMBF ). The authors are responsible for the content of the publication.

ORCID iD

Susanne Narciss

Author Biographies

Susanne Narciss is full professor and head of the research team “Psychology of Learning and Instruction (PsyLI)” at the Technische Universitaet Dresden. She coordinates and teaches within the psychology programs for the teacher students as well as for the BA and MA psychology programs. The PsyLI-team is conducting psychological research on issues related to life-long learning and instruction with and within socio-technical systems. A core purpose is to contribute conceptual, theoretical, and empirical insights on the issues of how learning tasks and formative feedback strategies foster learners in using (digital) information resources in smart ways to acquire knowledge and competencies.

Hermann Körndle is professor emeritus at the Faculty of Psychology at the Technische Universitaet Dresden. He has an extensive background in applied cognitive and instructional psychology. From October 1993 until March 2016 he was full professor and head of the PsyLI-Team. He is currently engaged in research on the factors in and effects of technology-enhanced interactive learning tasks; on how to promote core teaching competencies concerning the task and feedback culture in classrooms; and on innovative learning scenarios for vocational education.

Elsa Hammer was a member of the PsyLI-Team at the Technische Universitaet Dresden from 2017 to 2019. She taught courses in the MA psychology programs and also in the psychology program for the teacher students. Her research interests include issues related to the promotion of (preservice) teachers’ psychological knowledge and skills. She was part of the research group that elaborated, implemented, and evaluated the course on formative feedback strategies described in the present paper.

Gregor Damnik has completed his PhD in instructional psychology at the Faculty of Psychology as a member of the PsyLI-Team. Since July, 2019 he is teaching and researching at the chair of “Didactics of Computer Science” at the Technische Universitaet Dresden. He teaches courses in “learning with digital media” and “task construction”. His research interests include higher education and instruction, and automatic task generation, as well as program evaluation.

Kerstin Kisielski is a research assistant and PhD student of the PsyLI-Team at the Technische Universitaet Dresden. She teaches within the psychology program for the teacher students. Her research interest focuses on task-specific learning motivation. More specifically, her research examines the effects of learning task design on students’ expectancy and value-related beliefs.

References

Amobi

F. A.

Irwin

(2009). Implementing on-campus microteaching to elicit pre-service teachers reflection on teaching actions: Fresh perspective on an established practice. Journal of the Scholarship of Teaching and Learning, 9(1), 27–34.

Blomberg

Renkl

Gamoran

S. M.

Borko

Seidel

(2013). Five research-based heuristics for using video in pre-service teacher education. Journal for Educational Research Online, 5(1), 90–114.

Butler

D. L.

Winne

P. H.

(1995). Feedback and self-regulated learning: A theoretical synthesis. Review of Educational Research, 65(3), 245–281.

Desimone, L. M. (2009). Improving Impact Studies of Teachers' Professional Development: Toward Better Conceptualizations and Measures. Educational Researcher, 38(3), 181--199. doi:10.3102/0013189X08331140

Elvira

Imants

Dankbaar

Segers

(2017). Designing education for professional expertise development. Scandinavian Journal of Educational Research, 61(2), 187–204.

Hattie

(2012). Visible learning for teachers: Maximizing impact on learning. Routledge.

Höppner

Dotzler

Körndle

Narciss

(2019). Training mit Microteaching zur Entwicklung und zum Einsatz formativer Feedbackstrategien in Lehr-Lernsituationen (Training for the development and implementation of formative feedback strategies with Microteaching). In G. Uhde & B. Thies (Eds.), Kompetenzentwicklung im Lehramtsstudium durch professionelles Training (Competence development in teacher education through professionel training programs) (pp. 23-35). TU Braunschweig. https://doi.org/10.24355/dbbs.084-201901231126-0

Kleineberg

(2017). Erfolg von Lehrertrainings zum Einsatz von Feedbackstrategien im Unterricht kompetenzorientiert messen (Competency-based assessment of a teacher training on formative assessment and feedback). (Unpublished research report.) TU Dresden.

Kluger

A. N.

DeNisi

(1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119(2), 254.

10.

Kramer

König

Strauß

Kaspar

(2020). Classroom videos or transcripts? A quasi-experimental study to assess the effects of media-based learning on pre-service teachers’ situation-specific skills of classroom management. International Journal of Educational Research, 103, 101624. https://doi.org/10.1016/j.ijer.2020.101624

11.

Mostert, M. P. (2007). Challenges of case-based teaching. The Behavior Analyst Today, 8(4), 434--442. https://doi.org/10.1037/h0100632

12.

Narciss

(2012). Feedback in instructional contexts. In N. M. Seel (Ed.), Encyclopedia of the learning sciences (Volume F(6), S. 1285–1289). Springer Science & Business Media, LLC.

13.

Narciss

(2013). Designing and evaluating tutoring feedback strategies for digital learning. Digital Education Review, ( 23), 7–26. https://revistes.ub.edu/index.php/der/article/download/11284/pdf

14.

Narciss

(2017). Conditions and effects of feedback viewed through the lens of the interactive tutoring feedback model. In D. Carless, S. M. Bridges, C. K. Y. Chan, & R. Glofcheski (Eds.), Scaling up assessment for learning in higher education (pp. 173–189). Springer.

15.

Orr

Weekley

(2019). Teaching with case studies in higher education. In A. Baron & K. McNeal (Eds.). Case study methodology in higher education (pp. 180-208). IGI Global.

16.

Ostrosky

M. M.

Mouzourou

Danner

Zaghlawan

H. Y.

(2013). Improving teacher practices using microteaching: Planful video recording and constructive feedback. Young Exceptional Children, 16(1), 16–29.

17.

Schott, F., & Ghanbari, S. A. (2012). Bildungsstandards, Kompetenzdiagnostik und kompetenzorientierter Unterricht zur Qualitätssicherung des Bildungswesens (Standards, Competency Diagnosis and Competency-based Teaching). Waxmann.

18.

Schütze

Rakoczy

Klieme

Besser

Leiss

(2017). Training effects on teachers’ feedback practice: The mediating function of feedback knowledge and the moderating role of self-efficacy. ZDM Mathematics Education, 49(3), 475–489. https://doi.org/10.1007/s11858-017-0855-7

19.

Schütze

Souvignier

Hasselhorn

(2018). Stichwort—formatives Assessment (Keyword—formative assessment). Zeitschrift für Erziehungswissenschaft, 21(4), 697–715.

20.

Seidel

Stürmer

Blomberg

Kobarg

Schwindt

(2011). Teacher learning from analysis of videotaped classroom situations: Does it make a difference whether teachers observe their own teaching or that of others? Teaching and Teacher Education, 27 (2), 259–267. https://doi.org/10.1016/j.tate.2010.08.009

21.

Shute

V. J.

(2008). Focus on formative feedback. Review of Educational Research, 78, 153–189. https://doi.org/10.3102/0034654307313795

22.

Ziener, G. (2006). Bildungsstandards in der Praxis. Kompetenzorientiert unterrichten (Educational Standards and their Application, Competency-oriented Teaching). Seelze-Velber.

Promoting Prospective Teacher Competencies for Designing,Implementing,Evaluating,and Adapting Interactive Formative Feedback Strategies

Abstract

Keywords

Introduction

Interactive Tutoring Feedback Strategies

Design of the Seminar Concept and Material

Goals and Study Objectives

Content, Instructional Material, and Structure

Evaluation of the Seminar

Participants

Design and Procedure

Measures and Data Analyses

Results

Students’ Perceptions of the Formative Feedback Seminars with Micro-Teaching Phases

Students’ Gains in Knowledge and Competencies on Formative Assessment and Feedback

Discussion

Limitations and Implications

Footnotes

Acknowledgement

Declaration of conflicting interests

Funding

ORCID iD

Author Biographies

References