Abstract
Reflective journals have emerged as an effective means of monitoring and developing reflective practice in higher education, as part of a wider metacognitive strategy to transform traditional learning approaches. In addition, assessment procedures of reflective journals appear to be an important factor in enhancing commitment to learning and reflection.
This paper explores the effects of the use of journal writing and formative feedback on the quality of reflective practice, by presenting a case study of reflective learning in a master’s degree program in the North of Italy, in 2014/2015.
The article contributes to the current debate about the effectiveness of one-to-one writing tuition by analyzing student writing through qualitative analysis and quantitative trend assessment. In particular, it provides evidence on formative feedback efficacy in developing the quality of reflective practice. Unlike much of the research into writing support, this study seeks to evaluate the impact of feedback use not only by measuring student perceptions of effectiveness or level of satisfaction, but also by assessing the development of reflective practices.
Introduction
Educational sciences have shown the importance of metacognitive competencies in learning processes and more recently defined them as a promising field for evidence-based learning (Dunn, Saville, Baker, & Marek, 2013). Reflective practice, as a form of metacognitive competence, involves questioning our own ways of being, relating, and acting (Hibbert & Cunliffe, 2015) and presupposes students’ engagement in critical reflection on beliefs, values, feelings, and implicit assumptions that are used in setting and solving a problem (Mezirow, 1991). This implies a shift from traditional ways of learning; in this perspective, the active role of the student in the learning process becomes essential (Boud & Walker, 1990), in line with American Psychological Association (APA) educational principles. As quality principle 1 of the principles for quality undergraduate education in psychology (American Psychological Association [APA], 2011; Halpern et al., 2010) attests, students are ultimately responsible for their own learning […] and must learn to engage in self-regulation to advance their own academic interests. (Dunn et al., 2013, p. 10)
In addition, research appears to suggest that students’ perceptions of assessment play a significant role in their learning behavior and that innovative assessment formats, such as diaries and journals, prompt a rethink into the fairness and transparency of assessing learning and understanding (Boud & Soler, 2016; Struyven, Dochy, & Janssens, 2005).
The diary has emerged as an effective means of developing and monitoring higher education students’ reflective practice (Hubbs & Brand, 2005; MacLeod & Cowieson, 2001): it enables the writer to capture reflection-in-action, and through this process, provides a tool to consciously reflect on tacit practice (Beveridge, 1997; Boud, 2001; Shepherd, 2006); it enables learners to understand their own learning process and to increase active involvement in learning and personal ownership of their development (Mortari, 2012; Thorpe, 2004; Threlfall, 2014). However, the literature shows a number of concerns about its use, related to students’ perception of journaling and assessment evaluation, partly due to different procedures of use and evaluation (Dyment & O’Connell, 2011).
The Use of Feedback in Journaling
Assessment could be continuous and formative, by providing ongoing feedback to students, rather than summative, by offering conclusive feedback upon the quality of reflections made within the journals. In educational practice, summative feedback has been commonly used in reflective journal writing. Rarely do we find any form of progressive feedback offered (Threlfall, 2014). However, Hughes (2011) underlies the important role of formative feedback as part of facilitating assessment for learning.
As has been noted in the literature, timely feedback on student journals can lead students to write more reflectively in subsequent journals. Regular feedback and training can promote the improvement of the quality of the reflective practice over time: students who have received feedback on multiple occasions can journal more deeply (Dyment & O’Connell, 2011; Kim, 2013; Power, 2012; Threlfall, 2014). In addition, formative feedback has perhaps the greatest influence on how students monitor their engagement with learning activities and assess ongoing progress and performance (Nicol & Macfarlane-Dick, 2006; Ramsden, 2003) and leaves control and responsibility for learning with the students, whilst tutors provide the facilitative scaffolding (Todd & McIlroy, 2014). The use of ongoing feedback strategies is essential to assist students’ self-directed learning skills at the early stage of their reflective journal writing process (Kim, 2013).
Even if the benefits of formative feedback are widely recognized, the same cannot be said for the approaches to its implementation, which are different in terms of degree of structuring, the reference to standards and guidelines, and the importance given to the emotional dimension of the reflective practice. Several authors have, in fact, denounced a move to a narrow interpretation of reflexivity that does not take into account the emotional aspects (Ruch, 2002), a “cognitivist strain” (Clegg, Tan, & Saeidi, 2002) that makes reflective practice mechanical and externally subscribed away from Schön’s notion of “professional artistry” 1 (Finlay, 2008). According to this shift, feedback is built around assessment criteria “to encouraging cognitive reflection on key subject issues rather than emotional reflection and personal learning experiences in general” (Kim, 2013, p. 257).
The risk is that elements of models of reflection are turned into checklists that students work through in a mechanical fashion without regard to their own uncertainties, questions, or meanings (Boud & Walker, 1998). Thus, the emotional aspects and perceptions of students and the enhancement of their reflective capabilities are not fully taken into account by educators (Rué, Font, & Cebrián, 2013). Even if it may benefit from referring to guidelines, feedback should stand out from a rationalistic approach and pursue an individual focus and intimacy in the relationship “to touch emotional as well as rational springs” (Gursansky, Quin, & Le Sueur, 2010, p. 789).
According to this perspective, since feedback is a matter of interaction between teachers and students in relation to their observations, interpretations, evaluations, and expectations about how students can improve (Van der Schaaf, Baartman, Prins, Oosterbaan, & Schaap, 2013), teachers should give students enough space to interact (Blatt, Confessore, Kallenberg, & Greenberg, 2008). In providing students with feedback it is advisable to avoid transmitting the impression of judging and criticizing what the student wrote (Walker, 2006). That could happen if the teacher provided comments that are too detailed, especially if negative (Schartel, 2012). Power (2012) underlies: Too much, and there is a danger of mechanical writing, or recipe following (Boud & Walker, 1998; Procee, 2006). Too little, and student anxiety and confusion are likely to have a detrimental effect on the quality of reflection, and the aim of encouraging exploration of thoughts and deep thinking becomes self-defeating. (p. 642)
Reflective Practice and Mentalization: A Theoretical Model to Evaluate Reflective Journals
Assessment procedures of reflective practice in journaling vary considerably: what they have in common is that they analyze the quality of reflexivity referring to different levels, in accordance with the main traditional models (e.g., Boud & Walker, 1998; Kember, McKay, Sinclair, & Wong, 2008; Mezirow, 1991; Wong, Kember, Chung, & Yan, 1995; see also the review of Dyment & O’Connell, 2011).
Agreeing with the claim of a precise and non-equivocal definition of the different levels of reflection (Kember et al., 2008) and considering the need to specify which indicators can be used to assess reflective practices in written texts, we proposed a model that defines reflective practice as the process of attributing mental states to oneself and to others and of explaining one’s own and others’ actions with reference to those mental states (Bruno, Galuppo, & Gilardi, 2011). These include both mental states with a non-epistemic nature (wishes, intentions, emotions) and those with an epistemic nature (beliefs, reasoning, inferences).
Crossing the tradition tied to Schön’s work (1983), 2 which defined reflexivity as a conversation with the situation, with that of the psychoanalytic perspective of reflective function (Fonagy & Target, 1996) and that of the theory of mind (Milligan, Astington, & Dack, 2007), we embrace the approach that emphasizes the non-mechanical nature of reflective practice.
We therefore argue that such conversation with the situation includes a dialogue with one’s own and others’ minds (Fonagy & Target, 1996). Since the cultural perspective of research on the theory of mind highlighted the relationship between mentalizing and language (Antonietti, Liverta Sempio, & Marchetti, 2006), we have proposed to consider the use of mental language as an indicator of the reflective practice, that is, a lexicon composed of terms (cognitive, volitional, emotional) that refer to the process of mentalizing (Bruno et al., 2011; Gilardi, Bruno, & Pezzotta, 2006).
The analysis of mental language lets us recognize, as a first step, the presence and the characteristics of the writers’ reflective practice and, in a second step, assess the quality of their reflective practice, categorized into five levels:
Category 0—non-reflective practice: the written text doesn’t have any mental term; Category 1—declarative reflective practice: the writer relates an event referring to mental states, but in a descriptive style; Category 2—relational reflective practice: the writer compares mental states between subjectivities, or over space and over time; Category 3—interpretative reflective practice: the subject explains a behavior or decision by referring to the underlying mental states; Category 4—critical reflective practice: mental states are used in a critical way to get new perspectives (Bruno & Gilardi, 2014).
In light of previous research that showed the influence of assessment procedures on the quality of reflective practices and the paucity of educational experiences on the use of ongoing feedback (Kim, 2013) and on its impact on the learning experience, this study aims to discuss the effectiveness of formative feedback on students’ reflective practices, by exploring the following: (1) its impact on the quality of reflective practice; (2) students’ perception of effectiveness of feedback and the meanings given to the use of the journal.
Our hypothesis is that the use of ongoing and recursive personalized feedback during the journal writing enhances reflective practices, as well as students’ engagement in the training process.
Method
Participants and Context
The context is a course of organizational psychology within a master’s degree program in the North of Italy, in the academic year 2014/2015. Twenty-four students attended the course, mostly (80%) females, all with a bachelor’s degree, 13 of them in psychology, 11 in social work. The course lasted 10 weeks (18 meetings, 3 hours each).
During the course, students were invited to write a journal entry without any spatial limitation about their learning experience every week. On the fourth week, and then from the sixth week to the end of the course, the teacher and the tutor of the course (i.e., the two authors) provided each student with personalized feedback on his/her journal entry each week. Some students did not receive their feedback weekly, either because they were absent from class or because they did not write due to their absence from the previous meeting or for personal reasons.
Amount of Feedback Received By Each Student During the Course.
Students were provided with our feedback at the end of the lesson or during the break: they were met individually, for 10–15 minutes. In particular, the evaluation of the quality of reflective practice was presented, by comparing it to the previous one, and students were invited to increase their reflective practice level, referring to the above-mentioned coding scheme. Effort was made to avoid producing a judgmental effect: for example, commenting on the quality rather than the levels of reflective practice was preferred (i.e., “this time you have used a more descriptive style of reflective practice…”). Especially in the first feedback, the difference between reflective writing and standard academic writing was highlighted and students were invited to discuss our evaluation. Students were also exhorted to reflect on their experience of the course, explaining thoughts, emotions, and ways of thinking related to the course, rather than using the diary to give a summary of the topics of the lectures.
Following the completion of the course, all students were asked for written informed consent to have their journal entries included in this study. The assessment of the journals did not contribute to the final examination grade since, according to Wallace (1995), when reflective abilities are formally assessed, the students write down whatever they think it is that teaching staff expect, rather than truly exploring and reflecting on their own experience.
Data Analyses
Twenty-three diaries were considered for the analysis (one was excluded because the student had written less than 50% of the entries). This resulted in a pool of 206 journal entries for analysis, due to some missing journal entries.
This study employed a mixed method design, through the following different steps: content analysis, to evaluate the levels of reflective practice; quantitative analysis, to explore feedback impact on reflective practice; thematic analysis, to investigate learning processes activated by feedback and journal use in students’ experiences.
Content Analysis
As suggested by Kember et al. (2008), the analysis unit was the journal entry. Every entry was identified by the number of the week. The analysis included the following steps for each journal entry.
Analysis of mental language: detection of mental words. Categorizing the entry in one of the five levels of reflective practice of the coding scheme. Consensual validation through cross-coding. The two authors independently coded the data, discussed their difference of opinion, and reached a joint conclusion on how the entry should be coded.
Coding data were subjected to a quantitative analysis.
Quantitative Analysis
Descriptive and inferential data analyses of the distribution of the five levels in the corpus and in each entry were conducted to evaluate the impact of the feedback on the quality of reflective practice. The quality of reflective practice was considered as an ordinal variable, in reference to the five levels mentioned above. For this reason non-parametric tests were used.
Thematic Analysis
Entries in which students wrote about journaling were analyzed to investigate the developmental process of reflective practice and to gain comprehension of students’ subjective experience, also in terms of satisfaction for the assessment strategies. Using a data-led approach, the relevant material was selected for each entry and was subsequently encoded. After analyzing each journal entry, the codes were compared and similar ones were grouped into broader categories (Braun & Clarke, 2006).
Findings
Quantitative Results
Findings describe students’ emerging reflective practices and their development during the training process. Figure 1 shows the distribution of the five levels in the whole corpus.
Cumulative classification of students' reflective practice within journals.
Figure 2 shows the frequency of levels 0–4 for each week.
Weekly classification of students' reflective practice within journals.
The data shows a high variability in the use of the categories of reflective practice. The analysis of the distribution of the five levels shows two major trends: the decrease of the frequency of levels 0 and 1 and the increase of the frequency of level 4 in the final phase of the course. In the last two weeks, no entries at levels 0 and 1 are present, while the frequency of level 4 is more than double in comparison to the first three entries.
The Friedman Test was performed to estimate the differences among the mean rank of the categories used during the 10 weeks. The results show a significant difference: X2 (2, n = 13) = 52.94, p = .000. A subsequent post hoc test, performed with Bonferroni correction of the significance level, showed a significant difference between entry 1 and entry 9 (z = −4.08, p < .002), between entry 1 and entry 10 (z = −4.18, p < .001), between entry 2 and entry 9 (z = −3.50, p < .021), between entry 2 and entry 10 (z = −3.59, p < .015), between entry 3 and entry 9 (z = −4.83, p < .000), and between entry 3 and entry 10 (z = −4.92, p < .000), with higher categories in entries 9 and 10.
To explore the relation between the feedback and the quality of reflective practice, a correlation Spearman Rho Test between the number of feedbacks provided to each student and his/her level of reflective practice at the last entry (number 10) was performed. The last entry was chosen as an outcome measure, by assuming the developmental nature of reflective practice. The amount of personalized feedback received by each student shows a positive relationship with the reflective practice by revealing a high correlation (rho = .512, n = 18, p = .015).
Qualitative Results
The thematic analysis shows that journaling and feedback are perceived as useful devices in students’ learning experience. The use of the reflective journal sustains learning in two areas: “reflective practice” and “professional development.” The use of feedback is related to students’ satisfaction in two areas: “sense-making of journal writing” and “responsibility for learning.”
Journaling
Journaling and Reflective Practice
Students claim that the journal was helpful in developing different reflective practices, both cognitive and emotional: I have to admit that writing and re-reading episodes of my day provides me with much more information than I would gain from a simple unstructured reflection of what has been done. (stud 3, m) For someone to open the jar of their emotions, in the classroom or through their diary, it was simple; for others (like me) it took time, exercise and reflection; all of us realized the potential of feelings as an instrument which enables us to learn, to know and to possess knowledge. (stud 4, f)
Journaling and Professional Development
Writing a journal could give such a great value to future and current professional contexts that some students have introduced daily writing about work experiences to enhance their ability of observation, and others declare their intention to maintain this practice to increase the educational opportunity of their internship. I have gradually understood the utility of journaling and in this period I’m trying to write “two lines” at the end of my work day. (stud 3, m) In my baggage I would also put the ability to reflect on action and in action, refined by the weekly journal writing, and I hope it could be particularly useful for me to manage the different situations and different contexts in which I will work. (stud 21, f)
Feedback
Feedback and Sense-Making of Journal Writing
Feedback seems to help students make sense of the weekly writing and in this way to diminish possible discomfort deriving from an unusual teaching practice, which generated an initial uncertainty for some students. Only when we received the feedback, did I finally realize the aim of this tool that was becoming a mysterious thing. (stud 20, f) In time, despite my limitations, through feedback, I began to understand what it was to write a journal, so I changed and I began to write freely, trying to open my mind. (stud 16, m)
Feedback and Responsibility for Learning
Personalized feedback generates a sense of responsibility for learning and for the relationship with the context-class. In this way students feel they are considered as individuals. The personalized feedback to each journal entry made me more responsible because, feeling respected, I took my participation in lessons more seriously and I tried to make an effort in every activity, not only for personal satisfaction, but also to give something to the group in terms of constructive sharing of my experiences. (stud 12, f) Thanks for making us feel real “co-producers” of the course. Thanks for having thought about us all this time, even outside the classes, for believing in us. (stud 6, f)
Discussion
This study extends previous research on the efficacy of diary and assessment procedures by examining the quality of reflective practice over time and its relation to feedback strategies.
As for the quality of reflective practice, quantitative trend assessment shows a progressive presence of more sophisticated categories throughout the 10 weeks of the course. In particular, a difference was found in the use of the categories between the initial weeks of the course and the final ones, which supports the literature that links the potential development of reflective practice to the use of journals (Boud, 2001).
Also, qualitative analysis indicates that students feel they are more reflective, more aware of the influence of emotion and personal thought for learning. Keeping a reflective journal provides students with an opportunity to link not only theory and practice, but also to integrate intrapersonal, interpersonal, cognitive, emotive, and professional aspects of themselves.
Consistent with previous studies (Dyment & O’Connell, 2011; Kim, 2013; Power, 2012) that suggested the importance of feedback, but in a speculative way, according to Threlfall (2014), these results can be attributed also to teachers’ engagement in providing students with personalized ongoing feedback to the journal entries, by showing empirical evidence.
In fact, as for the relation of reflective practice quality to feedback strategies, quantitative data highlights that students that had received feedback on multiple occasions use more sophisticated categories at the end of the course, and that formative feedback can be effective to develop the quality of reflective practice.
In addition, the results of thematic analysis illustrate that the quality of the relationship between students and teachers, built on the feedback process, promotes learners’ commitment in journaling and also deeper reflection. The efficacy of formative assessment is related to the feeling of being respected and acknowledged (Power, 2012). Already other studies have highlighted that trust between students and teacher can be a facilitating factor for reflective practice (Paget, 2001); furthermore, in this study, data suggests that trust, which emerged from personalized feedback, can sustain commitment to learning. Data confirms that the iterative student–teacher relation in the recursive feedback process has a fundamental part in reinforcing the student’s role as an active learner, his/her motivation, intellectual commitment, reflection on performance progress, and reflective practices (Higgins, Hartley, & Skelton, 2002; Kember, 1997), in line with APA educational principles (American Psychological Association, 2011).
Moreover, there is a connection between learners’ commitment in journaling and the feedback strategies that we implemented, in relation to the analysis model (Bruno & Gilardi, 2014): the focus on the way students use mental states produces a process of cognitive and emotive mirroring and allows us to personalize the assessment process and to avoid transforming reflective practice in an externally subscribed and mechanical process (Ruch, 2002). In fact, the analysis model proposed to assess students’ reflective practice can help teachers in providing feedback and reducing the sense of criticism and judgment that can inhibit the reflexivity (Walker, 2006).
Conclusion
This study explores the benefits deriving from the use of teachers’ feedback to the writing process by providing personalized feedback to each student after the weekly writing entry.
A limitation of the study concerns the use of the journal in academic contexts. In an environment based on competitive, cognitively oriented examinations, students may write to obtain approval, rather than exploring their own experience (Boud & Walker, 1998). To reduce the effects of this limitation, the analysis model assesses reflective practices by detecting the mental language. The latter offers the advantage of not being highly socially desirable, since such a lexicon is made up of terms that the writer may use without being aware of doing something desirable for the reader.
Another limitation is that applying the model and providing feedback are time consuming (Gursansky et al., 2010; Lay & McGuire, 2009; Threlfall, 2014) when the class is numerous. If such feedback is perceived as too time consuming, an option may be that teachers could ask students to choose the sections or entries on which they would most appreciate instructor feedback (Moon, 1999). The authors believe that the assessment of the outcome of the students’ writing is professional and ethical and should be part of teaching reflective practices. Teaching one-to-one is a costly approach and it is crucial to highlight the benefits of this strategy. This article contributes to the debate about the effectiveness of one-to-one writing tuition not only by measuring student perceptions of effectiveness or level of satisfaction, but also by assessing the development of the use of reflective practice, thus sustaining the principles of evidence-based learning.
Footnotes
Acknowledgements
We would like to thank all the students involved in this research. We would also like to sincerely thank the reviewers of this manuscript for their very helpful comments, which have improved the paper.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
