Abstract
Feedback can occur before and after assessment submission, but needs to be useful in order for students to improve their subsequent performance. Arguably, undergraduate students, and particularly international, online and new students, are especially in need of feedback to effectively engage in academic and disciplinary expectations. Therefore, this article draws on survey data from students, disaggregated by mode of study, citizenship of enrolment and year of study, to explore their experiences of feedback usefulness both before and after assessment submission. Overall, undergraduate students were positive; however, this perception decreased according to their year level. Comparisons between online and international students also revealed key differences. A conclusion is that undergraduate students cannot be treated homogeneously, and educators need to attend to the feedback experiences of different student groups as they progress through their programme.
Feedback: process not product
Feedback is known to have a powerful influence on learning (Hattie and Timperley, 2007), but only if learners can make use of it (Winstone et al., 2017). It is a common misconception that feedback is the delivery of information from educator to learner after submission of a task. It is a familiar cry of educators to say ‘here is your feedback’, and that of institutional policy to require feedback to be ‘given’ to students within certain time frames. These actions reinforce a teacher-centred perspective, focused more on what the teacher does and the labour of giving, rather than a learner-centred perspective which recognizes the active role of students in their own learning (Carless and Boud, 2018; Evans, 2013; Nicol, 2010). It is argued that feedback should not be thought of simply as an artefact that is given by educators to students in a one-way flow of information. Instead, it should be thought of as a learner-centred process, in which learners need to be able to make sense of performance information and then act upon it in order to improve their future performance (Boud and Molloy, 2013; Carless, 2015; Nicol and Macfarlane-Dick, 2006). Importantly then, the feedback comments within that process need to be useful to the learner – that is, understandable, sufficiently detailed and usable in order to affect change (Ryan et al., 2019).
Feedback is about improving learners’ understanding, skills or other capabilities such as study habits, motivation and self-regulation (Nicol and Macfarlane-Dick, 2006; Sadler, 2010). However, one of the greatest criticisms of current feedback practices is that comments are not actionable or usable (Li and De Luca, 2014; Orrell, 2006; Walker, 2009). In order for feedback information to be useful, students need to find ways to be able to action that feedback and test their new understanding (Boud and Molloy, 2013; Gibbs and Simpson, 2004; Pitt and Norton, 2017). Educators can help in this process by designing learning activities, including assessments, to provide contexts for learners to generate feedback information and to subsequently action it. Based on this argument, feedback should not necessarily be thought of as something that comes after submission, but rather something that occurs between – or connects – performances (Boud and Molloy, 2013).
To enhance their usefulness, feedback comments should be future oriented; that is, looking at current performance with a view of how it can inform future action. That future action could be the next informal assessment in class, or it may be something that occurs in subsequent subjects, or even in the workplace. When thinking about class design, feedback is most useful when it is positioned between tasks, allowing learners to promptly act on the information and test their understanding. The initial task does not need to be the submission of assessment. It could be in the form of a variety of tasks, such as essay drafts or quizzes, which expose the learner’s understanding or skills. Given that the purpose of feedback is to improve future performance, there is a need to further explore perceptions of feedback usefulness, both before and after assessment.
Learners’ experience of the usefulness of feedback
Studies that have explored feedback usefulness often highlight the notion that learners may not always understand comments provided by educators (Henderson et al., 2019; Hendry et al., 2016; Orsmond and Merry, 2011; Price et al., 2010; Rand, 2017). Misunderstandings may occur for various reasons, including not being able to decipher messy handwritten comments in the margins of essays (Ferguson, 2011; Ryan et al., 2016), the use of unnecessarily complex academic language (Chanock, 2000; Higgins et al., 2001; Mutch, 2003; Sadler, 2010; Winstone et al., 2017), information that is inconsistent between educators, for example, personal preferences of educators in relation to style or formatting of written work (Rand, 2017), unclear language that is designed to mitigate criticism (Hyland and Hyland, 2001) and comments that are indirect, vague or unconstructive (Henderson et al., 2019; Zhang and Zheng, 2018). Interestingly, Zhang and Zheng (2018) reported that some educators consider ‘implicit suggestions using vague language [to be] the most useful feedback’ (p. 1129), arguing that they encourage learners to engage in their own deeper thinking rather than being provided with answers. However, in many cases, learners feel disappointed or frustrated by such vague remarks and may be inclined to disregard the information entirely (Mahfoodh, 2017; Zhang and Zheng, 2018).
Others have noted the need for comments to be detailed enough for the learner to gain useful meaning from them (Hepplestone and Chikwa, 2016). For example, Ferris (1997) found that learners were more likely to implement revisions to their work when they received feedback comments that were longer and more specific than when they received shorter or general comments. This makes sense, as learners are limited in their ability to act upon single-word examples of praise (‘good!’) or criticism (‘wrong!’), ticks, circled text or underlines (Price et al., 2010). It is well known that educators do not always have the capacity to provide in-depth comments on assessment tasks, due to time and labour pressures. However, a lack of appropriate detail can hamper the utility of feedback comments for learners and may even be perceived as reflecting a lack of care (Price et al., 2010).
Providing detailed and understandable feedback comments is not enough to ensure that they will be useful; the information must also be actionable (Hepplestone and Chikwa, 2016). Oftentimes, learners do not pay much attention to the feedback information because their next assessment task does not have similar or linked learning outcomes, or the feedback information is provided after the teaching period has finished (Rand, 2017). This is becoming a common problem as higher education institutions increasingly offer more modularized subjects, where assessments are often bunched towards the end of semester or the learning outcomes of assessments do not apply to subsequent subjects in the degree programme (Carless et al., 2011; Deepwell and Benfield, 2012). In light of these changes, it is critical for educators to consider the timing and sequencing of feedback comments. In many cases, the most useful time for learners to receive feedback comments is prior to the submission of a piece of assessed work (Boud and Molloy, 2013; Court, 2014; Covic and Jones, 2007; Gibbs and Simpson, 2004). This can be as simple as providing feedback comments on an essay draft, which learners can then use in a final essay. Learners tend to appreciate comments that feed forward to another task and inform them about the expectations of the educator with regard to the quality of that task (Hendry et al., 2016).
Feedback usefulness and student diversity
Learners can have diverse needs when it comes to learning, but they all deserve the opportunity to benefit from feedback processes. It is therefore imperative that educators who design feedback take these individual needs into consideration. This issue is particularly salient since university campuses are being increasingly populated by students from different social and cultural backgrounds, with different experiences and academic needs. This increased diversity brings challenges to universities. While initiatives have been developed to enhance the learning experiences of students whose demographic, socioeconomic, cultural or enrolment patterns differ from the majority of the student body, it is surprising to find scant research examining the experiences of these groups of students in relation to feedback. Ali et al. (2017) highlight the need for such work, arguing that these types of students may not have access to the same cultural capital as most other students. Furthermore, Sambell and Hubbard (2004) note that these students may not learn in the same ways as the majority of the student body, may not be as comfortable approaching academic staff for help, and may not have the same perceptions and expectations of feedback.
International students are one such group. International students may have difficulties deciphering complex academic language (Hyland and Hyland, 2006; Warner and Miller, 2015), and their perceptions of comments may be coloured by their own unique cultural expectations and experiences, particularly with regard to critical content (Hyland and Hyland, 2006; Tian and Lowe, 2013; Warner, 2011; Warner and Miller, 2015). There is some evidence that educators understand that international students, particularly those who are non-native English speakers, may have additional feedback needs when compared with domestic students. For example, educators tend to spend more time providing international students with comprehensive feedback focused on grammar and structure of arguments (Ferguson, 2011; Sanchez and Dunworth, 2015). Evans and Waring (2011) reported that international students were more likely than domestic students to value a high level of detail in their feedback comments, while domestic students were more likely than international students to prefer comments that were framed positively and focused on good aspects of the students’ performance. International students were also more likely than domestic students to seek feedback comments on pre-submission work, such as drafts. Consideration of international students’ learning needs is important, since they represent a substantial proportion of the student body in many countries (UNESCO Institute for Statistics, 2019). For example, in 2017, 19% of higher education students in the United Kingdom and 29% of students in Australia were classified as international citizens (Australian Government Department of Education and Training, 2018a; Higher Education Statistics Agency, 2018).
Another such group of students are those enrolled as online students or via distance learning. Online students are known to have unique issues with feedback. For example, they tend to be somewhat socially isolated from the wider university community (Orlando, 2016) and thus benefit from detailed feedback comments and interactive dialogues with their educators (Wolsey, 2008). They may also feel highly dependent upon feedback in order to improve (Furnborough and Truman, 2009) and put significant weight on the feedback comments they receive (Ortiz-Rodríguez et al., 2005).
It is also important to consider that students may have diverse feedback needs as they progress through their course. Students in their first year of university often find feedback to be different to what they were used to in secondary school and thus have difficulties using it effectively (Robinson et al., 2013). Due to this, educators should provide feedback comments that will help first-year students to transition to the feedback cultures in higher education (Cramp, 2011; Poulos and Mahony, 2008). Students nearing the end of their programmes may also have different needs with regard to feedback. For example, they may require more individualized feedback as they meet the increasing demands and pressures that occur near the end of their programme. However, they should be becoming more capable learners as they proceed through their programme, so it may be that the same feedback which scaffolds work for novice students tends to impede the learning of more expert students; a concept suggested in discussions of cognitive load theory (Clark et al., 2005). At this juncture, only limited research has addressed these possibilities. One such study by Ali et al. (2017) found that year of study was the sole significant predictor of students’ perceptions of and engagement with feedback; as students progressed through their programme, they became increasingly disengaged with feedback. The authors therefore argued that educators need to find year-specific ways to encourage students to enthusiastically engage with feedback.
While many have studied the usefulness of feedback (Ferguson, 2011; Gibbs and Simpson, 2004; Li and De Luca, 2014; Winstone et al., 2017), limited research has examined whether perceptions of feedback usefulness differ between particular student groups or at different points in course of a programme. This is a significant gap in the literature, as there is no single model that will guarantee feedback to be useful for all students (Sadler, 2010). There is therefore a need to further explore undergraduate students’ perceptions of the usefulness of feedback comments received on an assessment task, both pre- and post-submission, and to determine whether students’ perceptions of feedback usefulness differs according to their mode of study (online/on-campus), citizenship of enrolment (domestic/international), or year level (number of years of study).
Method
Survey data were collected as part of a nationally funded research project. This project involved two Australian universities and had the broad focus of investigating the feedback experiences and practices of university staff and students when it came to coursework.
Participants
More than 67,000 students from two Australian universities were invited to participate in the online survey, and 10% (n = 6744) of students volunteered. From those, 67% (n = 4514) completed the survey. Data obtained from 3002 undergraduate respondents were used in this article, of which 69% were women and 81% were aged between 17 and 24 years. The sample comprised 51% students from Humanities and Social Sciences; 24% from Health; 24% from Sciences, Technology, Engineering and Mathematics and 1% from other disciplines, which is broadly in line with the country as a whole (Australian Government Department of Education and Training, 2018b). The majority of the sample were enrolled fulltime (89%; slightly more than in the country as a whole), on-campus (89%; broadly in line with the country as a whole) and as domestic students (78%; broadly in line with the country as a whole). A total of 39% had spent 1 year or less in their programme; 26%, 2 years; 21%, 3 years and 14%, 4 years or more (comparison data for the country as a whole were not available).
Materials
The survey instrument is available at http://der.monash.edu/feedback/wp-content/uploads/Feedback_for_Learning_Survey.pdf and is free to use under a Creative Commons ShareAlike4.0 International License. The survey items were developed through extensive consultation with the literature and feedback experts. The survey was tested for content validity by local and international feedback experts, university staff and students (see Ryan and Henderson, 2018 for further details).
Analyses are drawn from a small selection of survey items. These include three demographics items: mode of study (on-campus or online), enrolment by citizenship (international or domestic) and length of time enrolled in current programme of study (1 = less than 1 year, 2 = 1 year, 3 = 2 years, 4 = 3 years, 5 = 4 years, 6 = 5 years, 7 = more than 5 years). There were also two multiple-choice items (i.e. select all that apply) relating to the type of assessment task (written essay, laboratory/practical test, oral presentation, short quiz/test, portfolio/project, journal/blog/reflective piece, exam, other) and mode of post-submission feedback comments (electronic annotations, handwritten comments, marking sheet/rubric, face-to-face, digital recording, other), and four items measuring usefulness of the most recent feedback comments that students had received from academic staff, both pre- and post-submission. The pre-submission item was directed only at a subgroup of students who mentioned that they received such comments and asked, ‘How useful were the comments you received from university academic staff before submission?’ This item used a 5-point response scale where 1 = Not at all and 5 = Extremely. The post-submission items were directed at the entire sample of undergraduates and asked them to indicate their level of agreement with the following statements: ‘I understood the comments I received from my assessor after submission’, ‘The feedback comments I received from my assessor were detailed’ and ‘I will use/have used the comments I received from my assessor after submission to improve subsequent work’. These items used a 5-point response scale where 1 = Strongly disagree and 5 = Strongly agree.
Procedure
Approval was received from the human research ethics committees of both universities. Recruitment of students occurred by providing the survey link through bulk emails, notifications posted on online learning management systems and on-campus advertising (e.g. flyers, posters and electronic notices). A chance to win one of four AU$400 gift cards (two per university) was offered as an incentive.
Analyses
Non-parametric tests were used to conduct all inferential analyses, as the survey items included ordinal data, and there were large differences in sample sizes between the comparison groups (Kuzon et al., 1996). As such, mean ranks are used in place of raw means when the results of between-groups comparisons are reported. As multiple hypotheses were being tested using the same dataset, the Holm–Bonferroni sequential correction was applied to all p values (with the omnibus criterion of significance set at 0.05) (Gaetano, 2018). The interpretation of all effect sizes were guided by Cohen’s (1988) criteria. For each individual statistical test, not applicable or not able to judge responses were removed from the feedback perception questions prior to analysis. Mann–Whitney U tests were used to test for differences in feedback perceptions between groups for mode of study and citizenship, while Spearman’s rank correlation coefficients were calculated to assess whether experiences with feedback changed according to the number of years each group of students had been enrolled in their programme.
The percentage breakdown of type of assessment tasks were written essay (52%), laboratory/practical text (14%), oral presentation (12%), short quiz/text (11%), portfolio/project (10%), journal blog/reflective piece (7%), examination (5%) and 15% other. The modes of post-submission feedback were electronic annotations (54%), handwritten comments on a hard copy of assessment (23%), on a marking sheet/rubric (40%), face-to-face (12%), digital recording (5%) and 7% other.
Results
Usefulness of feedback comments received pre-submission
Analysis revealed that there were no significant differences between the mean ranks of students who are enrolled to attend university on-campus and those who attend solely via online study or international and domestic students (see Table 1).
Percentage breakdown, mean ranks and results of Mann–Whitney U tests for all undergraduates, on-campus, online, domestic and international students for experiences of the usefulness of pre-submission comments received from academic staff.
Mean ranks and results of Mann–Whitney U tests are not reported for all undergraduates, as these statistics can only be calculated when there are two or more groups.
Spearman’s rank correlations revealed that there were no significant relationships between year of study and usefulness of pre-submission feedback comments for any of the four cohorts (rs ranged from −0.23 to 0.0; p values ranged from 0.75 to 1.0), which suggests that experiences with usefulness of comments prior to submission remained stable across students from different year levels.
Usefulness of feedback comments received post-submission
All students
The majority of students reported that the comments they received were useful (see Table 2). Spearman’s rank correlations indicated that there was no significant difference over years of study for students’ ability to understand the comments (rs = −0.03, p = 1.0), but there was a significant negative relationship, with a small effect, between years of study and the items relating to detail (rs = −0.06, p = 0.03) and intention to use comments to improve (rs = −0.09, p = 0.03). In other words, students who were further into their programmes were less likely to agree that the feedback comments were detailed and usable.
Percentage breakdown for usefulness of feedback post-submission for all undergraduate students.
Mode of study: on-campus and online students
Table 3 shows that online students had higher mean ranks than on-campus students for the detail of the comments, and the difference between the two groups was significant with a small effect. However, while online students had higher mean ranks than on-campus students for the items relating to understanding and intention to use the comments, the differences were not significant.
Percentage breakdown, mean ranks and results of Mann–Whitney U tests for usefulness of feedback post-submission split by mode of study.
Difference is significant at the 0.05 level.
Spearman’s rank correlation tests revealed that for on-campus students, there was a significant negative relationship with a small effect between number of years enrolled and level of agreement that the comments were detailed (rs = −0.07, p = 0.03) and that they intended to use the comments to improve subsequent work (rs = −0.09, p = 0.03). There was no significant relationship between number of years enrolled and understanding the comments (rs = −0.04, p = 0.64). These results indicate that on-campus students were less likely to agree that the comments were usable and detailed as their years enrolled increased. For online students, there were no significant relationships between years enrolled and any of the survey items (rs ranged from 0.07 to 0.01; all p values = 1.0).
Citizenship of enrolment: domestic and international students
Table 4 shows that international students had slightly lower mean ranks than domestic students for understanding, but this difference was not significant. However, international students had significantly higher mean ranks (with a small effect) than domestic students in relation to detail of feedback. International students’ mean ranks were also slightly higher for their intention to use comments to improve, but the difference was not significant.
Percentage breakdown, mean ranks and results of Mann–Whitney U tests for usefulness of feedback post-submission split by citizenship of enrolment.
Difference is significant at the 0.05 level.
In comparison, there were slightly different results according to citizenship across year levels. For domestic students, there was a small, but significant, negative relationship between years enrolled and level of agreement that they would use/had used the comments to improve future work (rs = 0.09; p = 0.03). There were no significant relationships between years enrolled for domestic students and the items relating to understanding (rs = −0.04; p = 0.91) and detail (rs = −0.06; p = 0.13). For international students, there were no significant relationships between year of study and any of the three usefulness items (rs ranged from 0.08 to 0.03; p values ranged from 0.75 to 1.0).
Discussion
This article reports on undergraduate students’ perceptions of the usefulness of feedback comments they had recently received on an assessment task, both pre- and post-submission, and whether perceptions differed according to their mode of study, citizenship of enrolment or year level. As identified earlier, feedback is understood to be a process in which information about the quality of a performance should lead to a change in student work or learning strategies. Arguably, for the most useful impact, feedback should occur prior to the assessment submission or completion. This means that educators are challenged with finding ways to elicit student performance early, and to facilitate feedback which leads to improved submissions.
There were no significant differences between student types in their experience of pre-submission feedback usefulness. Most students who received feedback prior to submission reported that it was moderately, very or extremely useful. Despite the value of early feedback, a large proportion (71%) of students in the study described here did not have the opportunity through feedback to gauge, and improve, their performance prior to submission of their assessment tasks. Therefore, these results suggest that educators in institutions where this is not done should explore more options to increase feedback pre-submission.
With regard to perceptions of feedback comments received post-submission, a comparatively high proportion of students (22.1%) disagreed that the comments were detailed, while 25.7% of the sample were undecided, or disagreed, that they had used or would use the feedback to improve their future work. These results provide a valuable reminder to educators to ensure that comments are not only understandable, but also detailed and – most especially – actionable. For these undergraduates as a whole, perceptions of detail and usability decreased as they progressed through their programme. It is unsurprising that students at different points in their programme may need different levels of detail in feedback comments, and have different levels of appreciation for what they mean to them and what, if anything, they might do, with that information.
Online students were more positive than on-campus students about the detail of the feedback comments; however, it was the on-campus students who had less positive perceptions of detail in the later years of their programme. These results may reflect a tendency for educators to provide online students with more detailed comments than on-campus students. In the absence of other opportunities to check their progress and ideas, such as those enjoyed by on-campus students, the feedback received prior to submission can be a very important influence on online students’ orientation to work. However, it is also possible that on-campus students had additional opportunities through their classes and interactions with peers to discuss their work. In this light, it is plausible that on-campus students build additional mechanisms for feedback which results in them re-evaluating the usefulness of the educator comments, while in comparison, online students remain dependent on the relatively few instances of educator-based feedback that they receive. Clearly, there is a need for further research to better understand why online students feel the feedback they receive is more detailed, and why on-campus students in particular are less positive in the later years of their programme.
International students provided higher ratings on the detail of post-submission feedback comments when compared with domestic students. There are several possible reasons for this result. First, they may reflect a tendency for domestic students to only passively engage with feedback, perhaps due to the idea that feedback comments are typically used only to justify or demonstrate a grade. As such, domestic students may come to view feedback comments as something to be ignored rather than being improvement focused. Second, educators may actually be spending more time providing detailed feedback to international students (Ferguson, 2011; Sanchez and Dunworth, 2015). If so, it would make sense that international students perceived the comments to be more detailed. Finally, the differential results may reflect inherent differences between domestic and international students with regard to their feedback preferences (Evans and Waring, 2011). A striking pattern here is that domestic students perceived the feedback to be less usable across their programmes, while international students did not. This result indicates that educators may be serving most domestic and international first-year students well in their feedback comments, but failing to tailor it as effectively for domestic students in later years. While further research is needed to explore this result, it may be due to educators providing international students with feedback that focuses on their language skills regardless of their year level, while domestic students are receiving less useable comments. It is also quite likely that the negative trend in perceived usability for domestic students is simply because most assessments are not designed to be followed by another task that allows the feedback comments to be utilized.
This study has certain limitations. The sample was self-selected. In addition, the data presented were cross-sectional, so future research measuring students’ perceptions and experiences with feedback over time should be longitudinal. It is also worth noting that perceptions of usefulness were dependent on respondents’ memory. Although the students were asked about their most recent assessment task, they may have forgotten how useful or not useful their most recent feedback comments were. This study also grouped all international students together, despite the fact that there is likely to be a diversity of cultural and language experiences among them. There was also a small degree of cross-over between international and online students, which may have had a confounding effect. Further research is needed to tease apart these differences, and examine their impact on feedback experiences. Also, while this study examined the mode of feedback for post-submission comments, the same information was not collected for pre-submission comments. Importantly, all of the statistically significant results reported had effect sizes that were less than 0.1, so the practical significance of differences or relationships was low. In addition, this study only focused on undergraduates from only two universities in Australia. More research is needed to establish whether the results of this study are applicable for postgraduates, in other countries and cultures, and in other types of universities. Further research could also examine the impact of the type of assessment and the mode of feedback delivery.
Online students were more likely than on-campus students to agree that the comments were detailed. The same pattern was true of international students compared with domestic students. All groups of undergraduate students indicated a negative trend with regard to the usability of feedback comments over the course of their programme. This negative trend was particularly pronounced in domestic students. As a consequence, we are left with many questions that need to be addressed by future research. One such question is whether students’ perceptions are being influenced as a factor of their enrolment. For instance, online students may be so isolated from other feedback opportunities that they remain dependent on educators. As such, their judgements of the usability of educator-provided feedback are less likely to shift over time. Further research needs to be conducted particularly in relation to the nature of the feedback experience of online/on-campus and domestic/international students both pre- and post-assignment submission, as well as over time. Finally, the views of students were gathered about the usefulness of feedback comments they had received on an assessment task immediately following its submission, and so future work is needed on the impact of the feedback over time.
Fundamentally, in order for feedback to be useful, students need to understand the information, it needs to be sufficiently detailed and it needs to be usable. Although this study raises many questions, it highlights the importance of paying attention to the design and usefulness of feedback both pre- and post-submission, as well as considering the experiences of different student groups over time. Educators need to question whether the feedback practices in the first year are useful for those same students in the later years of their course. Furthermore, educators need to consider that different cohorts of students interpret feedback comments in different ways, and so there is always a need to do what we can to ensure that those comments are crafted for usefulness according to different students’ needs.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship and/or publication of this article: This work was supported by the Australian Government Department of Education and Training (Grant No. ID16-5366).
