Factors Affecting Students’ Likelihood to Access Feedback

Abstract

Previous studies in higher education have found that a considerable number of students do not access feedback. Here, we use assessment system data on nearly 300,000 assignment submissions to statistically analyze demographic and timing factors leading to lower likelihood of feedback collection. The most significant factors were student performance, gender, course level, assignment number, and timeliness relative to the next assessment date. Deprivation was not found to be significant once other factors were accounted for. The insights gained from this study suggest dedicated time for reviewing feedback in course design and consideration of demographics in engagement interventions are needed.

Feedback is recognized as a fundamental component of teaching but is an active area of research due to students reporting dissatisfaction with quality or timeliness and teachers questioning whether effort translates to value (Henderson et al., 2019). Previous work shows where feedback has not enhanced performance because of the student not valuing, engaging with, or understanding it (Carless & Boud, 2018). For students to benefit from feedback, they need to access it. A small feedback collection study in medical students (Sinclair & Cleland, 2007) found only 46% collected formative feedback.

The Open University is a distance learning institution with over 100,000 students, the largest undergraduate student population of any United Kingdom university. Students have a higher average age and more diverse backgrounds. Given the nature of the institution and its students, student success has a greater reliance on engagement with feedback. The university has an online assessment submission system that tracks assignment submissions and return times. Students can see their assignment score once it has been returned but need to download their marked assignment to obtain feedback, and this download is recorded. This study analyzed around 300,000 assessment submissions to gain insight into factors influencing the rate of feedback collection. It assessed the significance of nine factors in three key areas covering demographic factors (gender, ethnicity, Index of Multiple Deprivation [IMD]), assessment-related factors (level; whether the assignment was the first, second, or third in the course; performance of student on the assignment), and timeliness factors (whether the student had an extension, the return time from assignment cutoff date, whether the return was before or after the next assessment cutoff dates for any of modules studied). This study is the first to analyze large-scale empirical evidence on student feedback collection.

Method

The study included 88,000 undergraduate students actively studying at The Open University in October 2022 at Levels 1, 2, and 3 (corresponding to Years 1, 2, and 3 in other universities) and included 296,628 assignments. Sixty-four percent of the students included in the study were female, 3% were Black, and 18% were in the lowest quintile of postcodes in the IMD (see supplementary material available on the journal website).

Statistical Analysis

Separate χ² tests were conducted to determine if each of the nine factors was individually significant in explaining the differences in collection rates. A stepwise logistic regression, including each of the nine factors as potential predictors, was then undertaken to determine the order of importance of the nine factors and which factors remained significant. Finally, first-level interactions between the factors were also included in the model. All analysis was undertaken using SAS Enterprise Guide 7.1. In the individual χ² analysis, p values were used to determine significance, and goodness of fit for the logistic models was determined using Hosmer and Lemeshow goodness of fit (Hosmer & Lemeshow, 2013) and Somers’s D.

Results

All nine factors were found to be individually significant (p < .001). Variation by factor is shown in the proportion of marked assessments not collected (see Figure 1). The largest variation is for assignment score (6% noncollection in the highest category and 25% in the lowest).

Figure 1.

Noncollection rates by factor showing rates for each category.

A stepwise regression model identified the order of importance of the nine factors to be assignment score, gender, assignment number, whether feedback was returned before or after the cutoff date for any assessment on any course studied, course level, ethnicity, whether the student had an extension, and time taken to return the assignment. The stepwise regression did not include IMD, which is interpreted as meaning this is no longer significant once the other factors had been taken into account. Including the first-level interactions revealed 28 significant effects (see supplementary Table 1 available on the journal website). Table 1 shows the first 10. Hosmer and Lemeshow goodness of fit had a χ² value of 5.2 on 8 df, and Somers’ D was 0.4 (for further details, see supplementary material available on the journal website). These measures indicate that that model was a good fit for the data.

Table 1

First 10 Entries in Stepwise Logistic Regression With First-Level Interactions

Order of Entry to Model	Variable	df	χ2 Score
1	Score	3	7,294
2	Gender	1	3,343
3	Assignment number	2	1,725
4	Proximity to next assignment	1	567
5	Course level	2	407
6	Gender × Course Level	2	386
7	Ethnicity	2	342
8	Proximity to Next Assignment × Assignment Number	2	337
9	Extension granted	1	214
10	Time to return assignment	2	180

Note. The p values for all variables are <.0001.

Discussion

This study analyzed factors related to the likelihood that students would collect their marked assessment feedback. The most significant factor was found to be the score the student achieved, indicating those at greater risk of poor performance were less likely to collect feedback. Previous research has also found poor performance as the main reason why students of all backgrounds drop out of STEM degrees (Theobald et al., 2020). This link between poor performance and feedback collection suggests interventions encouraging poorer performers to access feedback should be explored further.

Previous work has found engagement varies by student demographic characteristics, with Black students nearly 1.5 times more likely to drop out (Social Market Foundation, 2017). Although the percentage of Black students included in the study was small, given the large volume of assessments included, we were able to determine that ethnicity was a significant factor influencing the likelihood of feedback collection, with Black students having, on average, lower rates of collection. Also significant was gender, with male collection rates being lower. In contrast, IMD was not found to be a significant factor.

Although all timeliness factors assessed showed that in general, faster return times lead to a greater likelihood of the student collecting their feedback, the most significant of these factors was whether they received the feedback before the cutoff date of the next assignment they needed to submit, irrespective of whether it was for the same course or a different one. This showed that there is a risk that students mentally move on and do not allocate time to review feedback. Consideration should be given to having dedicated times for reviewing feedback, particularly for courses that have a modular design.

In conclusion, these findings suggest important implications for student support and course design that warrant further exploration.

Supplemental Material

sj-pdf-1-edr-10.3102_0013189X241285416 – Supplemental material for Factors Affecting Students’ Likelihood to Access Feedback

Supplemental material, sj-pdf-1-edr-10.3102_0013189X241285416 for Factors Affecting Students’ Likelihood to Access Feedback by Colette Christiansen, Carol Calvert and Clare Morris in Educational Researcher