Abstract
This programme evaluation study examined the changes in writing motivation, self-regulation and integrated writing task scores of international students over the course of a pre-sessional course on English for academic purposes (EAP). In addition, the study also investigated the relationship between writing task motivation, self-regulation, and essay scores at the beginning and end of the course. A quasi-experimental pretest–posttest design was utilized. Participants’ (n = 64) motivation and self-regulation were assessed at the beginning and the end of the month-long course using self-report questionnaires. Furthermore, participants completed an integrated writing task in the first and final weeks of the course. Descriptive statistics and Wilcoxon signed-rank tests showed motivation and self-regulatory strategy use to remain stable over time, except for self-efficacy measures which increased significantly. In addition, scores on the integrated writing task increased significantly. Spearman’s rank correlation analysis confirmed the strong inter-relationship between self-efficacy and self-regulation. At the end of the course, mastery goals, performance-approach goals, and utility value were found to be significantly correlated to essay scores. Further results and implications for foundation course developers will be covered in the article.
Keywords
I Introduction
Intensive foundation programmes, such as English for academic purposes (EAP) courses, are ubiquitous in Anglo-Western universities (Hyland, 2018). Such courses aim to prepare students who use English as a second language (L2) for tertiary level study. To ensure the effectiveness of foundation courses, it is necessary for course providers to conduct programme evaluations to both assess the quality of the courses and to assist in the development of future provision (Kiely, 2009). The primary aim of language programme evaluation is to determine the effectiveness of programme provision (Kiely, 2009). Programme evaluation is defined by Norris (2006) as ‘the gathering of information about any of the variety of elements that constitute educational programs, for a variety of purposes that primarily include understanding, demonstrating, improving, and judging program value’ (p. 579). Through gaining insight into the achievements of a language teaching programme, course developers can gather data that should assist them in improving the courses that they deliver (Davis, 2012). In terms of evaluating EAP courses, course providers can conduct effectiveness evaluations that focus on specific and measurable aspects of the course such as test performance (Norris, 2016).
The current research is part of a programme evaluation of an intensive month-long EAP course in the UK and focuses on changes in students’ writing motivation, and writing self-regulation, and the relationship between writing motivation, writing self-regulation and essay scores. Evaluating the effectiveness of an EAP course in terms of writing development is important as writing is the main medium of assessment at tertiary level, and hence one’s success at university is dependent to a large degree on how well they can display competence in writing (Hyland, 2017). In addition, in evaluating the effectiveness of EAP provision it is important to determine whether such courses impact learners’ writing motivation and self-regulation. Both motivation and self-regulation can have a positive impact on writing task performance (Teng & Zhang, 2017; Woodrow, 2011), and instruction on academic writing courses can develop both motivation and self-regulatory strategy use (Ching, 2002; Zhang, 2018). Therefore, in this current research I will investigate the development of motivation, self-regulation, and essay writing over an EAP course to demonstrate whether there are any meaningful relationships between these variables, and to discover if these variables change in the space of a 4-week pre-sessional course. The pedagogic implications of the current research are wide reaching due to the popularity of foundation academic writing courses in universities around the world and the need for educators to find effective techniques to help international students with the difficulties that they often encounter when writing in an L2 context.
II Literature review
1 Motivation
A commonly cited definition of motivation describes it as ‘the process whereby goal-directed activities are instigated and sustained’ (Schunk, Meece, & Pintrich, 2014, p. 5). In other words, motivation is the force that helps an individual to achieve their goals from the beginning to the completion of a goal. The majority of research involving writing task motivation in both second language acquisition (SLA) and educational psychology has focused on the relationship between writing task performance and learner goals, values, and beliefs, which are commonly operationalized as achievement goals, self-efficacy, and subjective task value, respectively. These motivational concepts will be discussed below.
2 Achievement goals
Students have underlying aims that influence how they perform in achievement activities such as writing an essay. The focus of achievement goal theory is an individual’s goal orientation, which refers to ‘the purpose and focus of an individual’s engagement in achievement activities’ (Schunk, 2012, p. 374). In the dichotomous model of achievement goals, a person will exhibit either a mastery or performance orientation towards a task. According to Ames and Archer (1988), with a mastery goal orientation an individual places importance on the development of new skills and ‘the process of learning itself is valued, and the attainment of mastery is seen as dependent on effort’ (p. 260). With mastery goals, learners gain satisfaction from hard work and challenging activities, see mistakes as part of the learning process, expend effort due to the enjoyment of learning something novel, and evaluate their performance in absolute terms (Ames & Archer, 1988). With a performance goal orientation, an individual is primarily concerned with the judgment of their ability by others and deem themselves to be successful if they can outperform others or by successfully performing a task with minimal required effort (Ames & Archer, 1988). Individuals with a performance goal orientation tend to become anxious when they make mistakes and define success as obtaining normatively high grades (Ames and Archer, 1988). Performance goals have further been separated into performance-approach and performance-avoid goals (Elliot, 1999). Performance-approach goals are concerned with gaining positive judgments of competence from others, and performance-avoid goals are directed towards avoiding negative judgments from others (Elliot & Church, 1997).
Several studies investigating achievement goals have been carried out in L2 settings. For example, Woodrow (2006) created a model of adaptive learning using questionnaire data from 275 pre-sessional EAP students at an Australian university. In Woodrow’s (2006) final model of adaptive learning, successful learners exhibited a mastery orientation, positive affect and were more likely to use self-regulatory strategies. Furthermore, mastery goals were significantly correlated with performance on an oral English test. On the other hand, less successful learners tended to have a performance-avoid orientation, showed less positive affect, and used fewer self-regulatory strategies. A significant negative correlation was found between performance-approach goals and oral test scores. In another study of EAP students, Woodrow (2013) found high levels of mastery orientation during a pre-sessional EAP course that dropped significantly after the students started their degree programmes. Woodrow’s (2013) study showed that mastery goals in L2 students may not be stable, especially when changing learning contexts. This is in contrast with studies of undergraduate students in first language (L1) tertiary settings that have found achievement goals to be generally stable traits over the course of an academic year (Tuominen-Soini, Salmela-Aro, & Niemivirta, 2011).
Further studies in L2 settings have found mastery goals to be negatively correlated with task disengagement, and performance-avoid goals to be positively correlated with task disengagement (Liem, Lau, & Nie, 2008), which shows that learners with a mastery goal orientation tend to persist through challenging or boring tasks, while those with performance-avoid goals tend not to. Connections have also been found between achievement goals and strategy use, with mastery-oriented students using more metacognitive reading strategies (Ghavam, Rastegar, & Razmi, 2011) and both deep and surface learning strategies (Liem et al., 2008). In terms of writing and achievement goals in L2 settings, studies are limited. One study by Chea and Shumow (2017), using a cohort of 244 Cambodian undergraduates, found significant correlations between mastery goals and self-efficacy in writing. In addition, there was a significant relationship between mastery goals orientation and scores on a paragraph writing task. No significant correlations were found between performance-approach goals and writing achievement in Chea and Shumow’s (2017) research. In general, studies of achievement goals in L2 settings have been in line with achievement goal theory in which mastery goals are related to adaptive learning behaviours and performance goals are connected to maladaptive learning behaviours.
Research into achievement goals in L2 contexts has consistently found positive correlations between mastery goals and learning outcomes, and negative correlations between performance-avoid goals and learning outcomes. However, there is only a limited body of research that addresses the impact of achievement goals on essay writing performance at tertiary level. As the main focus of university foundation courses is the development of academic writing skills, one of the aims of the current research to investigate the nature of the relationship between achievement goals and essay writing outcomes in an EAP setting.
3 Self-efficacy
As well as goals, a learner’s beliefs about their ability to perform a given task are similarly important. Such beliefs are referred to as self-efficacy beliefs and are defined as ‘the conviction that one can successfully execute the behavior required to produce the outcomes’ (Bandura, 1977, p. 193). In other words, self-efficacy is a belief that one has the necessary skills to complete a task or accomplish a goal. According to Bandura (1994; 1997), self-efficacy stems from four sources. First, mastery experience is the act of actively engaging in an activity in a meaningful way. Second, vicarious experience is observing others such as classmates and teachers and using them as models. The third source of self-efficacy is social persuasion which relates to feedback, and fourth is physiological and affective states which pertains to how an individual feels, both physically and emotionally. It is important to note that these four antecedents of self-efficacy are not trait-like, and so a teacher is able to help a learner develop their self-efficacy over time (Johnson, Edwards, & Dai, 2014).
Studies of self-efficacy in L2 contexts have found that self-efficacy in writing can increase over a course of study. Zhang (2018) charted the development of 59 graduate students over the course of a 14-week sessional academic writing course. At the end of the course, questionnaire results highlighted a significant increase in writing self-efficacy, and interview data showed that the writing course had increased students’ confidence in writing through specific practice (mastery experience) and tutor feedback (social persuasion). Tutor feedback was also identified as being a source of writing self-efficacy development in a study of 67 Japanese university students conducted by Ruegg (2018). Students in Ruegg’s (2018) study who received tutor feedback on every preliminary draft they wrote over the year of an academic writing course significantly increased their writing self-efficacy. Interestingly, in a study that did not find an increase in learners’ self-efficacy over an EAP course, Piniel and Csizér (2015) noted that students on this particular course often complained about a lack of feedback on their written work, highlighting the importance of social persuasion in self-efficacy development.
Further studies involving self-efficacy and writing in L2 settings have discovered correlations between writing self-efficacy and writing performance (Mills & Peron, 2008; Woodrow, 2011). Perhaps the most comprehensive study of writing self-efficacy amongst language learners to date is that of Woodrow (2011). Using a mixed methods research design, Woodrow examined the writing self-efficacy and anxiety in English of 738 Chinese university students. Participants completed a writing self-efficacy questionnaire that comprised of can do statements related to micro skills such as vocabulary usage and macro skills such as paragraph organization. The questionnaire also included open-ended items, which assessed the participants’ perceptions of motivation. To assess writing performance, participants completed an argumentative essay. Through structural equation modeling, writing self-efficacy was shown to positively predict writing performance. Furthermore, writing anxiety was found to have a strong negative relation to both writing self-efficacy and writing performance. Through analysis of the open-ended questions, Woodrow found self-efficacious students exhibited more effort in their academic work and also exhibited more intrinsic reasons for studying, which helped to explain the strong writing performance of self-efficacious students.
As mentioned by Woodrow (2011), self-efficacy and international students’ writing achievement are still relatively underresearched. Furthermore, the majority of studies on self-efficacy and writing have been conducted with cross-sectional designs or have not involved participants in study abroad contexts (e.g. Zhang, 2018). It is hypothesized that international students may exhibit lower levels of self-efficacy than students studying in their home country and that due to the intensive and focused nature of an EAP course, EAP students may show large increase in their academic writing self-efficacy through participating in an EAP course.
4 Subjective task value
Another important factor that energizes a student completing a task is the perceived value that they attach to a task. Subjective task value is defined broadly as ‘a function of both the perceived qualities of the task and the individual’s needs, goals, and self-perceptions’ (Eccles et al., 1983, p. 90). Subjective task value is most commonly operationalized as intrinsic value and utility value. Intrinsic value is defined as the ‘the enjoyment the individual gets from performing the activity or the subjective interest the individual has in the subject’ (Eccles & Wigfield, 2002, p. 120), and utility value refers to how useful a task is in relation to an individual’s current or future plans and goals (Eccles, 2005). A task may have relative value if it is consistent with an individual’s goals while at the same time being of little inherent interest (Eccles & Wigfield, 2002).
Tertiary level studies have shown a link between subjective task value and achievement outcomes. For example, over the course of a semester, Hulleman, Durik, Schweigert, and Harackiewicz (2008) identified intrinsic value and utility value to be predictive of initial course interest and utility value to be the strongest predictor of final grades. A limited amount of studies have been conducted in L2 contexts that have utilized subjective task value measures. One large scale study by Raoofi and Maroofi (2017) of 304 undergraduate students in Malaysia discovered intrinsic value was predictive of writing scores on a short descriptive essay (50 words) and a longer argumentative essay (350 words). Another study by Woodrow (2013) showed utility value to be the strongest source of motivation amongst pre-sessional EAP students. Summarizing the data from interviews, Woodrow (2013) concluded that the main aim of the students was overwhelmingly to get a well-paid job. Parental pressure also added to the prominence of utilitarian aims. Students explained that their parents spent a lot of money sending them to a foreign university, and they felt a duty to succeed in their studies and future career.
Although subjective task value is a well-researched field in educational psychology, few studies have been conducted with L2 students. Furthermore, the majority of research into the relationship between subjective task value and academic outcomes is based on research with young L2 learners, so the current research aims to discover the relevance of the task value construct in relation to L2 university students.
5 Self-regulation
Learners who are motivated are likely to use self-regulatory strategies when performing a task. Research into self-regulated learning has discovered correlations between self-regulation and achievement goals, self-efficacy, and subjective task value (Pintrich, 1999). Self-regulated learning is defined as ‘self-generated thoughts, feelings, and actions for attaining academic goals’ (Zimmerman, 1998, p. 73). Through approximately 30 years of research in various educational settings, theories of self-regulated learning have been shown to be a robust predictor of students’ academic task achievement (Dent & Koenka, 2016). A seminal triadic conceptualization of self-regulation was posited by Zimmerman (1986). According to Zimmerman, self-regulation is conducive to learning and overall academic achievement because self-regulated learners are ‘metacognitively, motivationally, and behaviorally active participants in their own learning process’ (p. 308).
Some studies have looked at self-regulation in L2 environments. In a large scale (n = 512) study of undergraduate students in China, Teng and Zhang (2017) found metacognitive and cognitive strategies to be directly correlated with scores on argumentative essays. Csizér and Tankó (2015) investigated the self-regulatory strategy use of English language undergraduates at a Hungarian university. The students reported moderate levels of self-regulatory strategy use. Furthermore, strategy use was not directly linked to academic achievement; however, control strategy use was directly correlated with motivation, writer anxiety, and self-efficacy. Csizér and Tankó (2015) concluded that the relative low levels of self-regulatory use amongst their participants may be due to a lack of awareness of the importance of strategy use or difficulties in using strategies in actual academic writing tasks. The impact of strategy use instruction has also been researched (Ching, 2002; Nguyen & Gu, 2013). Ching (2002) reported on a study in which 29 undergraduate students took part in a 7-week course on writing strategy use. After the 7-week course the participants had developed in their use of self-regulatory strategy skills, with the most prominent increase being found in essay planning. Furthermore, the writing strategy course also helped to develop the learners self-efficacy in writing, and in turn improved the way that the participants responded to negative criticism.
The research to date on the development of international students’ self-regulation is restricted to only a handful of studies, and similarly few studies have investigated the relationship between self-regulation and motivation in L2 higher education contexts. Due the important role that self-regulation plays in academic achievement, the current research aims to build upon the current limited knowledge of the self-regulatory strategies that novice L2 writers utilize and develop over the course of foundation writing programmes.
6 Aims of the study
The overall aim of the current research is to provide an illuminative evaluation of a pre-sessional EAP course. An illuminative evaluation focuses on analysing the efficacy of specific aspects of a course to gain a deeper understanding of the learning processes that happen on a course (Richards, 2001). As mentioned by Richards (2001), three potential lines of enquiry in a course evaluation are (1) finding out what students have learnt, (2) the extent to which students are motivated, and (3) the learning strategies that students acquire. This evaluation will focus specifically on writing motivation, self-regulation and essay writing scores, because it is clear that both writing motivation and self-regulation can have a positive impact on writing task performance, and that instruction can develop both writing motivation and self-regulatory strategy use. Therefore, this current programme evaluation study aims to discover whether there are any changes in students’ writing motivation and self-regulation and whether these variables have any relationship to essay scores over the course of a month-long intensive EAP programme. The results of the evaluation should give an indication of the efficacy of the EAP course in developing international students writing motivation, self-regulation, and writing skills. By gaining an insight into the motivation and self-regulation of novice international students, teachers and course developers will be better informed in how to cater for their students’ needs who often have difficulties adjusting to their new academic study environment. The current research utilized a quasi-experimental pretest–posttest design which is commonly used in effectiveness evaluation research (Norris, 2016). As mentioned by Norris and Watanabe (2013), contextual barriers often do not allow for the application of experimental research on programme evaluation studies.
The following research questions will be addressed:
Research question 1: Does motivation, self-regulation, and essay scores change over a pre-sessional EAP course?
Research question 2: What is the relationship between motivation, self-regulation, and essay scores at the beginning and end of a pre-sessional EAP course?
III Method
1 Research context
The current study was conducted during a four-week pre-sessional EAP course at a UK university. The primary aim of the course is to prepare international students for their future degree studies, and, second, to help them adjust to their new learning environment. The course is mandatory for international students with conditional offers who do not meet the English language entry requirement of their degree programme.
The course is full time and consists of 15 hours per week of class time and an additional 15–20 hours of independent study. The programme consists of three modules: academic reading and writing (ARW), listening reading and discussion (LRD), and oral presentations. ARW modules account for the largest proportion of classroom instruction as reading and writing are considered the most important skills in university study and tend to be the most challenging skills for international to acquire proficiency in. The course takes a process-genre approach to the teaching of writing skills. For example, in the teaching of introductions, a model introduction is provided, and the various moves of a typical introduction are highlighted to the students. The students then use this information to form their own introductions based on the essay topic of the week. The students receive feedback on their introductions from the teacher and then redraft their work according to the teacher’s feedback. Assessment on the course takes the form of three written assignments that are completed in the first, second, and third weeks of the course. The assignments are graded in difficulty and are designed to reflect the input given in ARW classes in the corresponding week. Furthermore, students take a written test in the form of an argumentative essay on the first day as a means of initial assessment. Both written and oral feedback is given to students after each assignment in individual tutorials.
2 Participants
A total of 64 students took part in the study. Table 1 presents demographic information concerning the participants of the study and shows that majority of participants were female postgraduate students. All of the participants were from mainland China and had not previously studied in the UK. Overall IELTS scores were on average 6.37, which corresponds to B2 level (independent user) in the Common European Framework of Reference for Languages (IELTS, 2018). It is worth noting that the participants had lower scores on the productive skills of speaking and writing than the receptive skills of listening and reading. The bulk of students were entering into degrees in the Management School which includes courses such as accounting and finance. Courses in the Faculty of Arts and Social Sciences accounted for the second largest group of students’ future degrees.
Participants’ demographic data at the quantitative stage (SD in parentheses).
IV Instruments
1 Academic writing motivation and self-regulation questionnaire (AWMSRQ)
To measure the students’ motivational and self-regulatory profiles, the AWMSRQ was utilized (Wilby, 2019). This instrument focused specifically on measuring L2 university students’ motivation and self-regulation in relation to academic writing. The final version of the AMSRQ consisted of 41 Likert scale questions (see Appendix 1). The items measuring motivation in the AMSRQ comprised of constructs that assessed the participants’ beliefs, values, and goals in relation to academic writing. In the literature on motivation, a writer’s beliefs, values, and goals have been shown to influence a student’s effort and persistence whilst completing a writing task (Schunk, 2012). In the AMSRQ, beliefs, values, and goals were operationalized as self-efficacy beliefs, expectancy-value, and achievement goals, respectively. These constructs were chosen due to their prominence in both the theoretical and empirical literature on task motivation, and the positive impact that they have been shown to have on students’ writing task performance (see Kormos & Wilby, 2019).
The achievement goals scales included items pertaining to mastery goals, performance-approach goals, and performance-avoid goals. The items in these scales were adapted from Midgley et al. (2000) to focus on aspects of academic writing. The self-efficacy scale consisted of items relating to general academic writing self-efficacy. This scale was created specifically for the current study and was based on key aspects of academic writing ability from Bailey (2018). As per Bandura’s (2006) instructions, the self-efficacy items began with the phrase I can. The final items in the questionnaire related to motivation measure, intrinsic and utility value. These scales were adapted from studies by Conley (2012) and Kosovich, Hulleman, Barron, and Getty (2015) to focus specifically on the academic writing domain. Table 2 presents reliability scores at Time 1 (T1) and Time 2 (T2) and examples of the scales in the questionnaire.
Academic writing motivation and self-regulation questionnaire (AWMSRQ) reliability and examples of scales.
The items in the self-regulation scale measured the participants’ use of metacognitive strategies during, while, and post writing task. Metacognitive strategies are defined as ‘thinking about thinking’ (Anderson, 2002) and relate to a learner’s reflections and awareness of the self-regulatory strategies that they employ during a task (Anderson, 2005). Reflections on one’s cognitive processes during a writing task allows the students to control their use of strategies, which leads to a greater utilization of self-regulatory strategies that can assist them in successfully completing a task (Anderson, 2005). The self-regulation scale included questions about planning, monitoring, self-control, and self-reflection. These questions were based on items in the metacognitive strategy use in writing scale from Boekaerts and Rozendaal (2007).
The questionnaire ended with a set of demographic questions that cover gender, age, nationality, first language, degree programme, and IELTS scores. Due to the large cohort of Chinese students on the EAP course, the questionnaires were translated into Chinese as the quality of data received is higher when a questionnaire is in the participants’ L1 (Dörnyei & Csizér, 2012). The translator was a Chinese national who had studied translation at undergraduate level and had experience of translating documents from English to Chinese. The final questionnaire included both English and Chinese versions of each item.
2 The writing tasks
Two writing tasks were created to collect data on the participants’ source use in which students were required to complete an integrated writing task of between 250 and 300 words. Integrated writing tasks practice the kind of reading and writing skills that students will perform on their degree programmes and cover a number of key skills, as outlined by Shi (2018): By moving back and forth between reading and writing, students are engaged in locating or extracting source information, summarizing or synthesizing multiple ideas, relating or contrasting different understandings, evaluating or critiquing others’ views, and restructuring or integrating source texts into their own writing. (p.1)
The topics chosen for the tasks were related to education as it is an area of general interest in which the participants will have had some experience to base their arguments. Task 1 focused on whether e-learning will replace classroom learning, and Task 2 asked the participants to give their opinion about whether all schools should be same-sex. The source excerpts were chosen mainly from academic journals and either gave an opinion or provided statistics. The sources provided a roughly equal amount of arguments for and against the topic. On average, the source excerpts were 45.57 words for Task 1 and 35.29 words for Task 2. The input sources were analysed for readability using Coh-Metix (McNamara, Louwerse, Cai, Graesser, 2005) and were found to have similar readability according to Flesch reading ease (Task 1: 35.52; Task 2: 37.86) and Coh-Metrix L2 reading index measures (Task 1: 12.07; Task 2: 11.81).
3 Piloting of questionnaire and writing tasks
The questionnaire went through two rounds of piloting, with different students at each stage of piloting. The participants who took part in the piloting were all Chinese postgraduate students from the Department of Linguistics, who were in the final stages of their master’s degrees. Both piloting sessions consisted of think-alouds in which the participants completed the questionnaire while verbalizing their thought processes. In the verbalization process, students were also asked to report anything they did not understand or that might need changing. The first session resulted in several changes being suggested to the Chinese translation. For example, in the original translation, the Chinese symbol used for academic writing actually meant literature in general, so the participants wrote down the symbol that was used for academic writing. All the suggestions were written in Chinese to aid the translator in editing the questionnaire. After the questionnaire was edited, it went through a second round of piloting, in which only two errors in symbol choice were highlighted. After the second round of piloting, the translator attended to the errors in symbol choice resulting in the final version of the questionnaire. Furthermore, the piloting participants were also asked to complete both writing tasks. The completed samples were checked to see if students had answered the questions appropriately and attempted to use the sources, and on both counts the participants had completed the tasks as intended.
4 Data collection procedures
Participants completed the writing tasks at the beginning of the first and fourth weeks of the EAP course. To control for the effects of task, the research took a counterbalanced approach to the distribution of the writing tests. At T1 half the students were given Task 1, and the other half were given Task 2; this was reversed at T2. A time limit of one hour was given for the writing tasks. The writing tasks were completed in classrooms and were integrated into the course schedule and used for assessment purposes by the teachers on the course. After completion of the writing tasks, the finished essays were collated by class teachers and handed to the researcher. The questionnaire sessions lasted for up to one hour and were run after the writing task sessions in a lecture theatre. After the participants completed the questionnaires, they were collected by the researcher. Only students who agreed to take part in the study attended the questionnaire sessions.
5 Data analysis
The rubric used for rating the writing samples was the reading-into-writing rubric from the Trinity College Integrated Skills in English Exam (Chan, Inoue, & Taylor, 2015). The rubric measures writing performance according to 4 analytic criteria: reading for writing, task fulfilment, organization and structure, and language control. Each criterion is scored on a scale from one to four, and a global score is calculated by averaging the marks of the four measures. All essays were made anonymous by removing names and personal details and coded for later reference. The essays were assessed by the researcher and two experienced raters who both had experience of teaching and coordinating on EAP courses. Global scores were inputted into SPSS version 24 and significant inter-rater correlations were found for all raters’ scores (T1 corr. = 0.85, p < 0.001; T2 corr. = 0.82, p < 0.001).
Using SPSS version 24 the following analyses were conducted. As the variables in the study were not normally distributed, non-parametric tests were used. First, descriptive statistics and Wilcoxon signed-rank tests were conducted on the motivation measures, self-regulation measures, and writing scores at T1 and T2. To account for the increased possibility of type-I error due to multiple testing, a Bonferroni adjusted alpha level of 0.00625 (0.05/8) was utilized. Effect sizes were calculated using the formula r = Z/√N (Rosenthal, 1994) and were interpreted using Cohen’s (1977) guidelines: .2 = small; .5 = medium; .8 = large. Next, Spearman’s rank correlation analyses were performed at both times. Finally, to see whether there were any statistically significant differences between the correlations at T1 and T2 Fisher r-to-z scores were calculated.
V Results
1 Changes in motivation, self-regulation, and writing achievement
The changes in participants’ motivation, self-regulation and essay writing scores between T1 and T2 are detailed in Table 3. Wilcoxon signed-rank tests indicated that the mean score of self-efficacy increased with a large effect size and statistical significance (Z = –6.06, p < .000, r = –.76). Furthermore, essay scores increased significantly and to a large degree over the course of the EAP programme (Z = –5.94, p < .000, r = –.77).
Descriptive statistics and results of Wilcoxon signed-rank tests.
Notes. * significant based on Bonferroni correction (p = 0.00625).
2 The relationship between motivation, self-regulation, and writing achievement at T1 and T2
The results of correlation analysis involving the motivation, self-regulation, and writing measures at T1 are displayed in Table 4. The strongest correlation was between performance-approach and performance-avoid (r = .59, p < .001). The second strongest correlation was found between self-efficacy and self-regulation (r = .55, p < .001). Further correlations were found with intrinsic value and utility value (r = .49, p < .001), mastery goals and utility value (r = 0.46, p < .001), and performance approach goals and utility value (r = .42, p < .01). There were no significant correlations to report between motivation and self-regulation and writing scores.
T1 Correlations.
Notes. * p <.05; ** p < .01; *** p < .001.
Looking at the data for T2 (Table 5), a number of changes can be found. First, there were more significant correlations in T2 than T1. Second, the strongest correlation was found between self-efficacy and self-regulation (r = 0.65, p < .001). Third, both mastery goals and performance-avoid goals recorded significant correlations with all the other constructs at T2. Finally, three measures of motivation – mastery goals (r = 0.27, p < 0.05), performance-approach goals (r = 0.29, p < 0.05), and utility value (r = 0.39, p < 0.01) – correlated significantly with essay scores.
T2 correlations.
Notes. * p <.05; ** p < .01; *** p < .001.
The relationship between the variables in the study remained relatively stable over the course. The correlations between mastery goals and self-efficacy (z = –2.04, p < .01), and mastery goals and self-regulation (z = –2.86, p < .01) increased significantly between T1 and T2. Furthermore, increases were found between the correlations of self-efficacy and performance-approach goals (z = –3.24, p < .01). In addition, the correlations between utility value and self-efficacy also increased (z = –1.89, p < .05) over the 4-week period.
VI Discussion
1 Changes in motivation and self-regulation
The results of the study show an overall increase in mean scores for self-efficacy. The findings relating to self-efficacy in this study mirror those of previous studies that have examined the impact of tertiary level academic writing skills courses on learners’ writing self-efficacy development (Ruegg, 2018; Zhang, 2018). There are several possible contextual explanations for the increase in self-efficacy. First, the foundation course gives students many opportunities for mastery experience, which is the main source of self-efficacy (Bandura, 1997). The primary focus of the course is academic writing, with 25 in-class hours of tuition assigned to the development of academic writing. It is highly likely that such an intensive focus on writing would account for increases in academic writing self-efficacy. This explanation is in agreement with the results of Zhang’s (2018) study. The participants in Zhang’s study reported strong increases in academic writing self-efficacy (d = 1.59) at the end of a writing course. In follow up interviews, participants in Zhang’s (2018) study mentioned the positive role that specific instruction in academic writing skills had had on their confidence as academic writers. Bandura (1994) argues that mastery experience should be sustained and consistent for it to develop into self-efficacy. The EAP course of the study takes a process-genre approach to writing, and accordingly throughout the weeks the students are working on various parts of their writing through classroom activities and homework. Consistent writing practice through a process-genre approach was also mentioned by Mills and Péron (2008) as an effective method of increasing writing self-efficacy.
For mastery experience to be effective, students must feel a sense of accomplishment through overcoming challenges (Bandura, 1997). This sense of progression is achieved in the course through the grading of course content and assessment difficulty. In the first week, students learn how to compare ideas from academic texts and practice the macro structures of essay writing. In the following weeks students are introduced to more higher order concepts such as critical thinking and mitigation. Furthermore, the students are provided with scaffolding for the assignments as the focus on input during the course relates specifically to the course assignments. As a result, by the end of the course, even though the assignment is conceptually more difficult, general self-efficacy in academic writing remains high, because as a learner’s self-efficacy increases, so too does their belief in and desire to complete more demanding tasks (Bandura, 1997).
A further source of self-efficacy that is provided on the course is social persuasion through tutor feedback. Specific and systematic feedback was found to increase self-efficacy in Japanese English as a foreign language students on a writing development programme (Ruegg, 2018). Ruegg (2018) discovered oral feedback from tutors was particularly effective in nurturing self-efficacy. In the current study, for each of the three assignments student complete on the course, both written and oral feedback is given. Course teachers are told to be constructive in their feedback and to focus on structural aspects of essays rather than language errors. This has the effect of providing scaffolding for the learners, which in turn is likely to foster self-efficacy (Bandura, 1997).
The last point relating to feedback concerns the formative nature of assessment on the course. No final grade is given on the course, and the goal of the programme is developmental in nature. Not being graded allows for greater experimentation by students in areas of writing that are novel or challenging, such as paraphrasing. Mills and Peron (2008) stated that when students are not graded they tend to take risks and there is less chance of learners losing confidence in their writing. This relates to how affective states can influence the development of self-efficacy and the assumption that ‘positive mood enhances perceived self-efficacy, despondent mood diminishes it’ (Bandura, 1994, p.4). Through graded tasks, constructive feedback, and formative assessment, students are likely to feel less stress and negative emotions, which in turn helps enhance a student’s beliefs in their abilities (Bandura, 1997). Similarly, it is also highly likely that the reasons for increases in overall writing achievement mirror those of increases in self-efficacy. In other words, due to the specific and intensive focus on writing and detailed written and oral feedback throughout the course, an environment conducive to writing development is fostered.
Task value items remained stable over the course. Utility value means were relatively high at T1 and remained so over the course. High levels of utility value were also recorded by Chinese students on a pre-sessional EAP course in China (Woodrow, 2013). This is not surprising in the current study, as students were likely to be aware of the importance of writing to their future studies at the beginning of the course and were reminded of this throughout the EAP course. Pintrich (1999) points out that utility value can relate to goals of various proximities. Hence students on the EAP course could see both the immediate value of academic writing (passing the EAP course), and the more long-term benefits (getting a degree, getting a good job). The stability of utility value over the course can be explained by the stability of context. Eccles (2005) states that task utility is domain specific, and because of the EAP course having a narrow and consistent focus, the scores for utility value remained stable.
On the other hand, intrinsic value was relatively low at T1 and remained low in T2. Similarly, Woodrow (2013) found intrinsic value to be the lowest scoring item in her study. As the EAP course is mandatory for learners who have not met the language requirements of their course, it is safe to assume that the motivation to attend the course was extrinsic rather than intrinsic. In addition, the course was very intensive with a heavy load of writing, hence students may have been bored or have had negative feelings towards having to complete such a large amount of writing and attend such a large volume of classes on a similar subject. Furthermore, Pintrich (1999) describes intrinsic value as being a domain specific and relatively stable trait like variable, which can explain the lack of significant change in intrinsic value over the course.
Over the foundation programme, scores for achievement goals also remained stable. As the course uses formative assessment and no grades are given for essays, it was assumed that mastery goals would increase, and performance goals would decrease. However, the stability of achievement goals is supported in the literature, especially when the context remains stable (Tuominen-Soini et al., 2011), such as a short foundation course. Although mastery goals remained stable, this was likely to be beneficial for the students as mastery goals have been related to adaptive learning outcomes (Woodrow, 2006). Therefore, it is possible to conclude that elements of the course, such as a lack of grading, may have helped to maintain high levels of mastery orientation.
In addition to achievement goals, self-regulation did not change significantly over the course. This result is different to previous studies (Ching, 2002; Nguyen & Gu, 2013) that have identified increases in self-regulatory strategy use over time. The differing results can be explained due to there being no specific teaching of self-regulatory strategies on the course, unlike the studies mentioned that provided specific input in the use of self-regulatory strategies. Another explanation for the stability of self-regulation is that the students may have lacked the time to reflect on their own work. Reflection is a crucial element of the self-regulatory process as it feeds back into to the initial forethought phase (Pintrich, 1999). As a result, the participants may not have been aware of the self-regulatory processes that were in play throughout the course and thus reported no development when answering the questionnaire at T2.
2 The relationship between motivation, self-regulation, and essay scores
The current study found a significantly strong relationship between self-regulation and self-efficacy at both T1 and T2, with the strength of the relationship remaining strong and stable over time. These findings confirm the association between self-regulation and self-efficacy laid out in social cognitive theory (Bandura, 1997), and are consistent with research in tertiary settings (e.g. Csizér & Tankó, 2015). The relationship between self-efficacy and self-regulation exits because self-efficacious students have the necessary agency to use self-regulatory strategies when performing academic tasks (Zimmerman, 1998). Furthermore, the correlations between mastery goals and self-regulation increased significantly over the course which suggests that as students develop mastery in their studies, they become more aware of the benefits of utilizing self-regulatory strategies (Pintrich 1999).
At T1 of the study, there were no significant correlations between the motivation and self-regulation constructs and essay scores. The lack of correlations between these measures at T1 may be due to the novice academic writers being unaware of their own use of motivation and self-regulation while writing. However, at T2, there were correlations between the writing scores and both mastery and performance-approach goals, and utility value. The correlation between utility value and writing scores supports previous research into utility value and academic achievement at tertiary level (Hulleman et al., 2008). A possible explanation for the development of the relationship between utility value and writing achievement is because the goal of an EAP course is to prepare students for tertiary education. The students are made aware that the skills learnt on the course will be essential for success on their future course. As a result, by the end of the course, the students are very clear about the value of academic writing in relation to their future success which leads to a relationship between a realization of the value of writing and writing performance.
The correlations between mastery goals and performance-avoid measures and writing scores suggests that the participants exhibit a multi-goal approach when completing a writing task. The multi-goal approach is further evidenced by the positive relationship between mastery goals and performance-approach goals measures. Both mastery goals and performance-approach goals were found to motivate students in oral tests (Woodrow, 2006). Furthermore, Harackiewicz, Barron, Tauer, and Elliot (2002) found a combination of both mastery and performance goals to be optimal for academic achievement over a college course. The results of the current study indicate that EAP programmes should nurture both a students’ desire to improve their writing skills, and also their desire to achieve high scores in relation to their peers.
VII Conclusions
Through conducting a programme evaluation of an intense month-long foundation EAP course, a number of conclusions can be drawn. To summarize the findings of the study, the resulting data from the AWMSRQ at T1 and T2 show that the participants’ self-efficacy in academic writing increased dramatically over the EAP course. In addition, all motivation and self-regulation measures apart from self-efficacy measures remained stable over the 4 weeks which suggests that: (1) they are relatively stable traits, or (2) they take longer than 4 weeks to develop, or (3) the course neither enhanced or decreased the role of these individual differences. Furthermore, the participants’ scores on an integrated writing task increased over the course of the EAP programme.
The current study has shed light on a number of issues regarding the efficacy of intensive foundation programmes. First, through focused instruction and essay writing practice, novice students with limited experience of writing from sources in English can develop greatly in their abilities in writing an integrated writing task. These findings indicate that there was learning transfer as a result of the EAP instruction as the participants could apply their recently acquired knowledge to a test taken under timed conditions. Previous research has also shown that EAP instruction can lead to learning transfer on subsequent degree programmes (Terraschke & Wahid, 2011). An important conclusion to draw from the current findings is that by the end of the course the students were better able to perform on an integrated writing task, which is a cognitively demanding task that requires the application of a number of cognitive strategies, with the added pressure of a strict time limit (Shi, 2018). It can therefore be assumed that attendance on a pre-sessional course can help students to develop in their cognitive processing of completing tasks that involve writing from sources.
Second, the current research confirms previous research into the development of self-efficacy in writing through dedicated instruction in academic writing (Zhang, 2018). The current study expands on the research into academic writing self-efficacy by showing that novice writers can develop in their writing self-efficacy beliefs even over a short period of instruction. Therefore, the assumption can be made that pre-sessional courses can provide an environment in which academic writing self-efficacy is fostered. It is also likely that a process-genre approach can provide the optimal conditions for nurturing novice L2 writers academic writing self-efficacy. As found in the current research, a process-genre approach provides mastery experience through the drafting and re-drafting process. Vicarious experience, i.e. providing models of writing to students, is one of the main elements of a genre approach and helps in scaffolding the students to write beyond their current knowledge or proficiency levels. Central to a process-genre approach is feedback on drafts and final products that focuses on developing the writer’s genre knowledge. Pre-sessional courses further provide supportive environments that aim to support and scaffold students leading to more positive affective states (Mills & Peron, 2008).
Finally, it was found that mastery goals, performance-approach goals, and utility value are related to performance measures of integrated writing. In the field of SLA, few studies have utilized measures of motivation in relation to academic writing outcomes, although studies of motivation and academic outcomes have been widely researched in the educational psychology field (Eccles & Wigfield, 2002). As the current research has found, utility value is a significant motivational force for L2 writers at university level. International students are aware that academic writing is fundamental to their success in both their future studies and careers beyond university. Furthermore, few studies in SLA research have investigated achievement goal measures in conjunction with academic writing outcomes. The current research has shown that novice writers who aim to develop their skills while writing, and at the same time focus on getting high grades, can achieve higher scores in essays.
1 Pedagogical implications
The resulting data from the programme evaluation offers a number implications for course developers. It is recommended that EAP courses adopt a process-genre approach to academic writing. As seen in the current study, students tend to enter an EAP courses with low self-beliefs in their writing abilities, and by providing a programme that offers students ample opportunity for practice (Mills & Peron, 2008) and continuous constructive feedback (Ruegg, 2018), students writing self-efficacy can be developed (Bandura, 1997). In addition, as the students reported no significant increases in self-regulation it is suggested that EAP courses focus more on explicit teaching of self-regulatory strategies. Focused instruction on developing self-regulatory strategies has proven to be effective in improving writing quality at tertiary level (Ching, 2002). Finally, due to the positive relationship between utility value and writing scores, the utility of the EAP course and academic writing should be nurtured in students. A number of steps can be taken to help ensure that utility value is fostered. At the initial stages of course development, it is essential to conduct a needs analysis to make sure course content is relevant to the target needs of the students (Hutchinson & Waters, 1987). During the course, it is important to maintain a link between the content of the EAP course and the students’ future course of study (Woodrow, 2013). This can be done through using authentic sources as models for writing and creating writing tasks that allow students to explore themes of interest in relation to their post-EAP studies. Furthermore, EAP courses can host guest lectures from content teachers, which further bridges the link between EAP and university courses. If students appreciate the connection between the EAP course and their future course of study, then they are potentially more likely to produce higher quality assignments.
Both mastery goals and performance goals should also be developed in EAP courses. Taking elements of a process approach in which students create drafts can create an environment in which students are focusing on perfecting specific elements of their writing, and are thus writing to develop and improve their writing skills instead of focusing solely on an end product. At the same time, performance goals can be fostered by introducing students to successful and less successful examples of essays from past students. Through knowing what constitutes a successful and less successful essay, the students then have goals at which they can aim to compare their performance against.
2 Limitations
First, it was not practical or ethical to have a control group due to the limited number of international students available who were all required to receive the same input on the EAP course. Therefore, the results of the study should be interpreted tentatively. However, qualitative data obtained from this course (Wilby, 2019) suggest that the quantitative data is sound. Second, as the sample consists of only Chinese students, caution must be applied, as the results might not be transferrable to more diverse classrooms. However, due to the high numbers of Chinese students in Anglo-Western foundation courses (HESA, n.d.), it is likely that the prominence of Chinese students in the current study is somewhat representative of many similar courses. Comparison studies utilizing a wider demographic of participants, including students who come from educational backgrounds that are more similar to those of the UK, such as other countries in Europe, could provide a richer level of data. Such studies may highlight the impact of educational background on motivation self-regulation, and academic writing development.
