Blended Course Evaluation in the Context of English for Specific Purposes: Accountability and Development

Abstract

Blended learning utilizes the affordances of information and communication technology to integrate online learning with face-to-face teaching. It facilitates to meet students’ disciplinary learning needs and helps them achieve the intended learning outcomes so that they can advance in their professional study. The present study focuses on developing and evaluating a blended course implemented in the context of English for Specific Purposes (ESP) from the perspective of accountability and development, with the purpose of assessing the effectiveness of the course from three aspects: the satisfaction of students’ learning requirements, the achievement of their learning outcomes, and the ongoing refinement of the course. Data of the course evaluation survey were quantitatively analyzed by descriptive statistics and data of students’ learning reflections were qualitatively analyzed by thematic analysis. Results indicate that, firstly, students are satisfied with the course design and its implementation; secondly, the course modification should focus on adding academic presentations, teaching in English with trans-semiotising approach and consolidating the disciplinary community, which in turn activate students’ knowledge-sharing and critical thinking. The study offers a systematic framework to evaluate the ESP blended course, which also has implications for evaluating blended courses in other language curricula.

Introduction

Blended learning is an innovative teaching paradigm with technology, which seeks significantly to engage students in ways that classroom teaching and online learning are combined systematically. This modality of integrated synchronous face-to-face and asynchronous online learning provides flexibility to both instructors and learners (Garrison, 2017; Vaughan et al., 2013). Through blended learning, class discussion, task collaboration, and interactive activities can be realized either synchronously or asynchronously (Bower et al., 2017; Çakır & Bichelmeyer, 2016; Diep et al., 2017). Both researchers and practitioners suggest that blended learning should integrate instructional approaches and learning strategies to provide feedback on course development and students’ learning experiences (Diep et al., 2017; Hrastinski, 2019; Zacharis, 2015). Therefore, it can be assumed that by integrating face-to-face with online learning, communication between learners and instructors can be both synchronous and asynchronous, and learners will have time to reflect on the learning content and share their understandings in the learning process. In the context of English for Specific Purposes (ESP), learners need not only to interact and share understandings with their peers and/or the instructor, but also to learn the generic disciplinary knowledge (e.g., research article writing) and related academic skills (e.g., academic presentation) (Zhang, 2020). To understand the course effectiveness, evaluation is inevitably addressed to answer questions such as how well the students performed, how thoroughly the course serves the requirements of learners, teachers and other stakeholders, and how the course effectively meets the learning objectives (Tufail & Embi, 2016).

The present study focuses on the ways of evaluating a blended ESP course, English for Agriculture and Forestry, from the scopes of accountability and development (Norris, 2016; Nunan, 1992; Tufail & Embi, 2016; Weir, 1995) to address the research questions:

1. Are students’ learning needs satisfactorily met? What are their learning outcomes?

2. Do the course curriculum and the instructor need modifications? If yes, what are they?

The results from learning outcome assessment evidently illustrate to stakeholders that students’ learning needs have been satisfactorily met and they gained satisfactory learning outcomes. In addition, it is valuable for cyclical practice, in which the instructor can refine the ESP course from the aspects of adding academic presentations, using English as the instructional language, consolidating the disciplinary community for students to share knowledge, and fostering their critical thinking. This research attests the validity and reliability of the evaluation framework for the ESP course. Moreover, it promotes the studies about the effectiveness of establishing the disciplinary community in the context of ESP learning.

Literature Review

A number of researchers have conducted empirical studies to evaluate blended courses in various kinds of disciplines (Halverson et al., 2014; Kavadella et al., 2012; Manyazewal et al., 2017; Zampirolli et al., 2018). Manyazewal et al. (2017) evaluated a blended course on tuberculosis front-line health care. The end-course survey investigated the participants’ previous computer skills, their viewpoints toward online and face-to-face sessions, and their learning experiences of attending the course. The findings reveal that although a number of participants do not get access to the computers and have no experience of taking online courses during their professional services, the attendance of this blended course is still in a high rate and a great number of participants recommend this course to other health care professionals. In addition, this blended course helps more participants get trained without disrupting their work schedules. Zampirolli et al. (2018) implemented a blended course in the engineering education and evaluated the course via formative and summative assessments, in which quizzes, essays, and classroom activities in groups were involved. The findings show that fewer students fail in the blended course than in the traditional face-to-face class, and the number of students who get grade “A” or “B” is lower in the blended course compared with the face-to-face classroom learning. However, the study does not present the intended factors influencing students’ blended learning performances. Kavadella et al. (2012) and Halverson et al. (2014) evaluated blended courses in the medical field. Their studies suggest that students highly value the course organization and learning materials that activate their learning motivation and activity engagement. Additionally, in the learning process, students share understandings, refine their medical skills, and improve the quality for serving their patients.

In the context of ESP, blended learning paradigm has been adopted mainly for initiating students into a disciplinary community which requires students’ general disciplinary knowledge and related academic skills (Zhang, 2020). However, a few researchers focused on evaluating blended course in the context of ESP. Bueno-Alastuey and López Pérez (2014) evaluated an ESP blended course of Human Resources. The usefulness of Information and Communication Technologies (ICT) implemented in the blended course from the students’ perspective. The results show that students who use ICT more frequently are in a large portion of improving their pronunciation and productive skills. The study also suggests adding training and guides to blended courses with the purpose of encouraging students to use ICT. Klimova (2017) evaluated the blended course of Business English. The results of the questionnaire survey and online course report indicate the prevalence of the blended learning mode and students’ satisfaction with the online learning content, but the outcome of the end-term test does not show much effectiveness nor much improvement in business vocabulary.

Up to this point, there has been little research on the ways of evaluating blended ESP courses to assess whether they do in fact meet the students’ diverse needs, whether the course de facto achieves the intended learning outcomes, and what the course modification should focus on so that students can better apply what they have learned to their professional study. The present study answers these questions from accountability and development. Accountability demonstrates to stakeholders whether the course has achieved the intended outcomes, and development provides valuable information for the ongoing refinement of the course. The next section considers these facets in more detail.

Accountability and Development in the Course Evaluation

Accountability is defined by Norris (2016) as “a program evaluation to do with a heightened emphasis on holding public schools, teachers, and students accountable to certain expectations for adequate performance, primarily in terms of learning achievements” (p. 172). To account for learning achievements, in some researchers’ advocation, is test-based. They argue that student testing is a key element of accountability, and the importance of test-based accountability will not decline (Hout & Elliott, 2011; Linn, 2000). The results of tests may reveal some issues in teaching and learning but cannot cover all. There are many other influential factors that need to be considered, such as validity and reliability of the test, students’ uneven levels, and paper grading even with rubrics. However, Popham (1999) reminds us that students’ achievements on standardized tests are in fact influenced by three causative factors: (a). knowledge taught in school, (b). student’s intellectual ability, and (c). students’ out-of-school learning. Therefore, the quality of a course and of the instruction in the course is only one factor in obtaining good test results. To sum up in the meantime, accountability evaluation can be test-based but this is neither necessary nor sufficient in itself, because there are many influential factors which are hard to control. In this respect, tasks can be taken as a means to do course evaluation for accountability (Ektabani, 2011). In this blended course, the tasks completed by students were graded with rubrics and the students were provided with the instructor’s feedback. The task grades were taken as references to account for the students’ learning achievements. In addition, the other means of gathering data, such as the questionnaire survey, observations, and students’ evaluation are valuable for gathering information to present students’ learning attainment to the English department of the university which offers the course. This information also provides indications to the course instructor for making decisions to modify the course. This last goal of the course evaluation is known as development (Norris, 2016; Tufail & Embi, 2016; Zohrabi, 2012).

Evaluation for development serves the purpose of curriculum development and teacher development. Curriculum evaluation refers to “the collection of information on which judgment might be made about the worth and the effectiveness of a particular program. It includes, of course, actually making those judgments so that decision might be made about the future of program, whether to retain the program as it stands, modify it or throw it out altogether” (Hussain et al., 2011, p. 265). This aligns with White (1971) who argues curriculum evaluation is an essential step of constructing and reconstructing curricula for the purpose of examining whether the curriculum objectives have been achieved so that the instructor or stakeholders can make modifications where necessary. In this regard, evaluation for curriculum development serves two important functions: first, it offers a means of collecting information that can be applied to improve a course, and secondly, it offers underpinnings for making decisions about how to use the curriculum effectively and what should be on-going in the implementation.

The evaluation of this blended course was conducted from two perspectives, for accountability and future development. A range of evidence was taken into consideration, beyond student performance data. The factors taken into account when conducting the evaluation are presented in the following section, including the students’ course grades, questionnaire survey of course curriculum (i.e., course objectives, course content, tasks and activities, learner support, learning mode, learning assessment, and learning platform), students’ learning reflections, and teaching observation. Below, Figure 1 summarizes the framework for evaluating the blended course systematically.

Figure 1.

Framework for the blended course evaluation.

Designing the ESP Blended Learning Course

The ESP course, English for Agriculture and Forestry, is instructed in English for graduate students majoring in Agriculture and Forestry. It has been open to graduate students for five years since 2014. The early implemented course failed to engage students and also failed to meet their learning needs, which included the improvement of their essay writing ability in English, the requirement to have their papers published in indexed international journals, and the enhancement of their academic communicative skills so that they can advance independently in their professional studies. The curriculum designed for this course is task-based. A task-based curriculum is not based on language features but related to the language in use for performing specific activities (Ramírez, 1995). Hence, the task-based curriculum is designed according to the functions. The contents of tasks are based on students’ learning needs and reasonably challenge their learning ability (see Appendix A).

In the redesigned blended learning course, two teaching periods (four hours) for the course each week and 26 periods over the course of a semester, as set by the course offering Department—English Department in an agricultural university in mainland China. According to the time schedule, one period per week is set for students to learn online independently out of the campus, and one period is for classroom face-to-face interaction. The online learning materials consist of video lectures, online websites relevant to agriculture and forestry, academic articles from agriculture and forestry journals, and reference books about academic writing for ESP. The video lectures were scripted and made by the present researcher, who is also the course instructor. The video lectures were delivered via the mobile learning platform Mosoink which can be easily downloaded on a mobile phone and accessed either in or outside of the campus when there is a signal. Students can talk to the instructor privately one-to-one anytime and anywhere in Mosoink as in WeChat, China’s most popular social media application. The students are expected to watch videos, raise questions, discuss online, and take part in online activities. The face-to-face class session deals with the questions that students raise online, clarifies the video contents that they do not understand, enables face-to-face discussions, and affords possibilities of live presentations by individuals or groups. The instructor addresses students’ common problems with their tasks in class and also gives the feedback to individual student’s task via Mosoink. The role of teacher is as a facilitator and students are the performers in their learning process. They can ask for the teacher’s assistance online or in person, if they encounter some difficulties in the process of completing the tasks. Based on the above conception, the blended learning model is designed as shown in Figure 2.

Figure 2.

Blended learning model.

Materials and Methods

A total of 88 first-year graduate students majoring in Agriculture and Forestry participated in the study. They took this blended course in the second semester of 2017 taught by the researcher. Data were collected from the students’ task grades, a questionnaire survey of the course evaluation, students’ comments, and teaching observation. Students grades and the data of the course evaluation survey were quantitatively analyzed to demonstrate that the students’ leaning needs have been satisfactorily met and those students gained decent learning outcomes. It was adapted from a set of survey questions for evaluating blended learning courses (The New Teacher Project, 2014) with a modification of items 19, 20, 23 to ensure the survey was more appropriate to the blended course (see Appendix B). The adapted version of the questionnaire was piloted among 20 students and it was decided that no items needed to be further revised. There were 30 items in the questionnaire which covered seven dimensions of course objectives (four items), materials and content (four items), learning assessment (five items), tasks and activities (four items), learner support (four items), learning mode (five items), learning platform (four items). The items were in the form of a 5-point Likert-type scale ranging from 1 = absolutely to 5 = not at all.

All the students (N = 88), responded to the survey delivered via Mosoink, and the response rate was 100%. The reliability of the questionnaire was measured by Cronbach’s Alpha. The reliability coefficients for the various components are: course objectives .807, course materials and content .861, learning assessment .866, tasks and activities .856, learner support .886, learning mode .887, learning platform .876. According to Kline (2000), a coefficient between .6 and .7 is acceptable, between .7 and .9 is good, and above .9 represents excellent internal consistency. Therefore, the internal consistency of the questionnaire is good, and the questionnaire has a high reliability. Descriptive statistics were applied to calculate the mean value of each dimension which was used to draw a “radar chart” for data analysis. In addition, students’ learning reflection was conducted in the form of paragraph writing with the guidelines again given in Mosoink (see Appendix C). Students wrote their opinions regarding the implementation of the course and their engagement with it, with the purpose of gaining rich information about the effectiveness of the pedagogical intervention. Teaching observation recorded the process of teaching by the instructor, the talks between the instructor and the students after class in order to find the good points and deficits that need to be modified.

Thematic analysis (Braun & Clarke, 2006) was used to identify motifs recurring in the content of students’ learning reflections which along with the data from the task grades to demonstrate students’ learning outcomes and to account for the development of the curriculum and the instructor. Data were first assigned in-vivo codes. These in-vivo codes were then grouped under patterns, through which the themes were identified by comparing shared meanings and characteristics. The coding was done by the author from 86 texts and there were 26 distinct codes generated, through which five themes were identified. Results of each coding were checked by a research assistant to test the reliability of the coding system and the validity of the coding process, in which the codes were appropriately grouped under the thematic categories.

Results and Discussion

Students’ Task Grades

Students’ tasks were assigned according to the curriculum weekly and were graded with rubrics as an example shown in Appendix D. The total score for the tasks is 70, which was calculated on the basis of marks given for 13 tasks and an academic presentation. A summary of the students’ grades is shown below in Table 1.

Table 1.

Descriptive Statistics of the Students’ Task Grades.

Line of task grade	Number of students	Percentage
Above 60	70	79.54
59–50	14	15.90
Below 50	4	4.54

Table 1 shows that more than half of the students attained total score above 60, around one sixth students scored 50 to 59, and only four students scored below 50. The instructor checked the students’ attendance and their task completion records, finding that the low performing students missed several class sessions in order to do their research experiments outside the university and so they did not spend adequate time on their online learning or get involved in online discussion. The tasks they submitted were not good in quality (i.e., many grammatical errors and poorly structured writing). Their performances in the face-to-face class and online discussion also indicate that they still needed to spend more time on their English learning.

The boxplot (see Figure 3) shows the distribution of the students’ task grades. The information interpreted from the boxplot are: firstly, the short-shaped boxplot suggests that the students’ grades are relatively close and centrally distributed; and secondly, the middle quartile 64 is bigger than the mean value 63.18, which indicates that the students’ grades are left skewed (i.e., negative skewed), that is, the students’ grades approach the maximum value 68. Combined with the individual students’ grades, it can be found that a great number of students whose grades approach the maximum value of 68 is much greater than those who approach the minimum value 54. These results suggest that the students in general have gained satisfactory learning outcomes in their blended learning. However, the students’ task grades are only one point of reference when evaluating the students’ learning outcomes. These achievements need to be further explored from the questionnaire survey and their written evaluation.

Figure 3.

Boxplot of the students’ task grades.

Course Evaluation Survey

Descriptive statistics were used to analyze students’ perceptions towards the dimensions of course objectives, materials and contents, learning assessment, tasks and activities, learner support, learning mode, learning platform (see Table 2).

Table 2.

Descriptive Statistics of Course Evaluation.

Dimensions	Number of items	Mean	Standard deviation
Course objectives	4	1.6108	0.46556
Materials and contents	4	1.7045	0.73162
Learning assessment	5	1.4364	0.56205
Tasks and activities	4	1.5739	0.65451
Learner support	4	1.4205	0.57178
Learning mode	5	1.5136	0.61236
Learning platform	4	1.5653	0.68176

The results in Table 2 show that the mean values of the seven dimensions are all between 1.0 and 2.0, which indicates that students have a high approval degree of the dimensions, that is, they all describe the design and implementation of the blended course as effective. The mean values achieved from the seven dimensions were also depicted in a radar chart (see Figure 4) for visual analysis. This radar chart has seven axes representing seven dimensions along with the mean values of each dimension as points plotted on those axes. The spoke points closer to the center on an axis indicate a higher mean value. Therefore, it can be clearly seen that students gave a high rating to learning support and assessment, followed by learning mode, learning platform, tasks and activities, course objectives, and materials and contents. The results indicate that students are generally satisfied with the course design and its implementation, and these also provide quantitative baseline for the exploration of the qualitative information gathered through their written evaluation in the following sections.

Figure 4.

Radar chart of the course dimensions.

Students’ Learning Reflections

A total of 86 students completed their learning reflections which is coded under the themes of: a. learning materials and contents, b. instructor’s teaching (i.e., teaching mode, pace, and activities), c. learning platform, and d. suggestions for the course. The classification of students’ learning reflections is shown in Table 3.

Table 3.

Classification of Students’ Learning Reflections.

Categories (student number)	Examples	Proportion
Learning materials and contents (SN = 84)	Learning content is helpful/benefit a lot Academic writing is useful Useful to learn about writing lab report Search information from the professional website helps me know about Reading materials are useful Translation is also a useful content	97.67
Instructor’s teaching (SN = 81)	Teaching method is good Teaching mode is very special/very innovative The teacher helps me understand the in-class session Course arrangement is reasonable and step by step The teacher adjusts her teaching pace The teacher helps individual students	94.18
Learning platform—mosoink (SN = 82)	Creates a good environment Convenient to use A novel learning tool A good mobile APP Improve students’ learning efficiency Be fully involved in using the online Learning platform	95.34
Suggestions for the course improvement (SN = 80)	Teach in a little bit Chinese Combined both in Chinese and English Create more opportunities for students to express in English Organize some competitions Give students more opportunities to express their professional thoughts	93.02

Learning materials and contents

About 97.67% of students stated that the reading materials are helpful, and the content are useful for their disciplinary study, especially research article writing, laboratory report, English reading materials, e-mail to the journal editor, professional websites, translation, and CV. Excerpts are shown below:

Student 1. I think the content is helpful for my English learning, especially the academic article writing. I now know how to structure the article and am aware that it is different from writing in Chinese. And I also hope to publish my paper in an English journal.

Student 2. The learning content has greatly helped me to learn English and benefits me a lot. . . .Translation is also a useful learning content for us to practice using the English technical words.

Students 3. It is also useful for me to learn about writing laboratory report.. . . Searching and reading information from the professional websites also help me know about the development in my professional study, especially looking for the international journals.

Student 4. I think my greatest gains are in two aspects: academic writing and write the cover letter to the journal editor. Abstract is the most useful part, because no matter English paper or Chinese paper, abstract written in English is required. . . . Now I know how to write the cover letter to the editor.

Student 5. Through reading the English articles and papers, I now find my reading ability is better than before. . . . I also like the CV writing which relates to my job hunting.

From the students’ reflections above, it is evident that the learning materials and contents are useful and largely meet their learning needs. Students highly approve the research writing, series of modules, and this is consistent with the findings in the needs analysis: there was a demand to know how to write a full research paper from the abstract and introduction onwards. Others, such as e-mails to journal editors, reading professional websites, laboratory reports, CV, and translation were also designed in accordance with their stated learning needs. Writing the abstract of research papers is frequently mentioned in the students’ reflections, because they are also required to write an abstract in English for their Masters’ thesis. Learning from this course, they learned how better to structure their academic papers in English, and how to use appropriate terms and accurate grammar in their papers. Also relevant to the writing of research papers, they stated that sending e-mails to journal editors was useful and helpful in submitting their papers for consideration in international journals. The course was so successful that a process for submitting a research paper has been established for many students, that is, structuring the paper—writing the paper—finding a journal—submitting the paper. Students now have a system that guides them as professional researchers.

Instructor’s teaching

About 94.18% of students are satisfied with the instructor’s teaching method and style of delivering the course, that is, teaching pace, and tasks and activities. Excerpts from students’ evaluation are displayed as follows:

Student 6. The teacher’s teaching method is good. She uses the combination of face-to-face teaching and online learning. We can watch and listen to the video lectures in advance in Mosoink, and this helps me understand the in-class section. At the beginning of the class, I can only understand a few contents taught in English, but after one semester I find my listening ability improved significantly. . . . I am active to be involved in the online discussion and the classroom activities like academic presentation or present my work in front of the class. I can follow the teacher’s teaching pace.

Student 7. The course arranged by the teacher is reasonable and step by step. . . . There is a sufficient time for us to complete tasks, and the teacher grade them and give us feedback. I care about my grade a lot, because it reminds me to make progress. . . . The group report let us know the importance of collabration and also practices our PPT making and speech skills.

Student 8. The online video lecture makes us to learn autonomously. It pushes me to form my self-discipline in the learning process. . . . I am a hostess in the class academic performance. It is a good chance for me to practice my English abilities.

These students’ reflections suggest that they have generally accepted the innovative blended learning mode. They acknowledge that the combination of online and face-to-face learning modes is an effective way to stimulate students’ learning interest and optimize their learning environment. The students recognize that, first of all, it gives them flexibility in deciding where and when to learn, so that they can manage their own learning process; secondly, it fosters their autonomous learning habit during any off campus learning periods; thirdly, through the online communication, the instructor can interact with individual student privately and deal with his/her personal learning issues which is an efficient way of helping them progress; and fourthly, their English listening ability has been improved due to the instructor’s English teaching and the English video lectures. The reflections also demonstrate that when competence reaches a certain level, there is a payoff in confidence: the quality will make a change, and that is why some students stated after a period of learning, they “suddenly” found they could understand the English instruction. Moreover, the students appreciate being able to communicate or discuss with their classmates both online and in class, and they are aware that the blended learning mode helps them collaborate with each other and form a disciplinary community in the blended learning process. The establishment of the disciplinary community in turn cultivates their higher order thinking and enables them to negotiate and share knowledge with each other.

Learning platform

About 95.34% of students also gave a very high approval rating to the learning platform, Mosoink. They find Mosoink is user-friendly and claim that it makes their English learning efficient. Their learning interest is largely incentivized by the affordances of the online learning platform.

Student 9. I did my tasks in Mosoink. The platform makes communication between teachers and students more convenient because some students are too shy to ask questions in class. . . . It creates a good environment for us, which enables us to study at anytime and anywhere when there is a mobile signal.

Student 10. Mosoink is convenient to use. It has a function to remind us of submitting the assignments to Mosoink two days before the deadline. We can see each other’s assignment in Mosoink when the instructor asked us to do peer correction. It is helpful for us to learn from each other and I am motivated to be involved in the course.

Student 11. I felt a little strange when I first use Mosoink, but later found it helpful for us to download the learning materials and to communicate with the teacher, because some of the students do not like to communicate in class.

Student 12. Mosoink is a good mobile APP that makes it easy for the teacher to communicate with students and improves students’ learning efficiency. However, one problem is that you need to have a good network to use it. Sometimes, we go to the field in a remote place and the signal is not good, then it is hard to use it.

The enthusiasm with which the students rated their approval of the online learning platform, Mosoink, is remarkable. They all acknowledge that Mosoink is a convenient learning mobile application which can be used at any time, particularly anywhere. They assert that most powerful function is that they can easily install it on their mobile phone and no particular phone model is required. Students can easily find where the learning contents are, where to submit their assignments, and where to raise questions. They can communicate with the instructor privately as in WeChat, and this is what students appreciate most, as some students mentioned in the excerpts quoted above, that some of them are reluctant to communicate with the instructor or raise questions in class. The only problematic issue that most students mentioned was to do with the mobile signal. Sometimes but not always the mobile signal was weak in the classroom, and students had to wait for the app to open. Signal coverage was also a problem for those students who had undertaken field work in remote sites and so the instructor extended their submission deadlines until they had returned from their research trips.

Suggestions for the course improvement

About 93.02% of students mainly provided suggestions on activities that can encourage them to speak English and interact with each other. They also suggested that the instructor teach in both Chinese and English, rather than her present practice of giving instruction in English only.

Student 13. . . . Some students’ English is weak, so it is difficult for them to listen to the class taught in English. Therefore, I hope the teacher can speak some more Chinese in class.

Student 14. . . .I think teachers can give students more opportunities to stand in front of the class to express their professional thoughts with PPT. It can be videoed, and I think it will be very useful.

Student 15. I would like to participate in more group working tasks to learn from each other. The teacher can also organize some English-speaking competitions between students to increase fun of learning.

Student 16. . . . I am a hostess in the class academic conference. It is a good chance for me to practice my English abilities. I did a full preparation and I think I learned a lot. I hope that the teacher can create more opportunities for students in the classroom to make them express more in English.

The students’ reflections reveal that they would like to have more activities in which they can give professional PPT presentations and interact with their classmates in English. They also suggest that they might participate in some public speaking contests. Many students are keen on participating in an academic event held at the end of the semester as student 21 mentioned, a quasi-academic conference. It is evident that this kind of activity boosts students’ interest in speaking English. In future courses, the teacher will consider organizing more of these kinds of activity to motivate students’ English learning further by building on their current enthusiasms. As for the idea of using both English and Chinese, there are many arguments in favor and against the use of one or multiple languages in class. Some researchers (e.g., Richards & Rodgers, 2001; Sachdeva, 2003) maintain that monolingual instruction, the direct instruction, is effective because the direct method immerses learners in the target language, relating objects and actions with the words in the target language, and when the learners do not understand the meaning of the words, the teacher seeks recourse to gestures or mimes to make them understand. Through this method, the learners can be involved in an English language immersion learning environment to help them think and express themselves in English. Considering that most of students found that their English listening ability was perceptibly improved after a period of listening to the English instruction (see the students’ evaluation of the instructor’s teaching above), teaching solely in English will be continued in future.

Teaching Observation

The instructor’s teaching diary recorded the issues that arose in the teaching process and actions taken in response to these issues. Below are the reflective extracts from the instructor’s teaching diary, which summarize the teaching performances and students’ reactions.

Extract 1. English is my instructional language. The students can understand what I say in English basically, only sometimes I need to repeat some words or expressions, and I explain what I mean in another way when they do not understand. This situation happens mainly when I assign the tasks or activities. They are not clear about my requirements to them. For instance, one of the activities in the first module is to read the English professional website and list the information they find from it. I asked students to bullet the information points, not to write in a paragraph. Some students did not understand what I mean, so I tried to explain using body language and mime to make meaning. Consequently, a student responded in a low voice ‘只要求要点 {only information points are required}’. They got the point right.

It is noticeable that trans-semiotising (Lin & He, 2017) (i.e., to-and-fro from speech to gestures and other visual aids or materials) helps in meaning-making and communication in class. Although students respond in Chinese, they understand the basic idea, and this gives them space to build the interrelationship between the target language and their mother tongue. In future courses, English will be continuously used as the instructional language, during which visual prompts will be utilized to help with students’ understanding.

Extract 2. When they did a presentation on the professional website, they did well. They designed good PPT slides and sourced important information for the presentation from the professional websites. In the preparatory stage, students shared information with each other and discussed it, and this promoted a community of inquiry in which students constructed meaningful and shared understandings. In addition, the presentation motivated students to know about each other’s disciplinary information which enlarges their knowledge and activates classroom learning.

These reflective records also align with the students’ reflections in their evaluation, that is, they prefer oral presentations for boosting their interest of speaking English and motivating them to communicate with each other and to share understandings. During the process, the disciplinary community has been established and this is an evidence to testify the course success to the course offering department that students learned to construct knowledge through collaborative inquiry.

Extract 3. Some students they were shy and reluctant to speak up in class at the first few periods, but as they became familiar with me and their classmates, they started speaking up and raised questions in class. Students participate in the online discussion, raise questions in Mosoink, and express their own opinions. However, some students prefer face-to-face discussions in the classroom session. Sometimes they show their different opinions about various topics. For example, many students found that the generic moves to write a lab report are quite similar to those used in writing a research paper, and they asked about the differences between lab reports and academic papers. Bhatia (1991) identifies the differences between lab reports and academic papers. He argues that a lab report is for testing the existing theory or retesting the scientific findings, while a research paper is to introduce new knowledge or theories. However, a student did not agree with Bhatia’s perspective. She argued that the experiment she was doing now was also for finding something new. She challenges the scholar’s claim and expresses her thought based on her own practice and professional expertise.

It is noticeable that, firstly, although the students discuss online, they still adore face-to-face communication and teacher involvement, that is, students prefer the teacher’s involvement and comments that reinforce group identity and incentivize their enthusiasm to be involved in an intriguing, stimulating, and engaging learning environment; secondly, some students are inquisitive to challenge the scholar’s perspectives or eager to explore the answers to the questions. As an instructor, I respect their perspectives and guide them to seek for the answers to the questions independently, which are more valuable than directly telling them the results. In this way, students will be very impressed about what they eventually find out and it is hard to forget, and this is a merit that the course proposes, critical thinking, which is valuable to their learning.

Conclusion

In this study, the effectiveness of the ESP course evaluation is viewed from the perspective of two parties, students and the instructor, and the course is evaluated from the two aspects, accountability and development, of which accountability demonstrates to stakeholders the course has achieved the intended outcomes and development provides valuable information for ongoing refinement of the course, which can be concluded as follows:

Firstly, the evaluation in general shows that the course is effectively designed and implemented. The summative course grades are in line with the institutional expectations. More importantly, the students’ disciplinary needs, which are reflected by the learning outcomes, have been met. Secondly, students’ related academic skills have been practiced in the learning process where they learned to collaborate with each other to share their knowledge and think critically. This is a meaningful value of the course, which reaffirms the findings of the relative studies conducted by Diep et al. (2017) and Hrastinski (2019) mentioned in the introduction section. As Tsai et al. (2013) argues, fostering critical thinking among students was helpful for students to better understand the scientific process and to become more experimental and refutational in their scientific study. In the future, the instructor will be more involved in the online/ classroom session and design more academic activities to engage students in the disciplinary community so that they can actively interact with their peers, share their understandings, and make greater progress in their professional study. Thirdly, as the findings from the students’ evaluation discussed above, students may not be adapted to the English immersion learning environment at the beginning, but they gradually get used to it, and the results confirm that their English listening ability has been improved. In this regard, the instructor will continue using English as the instructional language both in the online leaning and face-face-classroom teaching to practice students’ English listening ability. In the process of using the direct method, trans-semiotising will be continuously used to help students understand the meaning of words or expressions. The findings, in general, help in the ESP course design process and the implementation of activities and tasks. The praxis also verifies the research results attained by Zhang (2020) that students’ disciplinary knowledge and skills, their motivation to apply what they have learned to their professional studies, and the greater satisfaction with their learning progress have been effectively promoted in the blended learning paradigm. Additionally, the results have attested the validity and reliability of the evaluation framework for the ESP course, and this could be kept probing for evaluating blended courses in other language curricula in the future studies.