Abstract
This article is premised on research that suggests there is a gap between technology use for teaching and learning and the technology used in assessment. Digital technology such as blogs, Facebook, Twitter, podcasts, and news feeds are increasingly used in teaching and learning. On the other hand, assessment is often only in traditional essay form and frequently via pen and paper. This article examines the complex nature of multimodal design and argues for assessment that has constructive alignment, which sets out the interdependence of learning outcomes, teaching, and assessment where all three are aligned as equal partners. The aim of the research described was to investigate whether teacher–learners believed there was alignment between learning outcomes, activities, and assessment that utilized digital media technology. This article presents findings from a small-scale case study that drew on qualitative data from Student Evaluations of the Paper and from online surveys from four cohorts of participating teachers in a postgraduate adult literacy, language, and numeracy professional development program. There were three important findings from the data that indicated the importance of firstly having alignment of digital technology and course design outcomes, activities, and assessments and that this was clearly visible to the participants. The second finding indicated the content was relevant to the teacher–learners and involved them in constructing their own learning through authentic and practical activities and assessment. The third finding indicated that course outcomes, activities, and assessment tasks were aligned with multi-literacy skills. It is argued that academics design courses that are constructively aligned connecting learners with outcomes, activities, and assessment, which include an explicit focus on teaching the multi-literacy skills required in a world that is becoming increasingly digital.
This article draws upon research carried out on a professional development Master’s course electronically delivered to teachers from different regions within New Zealand. The teachers registered on the program come from an eclectic range of professional backgrounds including secondary and tertiary education and from adult literacy and numeracy education in contexts that include prisons, the army, government, and industry-provided schemes for workplaces, as well as the unemployed, refugee, and youth groups.
The rationale for the research stems from findings from an initial pilot study in 2012 which challenged the authors to redesign assessment processes that addressed the following student comment “Converting the blogs which were enhanced with multimedia into an essay totally made all the work on the blogs pointless. Essays must be separate, the blogs must be assessed for how the tool is used to make information interesting, elicit feedback and provide continuity.” This challenged the authors to reexamine their own pedagogical approach to course design so that technology and its associated multimodal literacies could be included in the assessment process. The term multimodal refers to the modes that create meaning, such as visual, audio, linguistic, spatial, and gestural (West, 2019). This article examines the complex nature of design to include multimodal aspects and argues for assessment that has constructive alignment (Biggs, 2003) which sets out the interdependence of learning outcomes, teaching, and assessment with ideally, all three aligned as equal partners.
The following sections examine literature on assessment, the importance of having explicit teaching methods and learning outcomes, and the use of rubrics, all integrated within a context of multi-literacies where students communicate and create through e.g. reading, listening, graphics, writing, touch, and multimodal technologies. Such technologies as described by Towndrow et al. (2013: 329) “can be unfamiliar and uncomfortable territory for teachers, who, adopting familiar, comfortable choices of assessment strategies and procedures, would tend to overlook the multimodally designful qualities in their students’ work.” Following the literature review relevant to the above themes of multimodality and assessment, we detail the research methodology before analyzing the findings and discussing the implications and considerations for new thinking about assessment, technology, and multi-literacies.
Literature review
Multimodality and assessment
For the purposes of this article, we use the term multimodal (Curwood, 2012; Silseth and Gilje, 2017) to refer to a wide range of applications that enable users to share, comment, create, and discuss digital contents via text, visual, audio, tactile, gestural, and spatial representations. The term assessment is used to describe the process of gathering information on student learning and is said to be at the heart of the learning experience (Kirkwood and Price, 2016; Price et al., 2011). In addition, Gray et al. (2010) argue that the use of technology “raises significant challenges for assessment, posing a barrier to further adoption” (p. 106) and these challenges could be for technical, pedagogical, or logistical reasons. According to Gray et al. (2012) most assessment involving multimodal digital representations is of formative assessment that provides feedback so a student can improve a piece of work before it is graded, or otherwise is for low stakes assessment for few marks. Formative assessment refers to feedback and support given to students during the assessment process which is used to improve their competence (Silseth and Gilje, 2017). Summative assessment is a summary of student learning at a particular point in time and is for validation and accreditation (Gikandi et al., 2011). Gray et al. infer that the lack of published examples of multimodal digital summative assessment is because academics may be hesitant to reveal they have used “such novel technology” (p. 2) where aspects of integrity, reputation, intellectual property, accreditation frameworks, and external stakeholder expectations may be at stake. This article supports research (Gray et al., 2010; Kirkwood and Price, 2016) positing that technology-based assessment can work “for” and “as” learning and indeed offers ways to make assessment more authentic. Authentic assessment occurs within the context of an authentic activity with complex challenges and centers on an active learner that produces refined results or products and is associated with multiple learning indicators (Herrington and Herrington, 2006). For this to happen, summative assessment needs to be carefully designed so that scaffolding is built in from the outset with supportive formative feedback (Dabner et al., 2012). This article further argues that the traditional division between formative and summative assessment can be blurred and concurs with the contention of Gray et al. (2012) that in the “Web 2.0 environment, formative and summative assessment can appear like a double helix and perhaps become indistinguishable” (p. 11). This blurring of the differences between formative and summative assessment is because as the learning and assessment process become “visible” to the student (Rogerson-Revell, 2015) summative assessment will attach value to the process the students use to create their assessable products and will include feedback. For the assessment process to become what Hattie (2009) first described as “visible,” it is necessary that students and teachers understand the task, the marking criteria, the skills of writing, as well as multi-literacy skills such as placement of images and hyperlinks.
Multimodal literacy
There has been little empirical research to address the changing nature of literacy and the alternative ways of assessment in support of students’ new literacy practices in the digital age (Hung et al., 2013; Silseth and Gilje, 2017). There has been a shift for today’s students from a predominantly print focused environment to a visual, electronic, and digital one. According to Poutasi (2017), in many New Zealand schools, students are already using digital technology in nearly every aspect of their education and she predicted that by 2019 we would have a “tsunami” of students about to hit our New Zealand national exams who had never known anything but digital texts. Poutasi argues that we have to assess these students in ways that reflect their learning, which is increasingly digitally enabled.
The terms “new literacies” (Lankshear and Knobel, 2006; Leu et al., 2017), “multi-literacies” and “new basics” (Kalantzis et al., 2010) and “multimodalitiy” (Leu et al., 2017) represent the shift and describe the change and its impact on what we formally defined as “literacy.” The written modes of meaning now combine with visual, aural, gestural, and spatial patterns of meaning. To be successfully managed as part of a multimodal text, writing has to be understood differently from when it is in print text and Bezemer and Kress (2008), Bowen (2017), and West (2019) contend that the digital image is in fact replacing writing as the central mode for representation. They describe how the skills of reading images differ greatly to reading text that goes from left to right and top to bottom, while the image does not have such an order. For Bezemer and Kress and others (Lankshear and Knobel, 2006; Leu et al., 2017) communication is made up of a number of modes (image, sound, writing, color, music), which are always in use and that meaning, is in all of them. Thus, writing when incorporated into multimodal texts has to be understood differently from when it participates in print text alone. As West (2019) argues, writing in digital contexts has the potential to differ significantly from the traditional pen and paper contexts in that digital texts are shared with audiences who may further alter the text. This article therefore argues that courses need to be designed to include student learning about visual design of texts including the use of color, fonts, image placement, and size (Kalantzis et al., 2010; Kress, 2003; Leu et al., 2017), as well as the need for students to learn about the interpretation of multimodal texts and how sharing text can lead to altering, remixing or co-creation. The diversity of design skills brought about through the change from literacy to multi-literacies, therefore has pedagogical consequences for assessment.
Unfortunately, as argued by Van Haneghan (2011) and Towndrow et al. (2013: 328), the complex nature of multimodal meaning design is “as yet scarcely understood, particularly within classrooms and until we reach more nuanced understandings of multimodal design, assessments made by students’ digital multimedia products, and multimodal literacy capacities, will be of very limited value.” Jacobs (2013) also discusses the issue of the new approaches to assessment being time-consuming for teachers, arguing that they should not be an add-on, but rather an integral part in which the new basics and the old basics are taught in conjunction with each other. Walsh (2010: 211) on referring to Australian educational policy, argues while changes in digital communication provide avenues for reading and writing to be combined with for example images and music, national testing requirements are “still principally focused on the reading and writing of print-based texts” and as Gray et al. (2010: 107) caution, multimodal literacies could “fly under and beyond the radar of such a system.” This suggests course design requires the inclusion of explicit teaching and assessment criteria of the multimodal literacies and leads to the following reexamination of our course design.
We argue, as do other researchers (Aagaard and Lund, 2013; Gray et al., 2012; Silseth and Gilje, 2017) that the shift to multimodal literacies and the meaning these construct, requires a different form of assessment to traditional print forms. The interactivity and inclusion of design elements involving the selection, placement, and layout of images and text; use of hyperlinks; font size; and color and the meaning these construct, cannot be assessed by using assessment strategies that we have used in the past for written reports, essays, and examinations. Gray et al. (2012) contend that most digital media advocates offer little advice in any detail on how to carry out assessment using Web 2.0 or deal with differences in assessing it compared to other student writing and staff approaches to marking. In addition, according to Aagaard and Lund (2013) and Silseth and Gilje (2017), teachers seem to lack experience about how to assess multimodal texts. In this regard, Towndrow et al. (2013) describe how the teachers in their study downloaded often used assessment guidelines from the Internet with the result the researchers concluded that a prescribed rubric (an assessment tool that lists the criteria for a piece of work and states gradations for each criterion from excellent to poor) could be helpful for students working on multimodal projects.
However, as posited by Godhe (2013), criteria are often constituted as a framework describing what to include in an assignment but how these could be interpreted and made meaningful to student compositions in relation to various multimodal texts, is not addressed. In Lister’s (2014) analysis of 17 studies, learners wanted clear rubrics and examples of completed assignments for guidance. The creation of marking rubrics is not an easy one and there are many reasons for this. One reason concerns ownership of the produced work and brings up questions of fairness in marking group work. Also, there are issues of weightings toward different aspects of an assessment such as how to evaluate technical skills versus creative skills versus knowledge and critical thinking skills. However, this issue is also relevant to traditional essay writing scoring and Van Haneghan (2011) describes the dilemma for the teacher in grading writing that is technically correct but content weak.
While there are rubrics available on the Internet they are often decontextualized assessment guidelines and do not enable teachers to fully acknowledge students’ “inventiveness and creativity” (Silseth and Gilje, 2017: 4). Therefore, the development of a rubric is not a simple task and there needs to be well-designed models available in order for teachers to create valid and reliable ones. Hammett (2007: 351) has expanded on traditional rubrics to open up affordances of both page and screen in order to take account of the different symbol and semiotic systems that incorporate them. He uses the terms “extended” or “holistic” rubrics to describe them.
The challenge therefore for the authors was to design assessment that aligned with our learning outcomes and teaching methods that were also multimodal. There are few research cases which describe assessment of students using digital technology and multimodal literacies in higher education and this confirmed our experience and that of others (Gray et al., 2010), that there is a lack of examples to draw on. Gray et al.’s research into 17 cases of assessing Web 2.0 authoring in higher education sourced from recent educational conference proceedings or journal articles, concluded that many students needed to learn the basics of using the tools and ways to deeper learning. They also argue that traditional assessment does not focus on the ways that students choose to use the tool and concluded that students should not be asked to produce content in traditional academic assignment form.
We argue here that the multimodal skills necessary are often not explicitly or actively taught and it is important that we do this as well as design appropriate assessment. The design and conduct of assessments should therefore make it clear how multimodality integrates with other elements and forms of assessment, is linked to specified learning objectives, is supported by adequate instructions and marking rubrics, and provides feedback.
Background
The Online Learning and Teaching course described in this paper was designed and taught by the authors of this article in a university in Auckland, Aotearoa New Zealand. It was designed as a fully online, postgraduate course for Adult Language, Literacy, and Numeracy (ALLN) teachers and was one of the six in the Master of Adult Language, Literacy, and Numeracy program. Digital technologies and proficiency with these applications have been identified as essential to ALLN teachers’ practice (Davis and Fletcher, 2010) and the course was redesigned to reflect developments since 2004 when the program began and is described in the next section.
Course design and course activities
The literature suggests that technology-enhanced learning is most effective when framed within a learner-centered environment whereby students are encouraged to learn with multimodal representatives so as to generate their own content (Allen and Nelson, 2013; Brinthaupt et al., 2011). The underpinning course design was framed by social constructivist learning theory, utilizing Bain’s (2004) “baker’s dozen framework,” which is a set of 13 questions that may be used to guide course design. The questions encourage teachers to concentrate on what the learner is doing in contrast to what the teacher is doing during the learning, teaching, and assessment process. Of the 13 questions, the following two relate specifically to the aims of this article and have been summarized below.
How will I spell out the intellectual and professional standards I will be using in assessing student’s work, and why I use those standards? How will I create a natural critical learning environment in which I embed the skills and information I wish to teach in assignments that the students will find fascinating, authentic, and include opportunities for feedback?
These two questions became the research questions for the case study. The first one relates to the importance of having a pedagogy to describe why assessment criteria is used as well as having clear or visible guidelines. The second relates to the importance of relevant and authentic outcomes, activities, and assessments and the incorporation of multi-literacy skills.
The course was conducted through the Blackboard Learning Management System (LMS). The teacher–learners could co-construct wiki pages for themselves, posting a variety of multimodal resources, which provided evidence of their developing arguments as they reasoned together. Teacher–learners were required to create an open access blog to document their learning and assessment outputs. Four major course activities were developed, with activities 2 and 4 also designed as assessment outputs, and these are discussed next. The following activities have been summarized from Cartner and Hallas (2017).
Activity 1: Critique of the e-learning literature. For this activity, teacher–learners were asked to develop two researchable questions they had about learning and teaching with multimodal technologies for their own teaching context, and to post these in the wiki. They were required to investigate their own question 1, and a peer’s question 2, through a critical analysis of the literature, as well as tapping into educational blogs. The teacher–learners shared the findings of their investigation in their personal blog (http://blogspot.co.nz) providing an avenue for critique and comment on each other’s work. This activity provided much of the literature and subsequent ideas for blog posts which would contribute to activity 2 (assessment 1).
Activity 2 (assessment 1): Develop a philosophy of teaching and learning with technology. The teacher–learners were asked to articulate a personal philosophy of teaching and learning with technology. They were firstly expected to investigate theories of online learning, and philosophies of technology in practice. Secondly, teacher–learners were asked to write a series of blog posts articulating their perspective on the use of multimodal technologies for ALLN education, and to provide evidence that would show how their approach to teaching and choice of social media would impact students’ approaches to deep learning. The blog provided a platform for teacher–learners to document their learning, communicate with their peers and us, and an avenue to demonstrate conceptual change over time. The marking criteria rubric for Assessment 1 is provided in Supplemental Appendix 1.
Activity 3: Reflections on learning. In order to explore how they experienced their learning, teacher–learners were invited to reflect on the course activities and assessments. They were invited to fill out a Blackboard LMS anonymous survey and to fill out a University-wide Student Evaluation of the Paper (SEP). The self-reflection survey questions were adapted from Walker (2002: 126) “cycle of reflection” questions that had been originally developed for technology-related professional development. The two questions they responded to that relate to this research study were adapted as follows:
What concepts and skills do I understand as a result of completing the course activities/assessments? How has my thinking about social media for ALLN education been changed or challenged as a result of participating in the course activities/assessments?
The SEP questions asked students to write the strengths and suggestions for improvement on the paper’s organization and administration, assessment, learning and teaching methods, and resources.
Activity 4 (assessment 2): Investigation of a multimodal tool. The final activity asked teacher–learners to undertake an in-depth evaluation of a multimodal tool appropriate for their own ALLN teaching context. Teacher–learners selected a tool such as Twitter, Facebook, blogs, wikis, mobile technology, or Utubes, and provided a rationale for why it was worth investigating for ALLN education. They critically evaluated the tool for cultural and contextual appropriateness drawing on the work undertaken in activities 1, 2, and 3. Finally, they developed a learning activity that utilized the tool, clearly demonstrating how it was designed to advance their students’ ALLN education, and including evidence to support the design. The marking criteria rubric for Assessment 2 is provided as Supplemental Appendix 2.
Methodology
A case study approach to inquiry based on themes as advocated by Braun and Clarke (2006) was undertaken for this research. The reason for the case study was because it is congruent with research that examines issues in-depth (Richards and Morse, 2013: 28) and has “the common goal of generating new ways of seeing existing data.” Rather than the usual satisfaction survey (using options such as “agree” and “disagree”) we chose qualitative, open-ended questions that facilitated and illuminated teacher–learners’ understanding of complex learning situations “that cannot be made explicit in most other research designs” (Barone, 2004: 25). Therefore, Walker’s (2002) “cycle of reflection” questions were utilized in this case study to encourage teachers to become open to uncertainty in order to challenge their thinking about social media and pedagogical approaches. Accordingly, this case study draws on the voices of the participating ALLN teachers to provide evidence of the need for constructive alignment and for a multi-literacies approach. The following sections outline the design of the study, the participants, and method of data analysis.
Design of the study
This study drew on the qualitative data generated from a survey and the University-wide SEP undertaken in week 12. By week 12, teacher–learners had completed the course activities and assessments and could provide insights into how the teacher–learners' comments related to course outcomes, activities, assessments, alignment, and multi-literacy skills. The data were collected via the LMS anonymous survey tool.
Participants
Altogether there were four cohorts of teachers who participated in this research study between 2013 and 2017. The university’s ethics committee granted approval for the study, and all teachers in the four classes were invited to participate. Twenty-one teachers volunteered to take part. Their ages ranged from 34 to 67 years, with an average age of 50 years. The case study participants are referred to as teacher–learners in this article.
Data analysis
All the data from participant responses to the survey and the SEP were taken from the Learning Management System, read and discussed by both researchers. The purpose of the analysis was to identify differences and similarities, to find patterns and to search for common themes related to the alignment of learning activities, learning outcomes and assessment, and the explicit teaching of multimodal skills. The researchers in their initial analysis open coded the data into a number of themes such as assessment, technology, and the impact of learning theories. Participant responses were assigned to a theme or a number of themes. An illustration of this can be seen in the following example from one participant who wrote “I am experimenting with different types of media and learning how to implement these into my blogs and or discussion forums” (T1). This response would have been themed under use of technology and also in the impact of learning theories. Subsequent reading and analysis of the data reduced the number of themes to the three major ones relating to the course design being visible, relevant, and including the teaching of explicit multi-literacy skills.
Thus, the data were analyzed using an inductive, thematic analysis approach (Stake, 2005) following the phases of familiarizing with the data, generating initial codes, searching, and revising themes to defining and naming them (Braun and Clarke, 2006; Harrison et al., 2017). Multiple impressions of an instance (such as where teachers had referred to the need for student-centered learning) were aggregated and conclusions drawn about the issue that had emerged from the data. This is a subjective method of analysis; however, the nature of qualitative study relies on such a method in order to make meaning out of complex data (Harrison et al., 2017).
The next section of this paper reports the findings of the responses to the survey and SEPs, focusing on the ways in which the teacher–learners reported how well the paper spelt out learning outcomes, and details of assessment and learning activities (Research Question 1) and how well the paper created a natural critical learning environment students found fascinating, authentic, and included opportunities for feedback (Research Question 2). Anonymity has been ensured by assigning each teacher a number from 1 to 21.
Findings
The findings highlight key themes that emerged from the analysis of data related to course design, assessment, and multimodal literacy. Findings that relate to course design and assessment were that the design of the course and related learning theories, were visible to the students. The research findings also highlighted the need for the participants to have learning activities and assessment that are both authentic and practical. Findings on multimodal literacy relate to the importance of embedding the necessary multi-literacy skills that will enable the teacher–learners to successfully participate in the course activities and assessments that they can use in their own teaching contexts. These findings are discussed in more detail in the following sections.
Course design: Visibility
The visibility of the theory and principles that underpin the course was evidenced by the significant number of participant comments that specifically referred to the importance of course design with reference to constructivism and to authentic learning. Evidence that there was visibility can be seen in the number of times design terminology was referred to by the participants. An example of this is that “constructivism,” “construct,” or “social constructivism” were mentioned 23 times by 14 of the research participants. Also the term “collaboration,” which is associated with constructivist learning approaches, was mentioned 46 times in the participant data. In particular, teacher–learners identified constructivist learning theory, student-centered teaching, self-directed learning, and active, collaborative social media as essential elements in multimodal pedagogical design.
The following examples of participant comments indicated that teachers–learners’ believed that course design should be underpinned by a pedagogical design, in order that students construct their own knowledge. I believe it is about using a constructivist standpoint as a teacher, to allow students to construct knowledge as they go through the course and to assess this in collaboration with the student. (T9) This course has sparked my interest, and I intend now to dedicate some time to developing knowledge and skills using some of these other technology tools, but at the same time, being mindful that appropriate pedagogy needs to be applied, for learning and assessment to be effective. (T21) Course content was designed so that we could appreciate the potential of the tools through the use of them. (T7) assignment tasks were clear, easy to clarify (T20), related to the paper outcomes (T21), I am a visual learner and assessment tasks were easy to see and understand (T12), Careful thought given to the paper structure and activities (T2), and The marking rubrics clearly spelled out the requirements for the assessments, in terms of the knowledge we were required to demonstrate. (T21) It should not just be an add-on to provide motivation or interest (although it may prove to be a motivating tool). The key question needs to be around how the integration of social media is enhancing the learning opportunities for learners. (T6) We should eagerly embrace social media, but in a way that puts learning needs first. I realize how easy it is, if you are uninformed, to use new software/apps to jazz up a lesson without paying attention to the learning outcomes. I do think there is still a long way to go before we really come to grips with social media tools effectively. (T4) Maybe making it compulsory for participants on the course to comment on each other’s blogs and follow blogs as directed by the course co-ordinator. (T8) Making it compulsory to comment on wikis a minimum number of times and also to blog at set intervals or for a minimum amount. Some of us blogged throughout the period designated while some posted blogs the day the assignment was due which did not seem very fair on those following the directions given by the tutor. (T2)
Course design: Authentic and relevant
The finding that the assessment and learning and teaching activities should be both authentic and of practical use to the teacher–learners in their own contexts was given by 18 of the 21 teacher–learners. Evidence indicating this finding is shown in the following comments. In terms of authentic learning, I also came to see how these tools could be of benefit to learners in my own teaching practice, and how these tools could be used to meet their learning needs. (T21) I found the content and assignments related to the paper and my current context in which I work. The assignment that looked at Web 2.0 was very appropriate to the subject of Adult Language, Literacy and Numeracy because it helped to broaden and inform my understanding of some of the research being carried out in this area. (T20) My thinking around the use of social media as a teaching aid has strengthened after this paper because of the evidence that shows its relevance. (T9) The course was relevant to the workplace. (T2) My current cohort of learners in a workplace is very positive and receptive about using writing blogs and mobile technology. Researching about blogs gave me inspiration and insights on how blogs can be utilised for teaching and learning. (T3) The assessment activities also required me to constantly relate my new learning back to my own L&N teaching practice. This was useful, because it made me see the relevance (or sometimes irrelevance) and/or usefulness of these tools in relation to supporting learners in my specific learning environment. (T21)
Course design: Explicit teaching of multi-literacy skills
When considering their own literacy and numeracy teaching contexts, teacher–learners in this study expressed their understanding of the problems their students may face when introduced to a digital resource. In order to make use of multimodality in their teaching practice, they explained: it is important to understand the potential challenges of using social media and how to avoid the disadvantages (T2) and from another, […] reconsider the nature of online interactions … to set up the necessary level of digital culture – particularly in foundation students – this wasn’t something I had thought about before. (T5)
A number of participants commented on the blogging task and the skills required for blog writing as compared to essay writing. These include skills relating to blog layout, more informal structure, creating more summarized writing texts, collaboration, web searches, and skills for commenting on blogs. I am experimenting with different types of media and learning how to implement these into my blogs and or discussion forums. (T1) I am actually enjoying making blogs and appreciate that 500-700 words is a manageable bite and therefore, for me, more enjoyable than persevering with a several thousand word essay. (T10) I wanted to create a nicely presented blog so I decided to look up some of the more popular blog sites (looking at the layout, widgets included in the site). (T2) I loved the blogs and the way some of the weight was taken off highly structured academic writing, giving 21st Century real –world information sources more use. (T4) My writing is more focused and I have learnt to analyse the assignment questions more thoroughly. I am more conscious of the question behind the question and can more readily read an implied message. (T12) I especially liked the need to establish and maintain a blog and then to edit and submit the blog as the first assignment. (T1) I will definitely be using blogs as a literacy tool. It has also shown me that it is an excellent way for learners to develop writing skills. I need to be concise with my writing as no one wants to read a long and boring blog. Caution. I need to be careful what I put in a blog. As it goes in to the public arena. I cannot retract what I have put once it is out there. When something is put in writing, it is more concrete and could be misconstrued. (T15) One thing I realise about myself is that I still carry my “cultural attitude” of being “polite” and non-confrontations” even online. Even-blog writing has its rules. (T19)
Discussion
The findings indicate that the gap research had referred to in the introduction, between multimodal technology use for teaching and learning and the technology used in assessment practices was not evident through the comments by participants in this case study. This gap had been present in an earlier course as indicated by the student comment about blogs needing to be assessed differently to essays. The findings also suggest the participants understood the importance of the design of the course being visible. In addition, teacher–learner responses in our study described multimodal representations as needing to be an integral part of the course, linked to meaningful, authentic tasks and assessments, providing opportunities for learner participation and interaction that is collaborative and involves learners constructing knowledge with peers and digital resources. Consequently, the teacher–learners came to understand how they could capitalize on the affordances of multimodal representations. Finally, teacher–learner comments indicate they saw the importance for course design to include explicit teaching of the multi-literacy skills needed in order to combine image, reading, writing, and sound.
Course design: Visibility
The first key theme relates to the first research question relating to how the course design spells out the intellectual and professional standards used in assessing students’ work, and the reasons for those standards. Teacher–learners’ comments indicated the course design was visible with learning outcomes, and learning activities including all readings and blog exemplars, as well as the assessments and marking rubrics, available to them from the beginning of the course. In making the course design visible to teacher–learners, we provided a language and metalanguage to describe indicators of quality in multimodal text production and creation. These are provided in Supplemental Appendices 1 and 2. In this way, assessment is made integral to the learning process rather than remaining focused on end products as many more traditional print focused assessments do. The teacher–learners came to understand the design for the course utilized a constructivist approach and expressed support for literature (Cartner and Hallas, 2017; Kirkwood and Price, 2013; Timperley, 2008) which asserts that, in order to change teachers’ beliefs about multimodal course design, a shift from a technologically determined view to a pedagogically determined perspective is required. Teacher–learners also reflected on the idea that the use of social multimodal technologies required a change in pedagogy rather than be “an add-on” or to “jazz up” an existing paper.
Course design: Authentic and relevant
A second theme related to course design needing to involve authentic learning tasks and assessments that relate to the practical needs of the teacher–learners’ own students. Leu et al. (2017) explain how New Literacies theory describes the Internet as central to both our professional and personal lives and that use of the Internet requires new literacies to effectively meet its potential. They argue that in fact not to keep up with the rapid and continuous change will perpetuate the misalignments in assessment and instruction and that we will fall further in not meeting the present realities of multi-literacy.
The findings in this study support the above views and research question 2 which related to creating a natural critical learning environment with embedded skills and information for assignments that the students will find fascinating, authentic, and include opportunities for feedback. This was addressed in the case study where participants commented on blogs being useful learning spaces to practice grammar, writing skills, and social contact. The majority of the participants referred to the need for the course design to be authentic and practical as Bain (2004) supports where he describes how the best teachers create a natural critical learning environment. For Bain, this relates to the importance of questions and the importance of modeling the process of devising questions and showing how students can answer their own questions. As he describes it, the best teachers “embed the skills and information they wish to teach in assignments (questions and tasks) students will find fascinating - authentic tasks that will arouse curiosity, challenge students to rethink their assumptions and examine their mental modes of reality” (2004: 47). Many of the participant comments indicated strong agreement that the questioning aspects of Assessment 1, where students developed two researchable questions they had about learning and teaching with multimodal texts for their own teaching context, challenged them, and made the task a worthwhile and authentic one.
Course design: Explicit teaching of the multi-literacy skills
The third theme relates to the need to explicitly teach the multimodal skills necessary to meet the new literacies assessment criteria for the learning outcomes (Leu, et al., 2017) and relates to aspects of research question 2 concerning embedding skills and information. The findings from this case study support numerous researchers who agree that assessment techniques need to be designed to include the new kind of learning because traditional assessment techniques are inadequate to measure the kinds of skills required for multi-literacies and digital learning (Jacobs, 2013; Kalantzis et al., 2010; Towndrow et al., 2013). This article argues that the shift to multimodal literacies requires a different form of assessment to traditional print forms. Interactivity and inclusion of design elements involving the placement and layout of images and text, use of hyperlinks, font size, and color cannot be assessed by using assessment strategies that we have used in the past for written reports, essays, and examinations. This shift requires new approaches to assessment of the new skills our students are developing that brings about productive and innovative technology-enhanced learning (Aagaard and Lund, 2013). This new assessment according to Kalantzis et al. would suggest involving techniques that include project assessment, based on in-depth tasks that involve task plan, complex collation of material and presentation, broad knowledge, and a flexible solution. It would be designed to have the capabilities to measure multiple intelligences whether communicative, analytic, or creative, as well as performance assessment based on planning, doing, and completing of a task. In addition, the new assessment would assess organization and problem solving as well as group assessment to assess the collaborative capacities and finally portfolio assessment through documenting evidence of work undertaken.
These skills can be embedded throughout the course and the findings from this study support this. In Gray et al.’s (2012) study, academics were concerned student’s text building efforts were not academic enough. The teacher–learner comments on having blogs compulsory are supported in the Afshin Mansouri and Piki (2016) study, claiming the time and effort students put into blogs was thought to be unproductive if blogging was not compulsory. The length of blogs was also an issue in the case study and is also commented on by Afshin Mansouri and Piki stating that ones that were too long discouraged peers from reading them. These negative aspects of blogging are summarized into “when technology is employed in education, a range of cognitive, affective and social aspects need to be considered in addition to the technological ones” (p. 268). This case study’s findings support this view.
Conclusions
This article has described research into learning and teaching in the area of multimodality and assessment. The framework utilized in this article was based on Biggs (2003) constructive alignment and utilized Bain’s (2004) baker’s dozen questions to guide the design of course activities and assessment. The findings from this study reinforce the importance of the learner-focused, course design features embedded in Bain’s (2004) baker’s dozen framework. We conclude that good course design requires a context where learning outcomes, teaching and learning activities, and assessment are aligned, visible, and based on learning theories that are analyzed by learners. The co-creation with learners in developing extended or holistic rubrics supported constructive alignment of the learning outcomes, materials, and assessment. As a result of completing the online course activities and assessments, teacher–learners had developed a conception of course design with multimodal tools that was pedagogically focused and where multimodal skills were explicitly taught and embedded into a course that has application to the learner’s lives.
This study was carried out with a small sample of cohorts in one postgraduate course within a single institution; therefore, it is not appropriate to attempt to generalize its findings to all such courses. Our attempt at writing rubrics is still in a developmental phase which is understandable, given we are on what Leu et al. (2017: 12) describe as “the cusp of a new era in literacy theory, research, and practice” as we seek to prepare students for the new literacies and the Internet that define their future. However, the findings of our study will be useful for course designers, teacher educators, and teachers who wish to design courses that utilize multimodality within a pedagogically focused view of teaching and learning.
Supplemental Material
LDM899732 Supplemental Material - Supplemental material for Aligning assessment, technology, and multi-literacies
Supplemental material, LDM899732 Supplemental Material for Aligning assessment, technology, and multi-literacies by Helen Cartner and Julia Hallas in E-Learning and Digital Media
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Author Biographies
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
