Abstract
Multimodal composing is the construction of meanings through a combination of semiotic modes such as visual, audio, and textual to create multimodal texts. Studies have investigated the application of multimodal composing in the English language, in content area classrooms, and within various educational ESL and EFL contexts. However, the examination of international English learners’ (ELs) perspectives on multimodal composing and identity representation via multimodal texts at post-secondary level has not been fully explored. This study aims to investigate how a group of six international multilingual ELs in a university Intensive English Program (IEP) use multimodal semiotic resources to express ideas and how they reveal their social and literate identities through the processes and products of multimodal composing in a workshop. The study was situated in the framework of sociocultural and literate identities theory. Data were collected through semi-structured interviews, workshop observation notes, video-recorded workshop sessions, participants’ multimodal compositions, and the researcher’s logs. Social semiotic multimodal discourse analysis (MDA) and grounded theory were the primary data analysis methods. Participants revealed a variety of perspectives on multimodal composing. They were found to use a combination of words and images to compose paper-based and computer-based multimodal texts constructed from their personal journey learning English regardless of their age, gender, nationality, and social context. Various identities such as those of an international student, an English learner, a writer, a researcher, and a goal-achiever were observed in the data analysis process.
Keywords
Today, in the multimedia and technology-saturated world, language learners at different age and proficiency levels inevitably engage in various forms of multimodal reading and writing activities. To be literate in today’s language and content learning classrooms, students are expected to read and write print-based texts, multimodal texts in print and in various other forms, in many different contexts. Additionally, the advent of artificial intelligence (AI) has made speech recognition, image recognition, machine translation, human-machine dialogue, big data intelligent analysis, etc. a reality (Wang, 2020). Due to the multimodal nature of literacy activities, “classroom practices need to explicitly enable multimodal, collaborative and interactive learning opportunities between students, between teachers and students and between students and online learning spaces over time” (Edwards-Groves, 2011, p. 62). Therefore, teaching goals, activities and literacy instruction approaches all need to be adapted accordingly.
Digital forms of composition are quickly overtaking more traditional types of writing as the dominant mode of expression. As Selfe and Takayoshi (2007) note, “It is fast becoming a commonplace that the digital composing environment is challenging writing, writing instruction, and basic understandings of the different components of the rhetorical situation (writers, readers, texts) to change” (p. 1). Therefore, the practices of writing have shifted from composing written texts to constructing meanings by using multimodal semiotic resources such as images, sound, and other forms of communication. In particular, “students need to be experienced and skilled not only in reading (consuming) texts employing multiple modalities, but also in composing in multiple modalities, if they hope to communicate successfully within the digital communication networks. . .” (Selfe & Takayoshi, 2007, p. 3). Thus, to enhance writing skills, non-linguistic multimodal resources should be employed in writing instruction, together with linguistic resources, to engage learners, particularly English learners (ELs), to improve their English language and literacy skills.
Multimodality and ELs
Multimodality plays a critical role in the transformation of today’s literacy practices since it affects the ways meanings are communicated. According to Kress and van Leeuwen (2001), multimodality is “the use of several semiotic modes in the design of a semiotic product or event” (p. 20). Kress (2009) defines mode as “a socially shaped and culturally given resource for making meaning. Image, writing, layout, music, gesture, speech, moving image, soundtrack are examples of modes used in representation and communication” (p. 54). In other words, people from different cultures and social contexts use different semiotic resources within a mode to express meanings. Thus, it becomes a common practice that ELs combine several modes to communicate a message to complete a class assignment (Mestre-Mestre, 2015). Therefore, English language teachers should be more aware of the changes in ELs’ literacy practices and acknowledge the shift of practices in ELs’ language and literacy teaching (Mina, 2014).
The EL population is fast-growing in size, variety, and importance in the U.S. (National Center for Education Statistics, 2016). With the growing number of ELs in the U.S., ELs’ literacy practices and performance are inevitably affected by multimodal forms of communication and meaning construction. Although these ELs are in various stages of life and come from different linguistic backgrounds, they are all expected to gain the required level of English proficiency to be able to communicate in English at school in both oral and written forms. Therefore, it is crucial for these ELs to study the multimodal literacy practices in order to update their skills in reading and writing novel texts and literacy types. Multimodal meaning-making processes mandate a shift in the teaching of ELs “to acknowledge new ways of engaging students with L2 language, building on the literacies these students develop and practice” (Mina, 2014, p. 141) in their daily lives. To draw on these concepts, ELs’ practices of adopting semiotic resources in their writing process need to be studied to understand their construction of meaning and identity representation in specific social and cultural contexts.
ELs in the Intensive English Program (IEP)
An increasing number of non-resident international ELs have been admitted to U.S. universities. Those whose English language proficiency has not yet reached the level required for entry to a higher education context are placed in the university’s Intensive English Program (IEP) to receive intensive English training before they are admitted to the general degree-seeking programs or placed in English as a Second Language (ESL) programs to be better prepared for undergraduate and graduate programs of study. Thompson (2013) describes IEPs as “those programs in which the students are enrolled in full-time English classes that are not for university credit (pre-university courses)” (p. 212). Those ELs with English literacy skills inadequate to satisfy their various English learning needs require attention and assistance in and out of classroom spaces to enhance their English proficiency.
In this study, the IEP is a specially designed bridge program, which academically prepares learners of English as an additional language (EAL) who come to the U.S. to pursue their degrees in a higher education institution. The program provides EAL learners with intensive training in English-speaking, listening, reading, and writing to prepare them to enroll in the general degree-seeking programs in the university. Students in the IEP are enrolled as full-time, non-matriculated students and take classes to earn credits for non-degree-seeking purposes. Students are assigned to various levels (Level I to Level V) based on the results of their English placement test taken on entry into the program. Students must pass the Level V test to be qualified to apply for the regular degree-seeking programs in the university. Passing the test at Level V with required grades is equivalent to satisfying the requirement of the IELTS or TOEFL tests for undergraduate or graduate programs.
Literature Review
The application of multimodal composing within various educational contexts, including ESL and EFL contexts, to facilitate English and content areas’ learning has been documented in previous literature (Bou-Franch, 2012; Dzekoe, 2017; Grapin, 2019; Hafner, 2015; Jiang, 2017; B. E. Smith et al., 2021; Van Staden, 2011; D. Yang, 2012; Y. C. Yang & Wu, 2012). These multimodal literacy practices even travel beyond school boundaries to home and community settings to further enable learning (e.g., Boivin et al., 2014; Flottemesch, 2013; Westman, 2012; Wong, 2015). Additionally, computer-based multimodal feedback has also been used by instructors to help improve L2 learners’ writing skills (Elola & Oskoz, 2016).
Previous studies have explored the strengths and challenges of ELs’ academic writing practices (e.g., Lillis & Curry, 2010; Park, 2016). Moreover, the process of writing has been found to reveal voices and identities for L1 English Writers (Olinger, 2011; Seban & Tavsanli, 2015) as well as L2 English writers (Bhowmik, 2016; Maguire & Graves, 2001; Schneider, 2021; Yi, 2013). Notably, Vasudevan et al. (2010) state that, people’s “identities are not intrinsic or separate from social contexts and interactions; rather they are embodied and enacted in practice” (p. 445). In this respect, writers’ literate identities are shifted while they engage in various writing practices in different social and cultural contexts.
Multimodal composing portrays identities for writers with diverse social, cultural, and linguistic backgrounds. Particularly, it provides an approach to understand marginalized populations as well as minority groups in terms of who they are and how they perceive themselves in their specifically-situated environment (e.g., Shin & Cimasko, 2008; Wake, 2012; Wang, 2018). As an emerging genre of writing, multimodal composing substantially represents writers’ ownership of the text since writers must determine the materials to use, the resources to be referred to, and the forms of presentation to target audiences. Researchers investigated multimodal composing as identity demonstration and agentive identity expression for L1 writers of English (Hull & Katz, 2006; Krause, 2015; Shin & Cimasko, 2008; Wake, 2012) as well as for L2 learners of English (Ajayi, 2015; Angay-Crowder, 2016; Cimasko & Shin, 2017; Goulah, 2017; Mantegna, 2013; Mina, 2014).
In addition, Cummins et al. (2015) claim that identity-affirming literacy practices are likely to increase students’ literacy engagement. B. E. Smith et al. (2021) studied bilingual and emergent bilingual high school students’ perspectives on multimodal composing and found their views positive on composing with multiple modes with academic purposes as well as their expressions of bilingual/bicultural identities. As such, the power of multimodal composing in expressing, representing, and shifting writers’ identities have become an important topic in the literacy field.
Belcher (2012) has addressed the issue of IEP students being under-studied and their writing practices being insufficiently covered by current research. Even though previous studies have investigated IEP instructors’ use of social media networks or multimodal resources in mainstream classroom instruction (e.g., Fuchs & Akbar, 2013; Rance-Roney, 2008), few studies so far have investigated how international ELs in the post-secondary IEP context compose multimodally and how their identities are enacted through multimodal composing activities. As such, this study intends to contribute to the growing quantity of recent studies highlighting the possibilities of multimodal composing in ELs’ identity representation and language learning practices. For this investigation, three guiding research questions were specifically examined:
What are the multimodal resources ELs utilize in composing multimodal texts?
What are ELs’ perspectives of using multimodal resources to express themselves?
How are ELs’ identities portrayed in their multimodal compositions as well as their self-descriptions?
Theoretical Framework
Sociocultural and Literate Identities
Sociocultural and literate identities theory focuses on identity construction in social contexts where individuals interact with the social world using literacy activities such as reading and writing in various social and cultural contexts (Holland et al., 1998; E. N. Skinner & Hagood, 2008; Vasudevan et al., 2010; Wortham, 2006). A sociocultural perspective views identity “as dynamic and constantly changing across time and space” (Norton, 2006, p. 3). There are three key tenets that drive how identities are constructed from this perspective:
People construct their identities within a figured world (Holland et al., 1998).
Identities are not only constructed within the contexts individuals situate, but are inevitably shaped by the individual’s previous and current experience (Vasudevan et al., 2010).
Literate and social identities are interconnected (Sableski, 2007); literate identities can reflect the influence of particular cultural practices (Gee, 1996; Gee, 2010) as well as social practices.
Holland et al. (1998) assert that people construct their identities within contexts of “figured worlds” or what they call “identity in practice” (p. 271). As specified by Urrieta (2007), “People ‘figure’ who they are through the activities and in relation to the social types that populate these figured worlds and in social relationships with the people who perform these worlds” (p. 108). D. Skinner et al. (2001) also indicate that “without this knowledge (of figured worlds), it is difficult to understand the shifts or readings of events that the narrators construct or the reasons why they orchestrate voices as they do” (p. 13). To sum up, identities are built within the situated context where individuals act and interact while they are living and conducting cultural practices.
Sableski (2007) states that literate and social identities are interconnected. She defines literate identity as “the ways in which people form conceptions of themselves as readers and writers based on the discourses of which they are a part” (p. 20). In other words, students “figure” who they are while they are participating in reading and writing activities in different sociocultural contexts. However, their literate identities are not fixed; they are shaped and reshaped (Bloome & Dail, 1997; Martens & Adamson, 2001) while they interact with and draw on experiences from the outside world. Not only can they interpret themselves and build their literate identities through their relationship with their contextualized world, but they can also construct their literate identities through the experiences they imagine (Sumara & Luce-Kapler, 1996). Since students play a variety of roles within different social and cultural contexts, their literate identities also reflect the influence of particular cultural practices as well as social practices (Gee, 1996; Gee, 2010).
In this study in particular, the theory of sociocultural and literate identities helped understand how ELs’ identities were portrayed in their multimodal compositions, depicting their English learning experiences in different social and cultural contexts. While ELs wrote about their previous English learning experiences, they described how they ascertained who they were when they were interacting with contextualized culture, social activities, and people. Moreover, the findings on ELs’ English learning experiences reflect their perceptions about who they are in various literate activities, which are inevitably interconnected with their situated social and cultural life.
Methods
The Setting
The current inquiry was explored using a qualitative case study design. After approval from the university’s Institutional Review Board (IRB) which was required by the Human Research Protection Program (HRPP), a 5-week-long, 10-session, multimodal composing workshop was conducted in a computer lab in a Southern university in the U.S. Each session was 60 minutes long (an overview of workshop sessions is shown in Table 1).
Overview of Workshop Sessions.
The multimodal workshops offered free opportunities for ELs to choose their specific modal preferences (Kress, 2010) and multimodal composing paths (B. E. Smith et al., 2017). According to Prensky (2010), it is important to allow students’ choices because it provides opportunities to produce students’ creative best. Desktop computers were available in each workshop. Other available resources include digital cameras, iPads, tablets, laptops, pens, notepads, color pencils, crayons, construction paper, glue sticks, and scissors. Albers (2009) has noted that symbolic, metaphoric, and literal messages can be created and sent through visual means such as lines drawn by crayons, color pencils, paint, and other media. Additionally, to provide space to foster learning, choice making, and expression of ideas, Lewis Ellison (2017) suggests that the digital choices extend beyond the choices of digital and non-digital tools since the choices represent the learning to enact identity with autonomy.
In the workshops, participants used multimodal resources they selected to create texts to share their experiences of learning English in their home countries as well as in the U.S. in and out of the classrooms. Participants then discussed their multimodal compositions with other participants in the workshops. Participants composed and discussed one multimodal text every two sessions. They completed four multimodal texts in total over the 5-week time frame. The first session of the first week was an introduction to the workshop while the last session was a debriefing/reflection on what they had learned about themselves and multimodal composing. In general, participants composed their multimodal texts during the second session of the week. They shared and discussed with the group their completed multimodal texts during the first session of the following week. Each workshop was scaffolded so ELs could receive feedback from other participants and the researcher.
Participants
A group of six ELs from a Southern university IEP were invited to participate. These participants were volunteers from the university IEP’s level IV and level V groups, who represented 18.7% of the total enrolled international ELs in the program and six nationalities. (The demographic information of participants is shown in Table 2.) Pseudonyms were used to protect the rights and welfare of human research subjects recruited to participate in research activities. Before the first workshop session, each participant was initially interviewed in English. The purpose of the initial interview was to have the opportunity to get to know the participants, and at the same time, to obtain information about their knowledge of multimodal composing. Also, the purpose and process of the study were explained to each participant. They were asked to sign consent forms. They were also informed that they could opt out of the study at any time if they did not want to continue participating.
Demographics of Participants.
Data Collection
Data collected include the following: (1) observation notes were taken in each workshop session to keep record of the scenes, occurrences, context of the research site, participants’ mood and actions, and the conversations and interactions among participants. (2) Semi-structured interviews with each participant using open-ended questions were conducted. The initial interview was aimed at understanding the cultural, linguistic, and educational background of each participant, and their knowledge of multimodal composing. A follow-up interview after the final workshop session was conducted to investigate participants’ perceptions of multimodal composing and to probe for details and descriptions about participants’ mode choices and experiences. (3) Participants’ multimodal texts were examined to investigate how ELs select multimodal resources to convey meanings and how their identities were represented while they wrote about their English learning experiences multimodally. (4) Video recordings were made of the 10 workshop sessions to help review each session multiple times, to describe the occurrences in each session and to reflect on what is seen and unseen. (5) The researcher kept a log to reflect on participants’ mode choices, learning experiences, and challenges in each workshop session. Any changes, nuances, or unexpected occurrences that potentially affected the study were recorded as well.
Data Analysis Methods
Social semiotic multimodal discourse analysis (MDA; Jewitt, 2009, 2011; O’Halloran, 2011), and grounded theory (Charmaz, 2006; Glaser & Strauss, 1967) were the primary data analysis methods. Social semiotic MDA was used to analyze multimodal texts created by participants since this method is targeted at examining how multimodal writers purposefully select semiotic modes to design their multimodal texts. In addition, multilingual and multicultural designers’ identities were represented through their use of different modes and semiotic resources, which went beyond linguistic representation.
Based on grounded theory method, data generated from observation notes, interviews, workshop sessions and researcher’s log were analyzed. Close examination of the data and generalization of common themes were extracted from the data through coding (Charmaz, 2006). Grounded theory data coding involves two steps: the open coding phase that gives initial labels to the segments of data (word or line or incident), and the selective or focused coding phase that consists of reviewing high frequency initial codes or most prominent ones, comparing, categorizing, and abstracting the data (Charmaz, 2006; Charmaz & Belgrave, 2014).
Procedures of Multiple Sources of Data Analysis
Pre- and post-interviews
Interview transcripts were read through multiple times and coded according to questions and responses (incident). The coded data were read once more to look for similar codes, common themes, and to record frequently occurring themes. After codes were labeled, meanings were drawn from those themes and categorized. Interview excerpts, codes and the major findings were tabulated in the comprehensive Excel sheet (see Table 3) to search for themes that could possibly answer the research questions.
Excerpts, Codes, and Major Findings from Interview Data.
Observation notes
Notes generated for each participant were read multiple times to look for patterns ad themes. A narrative for each participant was composed based on the observation notes to describe how each of them worked on their multimodal texts during different sessions. The researcher stayed close to the data (Charmaz, 2006) and searched for themes and trends that could answer the research questions.
Video-recorded workshop sessions
The videos of the 10 workshop sessions were watched closely multiple times and descriptive notes were compiled from each session. The notes were maintained in the researcher’s log. The discussion questions and responses from each participant at different sessions were typed into an Excel spreadsheet. The information was reviewed multiple times before coding. During coding, participants’ responses were summarized in one column and assigned codes in another column. Codes were read and compared to be abstracted and categorized to generate findings. Themes were identified that supplied answers to the research questions.
Multimodal texts/compositions
The multimodal texts composed by each participant were reviewed multiple times. Notes were written to describe each participant’s multimodal texts and the researcher’s reflection on each text. Kress (2010) described the social making of mode in terms of “image, writing, layout, music, gesture, speech. . .” (p. 79) while analyzing participants’ multimodal texts (see Table 4) Albers’ (2009) suggestions of how to read visual texts included understanding the size, position, angle, focal role and lesser role of visual texts. Mode frequency in each participants’ four multimodal texts was tabulated.
Checklist of Multimodal Resources for Various Mode Representations.
The collection of multiple data sources and the application of different data analysis methods provided multiple lenses for the researcher to view and synthesize the data, which strengthened the trustworthiness of the study’s findings. The participants were invited to judge the accuracy of the interpretation of data and provide feedback that could be used to shape the later data analysis. After the initial coding of data and the generation of common themes, the researcher turned to a critical friend (Rossman & Rallis, 2003) to have a randomly selected 15% of data analyzed to check for inter-rater reliability. Discrepancies were resolved collaboratively to attain inter rater agreement before the researcher entered into further coding processes.
Findings
Various Perspectives on Multimodal Writing
Participants’ perceptions about writing drastically shifted over the course of the study. Before attending the workshops, none of them had heard of “multimodal composing” even though some of them had used multimodal resources in their courses to complete their academic writing tasks and in their daily lives as communication tools. After attending the workshops, several of them shared their perspectives that multimodal composing could help them express ideas more effectively than the five-paragraph essay because the former allowed the writer free mode choices. Two participants shared their understanding of using different writing forms based on audiences and purposes. Five participants agreed that multimodal composing processes helped them think logically, express ideas freely and efficiently, and write creatively. They all felt it was an interesting approach to writing and learning English. All but one participant shared their preferences for using words and images as their primary writing resources. Even though Leyla repeatedly mentioned that she would not choose to use colors or pictures, she was actually using multimodal resources of lines, squares, hand-drawing images, and arrows in her multimodal texts.
Participants held diverse opinions with respect to the practicability of multimodal writing. Dao shared that he did not see the use of multimodal resources as popular and encouraged in his classrooms, but he believed that the use of music, videos, and other multimodal resources would happen in the future. Sung Min described his thoughts of using multimodal resources to communicate meanings with people who speak different first languages and encouraged the use of multimodal resources when things could not be explained in words. Leyla made an interesting point that when instructors teach, they should use pictures, but when students learn, they should use words. She explained her points with her own experience and said that, as a learner, she preferred to look up words in the dictionary and find more related words to write essays. Tien suggested that more people should learn the technique of composing multimodally. In the case of writing, students could pull out the pictures or draw their ideas without the need to type words. She believed that information would be communicated more efficiently and swiftly using multimodal resources.
Use of a Combination of Words and Images to Describe English Learning Experiences
Participants chose words, colors, images, lines, and layout to design multimodal texts to describe their English learning experiences. They chose to create texts in ways they were familiar and comfortable with. Some of them mentioned they were attempting to find the “easiest way” to compose. Five participants shared their preferences for using multimodal resources to express their ideas. Even though Leyla shared her preference for using words, she described her experiences of using pictures and statistics to compose and present medical-related topics in her previous job, which means she was actually utilizing multimodal resources rather than words to create her texts. Four participants felt excited about creating multimodal texts because it was new and interesting to them. Leyla felt it was “ok” to create multimodal texts, but she thought it did not work for her to use pictures and color to learn English. Dao had the experience of using pictures to help him with classroom writing before attending the workshops. He usually drew pictures to generate and organize ideas before he started writing his essays for other classes. It was found that all participants chose multimodal resources that were available to them. In other words, they did not ask for more resources but chose to use those that they had access to.
Awareness of Audiences
All participants were aware that one important purpose of writing was to convey meaning to their audiences, and they were trying to do a better job showing, describing, or explaining something to make their compositions easily comprehensible for their audiences. Their awareness of the audience even made some of them struggle in terms of deciding what resources to choose. For instance, Dao explained that while he was trying to choose a music clip to express himself in his first multimodal text, he felt his understanding and feeling towards the specific music changed a few minutes later. For that reason, he wondered if his audience would also experience the same understanding and feelings. He then abandoned his original intent. Fanghui also mentioned the shift in her understanding of audience. She reported that she used to write for the purpose of making her audience think exactly the same as what she told them in her writing; however, now she has changed her mind. She would like to give her audience the freedom to think open-mindedly beyond her writing. She also suggested that multimodal writers should always display and present the target language to their audience but could use the audiences’ first languages to help with understanding.
Multiple Identities Revealed Through Multimodal Compositions and Self-Descriptions
Identities revealed through multimodal compositions
Participants in this study revealed various identities in their home countries, in the U.S., and as language learners through their multimodal compositions. In some cases, participants used multimodal texts to represent their journey in learning English and the shift of identities between their home countries and the U.S.
Referencing nationality or home country
In almost every case, participants used words and images that referenced their nationality or home country. Danilo used large-sized words to write the name of his national language “Spanish” and drew a picture to show the location of his country on the globe (see Figure 1). His affection for his home country and his national identity were revealed from his words and images. Dao used digital pictures that showed the real-life view of his hometown and the coffee house where he used to practice English, which represented his identity that connected closely to his home country and his previous experiences. Also, Tien used digital pictures that displayed an Asian-looking, black-haired female several places in her multimodal texts and a park, a coffee house in her home country, which represented her awareness of identity being a Vietnamese woman learning English in her home country (see Figure 2).
Danilo’s MM text I.
Tien’s MM text II.
Identities found in the new space
Participants also used their compositions to express and describe the identities they found in the new space out of their home countries. Sung Min drew pictures to display his inferior status of being checked at customs and being interviewed in his job-hunting process, which represented his identity as a foreigner and a helpless job hunter with limited English proficiency (see Figure 3). Danilo composed his experience of quitting a previous job and coming to the U.S. to be a full time English learner. He intended to immerse himself in the authentic English environment in order to study English. Leyla drew a curly-haired female image in one of her multimodal texts and wrote down her personal information in a text box showing her current identity of an English learner who passed the official language test and enrolled in the IEP level V class (see Figure 4).
Sung Min’s MM text II.
Leyla’s MM text I.
A shift of identities
In other cases, participants revealed a shift of identities between their home countries and their new living context. Sung Min drew pictures to represent his different learning experiences in the Korean and U.S. classrooms, displaying his identities of living, evaluating and shifting between these two classroom cultures. Fanghui chose digital pictures that represented her previous learning experiences of having classmates from an English-speaking country and now living with American roommates with whom she could practice more English. Tien used her own photos taken with friends in the U.S. next to the photo taken with friends at home to show her audience how she engaged with American friends to learn English and the culture, which displayed a shift of her identity in different social and cultural contexts.
Identities revealed through self-descriptions
Participants also described their perceptions about who they are in their home country contexts as well as in the U.S. during their respective processes of learning English. These self-descriptions of identities can be divided into the following five categories:
Self-identified as international students in the U.S.
All participants were aware of their identities as international students. Most of them mentioned English was not their native language but they had to learn it to survive and better communicate with people. Dao expressed his intention of “looking like a native English speaker” even though English is not his native language. He also expressed his feelings of not being integrated into U.S. culture. Some shared confusions, obstacles, and frustrations of being international students in the U.S. However, Danilo perceived being international as “a good thing” because he enjoyed learning the culture and fitting himself into this new context. Fanghui was prepared to be an international student and she was proud of communicating with her American roommate and local people without language barriers. Tien also perceived being an international student as a good opportunity to learn English in an English-speaking country.
Self-identified as home country citizens
Most participants described themselves as ordinary citizens in their home countries. Five of them mentioned the comfort and convenience of living in their home countries and compared their life situations in the U.S. They left their comfort zone to challenge themselves by learning an extra language, which they found frustrating both in their daily and academic lives. Sung Min described himself as wearing several identity hats including a young person, a son, and a good friend in his home country whereas others perceived themselves as no different from other citizens in their home countries.
Self-identified as English learners
All participants described themselves as English learners in their home countries as well as in the U.S. Four of them considered their home contexts as constraining their language learning since English was used only in the classrooms or at work, whereas the U.S. context supported their English learning because they had to use the language both in classrooms and in their daily lives. This English-learner identity has shifted from being just a learner in their home countries to being a learner and a language user in the U.S. This identity shift was motivated by their different learning contexts. For instance, Leyla mentioned a period of political unrest during the time she attended elementary school so “everyone was trying to be safe,” which interrupted and undermined her English learning. Tien also shared her experience of “following the train” when a large number of young people in Vietnam started to learn English due to the foreign investment in her home country. Danilo started his English learning journey in his middle school and resumed English learning at work due to the special need for English proficiency in his job.
Self-identified as multimodal writers and researchers
Ultimately, all participants perceived themselves as writers or researchers composing multimodal texts based on their previous learning experiences. Sung Min described his role as a researcher and a writer. He joked about his situation as a “guinea pig” who participated in the workshop. Leyla regarded herself a writer, a storyteller, and someone who was writing her autobiography. Danilo viewed his role as a researcher who was exploring multiple ways of language learning and multimodal resources of expressing himself. Dao also expressed his attempt to learn and explore new ways of making PowerPoint presentations using music and video clips, which he perceived as a creative writer.
Self-identified as goal-achievers through English learning
All participants mentioned their goals of coming to the U.S. to learn English. Dao came to improve his English and studied diligently to apply for the undergraduate program at the university. Danilo quit his job in Columbia and came to the U.S. to concentrate on learning English for the purpose of communicating fluently with his colleagues in his new job in the future. He took every opportunity to learn English even though he had been in the U.S. for only 2 months. Fanghui dreamed of becoming a lawyer in the U.S., so she set her goal on improving her English-speaking skills and competing with native English speakers. Sung Min intended to be a pilot, so he was aware of the importance of English. He came to the U.S. to polish his English so he could attend a pilot training school. Leyla came to the U.S. to apply to the university graduate program in business. Even though she self-identified as a lazy student, she made progress in her speaking and writing and was about to apply for the business program. Tien had the goal of getting her bachelor’s degree in accounting or nursing. She worked diligently to complete her class assignments, and she felt good about writing since she knew how to approach it. She also attended a pronunciation clinic to have her speech errors diagnosed and corrected.
Discussion
This study examined six international multilingual ELs’ perspectives on multimodal composing when creating and discussing four multimodal texts related to their previous and current English learning experiences as well as their identity represented in multimodal texts composed in 10 workshop sessions. Social semiotic MDA (Jewitt, 2009, 2011; O’Halloran, 2011) and grounded theory (Charmaz, 2006; Glaser & Strauss, 1967) analysis of interviews, workshop observation notes, and participants’ multimodal compositions revealed a promising view of composing via multiple modes for academic purposes. Students demonstrated how linguistic and non-linguistic modes were utilized to support their understanding of writing topics and innovative meaning construction. At the same time, the multimodal composing project provided a space for identity expression, agency exploration, collaboration, and meaningful social and literate identity construction or reconstruction.
When participants were asked about their opinions of using multimodal resources to express themselves, five of them related their responses to writing. Their perceptions of using multimodal resources to express ideas were affected by their literate histories, experiences, and social norms as well as “power, social difference, and cultural factors” in their specific institutional context (Bawarshi & Reiff, 2010, p. 200). Typically, when a standard genre of writing is taught in the classrooms, students are reluctant to go beyond the wall to take the risk of writing in a different genre. Additionally, since most of these participants had been in the U.S. for less than a year, they may not have been fully immersed in the academic community. As a result, they may have found it easier to assimilate to their perceived linear, print-based standards of university English for academic purposes (EAP) programs (Kapp & Bangeni, 2005).
Participants viewed multimodal composing as different from the writing they did in other classes. Several participants perceived multimodal composing as an out-of-classroom practice. Sung Min also mentioned the issue that multimodal writing may not be useful in dealing with serious classroom writing topics. Leyla’s resistance of choosing pictures, color, and other non-linguistic resources to compose writing tasks may have been due to her preference of being practical since she was primarily motivated to learn a more academic standard of writing for the program in which she was enrolled. However, Tien at the other end of the continuum was more of a risk-taker because she expressed her willingness to take a step forward and advocate a change in the current writing norm. Dao also supported the use of multimodal resources in the writing process. He had tried out the composing technique by using music and video clips in his PowerPoint presentations in class. These risk-takers would suggest a revolution to adjust the current writing norms in which a large number of writing teachers still adhere to the traditional paper-based genre and are reluctant to open up to multimodal genre options that best suit students’ writing purposes and interests.
This study revealed that most participants learned from their experiences or from others through the use of semiotic modes to write about their English learning experiences. Moreover, “semiotic modes have different potentials, so that they afford different kinds of possibilities of human expression and engagement with the world, and through this differential engagement with the world they facilitate differential possibilities of development” (Kress, 2000, p. 157). In other words, “semiotic modes have particular affordances that offer potentials and limitations for communication and representation” (Alexander et al., 2011/2012, p. 2). This means that the affordances of printed texts with linear and sequential logic may carry a more stable and less arbitrary meaning compared to visuals that afford “showing” or the depiction of meaning through images, which might be understood differently in varied sociocultural contexts and by different audiences.
Even though current academic writing practices encompass mainly the traditional print-based genre with words being the dominant mode of expression, multimodal writing is not totally divorced from academic writing. For instance, the exposure to multimodal resources and the experience of multimodal composing may bring new insights into ELs’ academic writing practices. Particularly, while composing an academic essay, instead of only seeing words as being read, printed, and comprehended, ELs who have multimodal composing experiences may perceive words as being listened to, sensed, smelled, and acted on. Multimodal composing practices may bring in multiple dimensions and possibilities to ELs’ mindset and assist with their academic writing practices. Furthermore, if academic writing is not limited to the final writing product, for instance, the essay itself, then there is an open space where ELs, like Dao and Danilo, can utilize multimodal resources to plan and draft their ideas prior to composing the final product.
Implications
ELs’ Social and Literate Identities are Constantly Shifted and Developed
When learning an additional language, L2 learners’ social and literate identities are constantly changing. These various identities depend largely on their self-portrayals based on their various life experiences (Dörnyei & Ushioda, 2009). Giving participants the opportunities to recall and retrieve previous language learning experiences and write about those experiences helps them learn from their own past. More importantly, the multimodal project helped them think about who they are in different sociocultural contexts and find out why they are learning English. Their compositions can be “benefited by lived experiences, which helped them generate relevant ideas to write about and improve the content” (Bhowmik, 2016, p. 289) of their multimodal texts.
Although a variety of identities were revealed from participants’ multimodal compositions and their self-descriptions, authorial identity that went beyond the use of words, pictures, and layout were not seen. In other words, no participants chose to apply sound, music, or video in any of their multimodal texts. Shin and Cimasko (2008) might help to explain participants’ choices in regard to their authorial identities. They argue that:
Unlike still images, audio design represents a strong departure from, and thus a potential breach of, traditional academic discourse norms. Multimodal texts that incorporate audio represent the construction of less academically viable author identities to those who are oriented toward more traditional views. (p. 385)
When recalling participants’ responses to the question of “which mode(s) or resources did you use the most frequently?” the majority answered “words.” Their responses suggested that even “non-linguistic elements still played a role that writing could not achieve on its own in the context of academic discourse. . . students drew on norms of traditional academic discourse and placed priority on linguistic design” (Shin & Cimasko, 2008, p. 388).
The development of ELs’ literate identities is a prolonged and complex sociocultural process. It is influenced by various factors of home and school literacy practices, L2 learning, race, and gender. However, classroom literacy practices are the most dominant factors in learning and identity construction (Martens & Adamson, 2001; F. C. Smith, 2008). Due to the importance of teachers’ classroom practices, findings for this study suggest a significant pedagogical stance that classroom teachers should set up a realistic goal, provide a meaningful learning climate, and prepare for practical literacy activities in the classrooms. Additionally, teachers should “recognize that students bring funds of knowledge to their learning communities and that teachers should incorporate students’ knowledge and experiences into classroom practice” (Boyd et al., 2006, p. 332). To develop students’ literate identities, teachers also need to be “committed to teaching and empowering students to recognize the intellectual and social competencies they bring into schools and provide pedagogies that will propel student learning” (Lewis Ellison & Wang, 2018, p. 16). Teachers need to understand students’ national, cultural, and linguistic backgrounds and use their backgrounds as valuable resources to foster the development of their literate identities.
Academic Writing Should Foster Possibilities of Multimodal Composing
To develop L2 learners’ writing skills, it is imperative that teachers and students open up to multiple possibilities provided by the multimodal resources students utilize in their daily lives. In the short term, academic writing may not reflect the multimodal method of composing, but in the long term, the discipline may come to reflect and represent the multimodal nature of texts depending on the purpose of the writing tasks. At the very least, multimodal composing plays the role of a precursor for academic writing tasks as it expands writers’ capacity for describing feeling, sensing, hearing, and touching while producing print-based texts in the classrooms.
One of the important features of the study was to give participants the autonomy to make choices among the available resources. Agency was shaped during participants’ processes of making and remaking their multimodal texts. Perceptions about applying multimodal composing to classroom writing practices have been advocated. Even though text-based writing norms predominate in the IEP to prepare students for higher levels of program studies, voices, perspectives and efforts of ELs as stakeholders in curricular implementations as well as pedagogical practices in literacy education should not be neglected. ELs’ voices and perspectives need to be fully understood to further explore the “learning potential of multimodal composition and supports needed for effectively integrating technology in the classroom” (B. E. Smith et al., 2021). Only through the advocacy of and empowerment to stakeholders can we transform and legitimize the multimodal composing practices in ELs’ academic preparation and evaluation in post-secondary institutions.
L2 English writers’ have been observed struggling to compose academic genres in classrooms. L2 writers who doubted whether multimodal genre should be used in the classrooms has also been recorded. Additionally, L2 writers who take the initiative to apply multimodal resources in academic writing has been revealed. Two quotes from participants describe the value of multimodal composition:
Sometimes you don’t have to write. You know, you can express your feeling. Maybe in the study you can save your time to write or type. You can do the picture you know. When everybody is learning this technique, it’s going to be easier, faster on the writing, to express your feeling you don’t have to sit there typing. It’s too long compared to pulling up the picture and draw. So once people know about this, it’s more easier to understand about things. (Tien post-workshop interview)
And as Fanghui put it: “If I put in something interesting and in different ways, it may attract the audience. So, I prefer to do writing in that way, and I hope my teacher would do that for me” (post-workshop interview).
It Is Time to Redefine What “Literate” Means to L2 Learners
Even though most participants experienced frustration and difficulties while composing the rhetorical academic writing genre in their classrooms, and their writing proficiency might be deemed low, they demonstrated few issues or challenges while composing the multimodal genre with paper or computers. As Carpenter (2009) posits, “When it comes to electronic texts, students know far more than they realize or can express, a fact that helps explain why college students are deemed rhetorically illiterate even when their texts suggest otherwise” (p. 146).
The multiple modal affordances and choices in participants’ composing practices make literacy learning authentic activities. When L2 students come to the classroom with their literacy evaluation based on their print-based texts, it may be arbitrary to identify them as being illiterate or low literate because their real literacy level was not fully revealed from their self-produced print-based texts. Therefore, what literate means to L2 learners needs to be shifted and redefined in today’s multi-literacy world. Krause’s (2015) definition of being literate is:
The ability to negotiate multiple forms of meaning in order to make sense of the world in order to generate new understandings. Being literate is the ability to see the overlapping and complementary nature of visual, auditory, spatial, gestural, and textual modes of communication. Ultimately, being literate means being able to adapt within the dynamic nature of our world and being able to negotiate and make meaning from experiences. This is not intended to imply print literacy isn’t important, it’s meant to imply it is simply not enough. (pp. 76, 77)
Being a language and literacy instructor of ELs myself, I propose the definition of literate to L2 learners as: a holistic evaluation of an individual not only focusing on his or her ability to interpret or compose print-based texts alone, but more significantly, on the evaluation of utilizing his or her knowledge of the sociocultural world and confidently and proficiently manipulating language and literacy skills that are multimodal in nature, and also living, interacting, and mediating within his or her situated social and academic world.
Conclusion
The findings in this study may be limited by participants’ reluctance to fully explore potential methods of multimodal composition. This study allowed participants’ preferences of mode choices, which may challenge some students with its openness and multimodal nature of workshop tasks. Additional research may need to focus on more intensive scaffolding or mediating students’ multimodal composing processes.
Furthermore, I offered the resources of pens, notepads, color pencils, crayons, construction paper, glue sticks, scissors, and digital devices which may have limited participants’ choices. However, I mentioned to all participants that they could ask for any other resources needed so that I could bring them in. However, no one asked for any additional resources but chose to use the available resources that were provided. The availability of resources might have constrained their choices to compose multimodal texts. Even though no one addressed time as an issue in their multimodal composing processes, it was reflected from participants’ explanations of not using certain semiotic resources such as sound, music, or cardboard due to the limitation of time provided in the workshop sessions. Consequently, future research may consider the design which allows ELs’ free choice of time.
This study examined a small group of six international students with various social, cultural, and linguistic backgrounds. Future studies with larger numbers and more varieties of participants may provide different insights into participants’ multimodal choices and may reveal patterns about the relationships between their choices and various experiences and backgrounds. The potential of multimodal composing for transforming and negotiating academic learning should continue to be explored. In addition, ELs’ multimodal composing for academic purposes should be understood in various content areas, contexts, and in different genres and digital devices. Researchers who are interested in language teachers’ multimodal composing practices and multimodal digital composition practices to facilitate language and content area learning, should design more studies regarding L2 teachers’ engagement. Also, scaffolding in content and language integrated classrooms with multimodal composing practices (e.g., Jiang et al., 2019; Sun et al., 2021) should continue to be investigated. Additionally, each participant is a unique learner, thus they can be re-examined and closely analyzed in the future as single cases.
