Abstract
This article reports on a qualitative assessment of intercultural competence (IC) in U.S. first-year writing (FYW) courses designed to increase intercultural exposure and interaction among domestic and international students. To measure students’ intercultural development via a series of reflective writings, we designed two innovative qualitative analysis tools: a grounded-theory coding scheme and a mapping procedure aligned to the Developmental Model of Intercultural Sensitivity. Our results show that qualitative assessment of reflective writing reveals dynamic, complex IC development trajectories, displaying nonlinearity, nondiscrete phases, and development within phases. Specifically, we noted that reflective writing helped students engage with and become attuned to aspects of cultural difference. Affordances of the FYW context indicated that students strongly engaged the cognitive domain of IC, and that this domain appears to be activated by reflective writing.
Keywords
Introduction
Amid the onset of COVID-19, intercultural competence scholar Darla K. Deardorff pointed out that images of the pandemic reminded us how much our lives depended on others in our communities and emphasized the need to renew efforts that address learning to live well together (Deardorff, 2020). Learning to live well together requires engaging with differences—whether cultural, linguistic, racial, or differences stemming from national origin, disability, and other identity markers. This can be uncomfortable. Often, individuals shy away from pursuing relationships with people who are different from them (King et al., 2011). Reluctance to engage may result from not wanting to say the wrong thing, a desire for familiar interaction with like-minded persons, more deep-seated prejudice against members of a certain group, or the recollection of experiencing discrimination or violence from other groups. Ethically and practically, then, diversity and inclusion initiatives must help individuals overcome their reluctance to engage across differences in ways that lead them to critique factors that create social divides.
One evidence-based framework that focuses on equipping people to interact in environments marked by differences is intercultural competence (IC): the attitudes, cognition, and behaviors necessary for effective interpersonal and intergroup interaction across cultural differences. We approach IC as a process of development of concrete skills, knowledges, and attitudes that foster effective engagement with difference (Dimitrov & Deardorff, 2023). Further, we value IC as a construct for difference because it equips people to understand both difference and similarity—including similarities within a difference and differences within a similarity (Dervin, 2020); balancing similarity and difference creates depth in intercultural encounters, reducing miscommunication, inappropriate acts, and stereotyped assumptions.
Assessment of IC development and evaluation of targeted interventions inform the design of programs that meet the practical and ethical imperative to teach university students to effectively contend with differences they encounter. Tracing the intercultural development of such interventions is frequently done via quantitative measures such as the Intercultural Development Inventory. This tool assesses individuals’ IC on a five-phase scale of levels, ranging from monoculturalism to interculturalism (Hammer, 2009). In its earliest use, the Intercultural Development Inventory dominated assessment (Lombardi, 2010), though other quantitative measures exist (e.g., Lee Olson & Kroeger, 2001; Spanierman et al., 2011). Even so, there remains an overreliance on linear or finite quantitative measures. Müller et al. (2020) found that 88.6% of 13,963 articles reporting studies on IC used such measures. These measures have limited, if any, ability to capture the “why” and “how” of development, leaving participants’ thought processes, reactions, and bumps in the road largely invisible.
This article focuses on an in-depth qualitative approach to evaluating college students’ IC development trajectories via reflective writing in first-year writing (FYW) courses. Our research responds to our commitment to create interventions that equip university students to contend with cultural differences, as well as to the general overuse of quantitative measures that do not fully capture the process of IC development.
Many U.S. universities have a required writing course that undergraduates take early in their degree. First-year writing (FYW) approaches are generally designed to facilitate a transition between secondary and postsecondary writing skills and expectations (Aull, 2015). Our approach is research-based writing, which we chose in part because it aligns with the goals, means, and outcomes of the institution’s FYW program (Introductory Composition at Purdue, 2024a); further, it allowed us to create assignments that engaged students in researching elements of the construct of difference thoughtfully and thus aids in developing intercultural competence.
Our approach to FYW, and IC, arose out of a genuine need that we observed on our campus: students tended to spend time with people like themselves, yielding Midwestern, Chinese, and Indian social “bubbles,” thus missing out on opportunities for rich cultural interaction. Our approach was initially pragmatic, but we were interested in how we could bridge Deardorff’s (2019) ocietal divides within our own discipline and teaching context. As FYW teachers, we noted many opportunities to address intercultural communication in our classrooms, with the goal of reducing the time students spent in these bubbles and equipping them to step outside these cultural comfort zones, which aligns with the values of a liberal arts education. The variety of FYW genres, including reflection, afforded the development of a method for continuous capture of intercultural development that could present different developmental profiles. IC assessment via reflective writing provides a window into qualitative and formative assessment of IC as well as a deeper look at the mechanisms that underlie IC development. That is, what we observed via reflective writing highlights the limitations of linear models of quantitative assessment and confirms that nonlinear conceptualizations of IC more accurately reflect empirically observable development pathways. Thus, the aim of this study was to develop a robust method of assessing IC development via reflective writing. The research question explored in this article is, how can reflective writing be leveraged to assess IC development in the FYW context?
Literature Review
Intercultural Approaches to First-Year Writing
Over the last decade, intercultural approaches to teaching FYW have become increasingly visible in U.S. composition studies. These approaches sustain the long tradition of engaging students in writing classrooms’ multicultural, multilingual realities. Contemporary intercultural approaches have been inspired by cross-cultural composition in the late 1990s and 2000s that enrolled both domestic and international ESL students, sometimes in relatively equal numbers, in the same writing course with interventions for “cross-cultural understanding, communication and collaborations” (Reichelt & Silva, 1996, p. 17). During the 2010s, teacher-scholars at U.S. 4-year universities (Willard-Traub, 2017), community colleges (Miller-Cochran, 2012), and on online platforms (Tseptsura, 2018) continued to develop cross-cultural composition pedagogy. Similar studies started mentioning “intercultural communicative competence” (Jordan, 2012; Lee & Jenks, 2016) or “intercultural competencies” (O’Brien & Alfano, 2015) as a learning outcome in FYW scholarship. These studies were often situated in discourses around diversification and internationalization of U.S. universities and writing programs (Williard-Traub, 2017). Internationalization agendas also expanded U.S. writing programs’ international operations through partnerships with writing programs in other countries, either in a main campus/branch campus relationship or a joint degree program (Martins, 2015; Rose & Weiser, 2018). Subsequently, these institutional and programmatic initiatives kindled conversations among writing administrators, scholars, and instructors about how to expand students’ academic and literacy repertoire to prepare them for the linguistically and culturally globalized, super-diversified educational and professional contexts (Benda et al., 2018). While first-year writing is primarily a U.S. model, intercultural writing pedagogy and attention to intercultural aspects of academic norms are also areas of focus in global contexts (Chuikova, 2020; Vinther & Slethaug, 2013). For example, renewed European attention to cultures of writing, and to making these cultural writing norms explicit, has resulted in new attention to writing pedagogy from an intercultural lens across the continent (Kruse et al., 2016). While these efforts are not explicitly focused on IC development via writing, they demonstrate growing international attention to the interaction between writing and interculturality.
Intercultural writing pedagogy has engaged many approaches to strengthen students’ language skills (including writing and general classroom engagement), rather than emphasizing IC as a direct learning outcome. Preceding intercultural writing pedagogies include cross-cultural composition (Matsuda & Silva, 1999), multicultural composition (Severino et al., 1997), intercultural rhetoric (Connor, 2011), and intercultural communicative competence in foreign language education (Byram, 2021). This added outcome necessitates the design of curricular interventions that cultivate students’ intercultural sensitivity to help them more effectively approach sociocultural and linguistic difference in the writing class and on campus (Jordan, 2012). In this previous work, IC and related constructs are discussed as holistic, general concepts without articulation of their components (e.g., cognitive, affective, and behavioral attributes). Articulating these components for assessment purposes matters, as it enables instructors to recognize evidence of intercultural competencies in students’ writings and determine their learning success.
Measuring Intercultural Development
Assessment of IC evaluates to what extent someone’s home culture defines reality, or whether the person can experience culture as fluid and related to context (Berg et al., 2012). There are two umbrella approaches to measuring IC: (a) the developmental paradigm (Bennett, 1993) and (b) the cognitive/affective/behavioral paradigm (Deardorff, 2006). Both include an emphasis on cognition and behavior. The first paradigm maps IC onto a series of developmental phases and involves moving from an ethnocentric mindset to an ethnorelative mindset (Bennett, 1993; Feng, 2016). The second paradigm of IC focuses on characteristics, such as encompassing feelings/emotions (affective), thinking/understanding (cognitive), and skills (behavioral) (Munezane, 2021). Developmental models identify particular characteristics, but instead of looking solely for characteristics that indicate strong IC, they also identify indicators of a spectrum of IC abilities. In other words, the difference between these two models is whether the scale is relative to the characteristics, or the characteristics are mapped onto a model that describes distinct levels or phases of competence. The article reports on assessment of IC through a developmental lens, the Developmental Model of Intercultural Sensitivity (DMIS) (Bennett, 1993); however, we also identify cognitive, affective, and behavioral indicators independently of specific phases through the qualitative assessment of reflective writing. The DMIS scale features a model of intercultural orientations toward difference ranging from ethnocentrism to ethnorelativism—denial, defense, minimization, acceptance, adaptation, and integration (Bennett, 2017). Each phase explains how occupants organize reality within culture(s). Movement between the phases is prompted by a need to become more capable of intercultural interaction. The advantage of this model is that it can capture a detailed view of how people move through the process of intercultural change. A definition of these phases within the context of FYW can be found in our earlier publication (Banat et al., 2022).
The DMIS model was adapted into a widely used IC assessment, the Intercultural Development Continuum by the creators of the Intercultural Development Inventory (Hammer, 2011). Rare qualitative studies that use the DMIS suggest that pathways of development are not inherently linear. For example, Acheson and Schneider-Bean (2019) argue that IC development can be better understood by a pendulum model, wherein individuals respond to challenging circumstances with a temporary retreat and later develop their IC, essentially swinging back and forth among phases, though showing holistic development over time. The order of stages and the movement patterns (Figure 1) represent the negotiation of similarity (denial and minimization on the left) and difference (acceptance and polarization on the right). In the middle is adaptation, the point of IC balance wherein similarity and difference are complexly understood in the context of intercultural interaction. One other rare example of capturing nonlinearity empirically is Bourjolly et al. (2005), who documented that, over a 10-month period, the DMIS pathways of 32 out of a set of 34 mental health workers were nonlinear—also measured via periodic reflective writings.
Pendulum Model of the Intercultural Development Continuum Orientations.
The most recent scholarship on IC development tends to contextualize existing models within broad questions about the purpose of IC in conflict-ridden social contexts (Ansara & Deardorff, 2022; Deardorff, 2019, 2020) or to consider operationalizing a more critical approach to IC: questioning economic justifications for IC as well as essentialist approaches and forwarding more social-justice-oriented approaches instead (López-Rocha, 2021; Zhao et al., 2018). Thus, there is a current opportunity to reconsider how measurement based on linear models of IC might reify arguments such as the following: IC is beneficial because it makes workers more able to collaborate, or IC development proves an institution is addressing intercultural issues. Our research is situated within these considerations, while we further worked to help students break down essentialist notions of difference within the United States’ increasingly conscientious political climate from 2016 forward.
The Role of Reflection in IC Development
Reflective writing is a staple approach to assessment in first-year writing and upper-level or discipline-specific courses. Literature on the value of reflective writing for development of students’ writing skills is well-established (Jankens, 2019). Scholarship on reflective writing in rhetoric and composition primarily focuses on the “mental activities of the composer in the composing process” and the value of metacognition for developing writing knowledge (Yancey, 2016, p. 3). Expanding on her earlier work on reflection’s role in assessment, Yancey (2016) presented the “third generation of reflection” in rhetoric and composition that showcased multifaceted roles and affordances of reflection in the higher education community. Relying on the National Survey of Student Engagement, Yancey (2016) demonstrated how integrative learning could allow students to connect prior knowledge and experiences, the curriculum, and broader societal issues. Reflection could thus prompt students to consider “the diverse perspectives of others as well as their own views while examining the views of others” (p. 9). Such ethnorelativistic reflection and evaluation of difference also lies at the heart of IC pedagogy. This is how we utilized reflection in our own curriculum, that is, to enable individuals to think deeply about what they have experienced and connect it to the current context and interactions. Yancey (2016) theorized reflection as a “Bakhtinian rhetorical exercise through which one engages with the cultural, to draw from it and give back to it an exercise of meaning-making” at both an individual and social level (p. 10), thereby offering us pedagogical possibilities and affordances for reflective writing beyond evaluating writing knowledge and its transfer to other contexts. These possibilities encourage innovation in pedagogy and inspire us to use reflection in FYW to support and observe IC.
Reflection and introspection have long been identified as critical to developing and maintaining IC abilities (Bourjolly et al., 2005; Feng, 2016; Ruben, 1976), although developing the former does not guarantee the latter. Indeed, explicitly fostering students’ reflective skills supports IC development in ways that cultural exposure alone does not. While cultural exposure only provides passive experiences of difference, reflection interventions integrated into intercultural training prompt metacognitive engagement in difference as these interventions require culture-crossers to make meaning of difference and arrive at a more sophisticated understanding of the encounter, which in turn benefits IC development (Deardorff & Arasaratnam-Smith, 2017). As a recommended practice in the field of intercultural learning, reflective inquiry can create a “staying power” for IC and advance students from the mere appreciation of difference to a more active interaction with difference (Wilbur, 2016, p. 69). Therefore, reflective writing intensifies the effects of experiential learning for intercultural development. Our previous work explores how building reflective and analytical skills can enhance students’ IC development (Banat et al., 2022).
Notably, curiosity, openness, and metacognition are crucial for developing IC (Deardorff, 2006) and are also identified by the Council of Writing Program Administrators and the National Council of Teachers of English as habits of mind that promote success in college writing (Council of Writing Program Administrators et al., 2011). As Clark notes, “reflection allows students to look uncertainty and change in the face and find in that growth and change a continuity that will follow them over time as they learn new things, have new experiences, and engage with the unknown” (cited in Yancey, 2016, p. 160). Written reflection therefore serves dual purposes for meeting the IC learning outcome: (1) students can make sense of the intercultural interventions in the curriculum and their concurrent experiences, and (2) develop a process of inquiry for sophisticated learning. Reflection allows students to contend with ambiguity and uncertainty as they continue to encounter new experiences, cultures, and communities within their universities (p. 160).
Methods
This project focuses on developing a qualitative method to observe students’ IC development, particularly their engagement with the four key interventions in our curriculum, past and concurrent life experiences, and characterizations of IC development.
Context and Participants
This IRB-approved study (1703019016) took place at a public land-grant institution in the United States. At the time of the study, this institution served a diverse student population: 53% of undergraduates were in-state students, of which about 18% were domestic minority students (including noninternational Asian, Black, indigenous, Latino, and multiracial students), and 16% were international students (Advisory Committee on Diversity, 2016; Neubert, 2017; Office of International Students and Scholars, 2017). Thus, the institutional location was more diverse in national origin than most public universities in the United States, and less diverse than average in terms of domestic minority students (Institute of International Education, 2023; National Center for Education Statistics, 2019; Pew Research Center, 2017).
Our university’s first-year writing (FYW) program includes general (“mainstream”) sections, international sections, and other specialized sections such as service-learning courses. All students must take FYW unless they have a score of 4 or 5 on the Advanced Placement Language and Composition exam. The university writing program offers guided self-placement (Introductory Composition at Purdue, 2022). With the input of advisors and test scores, students choose which first-year writing course they feel is most appropriate.
We recruited participants from eight FYW sections across three semesters: spring 2017, fall 2017, and spring 2018. Initial recruitment was offered to all students in our sections, which yielded 58 participants out of approximately 175 students. The sample was formed on an opt-in basis. Because of a data storage error, we completed analysis for 44 participants.
Data Collection
As we considered project set-up, we wanted to ensure that students who chose the international FYW section for language and social support received that support. We also needed to facilitate intercultural contact between international and domestic student populations. Thus, we paired an existing mainstream with an existing international section, rather than combining these sections. Sections were scheduled at the same time, facilitating a series of biweekly shared meetings. Further, since neither section was homogenous, students were interacting across cultural differences even within their “home” section. We—the four researchers (then graduate students) and coauthors—designed the curriculum and taught the classes, under the mentorship of an experienced faculty member and with guidance from two intercultural specialists at our institution. The curriculum relied on four interventions:
Linked sections with coteaching that fostered intercultural interaction
A multicultural reader
A research-based assignment sequence that required engagement with cultural difference
A series of five reflective writings—we titled the first four as reflective journals and the last as a course reflection
These four interventions balanced intercultural exposure (through readings and secondary research) as well as intercultural interaction (through the linked sections activities, out-of-class requirements for intercultural interaction, and primary research that fed into the assignments). Because we wanted to help students contend with inequality and social injustices, we selected readings to make visible some of the social divides that result from such situations (e.g., economic stratification, gender discrimination, movement across borders). Assignment sequences commonly integrate cultural inquiry into FYW genres like literacy narratives, research-based argumentative writing, multimodal composition, and reflective writing (Martins & Van Horn, 2018) to mentor students on navigating linguistic, sociocultural, and other types of difference (Benda et al., 2018). Reflective journals were placed strategically within interventions to scaffold reflective practice and provide opportunities to connect curricular interventions, classroom activities, and concurrent university experiences. The primary purpose of reflection was to offer students opportunities to analyze and evaluate their experiences with intercultural exposure and interaction. 1 Instructors used dialogic feedback to respond extensively to what students wrote about their encounters to foster more complex engagement with cultural encounters, whether those encounters were textual or interpersonal.
Shared goals, means, and outcomes (GMO) (Introductory Composition at Purdue, 2024b) helped us create a common curriculum for both classes. These GMOs focus on developing rhetorical awareness of diverse audiences, situations, and contexts; composing in a range of genres; critically thinking about writing through reading, analysis, and reflection; providing and incorporating feedback; performing research and evaluating claims; and engaging digital technologies. The intentional smaller enrollment of students in the international sections, and the limited number of common meetings with the general section, meant that the instructor for that class was able to offer L2-specific writing pedagogy with significant scaffolding. For example, though both sections created the same assignments, the drafting practices were different and more scaffolded in the international sections.
Analytical Procedures
Data Processing
Our data sources for this article are 219 reflective pieces (one participant did not turn in one of the five reflections). After the conclusion of the semester, student writing was downloaded from the learning management system and deidentified, removing student identifying information and instructor names. We saved each piece of writing as a Word document and also split it by sentence in an Excel spreadsheet.
Coding Scheme Development and Procedures
We adopted a collaborative grounded-theory approach to coding development, using an inductive approach to our data and engaging an iterative process wherein we returned to student writings across several phases of analysis to consider how we saw intercultural development present (or absent) in the data. We chose a grounded theory approach for the detailed coding because, although IC development is well-theorized, it was new to consider what reflective writing assignments from FYW might reveal (if anything) about IC development within an IC-focused curriculum. We felt that existing theory did not fully explain the process (Creswell & Poth, 2016) for engaging IC in FYW.
We did not come to the data set with a predetermined set framework for coding, but instead looked for themes, ideas, or phrases that might signal something about students’ cultural engagement. In the first round, we read the reflective writings from the initial eight participants from spring 2017. We individually adopted open thematic coding to develop a comprehensive list of thematic codes and negotiated some common themes, agreeing on an initial, but not closed, set of codes. For example, we considered whether initial codes spoke to the project aims, eliminating those that did not. We combined codes that were recurrent but worded differently. Then, we considered whether we would take a “lumping” or “splitting” approach (Saldaña, 2016). We chose splitting, since sentence-by-sentence analysis facilitated an easier way to develop and track interrater agreement, so that we could easily identify and discuss areas of disagreement. Each sentence could have multiple codes applied. Additionally, sentence-by-sentence splitting allowed us to quantitatively trace changes in code frequency over time, yielding a detailed look at how students individually and as a group engaged with aspects of the curriculum and displayed evidence/traces of IC. Throughout the process of coding development, we created coding memos to rationalize the code meaning and the process of applying it to text.
During summer 2018, we returned to our initial list of codes and to the reflective writings from the eight initial participants, transitioning from generative open coding into an axial phase wherein we considered the relationships and distinctiveness of the tentative codes, reaching agreement on a set of meaningful codes that addressed features of intercultural and writing development. Each teacher-researcher coded all reflective writings from this set of documents and we discussed our coding choices to arrive at a common understanding of the code meanings. All teachers coded some of their own students’ work and some work from other sections. At this point, we settled on a code book, defining each code uniquely and including examples from the reflective writings. As in grounded theory approaches, we were then able to group these codes conceptually; however, since we were working within an existing IC framework, we considered how our coding scheme resonated with that framework (which was used to shape the curriculum). This was a pragmatic departure from pure grounded theory. We noticed that the codes fell into five categories, four of which (cognitive, affective, behavioral, and multi-indicator) offered insight into students’ intercultural journeys and aligned with our chosen framework from Deardorff’s scholarship. The fifth category related to how they engaged with writing development.
Table 1 below shows how our codes function as IC indicators. 2
Codes and Indicators.
In fall 2018, we trained two undergraduate researchers in qualitative data analysis. They analyzed a portion of the data, providing an outsider’s look at the coding system’s coherence. They created reflective memos on the coding scheme to better define the codes and elaborate on what the codes revealed about IC development. Both researchers suggested adjustments to the coding system to ensure unique definition and consistent use of each code. After intensive sessions where we all coded together, and then further collaborative coding sessions with undergraduate researchers to ensure common understanding, we split the entire set of 219 reflective writings among our team of six. Our approach to intercoder agreement, drawn from Smagorinsky’s (2008) model of collaborative coding, involved two initial coders for each reflective piece. A third coder then resolved any discrepancies between the two primary coders. Rare instances where three coders could not agree were discussed by the whole group.
Once all reflective writings were coded, one graduate researcher wrote programs in Python to tabulate the numbers of thematic codes by students (for a profile of each individual) as well as by class and semester so that we could identify changes and trends. The qualitative codes were analyzed for the relative levels of frequency across the semester, and helped us systematically understand detailed features of intercultural development. We found the thematic coding especially helpful for understanding the cognitive aspects of IC development as well as how students engage cultural exposure and cultural interaction as a part of IC development. The thematic coding trends complement our more holistic analysis of development detailed in the next section.
Holistic Mapping on the DMIS
We applied the Developmental Model of Intercultural Sensitivity to students’ reflective writing so that we could observe development trajectories (or lack thereof) across the semester. We chose the DMIS because we appreciate the clarity and depth with which phases and indicators of these phases were described and felt confident that we would be able to align our method of analysis with broader DMIS scholarship. A developmental model further promoted the idea that all students could achieve growth (Mellizo, 2019) and helped us respond to resistant or defensive students with an eye toward what could open them to engaging IC.
The four graduate teacher-researchers met in person to “map” or place each of the five reflective writings on the DMIS phases for eight Spring 2017 participants. We discussed the foci and characteristics of each piece of writing and negotiated how to identify the signals of IC development phases. For the remaining semesters, we created a grid document that gave each of us a place to describe each participant’s reflective journal and explain why we thought that it displayed a particular phase of intercultural development, and then a spreadsheet that showed larger trends.
Table 2 shows a summary of one participant’s DMIS mapping. 3 We discussed any reflective writings where fewer than three graduate team members identified the same phase. In the example below, no further conversation was necessary. This dialogic process ensured a shared understanding of observing intercultural development in writing, and particularly how to identify journals where students displayed multiple phases of IC development simultaneously. We also consulted an IC specialist at our university’s intercultural research center to clarify some writings where we had evaluative uncertainties.
DMIS Mapping Sample.
Findings
Our research question considers means of IC assessment within the writing curriculum. Analysis indicated that an approach to tracing IC via writing, with multiple collection points across the time of intervention, shows that development is more complicated than linear progressing from one phase to the next. Reflective writing offered a lens on how development occurs in response to experiences before, within, and outside the class. Importantly, we observed nonlinear pathways of development, including pendulum swings and deepening within phases.
Thematic Coding and Analysis
Through thematic coding and analysis, we found that students engaged the cognitive domain most strongly, and we observed that many students referred to previous cultural experiences, reinterpreting them using the cognitive skills gained. At times, they also articulated how they will do so in the future. For example, students recognized that judgments about culturally “othered” people made before encountering class material and/or the university environment could be stereotypes. Toward the end of the course, students reported delaying judgment and being open to changing their perspective in new cultural encounters. One student wrote, “I’ve learned to do research before I make an impression of a specific culture issue, and change my perspective of reading it to the person in the culture context.” This was a commonly reported change across both the international and domestic participants.
Our analysis of the sentence-by-sentence thematic coding indicated that the most frequently used codes were in the cognitive domain (28%), with fewer in the affective (17%) and behavioral (16%) domains. To examine trends across the semester, we compiled the most prevalent codes of the three domains that we observed in students’ written reflections and showed their frequency over time in Table 3. Each code is listed by frequency of occurrence in each group of reflective writings; for example, for Journal 1, 13% of the total code instances were critical evaluation. Each piece of writing can demonstrate a range of codes.
Domains and Code Frequencies.
Among the cognitive codes, “critical evaluation” was consistently predominant in the four journal entries. “Critical evaluation” refers to interpretation, inference, synthesis, questioning, or analysis of cultural events or phenomena. Since critical thinking skills are instrumental to intercultural development, and students demonstrated “critical evaluation” consistently in their reflections, this suggests that reflective writing offers a means to engage, and observe, this facet of IC development.
We observed that students used new material from the class to reinterpret past experiences through an intercultural lens, which we traced via the “prior knowledge” cognitive code. While “prior knowledge” appeared more in the first two journal entries, its frequency decreased in the remaining entries. When IC development-oriented interventions were still new at the beginning of the course, students must rely on their previous cultural encounters as an entry point to engage with the intercultural learning activities. In these reflective journals, we observed students reinterpreting past experiences through the curricular material and their concurrent campus-based intercultural interactions.
For example, Lixin, an international student, identified that encountering class material (learning through course readings about ethnic stereotypes) and meeting Black American students made him question his previous limited exposure (via news media): Before coming to the US, I sometimes could hear news that there is a gun shot, or murder in the US, and many of them were done by African Americans. My mind automatically stereotyped African American as violence or only good at sport. After I came here, firstly I did not like to talk to them and stay away when I see a African American on street at night. However, one day, a black guy who lives next to my room, said hi to me, and made a self-introduction. Afterwards, we talked a lot, and became friends. He is very nice and kind, unlike what I thought he was. Then I realized I was affected by my stereotype, and this stereotype came from my ignorance, because I had never made a friend and chatted with a black guy before, and I even do not know them.
This student identifies a conscious shift in his attitudes and behavior. These sentences were coded with prior knowledge, stereotype, cultural identity, and critical evaluation (the cognitive domain of IC), openness and positive attitude change (affective domain), behavioral change, cultural interaction, and cultural exposure (behavioral domain). We saw a moderate and stable engagement of the affective domain throughout the semester and an increase in positive feelings about intercultural interaction by the end of the semester. While these phenomena in themselves were not an indicator of substantial intercultural progression, they were a positive sign because the affective domain is integral to intercultural development.
Among affective codes, “emotional response” had the highest frequency. Emotional responses were reported as catalysts for thinking about intercultural experiences; for instance, students reported discomfort with new experiences or frustration at being misunderstood. These students then considered why such emotional responses happened. Developing IC requires self-awareness of emotion, particularly acknowledging reluctance, fear, and discomfort (Guntersdorfer & Golubeva, 2018). Keeping the developmental and long-term nature of attitude change in mind, the modest upward trend of “attitude change” as an affective code in students’ reflections was also an important indicator of their intercultural progression. In our data set, nearly all reported attitude changes were positive. Positive attitude changes are a sign that students are responding to an intervention and/or intercultural encounter with a perspective shift toward relativistic appreciation of difference. The most common attitude shift identified was appreciation for cultural difference, or appreciation of the chance to interact with peers from other cultures. Another common reported shift was the transition from discomfort to relative comfort. Students rarely reported negative attitudes or negative shifts; rare instances involved course reflections where students briefly expressed that they did not really want an IC-focused FYW experience.
We found a much lower rate of behavioral indicator codes. Given that “attitude change” happened to a modest degree and generally precedes behavioral change, it is understandable that lower engagement with “behavioral change” was observed. For a 4-month writing course where IC is only one of the learning outcomes, we could not avoid contextual constraints that prevented us from implementing the best interventions for behavioral change in students. On the other hand, our analysis revealed that the two behavioral codes “cultural interactions” and “cultural exposure” appeared frequently. These trends indicate that students were immersed in the curriculum’s intercultural learning activities and showed heightened awareness of these contextual affordances in their reflective practices. Similar to the attitude change, students only reported behavioral changes that represented intercultural gains. This does not mean that defensive or polarized behavioral changes were absent from the student experience, but they were not included in the students’ reflections. It is possible that students did not want to include something negative; it is also possible that students who exhibited defensive behavioral changes did not have the cognitive IC ability to reflect on these defensive behaviors.
Overall, the thematic coding revealed that students engaged the cognitive domain of IC very strongly, with more moderate engagement with other domains. The detailed thematic coding allowed us to understand how engagement with aspects of IC can propel movement among phases of the DMIS scale.
Holistic Mapping on the DMIS Scale
Table 4 displays the overall developmental profiles measured through holistic mapping on the DMIS scale . Some students showed more than one trend (e.g., development and pendulum or maintenance and pendulum); thus, the number of profiles exceeds the number of participants.
Student Development Profiles.
The 15 students whose profiles exhibited back and forth (pendulum) motion across the semester either ended up within the same phase that they began (maintenance) or at a more ethnorelative phase (development). Nonresponsive students’ writing exhibited no, or very limited, reflection in at least three of the five reflective pieces. These students also exhibit low levels of intercultural development (defense or minimization) though some show acceptance. Reasons for nonresponsiveness were not always clear. It is possible students did not want to engage with the interventions or felt threatened by intercultural interaction. It is also possible that struggles with writing or reflective skills limited the extent to which we could observe their IC. As instructors, we likely did not meet every student at the point where they could engage IC development.
Across both sections, students who either showed overall DMIS scale development or maintained their initially ethnorelative stances (whether or not they had a pendulum swing in the middle) displayed more nuanced understandings of culture and cultural difference in the two final reflective pieces than they displayed in the initial pieces. This nuance also showed in students’ stronger abilities to critically analyze their home cultures and understand how home cultures shaped their present interactions. IC development requires a balance of self-understanding and other-understanding. A representative case of pendulum development and change in home culture analysis follows.
Lixin started the semester with a passive, surface-level acceptance of difference. He relates learning that stereotypes in both China and the United States are connected to assumptions about race or social class, and then explains how meeting Black dorm mates aided his realization about the harms of stereotyping. In the second journal, Lixin retreated to a defensive stance as he wrestled with the relationship between countries’ economic systems and the quality of life for citizens. While he did not state his culture was better, he reduced cultures to simple absolutes, asserting that one system was better as long as the country had enough money.
Through course readings and discussions — and through his new experiences in an American capitalist system — he learned about the historical and cultural features of different systems but struggled to consider them through an interculturally competent lens. These new ideas seem to have influenced his retreat into a defensive stage of understanding culture. In his first journal, he addressed the topic (stereotyping) at a more basic level and thus displayed features of acceptance; in this second journal, he was attempting to get at the root issues of the relationship between economic systems and culture, but did not quite make it. An attempt at more depth is a sign of growth, even though it means he displayed a lower IC phase. For his final journal, he wrote about class activity on cultural scales (e.g., implicit/explicit communication, individualist/collectivist orientation) and showed much deeper self-understanding, where he understood how his home culture shaped his interactions with cultures who had different features of explicit communication and individualism. He wrote, For example, when you find your friend just bought a luxury car, and you told him: “your car is so nice.” If your friend is an American, he would say thank you. However, if he is Chinese, he would probably say: “no, no, no, I just bought it at discount, and it is not nice.” When we read [between the] lines, the real meaning for this sentence is that “I do not have a nice car,” putting himself at a modest position, which saves your face. These words might be pretty confusing for foreigners, but that is how Chinese culture works.
Lixin recognized cultural differences in behavior and communication and connected those differences to broader underlying cultural values. He then described why his own culture’s communication is a certain way and explained what it might look like to an outsider with a different, less face-saving, standard of communication — features of acceptance with a level of depth not seen in his first reflection. The comparison reveals an ethnorelative evaluation of difference, even if appreciation of difference is not part of his testimonial. The fact that he engaged in comparison reveals signs of development. These examples are broadly repeated across the data set: progress can be measured in the depth of analysis of intercultural encounters as well as in movement between phases.
Discussion
There are three aspects of intercultural development that we believe are uniquely revealed through reflective writing: a combination of nonlinearity, nondiscreteness, and development within phases. Through reflective writing, we were able to observe the patterns of responsiveness (or resistance) to intercultural encounters. Together, these three aspects demonstrate that reflective writing provides a window into how individuals navigate and adjust to intercultural situations, and respond to new contexts that can challenge or deepen, and enhance or reverse, their overall IC development. One aspect, empirical observations of nonlinearity, has been reported in a case study by Acheson and Schneider-Bean (2019) and in occasional previous research. We observe this same pattern in a larger sample. Two other aspects, nondiscrete nature of IC development and development within phases point to a need to add more nuance to otherwise nonlinear models such as the pendulum model.
Nonlinearity
Our analysis observed that profiles often involve progress and retreat, or periods of stability and then change in both directions. For our population, swinging back and forth between phases was common, even within one piece of reflective writing. There are “magnets” and “anchors” that propel movement or balance on the pendulum. Magnets cause an over-focus on similarity or difference whereas anchors promote balance (Acheson-Clair & Schneider-Bean, 2020). Conflict between two individuals from different cultures could lead a person to focus on difference while positive connections with others acts would lead to a focus on similarity. A strong sense of self and emotional maturity are anchors that aid someone’s ability to find pendulum balance. In the adaptation-indicative writings, students describe self-confidence, enjoying interaction, and/or strong self-understanding—“anchors” that balance out understanding intercultural similarity and difference. For example, Emma, a domestic student, explains that at first, cultural interaction scared her—a magnet that pulled her away from developing IC. At the end, she stated that she had learned about an unfamiliar culture, . . .but most importantly I took a step out of my comfort zone, and overcame my anxieties. And as a result, I was able to meet a really awesome person that blossomed into a new friendship.
Another example of anchors can be seen in the following course reflection from Haoyu, who began the semester with a defensive (polarized) mindset: Now I understand a culture sometimes can be the representative of a civilization, which means if I want to achieve a further understanding, much more complicated information is required, and before I get enough information to see in a whole page, judgment is meaningless.. . . Not only my point of view about other cultures has been changed, I also found a new method to discover my own culture.
Here, Haoyu shows a balanced understanding of similarity and difference, reporting confidence in his sense of self and confidence about seeking information before reaching an understanding. Emma’s experiences were uncomfortable at first but then filled her with a positive sense of capability. Haoyu’s reflection mainly employs cognition to explain prerequisites for successful intercultural interactions while Emma’s reflection focused on her emotional response as the outcome of intercultural interactions. Although Haoyu’s reflection is analytical and Emma’s is descriptive, we witness enhanced confidence in both testimonials. These two types of confidence form anchors that help maintain balance when considering similarity and difference, even though these students may experience swings in future intercultural encounters that present new challenges.
We observed nonlinearity most through the cognitive engagement with IC. As noted in the findings, we had few reports of regression in behavioral changes and attitude changes, perhaps because of the context of the class and the use of reflective writing as a measure means that there is simply a larger amount of material engaging the cognitive facets of IC. Students do not necessarily need to be aware of their attitudes or behaviors for us to observe ethnocentricity or ethnorelativism in how they conceptualize or engage difference in their writing. Students need to be aware enough of attitude or behavioral change to report it and make meaning of it in writing. If students were ethnocentric in their behavior, they might have been hesitant to express those perceptions in reflective writing.
Nondiscreteness and Development Within Phases
In addition to the nonlinearity of development, we also noted that phases were not discrete and sometimes included internal development. Unlike previous qualitative IC development research, we account for the profiles of many students who showed characteristics of multiple DMIS phases at once, suggesting that even within the same person, IC is contextual and contested. This observation may be a function of allowing our coding system to accommodate mixed phases. For instance, Bourjolly et al. (2005) only assigned one phase to each reflective piece, assigning the higher phase if two were possible. Their choice potentially disguised an important piece of the development pathway that we did not want to lose. Mellizo (2019) briefly noted the presence of statements aligning with multiple phases in a few interviews with students in a music class, explaining, “On paper, this growth continuum appears very straightforward and linear. However, human behavior rarely fits neatly into such clear-cut categories” (p. 484). In our sample, 16% of the reflective texts showed a participant engaging two phases at once. 15% of the texts focused on writing development, so of the ones that primarily engaged the cultural aspects of the course, nearly 20% demonstrated this nondiscrete trend. Sometimes, the reflective text showed that the participant was on the border between two phases, and the writing captured a moment of transition. Other times, the participants simply showed characteristics of two phases at the same time when attempting to balance understanding similarities and differences.
For example, participants can show acceptance and defense in the same reflective journal, because certain reflection scenarios can portray the developmental process of learning through multiple stages as participants attempt to bridge cultural differences across various time periods referenced in their reflections. Or, participants may have had a more interculturally competent response to part of a situation, but exhibited defense or minimization in another part of that same situation. We found that students often engaged with the readings at a higher level of IC than with interpersonal interactions. This may be due to the higher stakes of interpersonal interactions. Acheson and Schneider-Bean (2019) note that successful bridges across differences resolve “the dissonance between self and other, achieving a dialectic of similarity and difference” (p. 50). This argument unveils a fluid, intricate developmental process. We also see this fluidity, and tension between understanding the self and the other, in many of our participants’ development.
Often, students made a statement that articulated difference and explored the root of that difference, but then ended with a statement of minimization. An international student, Aarshanti, stated, “In the end despite what culture or geographic location a person may come from, despite being drastically different from people from other places, there are a few common characteristics that connect us.” A domestic student, Brian, made a similar statement: “Throughout my time with my [FYW classmates], I learned that people of cultures and places different from mine are more similar to me than I could have thought they could be.” Brian then moved on to explain that also, there are different beliefs he often judges because they do not seem appropriate. Particularly for students who displayed acceptance, there were moments of minimization because students still experienced discomfort with difference or felt the need to use minimization to make connections across difference. The ideal balance between the two, understanding the contextual nature of interactions as shaped by both culture and individuality, would characterize adaptation. This dual-phase phenomenon is more evidence for a pendulum model of intercultural development as well as a challenge to the pendulum model, which still shows the phases as separate from one another.
We also saw change within phases, where students deepened their understanding of culture although they may not “progress” to the next phase. When conducting IC assessment, attending to within-phase development is critical; otherwise, responsiveness (or lack thereof) to the interventions may be missed and students’ trajectory shapes will not have the necessary nuance to reflect development. Profiles may simply appear stagnant, although the person is actively engaged in, reflecting on, and learning from intercultural encounters. Reflective writing’s affordances for promoting critical evaluation may be a mechanism for development within a phase. Someone who is critically evaluating behavior in an intercultural interaction, but is not developing toward a more advanced phase, may be working toward better understanding the “why” behind their current abilities or feelings. This reflective thinking takes time, which explains why some participants might spend more time within one phase of development.
Criticality, Analysis, and Intercultural Development
Reflective writing prompted students to reconsider both past and concurrent experiences in light of their engagement with our curriculum. Students maintained high frequencies of cognitive capacities such as their ability to activate their prior knowledge and engage critical evaluation. Evidence of advanced cognitive capacities shows how involving students in reflective practice can strengthen students’ analytical and critical learning. Table 5 gives the rates of cognitive codes across the semester:
Rates of Cognitive Codes by Reflection.
Higher engagement with cognitive aspects is likely a function of the curriculum, the university context, and the way that we measure IC via writing. Writing, like IC, is a cognitive and sociocultural activity, and in it we ask students to do both types of work. A 4-month writing course is relatively short, and by design has more analytical cognitive activities, so it is unsurprising that students demonstrated the most intercultural engagement cognitively. This trend is congruent with the findings in previous studies: training interventions showed the most significant impacts on students’ cognitive intercultural development (Behrnd & Porzelt, 2012; Czerwionka et al., 2014).
We saw continued high engagement with cognitive codes throughout the culture-focused reflections and a drop in the final course reflection where many students focused on reiterating impactful writing concepts and practices. Importantly, this is a drop across the entire set, and those reflective pieces that continued to engage intercultural themes continued to have a high rate of cognitive codes. We think that the challenges presented by intercultural situations may have propelled students to inquire, analyze, and critique in their reflective writing—that is, to move beyond reporting.
One notable exception to deep cultural learning that surprised us given the FYW context was that students rarely interrogated the “correctness” of English academic writing conventions/expectations or saw them as culturally situated. Use of mother tongues was largely for process reasons, such as helping another student understand something, or for identifying equivalent figurative language. This same trend was observed by Zhang-Wu (2023), who noted that multilingual FYW students had doubts about the legitimacy of using their home language in formal assignments, but often used the mother tongue in the process of navigating course content and developing assignments. We noticed that it seemed safer to students to discuss language learning as a linguistic process than a cultural one. Domestic students, too, noted that English is a powerful language that benefits the speaker, but rarely critiqued that status.
However, after reading a piece by Anzalduá (1987) that used Chicano Spanish and English and one by Villelabeitia (2005) on mixed English-Arabic phrases, more students engaged language as a cultural phenomenon rather than a purely linguistic one. For example, one student explained that before reading these pieces, she would have assigned lower class status to someone who used Venezuelan slang. She concluded, If I judge based on my cultural identity another person from a different cultural identity I can be wrong.. . . Therefore, I realized that I can’t judge another person from a different cultural identity because I don’t know how exactly language plays a role in this person’s country.
Several domestic students reflecting on these same readings explained that they did not realize language was closely associated with culture — particularly, that it could be indicative of class and stigmatized. Thus, even within the FYW course, language was a particularly sticky topic for students, and readings that demonstrated the cultural implications of language and multilingualism were key to promoting critical analysis and spurring IC development related to this facet of culture. However, when applied to their own course-related language learning, students were not able to contextualize that learning within a cultural frame. This suggests students may not perceive language difference as a cultural phenomenon unless it is embedded in the curriculum and supported by formal classroom instruction.
Overall, reflective writing provided a heuristic for formative assessment that allowed us to track students’ continuous development of IC. It further offered students the opportunity to practice and engage ways of thinking that promoted more critical and analytical interpretation of cultural phenomena. As mentioned earlier, IC tests and scales present results of a student’s performance in IC domains and their advancement, or lack thereof. However, these scores are not comprehensively informative of factors related to or influencing students’ development, their interactions with instructional pedagogies, and their gradual advancement in learning. Our results have demonstrated how reflective writing is a responsive medium for mapping relationships among intercultural activities, student experiences, and perceptions of the curriculum, and for increasing understanding of their intercultural encounters. The main purpose of IC tests and scales is assessment; however, reflective writing has the affordance to teach students analytical and evaluative reasoning in addition to utilizing it for assessment. While reflective writing can have self-reporting limitations, it provides a window into how writers understand their actions and what connections they make between what happened and how they are thinking about it. Sheppard et al. (2022) differentiate between descriptive, analytical, and critical reflection, noting that the latter is difficult for students new to reflection to achieve. In our curriculum, we employed a described analyze-evaluate heuristic in the design of reflective writing prompts to train students to process their own feelings and behaviors when encountering cultural differences. Our emphasis on analysis and evaluation was reflected in our results, thus showing improvement in students’ cognitive capacities as an indicator of IC development.
Limitations
In our data set, we did not have any students whose language proficiency substantially impeded our ability to evaluate their IC; however, it is likely that there is a level of nuance and depth that some students would be able to display in their first language that was not captured in their second. As we reflect on our curricular approach, we note that in the future, we need to increase the amount of explicit teaching on reflective writing to better support students’ skill development in this area.
As four of us transitioned to new institutions across three countries, we also realized that we first implemented this curriculum in a near-ideal place — a university with substantial international and domestic linguistic, ethnic, and racial diversity (types of difference visible to students on a daily basis), a very motivated and high-achieving student body who lived mostly on or near campus, funding and support from the campus intercultural office, and many sections of first-year writing that facilitated easy scheduling. As PhD students, we had a low teaching load, which gave us time for labor-intensive coding and analysis. The infrastructural, environmental, and personal factors meant that we were unusually well-supported in a place where intercultural development was welcomed. In a different location, implementation would have been more complex and results could have revealed types of profiles not seen here. This has alerted us to how important it is to tailor a curriculum to the exact context of implementation, and that results should be interpreted in the context of local affordances and constraints.
Significance and Impact
The study suggests three important things: (1) that first-year writing courses offer contextual affordances that allow for integration of IC interventions and IC qualitative assessment with robust writing instruction, (2) that qualitative assessment through reflective writing offers promise for in-depth analysis of intercultural development, and (3) that qualitative assessment of reflective writing reveals more nuanced intercultural development profiles than quantitative measures based on linear models. Thus, the significance of this work is theoretical, methodological, and pedagogical.
Theoretical Impact: Models of Intercultural Development
Alongside Acheson and Schneider-Bean (2019), we advocate for revision of the DMIS to account for patterns of movement that do not resemble linear “progression,” such as extension of a pendulum model. A focus on how people respond to specific interventions and changing cultural contexts can inform such revision. Our work also connects the DMIS worldview scale with the more concrete written, cognitive, and behavioral indicators of these phases via our grounded coding scheme, strengthening connections between IC worldviews and how individuals live out these worldviews in real situations.
We further think that a revised model should account for development within phases. In future work, we hope to investigate this phenomenon, considering what might characterize the first instance of acceptance versus what acceptance would look like when someone has engaged more deeply. What might prompt transition from acceptance to adaptation, and what instead might strengthen and deepen acceptance? What is the relative value of strengthening a phase compared to movement between phases? It is important to understand the developmental profile of someone who has just moved into an acceptance phase, compared to someone who has spent some time in that phase and had the chance to practice the aligned skills. Future directions for further research include demarcating the basic and complex markers of intercultural mindsets and skills at the different phases.
Methodological Impact: Measuring Nonlinear Intercultural Development
Reflective writing provides an effective means for this type of exploration because it can include the nuances of nonlinear development — particularly with assessment methods like ours that do not force each reflective writing into one phase on the DMIS, nor do we privilege either the “higher” or “lower” phase, as both reveal important evidence about the writer’s developmental trajectory. Examining reflective writing that is created in response to an IC-focused curriculum shows why people in different developmental phases respond to intercultural situations in certain ways, and what kinds of experiences might spur someone to engage cultural difference more capably.
We advocate for investigating multiple data points across a sustained intervention, so that it is possible to get outside a mindset that solely focuses on linear “progress.” While developing stronger IC remains a goal, it is important to understand how cultural challenges may result in a temporary overcompensation toward either similarity or difference but are actually building toward stronger intercultural abilities. Attention to nonlinear development via qualitative reflective writing further opens exploration into what experiences might cause long-term damage to intercultural abilities, and why.
Pedagogical Impact: Integrating IC and First-Year Writing
Our work demonstrates one way to integrate IC into already common FYW assignments, where they can be employed without too much extra work once instructors are comfortable with an intercultural approach. Reflective writing is richly considered in writing studies scholarship, but not from a systematic intercultural lens.
The next steps of this work are underway. We have published our curriculum and extensive implementation guidance on an international intercultural learning platform and made it free for use and adaptation (Sims et al., 2023). One coauthor is now a faculty member at an institution with a two-writing course sequence for introductory composition and has implemented an adapted version of the curriculum. However, the newly implemented curriculum follows a different assignment sequence across two semesters of development to fit the specifications of that program. Moreover, 59.9% of the new institution’s domestic student population are of color, 61% first generation, 59% multilingual, 48% Pell Grant recipients (of full-time undergraduate students), 26% part-time students, and a large number who are immigrants or children of immigrants (Hughes, 2023; Office of Institutional Research, Assessment, and Planning, 2024; University of Massachusetts, 2020). This profile offers new affordances and challenges for IC learning in mainstream FYW classes that have a major representation of ethnically diverse, immigrant, and Generation 1.5 multilingual students. Thus, we gave more explicit attention to racial diversity when redesigning the curriculum. The results from this adaptation inform improvement of the IC integration heuristic at new institutional contexts with different realities.
Our classroom-based work happens within increasingly polarized political climates, nationally and internationally. UNESCO points out that the costs of poor intercultural competence are high: certainly, conflict and war, but the more day-to-day interpersonal costs are also substantial. More broadly, stronger IC should yield a more respectful social world where human rights are valued and upheld (Leeds-Hurwitz, 2013). We hope our work contributes to this goal. We see our role as educators extending beyond discipline-specific knowledges and skills, and toward a greater peace-building purpose that we believe underlies the project of education.
Footnotes
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was generously funded by the Council of Writing Program Administrators Targeted Research Grant, the Center for Intercultural Learning, Mentorship, Assessment, and Research (Purdue University), and the Purdue University Department of English.
