Abstract
The present study explored how native speakers (NSs) and non-native speakers (NNSs) of English exploit evaluative meaning-making resources of English when sharing their judgements and opinions on English movie descriptions. To this end, 10 native speakers of Australian English and 10 Iranian non-native speakers took part in on-line informal interviews, based on an approach known as snowball sampling. The study employed a combination of both qualitative and quantitative methods of analysis. The results revealed significant differences in the use of evaluative language between the two groups, with NNSs focusing more on Affect, Judgment and Appreciation respectively, while NSs prioritized Appreciation, followed by Affect and Judgment. The findings are discussed by reference to cultural and linguistic differences in the use of evaluative resources. Moreover, NSs relied more on invoked evaluation whereas NNSs showed a preference for inscribed evaluation. However, both groups utilized a higher frequency of positive descriptions in during their descriptions of English movies. These findings can provide valuable insights for language teaching and research, highlighting the significance of incorporating evaluative language resources into language instruction. The study further brought attention to the interconnectedness of diverse evaluative resources such as reference to emotions and judgements with complex linguistic and cultural domains.
Introduction
Verbal- especially spoken- communication as the default form of human communication can arguably be the predominant form in human societies. It is typically viewed as a process via which a message is conveyed and interpreted through the means of language and shared linguistic features. Verbal communication is generally performed for many and varied reasons such as exchanging information, delivering messages, making judgments, and expressing personal feelings and opinions. It has been argued that evaluative language is one of the most significant aspects of verbal communication (Bellés-Fortuño, 2018; Ngo, 2013). As a multidisciplinary enterprise, this perspective on language has recently received the attention of scholars from various academic domains such as computational linguistics/applications (Benamara et al., 2017), medicine (Bellés-Fortuño, 2018), and linguistics (Su & Hunston, 2019). The major impetus for this perspective arises from the fact that linguistic representations of evaluative meanings such as attitude, emotion, and opinion are one of the primary functions of language (Halliday, 1975). This facet of language is used to create and sustain relationships with writers/readers or speakers/hearers by confirming solidarity with their views or by leading or convincing them into a certain viewpoint (Ngo, 2013; Ngo & Unsworth, 2015). Furthermore, for non-native speakers of English such as EFL learners, the ability to state opinions and form judgments appropriately is not only deemed essential in coping well with academic activities but also requires them to use analytical and critical thinking skills successfully (Ngo & Unsworth, 2015; Unsworth & Mills, 2020), since the expression of emotions and opinions are essential in how human beings communicate with each other and how they perform their actions (Ortiz & Hernandez, 2014). Therefore, to prepare non-native speakers to be able to participate in verbal communication effectively in academic and professional environments, it seems to be important for them to be equipped with the linguistic understanding of evaluative functions of language resources so that they can utilize these evaluative resources in verbal interactions successfully.
With this in mind, the present study aims to examine and compare how native speakers of Australian English and non-native speakers of English (i.e., Iranian PhD holders in English Language Teaching) exploit evaluative language resources when they share their opinions and judgments on English movies and in the context of on-line informal interviews which elicit their opinions and reactions to movies they have watched. By doing so, we aim to identify how Iranian non-native speakers of English use their repertoire of evaluative devices to voice their feelings, appreciations, and judgements in English-language movie descriptions. The analysis is expected to highlight areas where non-native speakers’ use of spoken language could be further successfully developed and where classroom teaching would be viewed as valuable since the linguistic ability to verbalize one’s personal emotions and opinions precisely and effectively is viewed as one of the important issues in language teaching and research (Ngo, 2013). Accordingly, the following research questions were investigated in the study:
How do native speakers of Australian English and Iranian non-native speakers of English exploit evaluative meaning-making markers in their English movie descriptions? Do these evaluative resources differ in the two groups?
Literature Review
The primary distinction between Homo sapiens and other animals is clearly evident through their use of verbal communication (see Dessalles, 2016). For this reason, many believe that verbal communication should be a primary focus for both the scientific study of communication and for the scientific study of language (De Saussure & Rocci, 2016; Dessalles, 2016). In general, verbal communication is the process of exchanging information, notions, emotions, intentions, and thoughts through spoken words. It is a fundamental facet of human verbal interaction and can go beyond mere information transmission. For example, verbal communication in movie descriptions can involve more than just simple information exchange, encompassing the expression of thoughts, emotions, reflections, interpretations, analyses, and the sharing of experiences regarding different aspects of movies such as characters, plots, genres, themes, and settings. For this reason, it can allow individuals to articulate their excitement, surprise, joy, sadness or any other feelings evoked by the cinematic experience. To effectively express these personal feelings, opinions, and experiences, evaluative language can play a crucial role. For instance, instead of simply stating, this movie had a sad ending, one could exploit evaluative meaning-making resources to articulate it as this movie concluded with a poignant and heart-wrenching final that left me feeling profoundly moved or Heath Ledger’s Joker was a tour de force, featuring stellar performances by the talented cast… (Taken from the dataset of native speakers). Therefore, by exploiting evaluative meaning-making resources, descriptions can gain a greater sense of richness and depth, enhancing the overall impact. It may also provide additional layers of meaning beyond mere factual statements.
Evaluative language pertains to the linguistic expressions used by writers/speakers to convey their subjective perspectives, opinions and attitudes toward the content they present and the individuals they communicate with (J. R. Martin & White, 2005). It is of inherent interest, as language users frequently employ evaluative language to assess, appraise, and categorize objects and people in their daily lives (Fuoli et al., 2022; Taboada & Carretero, 2012). Despite its significance, there are limited studies that specifically focus on evaluative meaning-making resources in verbal communication during movie descriptions. In focusing our analysis on the evaluative resources used by speakers in the data, we rely on the categories of Attitude provided by the Appraisal framework (J. R. Martin & White 2005; J. R. Martin & Rose 2003) discussed in more detail below. Briefly, the Appraisal framework assists researchers in classifying language choices in terms of three main types of evaluative stances: Attitude, Engagement, and Graduation. Attitude classifies wording in terms of its evaluative charge (negative or positive) and three main kinds of semantic classes (Appreciation, Affect and Judgement), while Engagement values pertain to how speakers/writers project their addressees’ alignment with the values expressed. Graduation, in turn, focuses on the intensification or downgrading of these evaluative stances. In this study, the focus was on the type of Attitudes used by interviewees. Three main types of Attitudes are proposed under the Appraisal framework: Appreciation concerned with evaluation of events, objects, and material phenomena; Judgement which identifies evaluation of human behavior, and Affect which is concerned with references to observed emotional responses.
Most studies with movies as the topics, and using this approach predominantly revolve around evaluative resources in written movie reviews. For example, J. Li (2011) examined the characteristics of attitude in film reviews from the perspective of the Appraisal framework. He found that film reviews reflect different attitude characteristics due to different communicative purposes. Taboada et al. (2014) also examined evaluative language in online movie reviews written by non-expert critics, using a three-way corpus of English, German, and Spanish. The researchers selected 50 reviews for each language and found similarities in the distribution of Appraisal categories. Appreciation was found to be the most frequent type of Attitude, followed by Judgement and Affect. However, significant distributional differences were identified across the three languages, which could be attributed to factors such as lexico-grammatical choices, argumentative style, and sociocultural issues.
In another study by Y. Li (2021), a total of 205 movie review samples taken from two different websites (e.g., IMDb and Rotten Tomatoes) were analyzed. The researcher examined the use of evaluative resources in movie reviews as a way to gain insight into the attitudes of movie reviewers based on their comments. The findings of the study revealed a higher use of certain evaluative resources (e.g., Appreciation) in movie reviews, and its relationship with attitudes. The researcher also found that the emotional and sentiment expressed in audience reviews reflected their feelings toward the movies. Furthermore, the movie reviewers demonstrated a general tendency to provide positive and explicit description, showing their attitudes and evaluations of various aspects of the movie, including its quality, impact on the audience, and overall structure and performance. Based on these observations, the study concluded that language, particularly the use of evaluative language, is intrinsically linked to culture. This implies that an individual’s cultural background and perspective can influence their choice of evaluative resources and subsequent attitudes toward movies. Overall, the study sheds light on the intricate relationship between language, culture, and attitudes in the context of movie reviews.
More recently, Fuoli et al. (2022) investigated the use of metaphor and evaluation in 94 on-line movie reviews written by non-professional native critics. These reviews were created by film enthusiasts for their peers and were published on popular websites such as IMDb, Rotten Tomatoes or Metacritic. The researchers discovered that around half of their corpus’s metaphors performed evaluative functions, with creative metaphors (i.e., drawing together previously unrelated concepts) more likely to be evaluative, and metaphorical evaluation was consistently more negative than non-metaphorical evaluation. The researchers also reported a preference for inscribed evaluation (i.e., wordings which are easily recognized by native speakers as having either a negative or positive “charge” out of context) in both creative and conventional metaphors. Overall, the existing research on evaluative resources in verbal communication during movie descriptions has predominately focused on written, one-sided texts, and may have underestimated interactive, conversational, multi-party texts and socio-communicative functions of evaluative meanings (see White, 2008, 2016).
The present study was conducted to contribute further to work on the evaluative resources used by speakers on the topics of movies. Because previous studies have primarily concentrated on written contexts and have neglected the interactive aspects of evaluative markers in verbal communication, particularly during movie descriptions, our study hoped to make new insights into these areas available. In addition, we were interested to compare the use of evaluative resources in verbal descriptions between native speakers (NSs) and non-native speakers of English (NNSs) when describing movies interactively. This research area has not been as extensively investigated. Hence, the current study sought to empirically investigate how NSs and NNSs incorporate evaluative meaning-making resources when they engage in verbal descriptions of English movies. Understanding their use of such resources can provide insights into the dynamics of cross-cultural communication and shed light on potential differences and challenges encountered by NNSs in articulating their opinions during movie descriptions. Finally, attitudes, emotions, and judgements are intricately connected with language use and reflect one’s cultural background, personal experiences, and even worldview. When both NSs and NNSs utilize evaluative meaning-making markers in their communication, these language components can come into play. However, this research area still needs empirical verification.
Given the limited existing knowledge in this specific domain, the present study aims to examine and compare how NSs and NNSs utilize evaluative meaning-making markers when expressing their opinions, and judgments regarding English movies. Thus, this study, pedagogically speaking, has the potential to contribute to the linguistic foundation required for incorporating evaluative meaning-making markers into English as a Foreign Language (EFL) for Iranian students in both communicative-based classroom practices and language assessment. Therefore, an exploration into the use of evaluative language may cast new light on linguistic similarities and differences between NSs and NNSs.
Method
Research Design
The present study employed a combination of both qualitative and quantitative methods. Initially, semi-structured interviews were conducted to collect qualitative data and identify the evaluative resources. The qualitative analysis provided a deep understanding of the evaluative language used by both groups of participants. This was subsequently complemented by a quantitative analysis, which involved using the UAM Corpus Tool (O’Donnell, 2008) to quantify evaluative resources. In addition, it examined differences in the frequency of tokens across various categories expressed in verbal communication of movie descriptions resulting from the interview questions (see Appendix A).
Theoretical Framework
The present study examined and compared the use of evaluative meaning-making markers in verbal communication during movie descriptions by native speakers of Australian English and Iranian non-native speakers of English. For this study, evaluative markers were investigated through the linguistic lens of J. R. Martin and White’s (2005) Appraisal framework. Formulated within the theoretical framework of Systemic Functional Linguistics, Appraisal is regarded as a linguistic approach of evaluative language. The Appraisal framework provides a useful resource for analysts interested in discourse semantics dedicated to the analysis of interpersonal meanings. In general, the Appraisal framework maps evaluative resources along three axes of distinct sub-systems, that is, Attitude, Graduation, and Engagement, each of which is divided into a number of subdivisions (J. R. Martin & White, 2005). With respect to the (sub)categories of attitude, these turn on the semantics of communicating emotions, assessing people’s behavior, and evaluating things and occurrences. These subcategories of Attitude are referred to as Affect, Judgement, and Appreciation, respectively (J. R. Martin & White, 2005). For the purpose of the study reported here, we focus solely on the instances of Attitude, which is schematically shown in Figure 1. At the same time, it needs to be acknowledged that both inscribed and invoked attitudes were identified and these rely on inferences suggested by the co-text and cultural understandings (Don, 2016; Fuoli, 2018). In such cases, the researchers needed to discuss and agree with the categories identified.
Attitude system adopted from J. R. Martin and White (2005).
Affect refers to the speaker’s emotional feelings toward the subject being discussed and includes expressions of emotions such as happiness, anger, or sadness (e.g., I like it). Judgment involves the social, ethical, and moral evaluation of the speaker toward human behavior and actions (e.g., He is an efficient actor). Finally, Appreciation is concerned with the speaker’s assessment of the aesthetics of objects, events, situations, etc., and classes these targets of evaluation under areas such as quality, beauty, significance, admiration, or disapproval (This movie is fascinating) (J. R. Martin & White, 2005). These three subcategories of attitude encompass both positive and negative polarities. The Appraisal framework was chosen because it offers a systematic set of categories of all the possible evaluative resources in the language (Benamara et al., 2017; Unsworth & Mills, 2020) and can provide a rich description of how evaluation is expressed both explicitly and implicitly through language use (Benamara et al., 2017; Fuoli et al., 2022; Ngo & Unsworth, 2015; Unsworth & Mills, 2020).
Corpus
For the present study, we utilized two datasets consisting of on-line informal interviews centered around English-language movie descriptions. One dataset comprised interviews with 10 Iranian non-native speakers of English and the other featured interviews with 10 native speakers of Australian English. The members of these two groups were selected based on their availability and each having at least tertiary qualifications. The first researcher is located in Iran, while the third researcher is based in Australia. Although the original cohort of NSs was not restricted to Australian NSs only, for the comparison of the present study, the cohort was restricted to the two distinct cultural groups. It was also decided that the total number of participants be limited to 20 to enable the online interviews, the transcriptions, and the intense manual analysis to be conducted by the researchers within a suitable timeframe.
Participants
Ten Iranian PhD holders in ELT were invited to participate in the study, as shown in Table 1. The participants included eight males and two females, all of whom were born, raised, and lived in Iran at the time of the interview. Their ages ranged from 35 and 41 years old and Persian was their first language. The participants all earned doctorate degrees from Iranian state universities and held teaching positions in English-related majors (e.g., Applied Linguistics, Translation Studies, and Linguistics) at various universities. They have been immersed in the study of the English language, encompassing both learning and teaching, for a duration ranging from 15 to 22 years. The participants had not resided in any other English-speaking countries other than Iran.
Demographics of the Iranian non-native speaker participants.
The second group of participants was 10 native speakers of Australian English. They were born and raised in Australia and comprised four males and six females (see Table 2). They were residing in Australia at the time of the interview. Their ages ranged from 22 and 75 years old, and English was their first language. All participants had received their education from Australian universities. They held diverse academic degrees, with five having bachelor’s degrees, four having master’s degrees, and one having a doctorate degree.
Demographics of the Native Speaker Participants.
The researchers utilized a non-probability sampling technique known as the snowball method (Ary et al., 2018) to choose participants for both groups. This involved initial interviewees assisting researchers in recruiting their friends and acquaintances for subsequent interviews. Therefore, all the participants were either friends of the researchers or friends of the interviewees who chose to participate voluntarily. Prior to the commencement of the study, participants were sent an invitation email, outlining the research design, objectives, methodology, interview questions, interview environment, and movie titles. Upon confirming their participation, the researchers promptly contacted them to specify a mutually convenient interview time that aligns with their availability and preferences.
Movies
For the purpose of analysis, a list of movies was selected to elicit how both NSs and NNSs exploit evaluative language resources in their descriptions of English movies. The motivation behind this choice was that, firstly, movies are an important component of our daily lives (Schneider, 2012). Secondly, the present state of research on movies has received scant attention in entertainment research (Schneider, 2012; Vorderer, 2011). Thirdly, movies are regarded as an intellectual product with diverse evaluative dimensions (Carretero & Taboada, 2015). Finally, movies are a pervasive cultural phenomenon and an engaging medium that have the potential to evoke emotional and cognitive reactions in viewers (G. N. Martin, 2019).
To ensure a high degree of representativity, objectivity and comparability of the data, a set of criteria were set to select the movies. These criteria encompassed various cinematic components, such as awards, critical acclaim, professional evaluations, and word of mouth gathered from the Internet Movie Database (IMDb.com) (see Appendix B). IMDb.com is an on-line database that provides extensive information on movies, including reviews, production details, plot synopses, and user ratings. Based on these criteria, a pool of 20 movies was selected to be later introduced to the participants (see Appendix C). Both groups of participants were provided with the given list. However, for practical research considerations, they were requested to watch three different movies from the movie list provided.
Semi-structured Interview Questions
For the present study, a semi-structured on-line interview format was singled out and the interview questions were grouped into three open-ended types to facilitate exploration. The first type of questions consisted of basic, demographic questions that were designed to establish rapport and make the interviewees feel comfortable with the interview process (Dörnyei, 2007). The second type of questions was designed to collect information about the participant’s experiences, behaviors, opinions and values and sensory perceptions (such as what they have seen or heard) of the intended English movies. By employing such question types, the researchers were able to gain a more comprehensive understanding of the subject matter being investigated. The last set of questions served as closing question(s), allowing participants the opportunity to share any final thoughts or remarks (Ary et al., 2018; Dörnyei, 2007).
Initially, we formulated a set of 23 questions. However, to ensure accuracy and clarity, the interview questions underwent a thorough review process by one native speaker of American English with professional knowledge in film studies and one Iranian non-native speaker with expertise in applied linguistics. Based on the feedback received during this review process, four questions were subsequently excluded from the set. These questions included topics such as the factors influencing individuals’ choice to watch a specific movie and the distinctive qualities that make favorite movies memorable. Consequently, the researchers proceeded with a refined set of 19 interview questions. It is important to mention some questions (e.g., questions from 4 to 9; See Appendix A: Interview questions) were specifically tailored toward NNS in order to gather essential background information about them. The remaining 13 questions were directed to both groups (Appendix A). By using the same set of questions for each interviewee, the researchers were able to analyze and compare how different participants exploited the evaluative language resources in the same context.
Procedure
The Pilot Study
We performed a pilot study with three participants, two Iranian non-native speakers and one native speaker of Australian English who closely matched the characteristics of the intended participants in comparable circumstances. Firstly, we aimed to clarify and simplify the interview questions to make them clear, unbiased, jargon-free, and easily understandable for all participants (Cohen et al., 2018; Dörnyei, 2007), taking into account the individual’s linguistic and sociocultural factors that could have a bearing on their willingness to take part in an interview and affect their comprehension and response to questions (Cohen et al., 2018; Gablasova et al., 2019). As a result of this, we appended the word “why” to certain questions and included appropriate follow-up probes (e.g., Can you give an example?) to add more detail to the responses. Secondly, we received feedback from participants highlighting both positive and negative aspects including potential challenges. For example, one important challenge that arose was the issue of slow internet connectivity during on-line video interviews, which impacted the exchange of significant information between the researchers and interviewees, as well as impacting their emotions, mood, and ability to focus (Griffee, 2012). Moreover, two participants expressed their desire to discuss their favorite movies, prompting us to invite all participants to share their opinions on an additional movie that was not initially listed and was based on their preferences. Finally, the pilot study assisted in estimating the amount of data that could be collected and processed within the given timeframe of the study.
Skype Interview
The present study was conducted in 2021 during the COVID-19 pandemic, necessitating the use of mediated interviews due to geographical dispersion and logistical constraints. Skype video calling was employed to replicate an in-person interviewing experience, providing real-time interaction, convenience, and a safe environment. Synchronous interviews encouraged reticent participants to contribute more freely, reducing socially desirable responses (Tracy, 2020; Busher & James, 2012).
To examine how NSs and NNSs employed evaluative resources in their English movie descriptions, we developed an interview protocol, schematically illustrated in Figure 2. Before commencing the on-line interviews, participants were informed of the study objectives and reassured of anonymity and confidentiality. Participants were reminded that the study aimed to capture their thoughts and perspectives on the movies without evaluating their language proficiency. This inclusive approach created a comfortable environment, particularly for two Iranian participants who expressed concerns about being judged based on their language skills. Verbal and written consents were obtained to ensure ethical compliance and respect participants’ privacy rights (Cohen et al., 2018; Tracy, 2020).
On-line interview protocol.
Following the consent, participants took part in informal dyadic interviews with one of the researchers via Skype. These on-line interviews took place in the comfort of their own homes and incurred no financial cost for the participants. Each participant took part in three on-line informal interviews, where they described three English movies: two movies from the provided movie list and one movie based on their personal preference. The interviews were performed on separate occasions based on the interviewee’s preference to prevent them from becoming bored and fatigued by discussing the same topics repeatedly. This also assisted us in acquiring adequate depth and breadth (Dörnyei, 2007) and was important for the researchers to manage the participants’ involvement in the discussion to maintain focus on specific topics and reduce any repetition or overlap in conversation.
During the on-line informal interviews, participants were asked about the movies they had watched in order to elicit information on their use of evaluative language resources. In particular, the researchers were interested in how the participants shared their feelings, appreciation, judgements and the value in verbal descriptions. Throughout the interview process, the researchers also employed various interview techniques to collect further information, qualify responses, and clarify any misunderstandings that arose. These interview techniques included follow-up questions, elaborations, prompts, and probes, which are identified as effective ways to promote the quality and depth of qualitative data (Cohen et al., 2018; Tracy, 2020).
It is worth noting that the interviews were not conducted as a simple question-and-answer exchange between one of the researchers and the participants. Rather, the questions were utilized as a medium to encourage natural conversation and dynamic co-creation of information. Overall, each on-line interview had a variable duration, typically ranging from 15 to 37 min, and was audio-visually recorded for subsequent verbatim transcription. Overall, we amassed more than nine hours of recorded interviews.
Data Transcription
Upon collecting the interviews, a transcription process was initiated to convert spoken data into written text for analysis. Initially, each interview was assigned a file number, ranging from File #1(native speakers) to File #20 (non-native speakers). Then, the collected audio-visual data underwent manual (word-for-word) transcription following a conversation transcription guideline derived from Eggins and Slade (1997) to achieve consistency in transcribing interviews. Additionally, the basic transcription method was employed in this study, since the present study did not take note of tone, moves and acts, word pronunciation, or phonetic speech patterns in spoken conversations. The adopted transcription guideline is presented in Appendix D.
The transcription process entailed identifying clause boundaries by analyzing factors such as pauses (excluding hesitations), intonation (rising and falling pitch contour), and other contextual cues that often signify the initiation or completion of a clause (Ngo & Unsworth, 2015). Finally, the researchers thoroughly scrutinized the complete written transcripts multiple times while also watching the interviews. This approach allowed us to make necessary adaptations and gain a comprehensive understanding of the content within the datasets.
The interview transcripts underwent a “member check” process where they were sent back to the interviewees for accuracy validation (Cohen et al., 2018). However, within the native speakers’ dataset, several discrepancies such as transcription of contractions, words with same/ similar pronunciations, and the spelling of a particular word were detected and subsequently addressed. Another validation method employed was to maintain a spreadsheet as an “audit trail,” documenting and recording all research activities (Cohen et al., 2018). Overall, one of the researchers conducted the whole transcription process, which spanned a duration of more than six months. At a later time interval, the researchers re-watched 25% percent of the video interviews to monitor the quality of transcription and to note any differences. This step is crucial as transcription accuracy directly impacts the corpus’s reliability and usability (Gablasova et al., 2019).
After transcribing the data, the two corpora were converted into plain text files. These files were then inputted into UAM Corpus Tool (O’Donnell, 2008), which is widely used by researchers who work with the Appraisal framework (Fuoli, 2018). This corpus tool incorporates a pre-made Appraisal annotation scheme that includes features such as Affect, Judgement, and Appreciation, along with their subcategories (e.g., Affect: un/happiness, Appreciation: reaction, Judgement: tenacity). Starting from the “Document” layer, each file was then categorized as either a native or non-native file. This coding layer was essential for comparing the attitude repertoire of native and non-native participants. Overall, the transcripts of both datasets contained a total of 73, 218 words. Table 3 below shows detailed information regarding the datasets, sourced from the UAM Corpus Tool, the system used to annotate the datasets.
Datasets Characteristics.
Data Analysis Procedure
The process of annotating evaluative language is highly context-dependent and a subjective undertaking by its nature (Cavasso & Taboada, 2021; Fuoli et al., 2022; White, 2016). To address this inherent subjectivity and promote consistency in the annotation process, the researchers implemented a seven-step annotation procedure suggested by Fuoli (2018), which is schematically depicted in Figure 3. One important facet of this approach is its incorporation of an iterative procedure aimed at enhancing the clarity and reproducibility of the annotation guidelines (Fuoli et al., 2022).
An overview of the stepwise method for annotating Appraisal framework (Fuoli, 2018).
The researchers then formulated an annotation scheme manual to facilitate the analysis of evaluative meaning-making resources (step 2). As a crucial step, this annotation manual provided annotators with explicit instructions, criteria, and operational definitions, along with illustrative examples for various (sub) categories of evaluative meaning-making (step 3) to optimize reliability, replicability, and transparency (Fuoli, 2018). For example, for the purpose of this study, the evaluation unit of analysis, also known as unitizing (Fuoli, 2018), minimality (Cavasso & Taboada, 2021) or identifying markables or spans (Taboada et al., 2014), was defined according to Fuoli et al. (2022) as “a string of one or more words that conveys the writer’s positive or negative emotions, attitudes or judgments toward someone or something” (p. 16). Such a definition can embrace a wide range of expressions, without considering their length or the particular word class they belong to. In order for a segment of text to be classified as an instance of evaluation, it must include a recognizable target that is being evaluated (Fuoli et al., 2022). For this reason, transcribed texts were divided into spans of varying lengths, ranging from single words (unigrams) to constituents (e.g., noun phrases or verb phrases) and even entire sentences.
The manual also covered categories of positive/negative evaluations, authorial/non-authorial perspectives, as well as invoked/inscribed statements. For example, inscribed or direct evaluation conveys explicit positive or negative feelings and assessments while invoked or indirect evaluation implies the speaker’s assessment without explicit or direct expressions.
Example (1), extracted from the dataset of the Iranian non-native speakers of English, illustrates an instance of invoked evaluation, where there is no explicit evaluative wording employed to suggest whether the influence of festivals and prizes is perceived as “bad,” “good,” or “negative” or “positive.” The following statement merely indicates that these external factors have the potential power to impact the speaker’s decisions. Thus, even without evaluative words, the speaker’s mention of the festivals and prizes as determinants of their decisions implies an invoked/indirect evaluation. Overall, the process of invoked evaluation requires readers to draw pragmatic inferences that extend beyond the literal meaning of the text (Benamara et al., 2017; Don, 2016).
(1) …and also, the festivals, the prizes, they can also determine my choices sometimes. (File 18; General comment)
To accurately identify the evaluative language resources, we invited an experienced university instructor who possessed detailed familiarity with evaluative resources. To assess the consistency of the coding procedures, minimize bias during the annotation process, and identify potential areas for improvement, we performed two rounds of inter-coder agreement testing (steps 4 & 5). The first round involved randomly selecting two interviews collected during the pilot study. This selection comprised one interview with an Iranian non-native speaker and another interview with a native speaker of Australian English. For the second round, we randomly selected 10% of the data from the entire datasets (one interview from each group). Following each round, inter-coder reliability was conducted and a collaborative conversation occurred to address divergent notions regarding annotations. The results of the inter-coder agreement are presented in Table 4. We did not utilize complex measures such as Cohen’s Kappa and Fleiss’ Kappa as the majority of our labels are binary and a percentage agreement can be adequate for such cases (Cavasso & Taboada, 2021).
Results of the Inter-Coder Agreement Tests.
It is worth mentioning that the agreement comparison was also performed by determining the degree of agreement within attitude categories and subcategories. After reaching a point where we determined that further consistency improvements were not possible, we annotated the remaining datasets. The annotation process lasted around 2 months. The following screenshot shows how we used the UAM Corpus Tool to analyze the datasets (Figure 4).
Screenshot of the UAM corpus tool home screen developed by O’Donnell (2008).
Challenges in Identifying Evaluative Resources
The annotation process of evaluative meaning-making resources can pose several challenges. This is mainly due to the fact that evaluative language resources frequently have multiple plausible interpretations, and there might exist several valid category labels associated with them (Cavasso & Taboada, 2021; Fuoli, 2018).
Double-Coded Statements
One of the major challenges we encountered was double-coded statements within the dataset of the native speakers of Australian English, as shown in the following examples.
(2) Belfort is the anti-hero; a charismatic, greedy, sexist. (File 4; Wolf of Wall Street)
In example (2), we encountered a double-coded instance, where we had to decide on whether the interviewee evaluated Belfort’s “character” as a product/construct, or evaluated his specific behavior. Both interpretations can be plausible. For instance, the interviewee initially focused on evaluating Belfort’s actions, encompassing Judgement: social esteem: propriety: negative. However, the complexity increased when considering the use of the term “anti-hero,” a label for types of character found in novels or movies. Consequently, we confronted the challenge of classifying the label “anti-hero” as the target artefact as well. However, this seemed to be the feature of the movie description. Another challenge was related to interpreting the meaning of the words interesting and challenging, as found in the dataset of NNSs.
(3) Well, Anthony’s character I admire the most because he is interesting and challenging for a person his age to go through ups and downs of life like that as depicted in the movie… (File 20; The Father).
In example (3), the interviewee shows admiration for Anthony’s character and describes it as interesting and challenging. The challenge here lies in specifying whether “character” pertains to Anthony’s own personal qualities or the construction of a movie character as an artefact. While the use of the term “interesting” normally denotes Appreciation, it could be also classed as a form of Judgement, social esteem when applied to a human (he). On the other hand, the term challenging presents an even greater conundrum. While it might be interpreted as a positive attribute, referring to the character’s deviation from normality or their capacity to present something novel, it might also be perceived as affective inclination on the part of the (non-native) speaker who is struggling to express their positive attitude. Finally, the subordinate clause for a person to go through the ups and down like that as depicted in the movie… further adds another layer of complexity, indicating elements of Judgement: social esteem: tenacity. These examples are only two instances among several statements that shared the same issue of using lexical items normally associated with events, artefacts, products of human endeavors and applying them to human or personified entities, which we contend is a typical response regarding the personae depicted in movies, where viewers discuss characters’ actions as if they were in fact real people. These examples underscore the crucial need to clearly establish the rationale behind coding decisions in order to ensure and demonstrate consistency throughout the analysis.
In general, distinguishing the difference between Judgement and Appreciation can be difficult due to the blurred boundary between these sub-categories in some cases (Cavasso & Taboada, 2021; Fuoli, 2018; J. R. Martin &White, 2005). J. R. Martin and White (2005) suggest that this challenge emerges because both concepts can be viewed as stemming from the fundamental realm of Affect (see Figure 5). They further state that it is likely that we (as social animals) initially developed a language for discussing emotions, and then repurposed that for evaluative forms of evaluation. The interconnection between these subcategories is illustrated in Figure 5.
Interconnection between attitude domain adapted from J. R. Martin and White (2005, p. 45).
For the purpose of acquiring comparative statistics regarding evaluative resources such as Appreciation, Judgement, and Affect, we encountered the challenge of determining whether to treat them as distinct instances or demonstrate their subordinate role in relation to the expression of judgement. Therefore, we tended to treat them as distinct instances.
Cultural Challenge
Example (4), taken from the dataset of the NNSs, posed a cultural challenge.
(4) He was a mystic. He has this… that aura of spirituality around him and even though he was a mystic, he was also very powerful and good in battle… and… and… yeah… and I think… (File 13; Lord of the Rings)
In Persian culture, a mystic is often viewed as someone with deep spiritual knowledge and a direct connection to the divine. These individuals are respected for their wisdom and ability to guide others on the spiritual path. However, due to the complexity of mysticism and varying interpretations, its nature can differ. In the cultural context of Iran, being labeled as a mystic is generally seen as a positive attribute, indicating social esteem, capacity, and positivity. Such cases reminded us that it is crucial to note that interpretations may not be the same across both groups, and underscored the necessity of researchers having different cultural backgrounds to assist in interpretation.
Repetitions
In spoken language, repetitions can serve different purposes such as emphasizing a point (e.g., Taboada et al., 2014) or gaining time for thinking, particularly when language learners are concerned. Speakers also repeat expressions to gather their thoughts on either the content or the language being used. Nevertheless, there might be other reasons such as hesitations, graduation resources, pauses, and uncertainty that are not relevant to our study. We, therefore, considered them as redundancies, prevalent characteristics of casual language, and did not individually annotate repetitions. Below are examples (shown in bold italics) exploited by the NSs and NNSs.
(5) I think I
(6) I
Overall, while acknowledging that our interpretation of evaluative language resources is subjective and not absolute, we believed that our annotation scheme manual played an important role in enhancing the consistency and reliability of the annotation process, as well as the approach in which the researchers discussed potentially ambiguous expressions. By employing this manual, we also aimed to accurately recognize and classify evaluative meaning-making resources, thereby facilitating a thorough analysis. The extracts (7 & 8) below exemplify how we applied the Appraisal framework to analyze the datasets.
(7) I guess, a bit of disdain for the rich family in how they seem to sort of squander their wealth {Judgement, social esteem, propriety, negative, inscribed} or how they’re self-indulgent {Judgement, social esteem, propriety, negative, inscribed} in in how they treat their children slightly {Judgement, social esteem, propriety, negative, invoked} (File 9: Parasite)
(8) I just want to watch it more than 100 times {Appreciation, reaction, impact, positive}. I have… I think I have watched it for seven or eight times {Appreciation, reaction, impact, positive}. But I like to watch it more {Appreciation, reaction, impact, positive} (File 17: Godfather)
Results
Quantitative analysis of the data was performed through the UAM Corpus Tool. Based on its manual, proportions can be compared and interpreted using a scale of 0 to 3 “+” signs. These signs suggest the level of statistical significance in the difference between the mean of the features in one set and the mean of the other set. Put simply, as illustrated in Table 5, a single “+” signifies that out of any 10 results, one can be expected to be a false result (90% significance, or 10% chance of error). Additionally, this section presents the global counting of the proportions of various subcategories of Attitude. These subcategories include Attitude types (Affect, Judgement, and Appreciation), Polarity (Negative and Positive), and Explicitness (Inscribed and Invoked).
Interpretation of the Level of Statistical Significance.
To respond to the research question, which examined how NSs and NNSs exploit evaluative meaning-making resources in verbal discussions of movie descriptions and whether they differ in using evaluative resources, the results indicated that both groups utilized a total of 1,622 evaluative language resources in their verbal descriptions of movies. Nevertheless, there were significant differences in the use of these language resources between the two groups (see Table 6). NSs utilized a higher number of evaluative language resources, with a total of 1,031 instances observed in their movie descriptions while NNSs employed 591 evaluative language resources in their descriptions of movies. This difference was also found per 1,000 words/tokens, as shown in Table 7.
Number of Attitude Resources by Both Groups.
Number of Attitude Resources by Both Groups per 1,000 Words/Tokens.
In the dataset of NSs, the majority of Attitude spans (44.2%) fell under the Appreciation category, while in the dataset of NNSs, the dominant category within the Attitude spans was Affect, accounting for 43.8% (see Table 8).
Evaluative Resources Used by NSs and NNSs.
There were also significant differences in the use of Affect and Appreciation categories between the groups, whereas no significant difference was found in terms of Judgement. Interestingly, when analyzing 1,000 words/tokens, a significant difference emerged between the two groups in the employment of Judgement category (see Table 9).
Evaluative Resources Used by NSs and NNSs per 1,000 Words/Tokens.
Further analysis of the three subcategories of Attitude uncovered a significant difference in polarity between the two groups. It was found that NNSs used a higher frequency of positive descriptions compared to negative ones, while NSs tended to employ more negative descriptions than positive ones (see Table 10). Furthermore, statistical information regarding inscribed and invoked evaluations is also presented in Table 10. The results indicated that NSs predominately used inscribed evaluation (91.9%) and invoked evaluation (8.1%), while NNSs showed a higher preference for inscribed evaluation (95.1%) and a lower use of invoked evaluation (4.9%). Importantly, there were significant differences between the groups in terms of both inscribed and evoked evaluations.
Statistical Information About Inscribed and Invoked Evaluations.
Discussion
The present study explored and compared how NSs and NNSs exploited evaluative meaning-making resources when they discuss English movies in answer to interview questions. Through this exploration, we gained insights into the preferred evaluative language resources employed by both NSs and NNSs when describing movies. Accordingly, the most prevalent evaluative resources used in movie descriptions are discussed and compared to determine the extent of similarity or difference based on our results.
In response to the research question, the results indicated that NSs utilized a higher number of evaluative language resources in their movie descriptions compared to NNSs. One possible reason for this is their language proficiency and extensive exposure to and experience with the target language. This may enable them to exploit a wider range of evaluative language resources more naturally and effortlessly to communicate their thoughts on English movies. NSs might also be more confident and comfortable in expressing their judgements, which could explain their tendency to use comparatively more evaluative language. Conversely, the more limited repertoire of evaluative devices available to NNSs could be the reason for their tendency to rely less on evaluative language resources to describe English movies. The following extracts (9&10) (bold italics) exemplify the use of evaluative resources by NNSs.
(9) …for me, when there is a fugitive, I… I
(10) The character… definitely Paul, the main character was
The above extracts feature NNSs who were asked to share their opinions about certain characters. These interviewees exemplify a limited range of evaluative language resources, primarily relying on very basic core vocabulary such as great and hate. Throughout their interviews, these words were frequently reiterated and the presence of hesitation (…) indicates that the NNSs interviewees were struggling with their evaluative language repertoire. The excessive use of core and highly generalized expressions such as hate/great may hinder NNSs from conveying their attitudes in a sophisticated and effective manner during their verbal discussions (Ngo, 2013). These findings also imply that NNSs with PhD degrees could still benefit from language support to enhance their ability to articulate their feelings and opinions accurately and effectively. Therefore, these findings may be consistent with Unsworth et al. (2012) and Ngo and Unsworth (2015), who reported that doctoral students who achieved exceptional scores on the International English Language Testing System assessment, which was required for admission to postgraduate studies, at the same time demonstrated limited proficiency in employing Appraisal resources.
The results further showed that while NSs prioritized Appreciation followed by Affect and Judgment, NNSs generally opted for Affect, Judgment and Appreciation, respectively. In other words, differences were manifest in all three categories. However, we chose to compare the Appreciation and Affect categories due to their frequent occurrence in each dataset and to maintain brevity. NSs incorporated Appreciation as the most frequent type of Attitude, followed by Affect and Judgement categories, while NNSs relied on Affect as the dominant category, followed by Appreciation and Judgement. These differences could be explained by several cultural and linguistic differences.
Firstly, our findings suggest that NSs tended to consider movies as artistic creations and showed a greater inclination toward utilizing evaluative resources to value the aesthetic facets of the movies such as cinematography, acting, script, and plot. In other words, NSs appreciated the artistic endeavors and craftsmanship involved in creating these works, thereby highlighting the technical aspects and artistic achievements. For example, the following extract highlights the movie as the focal point, emphasizing its impacts and offering a description of why the interviewee selected it and is intrigued by it. This discussion also revolves around the script, acting, and takes the perspective of a viewer. Note the following extract from the dataset of NSs:
(11) Well, I thought it (movie) was a very clever way of representing the confused state of the person with dementia and it’s represented visually, which is very rare in a film to do this by representing the inner thoughts and behaviors of a person visually because he can’t recognize his own daughter, and takes it to be somebody else and as addressing her as somebody else, and not his daughter. But and when… when… when that happens, we don’t see his original daughter, we see somebody else, that we see the image that he must have in his head of who that person is. So that’s a very clever technique in the film to represent what he’s thinking visually, by putting in it’s very confusing to watch, because you’re not sure what’s going on. (File 7; The Father)
Therefore, these findings can align with previous studies (Fuoli et al., 2022; Li, 2021; Ngo, 2013; Taboada et al., 2014), echoing their conclusions. However, it is important to note that our research genre focused on spoken movie descriptions while previous studies centered around written movie reviews (except for Ngo, 2013). Secondly, the differences between NSs and NNSs might also arise from NSs perceiving movies and their characters as objects or constructions, rather than seeing them as real individuals with genuine lives and the depicted events as having real consequences. These differing perspectives ultimately influenced their responses, leading to Appreciation being the dominant category and Judgement being the least used. Therefore, while these findings cannot be extrapolated to all the individual participants of our study, it can be inferred that in the context of interview questions movies tend to evoke a greater sense of appreciation rather than emotional and judgmental reactions and the presence of Appreciation can be a distinctive characteristic of movie descriptions. On the other hand, NNSs relied on affective language to express their attitudes, possibly due to potential limitations in their linguistic repertoire such as lexical instantiations. This is evident from the frequent repetition of the same lexical instantiations (e.g., mysterious) for various evaluative purposes (e.g., he has a mysterious character vs. he is somehow mysterious) as depicted in Example 12.
(12) Yeah. About this main character, Andy, who I think is
In addition, NNS’s reliance on affective language led them to place greater emphasis on certain categories of attitudes such as Affect. Example (13) shows the abundant uses of affective resources (underlined) by NNSs with no other attitudinal lexis.
(13) Interviewer: Can you please tell me the reason why you chose that?
Interviewee: Actually.
And then I chose it by knowing that it’s an animated movie, but
animations. (File 13; Secret Life of Pets)
Another plausible reason for this is time constraints with on-line synchronous interviews. NNSs might find it more convenient and straightforward to engage in verbal discussions by using pre-constructed affective phrases such as I like/love/am interested in… to convey their general impression or emotions evoked by movies when they are not yet prepared to construct elaborate utterances. However, this mode of emotional expressions may be less refined or “processed” compared to the other modes such as writing (Fuoli et al., 2022). Hence, NNSs might encounter challenges of finding alternative ways to express their opinions.
Moreover, employing more affective language to express how movies made them feel (e.g., happy or sad) or how they were emotionally impacted (e.g., I was absorbed) may imply that NNSs prioritize expressing their emotional reactions over critically analyzing the filmic qualities (different features of movies) because NNSs often encounter difficulties in accurately expressing their emotions and appreciation due to the overlapping nature of Affect and Appreciation categories (Hood, 2019). Consequently, these difficulties can manifest in inadvertent slips in expressions, such as mistakenly substituting I am interested with It is interesting (Hood, 2019, p. 388; Unsworth & Mills, 2020).
More importantly, the cultural background of NNSs can play a pivotal role in shaping their language choices and their overall perceptions of cinematic experiences. For example, certain cultures like Persian place a greater value on emotional expressions, resulting in increased use of affective language in verbal communication (see Beeman, 2001; Gilan et al., 2022). Similarly, Naami and Farangi (2023) investigated attitudinal resources in Persian recounts (i.e., storytelling) and found Affect was the most frequently employed evaluative resource, followed by Appreciation and Judgement despite the variation in the genre.
Regarding the explicitness differences between NSs and NNSs, both groups tended to incorporate a higher number of inscribed evaluations in their movie descriptions. However, NSs relied more on invoked evaluation while NNSs demonstrated a greater tendency toward utilizing inscribed evaluation. This implies that NNSs tended to provide more explicit judgements or assessments in their movie descriptions. The following extracts (14 & 15) exemplify invoked and inscribed descriptions by NSs and NNSs, respectively. In extract (14), for example, the speaker refers to the character’s actions, which in turn lead to what the speaker implied was a positive outcome (underlined).
(14) …he was pushed to a point where then he became a problem and… and that he became the abuser of others and then that actually encouraged a whole lot of disadvantaged people to rise up (File 5: Joker).
(15) The film is a thrilling and captivating right from start to finish (File 17; Lord of the Rings).
Consequently, they chose to encode their evaluations explicitly, which represent the most straightforward form of expressing their evaluation (J. R. Martin & White, 2005). This approach allowed them to express their emotions more openly and directly without concerns about potential negative repercussions such as being judged negatively or facing embarrassment (Naami & Farangi, 2023). These findings may also suggest NNSs might deliberately adjust their evaluations in a clear and explicit manner as a strategy to show their comprehension of English movies. In contrast, NSs were more inclined to employ indirect or implied assessments that necessitate readers/listeners to infer the evaluative stance from the contextual cues or language used. The observed differences between the two groups might arise from the fact that language provides us with various choices to express an idea via different forms. However, NSs naturally have access to tools such as invocation and explicitness while NNSs (EFL learners) often rely on explicit devices taught to convey their intended meaning. Consequently, implicitness may be de-emphasized, resulting in a limited range of linguistic options available to NNSs.
As for the polarity, both groups utilized a higher frequency of positive descriptions in their verbal discussions of English movies, which could be attributed to the genre under study, as suggested by Potts (2011). Potts observed that certain genres such as movie reviews tend to exhibit a higher proportion of positive descriptions than negative ones. In his comprehensive analysis of 1.36 million movie reviews sourced from IMDb meta-data, approximately three-fourths (73%) of the reviews displayed positive evaluations. In the current investigation, we found that both NSs and NNSs expressed positive evaluations in 57.5% and 66.7% of their English movie descriptions, respectively. These findings suggest that positive evaluations are more prevalent in movie descriptions, potentially indicating a characteristic feature of the genre. Another possible explanation for the polarity differences could be the influence of prior experiences and personal preferences with movies, shaping participants’ descriptions. However, it is worth noting that our findings are consistent with previous studies (e.g., Li, 2021; Taboada & Carretero, 2012), which also observed a higher prevalence of positive descriptions. However, this contrasts with the results of Taboada et al.’s (2014) study, where an equal distribution of positive and negative descriptions was reported in written movie reviews.
On the other hand, NSs tended to utilize a greater number of negative descriptions compared to NNSs. This difference could be possibly attributed to the fact that negative evaluations often involve more complex figurative language such as metaphors, similes and other rhetorical devices (see Trnavac & Taboada, 2020) which may require sophisticated rhetorical abilities which are challenges for NNSs who may not be familiar with these expressions (Ngo, 2013). For example, instead of simply stating I don’t like James Bond movies, native speakers might opt for a more elaborate expression, as shown in Example 16.
(16) The James Bond type of film is the last I’d pay to see in the cinema. (File1; General comment).
Here figurative language in the form of simile is employed. The clause the last I’d pay to see in the cinema emphasizes the speaker’s complete lack of interest or willingness to pay money to view a James Bond-type movie and to mask the explicitly negative nature of their opinion (see Trnavac & Taboada, 2020), using what the appraisal framework terms “invoked” attitude which often depends on purely experiential, as opposed to evaluative lexis.
Overall, it is crucial to acknowledge that the perception of positivity and negativity can vary depending on the context, which involves both parties (the interpersonal dimension) and the topic of discussion (the ideational dimension). The dynamics of language, particularly those related to evaluative resources, highly rely on this contextual basis. Consequently, the positive or negative connotation of wordings can fluctuate due to multiple factors such as the overall context, related word groups, intended meaning, and the mode of language (e.g., spoken or written) used. Moreover, it is important to consider that the formulation of questions during research studies can further influence respondents to align their answers with the ideas presented in the inquiries. Therefore, it is essential for future studies to take these contextual factors into account in order to provide a more comprehensive understanding of how individuals perceive and interpret positivity and negativity.
Conclusion and Implications
The present study sought to examine and compare how NSs and NNSs exploit evaluative meaning-making resources when describing English movies. This study identified the preferred evaluative devices utilized by both groups during their English movie descriptions. The findings of the study suggest that NSs leaned toward exploiting the Appreciation category when describing their reactions to English movies while NNSs preferred the Affect category. The findings also suggest that NNSs may possess a limited repertoire of evaluative devices at their disposal. The implications of these findings can be valuable for university instructors, informing them about the interpersonal meaning-making resources that students need to develop in order to effectively expand their repertoire.
Additionally, these findings can provide insights for language teaching and research, highlighting the significance of incorporating evaluative language resources into language instruction. For example, educators can enrich their lesson planning by integrating instances of evaluative language extracted from movie descriptions into language teaching materials. Coupled with targeted exercises that require students to analyze and interpret these linguistic features, such as approach can enhance students’ comprehension skills. Educators are also encouraged to explore the creation of persuasive advertisements as a pedagogical tool. In this activity, students are tasked with developing advertisements for a product or service using the Appraisal framework to strategically incorporate evaluative language resources aimed at persuading the target audience. Through analyzing the impact of different linguistic choices on the effectiveness of the advertisement, students gain valuable insights into the power of language in influencing perceptions. This hands-on pedagogical approach not only allows students to hone their skills in articulating opinions effectively but also encourages them to consider the impact of cultural factors on language use. These practical illustrations can be integrated into language teaching strategies through the three-stage model of language development postulated by J. R. Martin and Rose (2003). This model delineates the progression of language development from dependent construction to joint construction and ultimately to independent construction, providing a structured framework for language learning and application.
The study also brings attention to the interconnectedness of diverse evaluative resources such as reference to emotions and judgements with complex linguistic and cultural domains. At the same time, it is important to consider that the specific context, genre, and topic of the discussion may impact the use of specific categories and distribution of attitudes. Certain topics may evoke certain evaluative responses rather than others. However, further research is necessary to gain a comprehensive understanding of the factors, contributing to the observed differences between NSs and NNSs in this context.
The present study has limitations that could be addressed in future research. These limitations include the limited dataset, small number of participants, and specific movie selection, all of which were due to the meticulous and time-consuming analysis process. Therefore, caution is needed when generalizing the findings to other contexts. Another important aspect to consider is the lack of consideration for gender differences among the participants, which could be accounted for in future studies, given the ability of researchers to conduct the same intensity of interview and analysis on a wider range of participants. Additionally, the study only focused on two groups (Australian and Persian speakers). While we acknowledge that the Australian group may not represent all native English speakers, the selection of speakers who were all tertiary educated suggests that their experience of English is no doubt global as well as academically influenced. It is also crucial to acknowledge the significance of incorporating a broader spectrum of cultural representations in shaping and directing future research endeavors. This recognition not only enhances the scope and thoroughness of scholarly inquiries but also cultivates a research landscape that is more inclusive and diverse. Furthermore, the study exclusively examined the Attitude components of the Appraisal framework. To gain further insights, future research could explore other Appraisal subsystems such as Graduation and Engagement. Moreover, future research can delve deeper into the influence of various movie genres or topics on the utilization of evaluative language. Finally, exploring how the context of the discussion affects the use of specific categories and attitudes when describing movies can provide a more comprehensive understanding of evaluative language in movie reviews.
As was noted in the Discussion section, NNSs seemed to strongly prefer Affect resources and set phrases in their responses. This suggests further investigation of such data using corpus methods, where such preferred lexical items or set phrases can be isolated in co-text and counted. Such studies also provide material for teachers who can use these kinds of output to illustrate claims regarding NNS use of English. Originally, our study was inspired by an interest in our students and enhancing their proficiency in English as a foreign language, in a cultural context that does not often allow them to engage in spoken discussion with native speakers of English. The data that we obtained from the study now provides us with material that can be used in future classroom instruction, especially considering the availability of corpus tools, and lately, automatic transcription software. Both NS transcripts and NNS transcripts could be used to investigate the prevalence of certain lexical items in context, allow learners to compare their responses to those of NSs, and in general, provide discussion points regarding the use of evaluative language in conveying viewpoints.
Footnotes
Appendix A
Appendix B
Appendix C
Appendix D
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
