Empowering critical digital literacy in EFL: Teachers’ evaluation of didactic materials involving the recognition of presupposed information

Abstract

This article proposes the use of a textual analysis technique involving the recognition of linguistic implicitness to promote Critical Digital Literacy (CDL) in the English as a foreign language (EFL) classroom. The technique, called Implicit Content Extraction (ICE), allows for the detection and analysis of (non-bona fide) presuppositions, thus enabling students to identify how much and what potentially manipulative information is being conveyed in a text. ICE has been tested on ready-to-use communicative materials. The materials were evaluated by EFL teachers (N = 10) who participated in a semi-structured interview. The analysis shows a positive reception of the materials, which meet the need for CDL, and a willingness on the part of teachers to implement the technique. Concerns are also raised about the explicit treatment of presuppositions, pointing to a low ability of students and teachers to analyse implicit strategies in a text.

I Introduction

In schools, education aims to empower students to participate in democratic life by making informed and deliberate choices. The first step towards this goal is to teach young students how to navigate the flood of information, misinformation (Rubin, 2019) and propaganda on social networks (Suarez-Alvarez, 2021). This is even more challenging when using a foreign language, as in the case of international advertising or other world issues. To meet this challenge, Critical Digital Literacy (hereafter CDL) skills are becoming increasingly important for education and intellectual development.

CDL skills are reflected in recent guidelines, such as those described in the Council of Europe’s Internet Literacy Handbook (Richardson et al., 2017), and frameworks, such as the Digital Competence Framework for European Citizens (Carretero et al., 2017). Critical media literacy will be assessed in the next PISA study in 2025, which will focus on learning in the digital world, including aspects of awareness of the use of social media and the gathering and filtering of online information (OECD, 2022).

To date, various techniques have been proposed to address this emerging educational need, but the main obstacle to their implementation in classrooms and teacher training is the lack of empirical evidence of their benefits in the K-12 context. This article aims to fill this gap in the secondary school environment through the creation of materials and their evaluation by teachers. Based on the technique of Implicit Content Extraction (ICE) (Sbisà, 2007, 2021), we developed two prototypes of didactic materials to improve students’ competence in understanding implicit messages. The materials were designed for a B2 level of the CEFR (Common European Framework of Reference for Languages) for receptive skills (i.e. the level required for secondary school graduation) and are supported by communicative tasks. The materials were evaluated by 10 teachers of English as a foreign language (EFL) in Austria and Italy. This article presents the results of this micro-evaluation and reports on the teachers’ ideas on how to support the development of CDL, focusing on the recognition of presuppositions, i.e. information that is taken for granted as being in the recipient’s background knowledge (Stalnaker, 1974b).

The article is structured as follows: Section II presents some a state-of-the-art review of CDL, implicit language and social media. The research questions and methodology, including the development and evaluation of the materials, are presented in Section III, and results are presented in Section IV. Section V discusses the data and is followed by concluding remarks in Section VI. In accordance with the principles of transparency and reproducibility, the teaching materials, interview guidelines, and taxonomies used in this study are publicly available online (for access, see https://doi.org/10.17605/OSF.IO/KBGPC).

II Background

1 Critical Digital Literacy in didactics

Educational actors have responded to the challenges of the information society by proposing models for the promotion of linguistic and metalinguistic competences. Despite their increasing popularity, their definitions remain ambiguous and elusive, interpreted differently in academic and policy contexts (Spante et al., 2018). The European Framework for Digital Competences DigComp 2.2 (Vuorikari et al., 2022) defines information literacy as the competence that allows ‘articulating information needs, locating and retrieving digital data, information and content’, and ‘assessing the relevance of the source and its content’. This is similar to another popular label, digital literacy, which is broadly defined to range from the functional use of technology (Gourlay et al., 2013) to ‘the confident and critical use of information society technologies (IST) for work, leisure and communication’ (European Commission, 2006, p. 16). In order to focus on this latter aspect of digital literacy, the original label of digital literacy has been developed into CDL. Roche (2017, p. 43) refers to CDL as a subcategory of digital literacy, emphasizing its function to ‘critically evaluate, use and create information through digital media in engagement with individuals and communities’. CDL can thus be located at the intersection of textual analysis of digital information and social engagement. We have chosen this label because we believe it most accurately describes the scope of our research. The potential of societies to resist misinformation is reflected in the Media Literacy Index (Lessenski, 2022). For the purposes of this article, we have chosen two countries – Austria and Italy – that have similar median scores on the Index, assuming that we would find similar and average dispositions to CDL in school contexts. In both countries, school curricula promote CDL, following the recommendations of the OECD (2021) and the European Commission (2022a). As a transdisciplinary competence, CDL meets the requirements of the curricula in Austria and Italy: The Austrian curriculum explicitly encourages the integration of media technologies in civic education, because a critical engagement with political communication (also in digital media) stimulates political participation (BGBl (Bundesgesetzblatt) II, 2 Jan 2023, No. 1 Annex A Curriculum AHS, p. 115). In Italy, a critical interdisciplinary approach to online media is promoted by the National Plan for Digital Education (Piano Nazionale per la scuola digitale, 2015, p. 79), a pillar of the school reform (Law 107/2015).

Empirical research on CDL has shown that in education systems that promote the identification of biased information, students are more likely to distinguish between facts and subjective opinions (Suarez-Alvarez, 2021). CDL education helps students to identify reliable sources of news and political information. As a result, students tend to become more politically engaged (Martens & Hobbs, 2015) and more aware of misinformation (Al Zou’bi, 2022). While these findings are encouraging, they are too broad to provide concrete solutions for practitioners. Indeed, the implementation of classroom activities that promote CDL has not yet been precisely designed. Fact-checking activities, one of the most common tasks used to promote CDL (European Commission, 2022b), lead to polarization of positions and avoid a comprehensive picture of an issue (Ceci & Williams, 2020). Other tools, such as crowdsourced judgments on the quality of news sources (Pennycook & Rand, 2019), are limited to information conveyed by institutional sources and overlook the spontaneous spread of information in social networks. Still others, such as source control or inoculation, i.e. exposing people to a refuted version of the misleading message beforehand, are effective against misleading arguments (Cook et al., 2017; Hämäläinen et al., 2023), but require a significant amount of prior knowledge, effort and time. While the use of lateral reading interventions (consulting more sources on a topic) is increasing, the evidence for their positive impact on students’ information literacy development remains limited. A study by Nygren et al. (2021) examined the outcomes of a two-hour intervention in four countries. The results showed a statistically significant improvement in adolescents’ ability to determine the credibility of fake images and videos. In addition, the intervention inspired students to incorporate digital tools into their assessment of the credibility of information. Notably, the intervention did not increase skepticism towards credible news sources.

While many of these attempts could be successfully applied to fake news, they overlook the fact that most individual choices are based on emotions, opinions, beliefs, and unconscious (cognitive) biases that are not explicitly communicated (Gigerenzer & Gaissmaier, 2011).

A more modest but promising goal might be to encourage learners to analyse how information is packaged before enabling them to check whether the information conveyed is true (Brocca, 2020; Brocca et al., 2020). In this way, the ethical aspects of the use of implicit and explicit communication are brought to the forefront of teaching practice (Rösch & Sühl-Strohmenger, 2016, p. 54).

2 Bridging the gap between educational needs and the knowledge of implicit language use

This study explores teachers’ responses to an innovative technique for improving CDL. By looking for implicit information in a text, CDL relies on a tool to measure and compare tendentiousness between texts, that is, the extent to which two or more texts can be compared based on the amount of implicit information they contain. The original model developed by Edoardo Lombardi Vallauri and Masia (2014) searched only for a subset of implicit strategies, among them presuppositions, implicatures, topicalizations and vague expressions. A major effort to systematize the relevant identification criteria for detecting such implicit strategies in a text was made in the three-year IMPAQTS project (Implicitness and Manipulation in Politics: Quantitatively Assessing the Tendentiousness of Speeches, coordinated by Roma Tre University, and funded by the Italian Ministry of Education, University and Research, project code: 2017STJCE9). The data collected through behavioral and neurophysiological experiments suggest that the implicit packaging of some information does indeed affect the way in which some content is represented in the mental model of the addressee (see Masia et al., 2023 for a discussion). For example, it has been found that the presuppositional wording of a question can distort information retrieval and induce false representations in the addressee’s mind (Loftus, 1975; Sanford, 2002). This means that the ability to detect and decode implicit meaning in order to grasp the speaker’s true communicative intention is of paramount importance when it comes to improving one’s understanding of a message.

The results obtained in the IMPAQTS project have also benefited from the inclusion of a procedure based on Sbisà’s research experience in lower secondary schools in Italy, where she tested the effects of ad hoc training on the recognition of implicit content (Sbisà, 2007). Her methodology consisted of first making students aware of what a presupposition or implicature is, and then having them identify such strategies in their textbooks and make the implicit meaning explicit (ICE). Sbisà’s experiment was carried out with Italian as L1, in humanities or science classes. Systematic data on the effects of the same approach in L2 language teaching are not yet available.

a Linguistic implicitness

Before outlining the rationale for our investigation, a few preliminary remarks on the phenomena of linguistic implicitness under consideration are in order. Broadly speaking, presupposition can be described as information that is taken for granted as common ground in conversation (Stalnaker, 1974a, 1974b). In most linguistic and philosophical approaches, presuppositions are typically ‘triggered’ by specific lexical categories or syntactic constructions, both known as presupposition triggers. These are generally identified in:

• • definite descriptions (e.g. The black cat; Sophie’s book; these dice, etc.);

• • change-of-state verbs (e.g. John has stopped smoking; Mary keeps on making noise, etc.);

• • factive and semi-factive predicates (e.g. It’s a pity you could not come to my party, etc.);

• • defining relative clauses (e.g. The shoes you bought are too expensive for me);

• • focus-sensitive adverbs (e.g. Also Jane went to college last year);

• • and many others (for a more exhaustive list, see Lombardi Vallauri, 2009; Sbisà, 2007).

As argued above, since implicit strategies can be more or less neutral depending on the context, it is important to distinguish between their different uses and, in particular, to recognize when they become potentially manipulative. In this case, they are also known as non-bona fide true (Lombardi Vallauri, 2016). Indeed, in the context of political or commercial propaganda, implicit strategies typically function as manipulative disguises of the truth (Masia, 2021).

b Implicit Content Extraction (ICE)

Over the last few decades, thanks mainly to the data made available by the newly emerging field of experimental pragmatics (through behavioral and neurophysiological approaches), understanding the implicit component of a text has gradually become a skill of paramount importance in building a native speaker’s language knowledge and competence. It should be noted that implicit language should be taught not only for its most salient linguistic manifestations, but also for the multiple functions it can perform in discourse (Lombardi Vallauri, 2019; Reboul, 2017, Garassino et al., 2019, 2022). Indeed, presumed meanings primarily serve an economic function, as they allow speakers to convey more than what is said on the surface of the text. But implicitness in language also fulfils other socio-interactional goals, such as reducing the risk of damaging and/or attacking the ‘face’ of our interlocutors (in the sense of P. Brown and Levinson, 1987) or allowing ‘plausible deniability’ (Pinker et al., 2008) of critical content. Given this, implicitness in language is usually taken for granted by speakers, despite its potential ambiguity.

From a didactic point of view, text analysis should involve becoming able to recognize different types of implicit meaning (e.g. presupposition, implicature, vagueness, etc.) and making their meaning explicit (i.e. unambiguously accessible, thus increasing the relevance of a message). One tool for achieving such a goal is ICE, as proposed by Sbisà (2007, 2021). This technique consists in recognizing and extracting the implicit content of a message. For example, if the reader encounters a sentence such as (1) (Sbisà, 2007, p. 75; the example, originally in Italian, is presented here in its English translation and slightly adapted).

• (1) The family crisis also stems from the narcissistic attitude of parents.

The teacher should help make students aware that this sentence is taken for granted, thereby treating as already shared knowledge the fact that there is a family crisis and that parents have a narcissistic attitude, i.e. implicit content conveyed by the definite descriptions the family crisis and the narcissistic attitude of parents. In other cases, as in (2), an example within the teaching materials used for this study, teachers should make students aware that even if the presupposed content is explicitly conveyed in the text (e.g. the addressee’s innocence), its linguistic encoding through a definite description makes it backgrounded in discourse and thus less likely to receive critical attention:

• (2) Lose your innocence in 3.9 seconds. The All-New BMW M4.

Sbisà acknowledges the importance of such a practice in enhancing students’ overall understanding of a text, let alone empowering their capability of distancing from potentially critical content (political opinions, description of advertised products) which cannot be ‘blindly’ taken as a true critical evaluation. It is important to highlight, however, that this didactic practice may be difficult to apply for teachers, as it requires a deep understanding of different implicit strategies in texts on the part of the teachers themselves. Moreover, the process of implicit recognition itself can be prone to bias, due to the possibility of drawing different inferences about the same textual material depending on one’s own encyclopaedic knowledge (especially in the case of implicatures) and the overlap of multiple implicit meanings within a single utterance or message.

Moreover, people may tend to adopt strategies akin to ‘rational ignorance’ (Downs, 1957) when engaging in everyday activities, such as reading a text. That is, they tend to achieve a ‘good enough’ understanding of the task at hand, optimizing cognitive and temporal costs. Thus, teachers should be aware of the need to motivate students throughout the activity (e.g. by choosing engaging and relevant material, etc.).

Teachers and students should also be aware that such an effort seems particularly justified in some contexts. As we will see in Section II.3, social media such as Twitter ¹ are inherently problematic as the brevity of their text almost naturally calls for a high density of implicit content (as shown in Lombardi Vallauri, 2019; Garassino et al., 2019), which in some cases are non-bona fide true (for a corpus-based analysis based on political tweets, see Garassino et al., 2022).

3 Implicit language in social media

Among the different media examined by the CDL, social media posts were identified as a widely used method of communication. However, their application in the classroom still poses challenges for teachers and remains a topic of debate (Van Den Beemt et al., 2020). Among the various social media platforms, this study focuses on Twitter due to its significant role in contemporary political and social communication. Twitter’s effectiveness lies in its interactive nature, which allows for the rapid dissemination of political opinions and the amplification of persuasive effects. Drawing on previous corpus-based research (Brocca et al., 2016; Garassino et al., 2019, 2022), we recognize that the 280-character limit of Twitter messages encourages politicians and influencers to use implicit strategies to streamline their messages. Tweets are therefore a particularly rich source of implicit messages, many of which allow questionable information to be spread without ever being challenged.

4 Teachers’ interaction with learning material

Teachers’ analysis of learning materials is a cross-cutting research issue, involving both pedagogical and subject-specific research. In Vygotsky’s (1978) socio-cultural theory, learning is seen as a social activity and tools, including educational materials, mediate the relationship between the object (learner) and the subject (teacher). The learning process involves a dynamic relationship to which both the teacher and the materials contribute. This perspective recognizes the capacity of materials to function as a medium that promotes not only the disciplinary progress of the student, but also the professional growth of the teacher. As teachers often see themselves as curators, transformers or producers of resources rather than mere implementers (Wang et al., 2021), materials can be used as a means of communicating with teachers rather than reaching students through teachers. The idea that materials can also improve teachers’ skills is supported by some practical experience: Tomlinson and Masuhara (2018, p. 53) recommend focusing on the ability of resources to contribute to teacher development, while assessing their quality. Davis et al. (2017) created curriculum materials in order to provide teachers with training while using them. This could be particularly important if the creators of the materials introduce a thematic or methodological innovation that teachers have not previously encountered.

At the beginning of a perspective in which materials function as a dynamic medium, Remillard (2005) posits the ‘enactment perspective’ in which teachers actively interpret and transform materials in collaboration with students during class. Empirical evidence suggests that teachers enact materials differently depending on their beliefs (Remillard & Bryans, 2004), their understanding of the underlying intentions (Drake & Sherin, 2009) and their ability to use the resources of the materials (M. Brown, 2009). The latter is closely linked to teachers’ access to subject-specific knowledge. The question of what and how much subject-specific knowledge teachers should know in order to teach their subject properly (Bartels, 2005, p. ix) has been widely discussed in applied linguistics (Bardovi-Harlig & Hartford, 1997; Temple Adger et al., 2002). However, empirical data does not support the frameworks that attempt to define KAL (Knowledge About Language) (Bigelow & Ranney, 2010). Although a high level of KAL may be beneficial, it is not essential for being an effective L2 teacher (Andrews & McNeill, 2005). Indeed, even L2 teachers with exceptional KAL proficiency may encounter difficulties when explaining errors or metalanguage. The difficulty in defining KAL needed in pedagogical contexts lies in the fact that linguists often face a conflict of interest (Bartels, 2004) when discussing this topic and the perspective of insiders, especially teachers, is often missing. On the contrary, a perspective that explores the interdependent relationship between disciplinary knowledge and pedagogical knowledge is more effective in describing classroom practices. The Pedagogical Content Knowledge (PCK) model proposed by Shulman (1986, 1987) fits this scenario as it describes the intersection of pedagogical and content knowledge. PCK involves understanding how to teach a particular subject comprehensively and engagingly to individual learners. PCK requires the ability to select and adapt teaching methods, assessments and resources to meet the needs of students in relation to the specific topic under consideration. Based on the PCK model, Kersting et al. (2016) explored the concept of ‘usable teaching knowledge’ as ‘the knowledge that teachers can access and use during teaching’ to make instructional decisions in their classroom. The study suggests the importance of a high level of pedagogical expertise to make content decisions on the spot. Remillard and Kim (2017) analysed teachers’ thought processes when planning lessons using curriculum resources. They introduced the ‘Knowledge of Curriculum-Embedded Mathematics’ (KCEM) framework to identify the subject-specific knowledge that teachers use while using instructional resources. The framework, derived from the study of the mathematics curriculum, has four overlapping dimensions:

• • Foundational mathematical ideas: Understanding the basic mathematical concepts and principles that underpin the curriculum materials.

• • Representations and connections among these ideas: The ability to connect and integrate different mathematical representations (like visual models and symbolic notations) used in the curriculum.

• • Problem complexity: Assessing and determining the relative difficulty of mathematical tasks and problems in the curriculum, and adapting them to suit different students’ needs.

• • Mathematical learning pathways: Understanding how mathematical concepts and skills are developed and sequenced over time within the curriculum.

These dimensions reveal common approaches to incorporating subject-specific concepts into designed materials and shed light on the structures and formats that teachers need to consider in order to plan pre-made materials effectively. Remillard and Kim focus on a mathematics curriculum. However, we believe that their framework also provides an analytical lens for understanding teachers’ evaluations of innovative materials in foreign language education. Adapting the KCEM model to CDL in FLE requires a creative interpretation of its dimensions.

• • Foundational Subject-Specific Ideas: In the FLE context, this translates into identifying essential linguistic structures and cultural content. The Foundational Subject-Specific Ideas of our study center around the concept of presupposition.

• • Representations and connections among ideas: This dimension involves the connections between the linguistic foundation and the media-pedagogical goals as, in our case study, to be able to read the implicit meaning in social media messages: in particular the role that presuppositions can play in manipulation. Teachers play a crucial role in guiding students to draw connections between the content in these media and the underlying linguistic structures, aligning with the content-based approaches.

• • Problem complexity: The focus lies on evaluating the linguistic and thematic complexity of texts in relation to student proficiency levels. When using authentic materials, it is crucial for teachers to select resources that are intellectually stimulating yet approachable, fostering critical thinking without overwhelming the learners.

• • Learning pathways: This dimension emphasizes the development of a structured sequence of activities.

In summary, adapting the KCEM model to explore the dynamics of teachers’ engagement with Critical Digital Literacy in Foreign Language Education significantly enriches our understanding of pedagogical practices. This approach not only sheds light on the depth and breadth of teachers’ professional knowledge in this domain but also offers a practical framework for effectively integrating digital literacy into language teaching.

III The research design

1 Research questions

In light of the foundational principles, this study embeds the activity of ICE within ready-to-use teaching materials. This integration prompts an exploration into teachers’ expertise, particularly in recognizing presupposed information in authentic contexts, thereby potentially empowering and consolidating Critical Digital Literacy (CDL). This article seeks to delineate teachers’ perceptions of ICE, guided by research questions that resonate with the dimensions of the KCEM model:

• • Research question 1: How can the materials assist students in recognizing strategies of implicit communication? This question aligns with the first, second, and third dimensions of the KCEM model, exploring the teachers’ foundational concepts underlying ICE, the connection between ICE and CDL, and the potential complexity faced by students.

• • Research question 2: In what ways would teachers implement ICE in classroom settings? This aligns with the fourth KCEM dimension, focusing on the sequential and developmental aspects of integrating ICE into the curriculum.

These research questions adopt an exploratory approach, crucial in a novel field like this research. For research question 1, hypothesizing is challenging due to the emerging nature of teaching implicit information recognition in foreign language contexts. Teachers’ perceptions might differ, influenced by their affinity for text analysis and diverse professional paths. This variance could range from a profound pragmatic background, possibly obtained through elective academic courses, to minimal exposure.

Regarding research question 2, the interview guidelines prod teachers to envisage practical classroom applications of the material, aiming for a detailed and practice-oriented assessment of ICE. This approach is anticipated to yield insights into the practical implementation of ICE, considering factors such as teachers’ background, time constraints, social learning dynamics, and students’ motivation. These factors are integral to the application of any new technique, especially when using authentic materials. The responses to this question are expected to offer valuable feedback for refining the teaching materials, aligning with the KCEM model’s emphasis on the progressive development of teaching materials content and instructional strategies.

2 Methodology

This paragraph describes the thematic background and sources of the text. The materials consist of two thematically different lessons. One is called Stop offending my friend, the other Reading between the lines: A political campaign. The first lesson deals with haters’ attacks on the climate activist Greta Thunberg as well as misinformation disseminated by climate change deniers. The second lesson deals with the US President Joe Biden and his campaign for the presidential elections, and more generally with political slogans. The first topic was chosen because Greta Thunberg is a well-known social media figure who is close to young students and who is promoting the current global debate on climate change. As such, it is relevant to the curriculum in terms of social justice, human rights and global citizenship and is expected to engage students. The second theme was chosen because it provides an opportunity for students to learn about the culture, values and institutions of an English-speaking country. Both themes are relevant to students’ interests, providing a window into global politics, language practice, and (inter)cultural understanding. The texts were collected from social networks and online news websites. Some longer texts, such as the inauguration speech by Joe Biden, were shortened. Twitter was chosen since research on implicit language in Twitter has a solid background (Garassino et al., 2019, 2022). However, this social network is typically not used by students (Dixon, 2022) and therefore it is less likely to interfere with students’ private sphere like other social networks. The tweets have been collected in April 2022 with the web-application Vicinitas.io searching for tweets in English with the hashtag #GretaThunberg or addressed to @gretaThunberg or to @FriendsofSciences (a climate change deniers’ group). The same research has been conducted for the second lesson searching for @Joebiden or @POTUS within the tweets produced in November 2020. Among the downloaded tweets (N > 1,000), a sub-corpus was created consisting of tweets containing presuppositional triggers. This paragraph describes the pragmatic background and extraction of presuppositions. In the first set of materials, the speech-acts are directed to attack someone (Greta Thunberg), and in the second they are directed to persuade about a certain politician’s position. Both types of speech acts are characterized by the presence of implicit discourse strategies, mostly presuppositions. Therefore, we selected texts in which the use of the implicit content was salient and easier to follow because it was linked to the lexicon. We chose the following presupposition triggers (in brackets we report the examples used in our materials):

• • change of state verbs (e.g. ‘I’m ready to build back better this country’);

• • iterative temporal adverbs (e.g. ‘Make America great again’);

• • definite descriptions (e.g. ‘Greta must work on her anger management problem’).

In the corpus, the selection was carried out using automatic vocabulary search systems: we searched verbs such as renew, stand up, innovate, break, melt, etc. adverbs such as again, always, etc., definite articles such as the, and demonstrative and personal pronouns, such as this, that, her, their, etc. The collected messages were manually selected for the use of non-bona fide true presuppositions, namely presuppositions whose content is questionable and non-necessarily true. The selected sub-corpus of non-bona fide true presuppositions provided the examples to be used in the didactic materials. For the additional materials on advertisements, a manual search was carried out using the Google search engine.

This paragraph describes the didactic approach and material architecture. The materials are structured into three sections:

• • The introductory part of the lesson presents the topic, motivates students and recalls their previous knowledge. After this short introduction, each material presents texts for the analysis (Figure 1): first, a global comprehension task and then a detailed analysis. In a second step, students are asked to inductively identify language forms (i.e. the presupposition triggers) which are relevant for the subsequent phase of the analysis.

• • The second section involves the implementation of a task, the result of which must be a linguistic product close to the students’ everyday experiences. In the first set of materials, the students must imagine helping a friend who has been insulted for his engagement as a student representative (Figure 2). In the second set, the students must prepare an election campaign as a students’ representative and imagine spreading it on social networks. The classmates who are not involved in the production of the slogans must recognize any implicit messages and must respond appropriately to the online posts.

• • The third section is dedicated to focusing on form. In this case, the teacher can move students to reflect explicitly on the forms used in the task phase. A short assessment, called ‘exit ticket’, is also included: students may quickly complete and deliver the ticket before leaving the class. Additional material focusing on advertising language is provided, raising consciousness about the pervasiveness of implicit communication and reflecting about its functions beyond social networks.

The linguistic focus is addressed through ICE. First, the original text is introduced; students should read and understand explicit meaning. Second, students should be aware of the presence of implicit information. Third, students should extract implicit information by paraphrasing it explicitly. Variations in the answer are possible. Teachers could consult the teacher’s version with the suggested solutions in grey. See Figure 3.

Figure 1.

The presented Twitter messages.

Figure 2.

Task activity.

Figure 3.

Example of an Implicit Content Extraction (ICE) activity.

Once implicit meaning has been discovered and paraphrased, students are encouraged to reflect as follows:

1. They can check the truth of the information. Example from the materials: ‘This information is false. Please check here: . . .’

2. They can be made aware, through focusing activities, that the content has been implicitly conveyed via a non-bona fide true assumption; e.g. ‘Focus on those parts of the text where the information is presented. How is the information communicated?’

3. They can propose hypotheses regarding the use of non-bona fide true presuppositions; e.g. ‘Why do haters hide some information and don’t say it overtly?’

4. They are encouraged to focus on form, gaining metapragmatic skills; e.g. ‘Can you associate indirect information with certain linguistic expressions in the text?’

The materials are based on communicative tasks and flanked by activities of a more formal nature. In the first and third sections, the additional materials, text analysis and other activities focus on form (Ellis, 2003, p. 258; D. Willis & Willis, 2007) with exercises in a presentation–practice–production architecture (Ellis, 2003, p. 5). It should be noted that the exercises are not isolated but are propaedeutic to the implementation of the task cycle in (2) above. In (3) above, the exercises do not tend to repeat language patterns but rather encourage reflection on language use. Focus on meaning and focus on form are thus integrated (Nunan, 2004, p. 34). The second section corresponds to the task cycle phase of Task Based Language Teaching (TBLT). The activities fulfill the characteristics of the TBLT approach (Ellis, 2003, pp. 9–10; Ellis, 2018, p. 157; J. Willis, 1996, p. 23) such as learner-centeredness, real-world reference, focus on pragmatic and communicative skills, outcome orientation, preparatory stages in peer work. The decision to adopt a Task Oriented Language Teaching (TOLT) approach is based on the need to provide teachers with more structure on which to base form-focused activities and help them adapt the materials to classroom routines (Bygate, 2020, p. 84). Constructing the materials within a TOLT framework allows for such advantages: on the one hand, in addition to focusing on a communicative output, the materials allow for encouraging students’ metalinguistic reflection through ICE. On the other hand, the guided practice sequences provide a useful aid for teachers. Indeed, the strong TBLT approach tends to make teachers uncertain about how and when they should explicitly focus on language (Ellis, 2018, p. 19; Mackey, 2004; Müller-Hartmann & Schocker, 2018). This may be particularly true in our case, where teachers may lack metapragmatic knowledge to identify presupposed information.

3 Data analysis material evaluation

a Micro-evaluation

Once the materials were ready, we started the evaluation. Moving away from evaluation based on closed criteria (e.g. checklists) evaluation through semi-structured interviews is preferable as it is a more flexible and holistic tool (Makundan & Ahour, 2010; McDonough et al., 2013), better suited to the exploratory nature of the study. We conducted a teacher-based micro-evaluation, aiming to obtain information about teachers’ perceptions regarding the recognition of presupposed information and its role in improving CDL. The evaluation was a pre-use evaluation (Tomlinson, 2013, p. 30), carried out by teachers before they used the materials with their students. To reduce the subjectivity biases in the evaluation, the analysis included ten teachers who were interviewed independently. To support the semi-structured interviews, interview guidelines were developed based on the literature on materials development for foreign language learning (Tomlinson, 2012).

The interview guidelines consist of two parts (see Appendix in supplemental material) that cover each research question: The questions in part one relate to the didactics of implicit language in social media and how the material may raise students’ awareness in detecting presupposed information (research question 1). The questions in part two relate to the implementation of the materials (research question 2). This last question forces the teachers to make explicit the procedure that they would use to implement the materials (Ellis, 2011, p. 231) and encourages them to examine the assumptions behind the material design.

b Population sample

Five teachers of English in Austria and five teachers of English in Italy working in upper secondary schools were recruited for the evaluation using a snowball system, asking a respondent to recommend other colleagues available for an interview. The teachers were heterogeneous in terms of age (Austrian teachers: Mean age: 33.2 years, Max = 59, Min = 23; Italian teachers: Mean = 47.6, Max = 60, Min = 25), gender (Austrian group: four female, Italian group: three female) and origin: Austrian teachers lived in the region of Tyrol, whereas Italian teachers were scattered throughout the country (Apulia, Liguria and Veneto). Regarding the type of school in the Austrian group, four high school teachers and one teacher from a technical institute were chosen. In the Italian group, four high school teachers and one teacher from a vocational institute were chosen.

c Data collection

Teachers were asked to read the materials and the suggested lesson plan before participating in the interview. All teachers were interviewed by the first author in their mother tongue (German or Italian), following the same guidelines. The interviews were conducted via video call between March and May 2022 and were audio recorded. The recordings lasted between 24 and 45 minutes, with a total of 353 minutes of conversation. The recordings were broadly (not phonetically) transcribed. The participants were informed via email about their rights according to the European General Data Protection Regulation (EU GDPR 2016/679) and had to explicitly consent to the scientific use and the publication of their anonymized data by ticking a box prior to proceeding to the interview. Personal data were pseudonymized manually in the transcription, and the cleaned transcriptions were stored according to the EU GDPR 2016/679.

d Data analysis

Data analysis was conducted according to Mayring’s qualitative content analysis (QCA) (Mayring, 2022). The main advantages of this method include content sensitivity, application in highly adaptable research designs, which can be particularly appropriate when dealing with innovative material. The analysis was software-supported by QCAmap (www.qcamap.org) (Fenzl & Mayring, 2017). In addition to being open source and web based, QCAmap results in a step-by-step QCA that allows multiple coders to work on the same source. As a first step, we developed a coding system: In contrast to current grounded theory (Ralph et al., 2015), which provides for the generation of theory solely from the data, QCA allows for the structuring of the categories obtained from the data in a stronger exchange with the theoretical background and the research questions (Mayring, 2022, p. 93). Thus, the categories created inductively according to the data were, in a second step, deductively restructured in relation to the research topics (critical digital literacy, didactics of the implicit and implementation of the ICE). The final coding system consisted of 23 codes (see Appendices in supplemental material), divided into the three research questions. The coding system was applied to the data and 266 relevant text passages were coded. Each interview was coded by two authors and the results underwent a peer-check process. In the few cases where the coding diverged, a consensus was reached among all three authors. Finally, data was exported in MS Excel to facilitate the presentation.

We checked the data for differences in the statements that could be attributed to age differences: both young and older teachers considered it important to improve the CDL. Both age groups also intended to use the materials equally. Variations between the Austrian and the Italian group are not considered in this article. It should be noted that the teacher working in a vocational school differed from the others in that he found the materials too difficult for his pupils.

IV Results

1 Didactics of implicit meaning in the materials

The statements about the teachers’ assessment of the didactics of implicit meaning in the materials are divided into four subcategories. The most frequently mentioned subcategory (36 mentions) relates to the way the materials deal with the detection of presuppositions. Seven teachers think that the ICE is presented as an engaging activity that stimulates students’ curiosity ² :

It encourages students to do research, it encourages them to be actively involved and gives them a clear task to respond to. (AUS2)

Six teachers also saw the analysis of advertisements as a good example of getting students to think about implicitness in a familiar text genre. Four teachers appreciated the strategy of ICE of extracting implicit meaning:

I think it is a very good exercise and one they certainly need to make explicit the type of implied content and the linguistic means by which this is achieved. (ITA5)

Regarding the analysis of implicit meaning, teachers identified potential challenges for the students in these materials. Four main problems were identified:

• • the subjectivity of some responses and the difficulty of testing the materials;

• • the general difficulty shown by students in understanding implicit meaning (partly due to lack of background in L1 education);

• • the need for more background information, especially to verify the truth of some implicit information;

• • focusing on forms (i.e. recognizing presupposition triggers) can confuse the students.

Finally, some teachers (n = 4) expressed their own difficulties in recognizing implicit meaning and pointed out the extra time they would need to prepare lessons based on the suggested materials.

2 Implementation of the ICE

Teachers gave less information about how they would carry out the presupposition recognition activities. Most of the statements concern the metapragmatic reflection phase: Teachers’ opinions are polarized. Three teachers affirmed that they would not explicitly focus on form, thus avoiding the activity of identifying presupposition triggers:

This reflection on the presupposition trigger . . . is useful, but it must remain a hint that can perhaps be taken up again. On the other hand, if we stop [the flow of the lesson] to do a grammatical analysis, there is a risk of diversifying too much, i.e. of taking the focus away from what we are doing. (ITA2)

Conversely, six teachers would encourage metapragmatic reflection and would prefer to focus on presupposition triggers. Seven teachers found it useful to focus on form in the exercises based on advertisements because the persuasive purpose of the advertisements is more obvious than in other text genres:

I would definitely use the advertisement and also especially the task where you sort of rewrite the ad again and make it a little more direct . . . there is again this reading between the lines, but here it is a little more obvious. (AUS5)

The second most frequently mentioned subcategory concerns the extent to which the teacher would encourage autonomous recognition of implicit meanings. Eight teachers indicated that the activity should be scaffolded by the teacher and/or were positive about the scaffolding included in the materials. Four teachers would encourage self-assessment, for example with the ‘exit ticket’ activity included as an appendix in both materials.

Teachers disagreed about whether the presupposition recognition activity could be assigned as homework. Two teachers affirmed that the activity requires independent reflection and should therefore be done at home. Two other teachers disagreed:

I would do it in class because I think that the students want to say a lot about it and maybe have opinions and then want to discuss it and if you do it alone at home t it might get lost a bit. (AUS3)

When reflecting on students’ reactions to the detection of presuppositions, some teachers (n = 6) noted that students would need (meta)linguistic skills:

I don’t think that they can say explicitly that it is a verb . . . I don’t think that they would also say that there is a definite article. (AUS3)

And for world knowledge:

Of course, you need background knowledge for [this content; authors]. Who was [the US president; authors] before Joe Biden? Why does he talk about building [back better this Country; authors]? (AUS5)

Some teachers (n = 4) stated that the activity of making presupposed content explicit could generate different paraphrases, so they would encourage group and plenary discussions to stimulate debate and homogenize the results. Two teachers stated that they would intervene by skimming students’ responses after the think/pair/share phase or during the discussion phase.

Finally, the teachers reflected on their own actions related to the implementation of the activity. Three teachers stated that they would use the ICE activity recursively, not only in the suggested texts but also in other lessons where implicit language might play a role. Four teachers stated that they would stimulate critical reflection by encouraging students to reflect on the messages they were able to paraphrase.

V Discussion

In this section, we will address the research questions by aligning the findings with the dimensions of the KCEM.

Research question 1: Foundational subject-specific ideas: Teachers have heterogeneous views on foundational subject-specific ideas: in particular they have different levels of knowledge about presuppositions and their role in manipulation. This confirms previous observations linking heterogeneity to personal attitudes (Remillard & Bryans, 2004) and differences in subject knowledge (Drake & Sherin, 2009). Some of the teachers expressed their own difficulties in recognizing linguistic implicitness (Section IV.1). Other teachers openly stated that they needed time to prepare and test the materials themselves. As mentioned by one teacher who reported difficulties in implementing ICE:

Teachers themselves are poorly trained in this regard, and so it is obviously very difficult to train students. (ITA5)

Connections between ideas: Teachers recognize that linguistic implicitness may be linked to persuasion. The materials make it possible to combine text analysis with CDL and are therefore a tool for raising students’ awareness of how questionable information can be expressed in texts.

Problem complexity: In addition to the encouraging findings noted above, the teachers’ statements highlight some concerns about the materials:

• • Concern 1: The materials presuppose a rich cultural background of the students, both in terms of world knowledge, linguistic and pragmatic competence, and confidence in dealing with controversial issues.

• • Concern 2: Students’ progress can be difficult to assess and evaluate. Teachers feel that drawing implicit meaning allows for multiple interpretations according to personal inclinations and knowledge. Therefore, it should not be evaluated.

To address concern 1, some teachers suggested anticipating the recognition of implicitness in advertising as a less challenging topic and postponing activities in less familiar areas (see Section IV.3). Some teachers also suggested omitting the metalinguistic reflection, and deepening the analysis only at the students’ request. The materials allow such a choice: the focus on meaning is predominant and the communicative goals of the lesson can be achieved without explicit reflection on forms, even if we have introduced an explicit treatment of presupposition triggers in the materials. The reason for this choice was to facilitate focus on form for teachers who may be new to TBLT and unfamiliar with teaching presuppositions. A more long-term solution to concern 1 would be to increase students’ competence in text analysis by multiplying the activities of implicit meaning detection also in L1 contexts and starting from lower education levels.

Regarding the difficulty of assessing students’ progress, concern 2, some teachers would skip the assessment altogether or leave the suggested activity (‘exit-ticket’) as a self-assessment.

Concerns 1 and 2 also reveal a critical dimension of teachers’ foundational subject-specific conceptions. Some teachers explain that students need background information to know if implicit information is correct. ICE makes it possible to paraphrase an implicit message into its explicit counterpart, which can be more easily challenged. Once the implicit message has been paraphrased, encyclopaedic knowledge helps to recognize whether a presupposition is bona fide true or not. The teachers failed to recognize the greater potential of ICE, which is the possibility of being fully aware of the messages conveyed in a non-explicit way. By recognizing the existence of non-bona fide true information, students recognize that the text is intended to convey a message that might bypass critical discussion. After using ICE, students can more easily question such information and form their own viewpoints. Concerns about assessment (concern 2) stem from the same misunderstanding: although ICE allows for multiple answers, these answers are usually a limited set of non-arbitrary answers, contrary to what some teachers have said. The evaluation of the answers is possible and can follow objective criteria, namely the correct recognition of implicit information. Teachers confused the activity of ICE with the activities of questioning the paraphrased message and checking its truthfulness. Again, unlike other CDL activities such as lateral reading, the main goal of ICE is not to check the truth of a statement, but to analyse how the message is conveyed.

Research question 2: Learning path: Teachers have different opinions about how the material would help students recognize the strategies of implicit communication (research question 1). Therefore, they would implement ICE differently in the classroom (research question 2). The deductive learning paradigm adopted in the materials is seen by teachers as useful in stimulating students to detect implicit meanings and their possible implications. Some teachers would include ICE in their long-term activity pool and use it recursively in different types of texts. Three issues are central in this context:

• • Issue 1: The polarization between some teachers who prefer to focus explicitly on metapragmatic background and other teachers who prefer to focus on meaning.

• • Issue 2: The positive evaluation of material that focus on advertising to train ICE.

• • Issue 3: Reflections on students’ autonomy and teachers’ scaffolding activities.

Issue 1 reflects the aforementioned challenge of integrating a focus on form within the TBLT framework, as discussed by Mackey (2004) and Müller-Hartmann and Schocker (2018). In addition, this point raises questions about the methods and feasibility of teaching pragmatics in a high school setting. The approach taken in the materials is inductive but explicit, as suggested by Taguchi (2010, p. 352), and allows for different implementations according to the students’ needs. The teachers’ statements show a general avoidance of explicit discussion of the pragmatic background underlying the materials. Some teachers are in favor of explicit reflection on the presupposition triggers only when the materials focus on advertisements (issue 2). Indeed, the discussion about the manipulative extent of advertising is not considered controversial, and therefore appears to provide a better basis for reflecting on language. Regarding issue 3, a minority of teachers would like to involve students in autonomous working sessions to allow space for individual reflection and (self-)assessment, while the majority would prefer to scaffold the process of ICE with teacher-based or material-based interventions. Most teachers would alternate short periods of individual or pair work with teacher-centered discussions in which students’ results are collected and reviewed by the teachers. This is made possible by the TOLT approach that underlies the materials. The alternation between independent work, pair work and teacher-centered phases shows that teachers are aware of the need for individual experimentation followed by teacher guidance. It also shows that the materials presented can be adapted to specific classroom needs.

VI Conclusions and limitations

This article is an empirical attempt to apply pragmatic knowledge to foreign language teaching. Much effort has been made to reduce the complexity of implicit language, as analysed in the literature, in the education of students, bridging the gap between the research community and the real world of practitioners. The evaluation of the materials obtained through expert interviews highlighted the relevance of the objectives and the need to fill a gap to promote CDL in FLE.

The approach adopted, based on task-oriented language teaching, allowed for the alternation of exercises focusing on form and tasks focusing on meaning. It is in the explicit treatment of pragmatics that teachers recognize the greatest difficulties and limitations of the materials. Teachers’ concerns about students’ ability to apply the ICE strategy corroborate Sbisà’s (2007) response-based data (pp. 171–190). Students need solid syntactic, lexical and textual skills to successfully understand implicit language. The activation of these competences needs to be promoted at all levels of mother tongue and foreign language teaching. Teachers may need additional training to use ICE effectively. To achieve this goal, the materials themselves could be further developed into educational resources (Davis et al., 2017) to provide teachers with training opportunities as they work with them.

Another observation can be made about the role of implicit communication in FLE. The CEFR Companion Volume (North & Piccardo, 2020, p. 110) collocates the ability to infer implicit opinions and explain subtle differences in the presentation of facts and arguments at the C1 level. We believe that this ability could be expected to the B1 or B2 level when applied to topics close to students’ interests and knowledge, such as advertising or topics related to social media. As stated by Sbisà (2007), and as also noted by some teachers in the interviews, the habit of reflecting on implicit meaning should not only be linked to L2 teaching, but should also be systematically addressed in L1 literacy and in all types of schools. The importance of critical reflection on implicit language is more crucial than ever. It is essential for improving CDL and for understanding information, news, and democratic participation.

The development of teaching materials is a complex process supported by ongoing evaluation, theoretical background, and empirical evidence. Due to the innovative nature of ICE, teacher-based evaluation was preferred over student-based or response-based evaluation at this stage. The data collected will be used to improve the material, and a second evaluation based on other data sources (such as response-based or learning/teaching diary-based evaluations) will follow.

Supplemental Material

sj-docx-1-ltr-10.1177_13621688241235019 – Supplemental material for Empowering critical digital literacy in EFL: Teachers’ evaluation of didactic materials involving the recognition of presupposed information

Supplemental material, sj-docx-1-ltr-10.1177_13621688241235019 for Empowering critical digital literacy in EFL: Teachers’ evaluation of didactic materials involving the recognition of presupposed information by Nicola Brocca, Viviana Masia and Davide Garassino in Language Teaching Research