Sage Journals: Discover world-class research

Abstract

The current article tries to address the alignment issue between English textbook of teaching and English language testing method of Grade 12 in the Kurdistan Region of Iraq (KRI). It aims at describing Grade 12 English ministerial test papers to find out what skills are targeted in the exam, and which language proficiency levels are measured in the exam according to the CEFR levels. The data was collected through Grade 12 English ministerial test documents (i.e., 2 test papers; each with 50 test items) of the 2023 to 2024 academic year of both first and second attempts. The article used quantitative research design and its data was analysed through items’ analysis by the researchers, and CEFR levels’ analysis by the AI-CEFR Analyser through assessing linguistic features and overall difficulties of the test items to estimate their CEFR levels, then after frequencies and percentages were implemented to interpret the results. The study findings revealed that, the English ministerial exam of Grade 12 cannot be considered as valid test, and the test fails to capture real language proficiency levels of the Grade 12 students. Also, the test cannot be considered as communicative English test, because it lacks the communicative requirements suggested by the CEFR framework. Finally, the study contributes to the ELT context in the Kurdistan Region of Iraq and provides pedagogical implications for the Ministry of Education-KRI to start reforming its English language testing method and move towards implementing communicative language assessment for English language at the public schools of KRI.

Plain Language Summary

Discovering Language Testing Problems Through an AI – Based Analyser

This article aims at describing Grade 12 English ministerial exam in the Kurdistan Region of Iraq (KRI), to find out what skills are targeted, and which language proficiency levels are measured according to the CEFR framework levels in the exam. The data was collected through official English language test documents (i.e., English ministerial test papers) of the 2023-2024 academic year of both 1st and 2nd attempts. The article used quantitative research design to interpret the results. The significant findings of the study revealed that, the English ministerial exam of Grade 12 cannot be considered as valid test, and it fails to capture real language proficiency levels of the Grade 12 students.

Keywords

Grade 12 English ministerial exam language proficiency levels CEFR framework communicative assessment

Introduction

Language testing and language teaching are closely related to each other, this means the way students are tested will affect teachers’ actual teaching in the classrooms and students’ learning of a language (Davies & Elder, 2007). The effects of language testing on language teaching and learning are either positive or negative. They are positive if a language testing formats, skills, contents and tasks go in parallel way with the official curriculum or textbook of teaching, but the effects will be negative if a language testing formats, skills, contents and tasks differ from the official textbook of teaching, and students study to pass the tests only, rather than to learn a language as real tool for communication (Schmitt, 2002).

Generally, the emerge of Communicative Competence theory by Hymes in the early 1970s affected language teaching and language testing hugely in a positive way (McNamara, 2000), and shifted language testing approach from integrative/Pragmatic Testing Approach to the Communicative Testing Approach (Weir, 1990), and it is the dominant approach since the early 1980s up to the present day (Mahmud & Bostanci, 2022). Additionally, in 2001, Council of Europe introduced and published CEFR framework; a set of guidelines to be used by European countries, as a standard framework for curriculum developments, teaching, learning and assessment of languages (Council of Europe, 2001). CEFR views language as communication, and necessary actions should be taken to develop learners’ different competences in language uses, including language teaching and testing (Council of Europe, 2001). Now, CEFR framework is recognised globally by many countries, and it is used as a reliable guide for users’ language proficiency (Abidin & Hashim, 2021).

With regards to the Kurdistan Region of Iraq, English language curriculum (or textbooks = Course), experienced radical changes more than a decade ago, and the Ministry of Education-KRI in partnership with Macmillan Education-UK introduced and implemented a new course of textbooks for teaching English in the public schools of the Kurdistan Region. The Course is called Sunrise and it is based on Communicative Approach to language teaching to develop students’ communicative skills, through practicing reading, writing, listening, and speaking skills in addition to grammar, vocabulary, and pronunciation subskills (Mahmud, 2013).

Having said that, the English language testing method in the Kurdistan Region is not reformed similar to the new English curriculum, and the traditional way of testing English language that was in place, remained as it was and students are assessed according to the previous traditional ways. Also, in the 2014 to 2015 academic year, Ministry of Education-KRI started implementing multiple choice method to be used in the public schools for testing and assessing students in all the school’s subjects including English language (Ministry of Education, 2015). Now, the Ministry of Education-KRI implements multiple choice method by 100% as the only technique in all the subjects, for assessing students’ levels in Grade 12 ministerial exams (Ministry of Education, 2022).

This radical shift, that is, the use of multiple- choice testing method alone by 100%, in the examination system affected English language education negatively, especially English language teaching and testing in Grade 12 of the secondary stage. Consequently, English language teachers now prepare students to get high marks in English language ministerial exam only, without focusing on their actual language learning and improving their language skills as required (Sabah, 2015).

Accordingly, the rational of this study is driven from the fact that, developing students’ communicative skills in English language is the primary goal of the Ministry of Education in the Kurdistan Region, and it is to be reached via teaching English language as a main subject in the KRI schools. But, this goal of the Ministry of Education is not supported by the current English language testing method in general, and Grade 12 testing method in particular, because there is no alignment between the actual English language coursebook of teaching (i.e., English textbook) and the English language testing method of Grade 12, especially, the English language testing method in the ministerial exams of the said Grade. Accordingly, the current article tries to explore this gap, also it shows the failure of Grade 12 English ministerial exam when it is mapped to the CEFR framework.

The reason for using CEFR framework levels as the base of our comparison, is because there are several reputable English language tests (such as: IELTS, Pearson PTE… etc.), which are aligned with the CEFR mapping and they are used as valid and reliable tests to measure individuals’ language proficiency levels. Additionally, in the Kurdistan Region’s ELT context, to the best knowledge of the researchers, no systematic study has been made so far to map Grade 12 English ministerial exam with the CEFR framework levels, therefore this is the task of this study. Finally, applying an AI based analysis, as a timely tool, with its results will help all stakeholders in the KRI to know where they are when comparing their English ministerial exam with the CEFR framework levels.

Aims of the Study

The current article aims at describing official English language test papers of Grade 12 in terms of targeted skills, and measured levels according to the CEFR framework levels.

Research Questions

The study is an attempt to get answers to the below research questions:

What are the targeted skills which the English ministerial exam measures?

What are the proficiency levels which the English ministerial exam measures in terms of the CEFR levels?

Review of Literature

An Overview About CEFR Framework

Common European Framework of Reference for Languages (CEFR) document, refers to a document which was prepared and written by the Language Policy Unit of the Council of Europe as a set of guidelines to describe learners’ achievements in foreign languages across Europe (Mousavi, 2009). CEFR is the result of more than 40 years of work on language education by the European Council, and in November 2001, it was recommended to be used as a setup system of validation of language ability (Mousavi, 2009).

The CEFR framework provides detailed explanations and descriptions of learner’s level by skills (Cambridge University Press, 2013), with its primary aim of providing a method for teaching and assessing of languages (Mousavi, 2009). The CEFR framework helps schools, syllabus designers, teachers, teacher trainers, exam writers, and learners in the process of language learning and assessment (Cambridge University Press, 2013). Currently, the CEFR framework has been accepted by more than 40 countries as a reliable framework to establish their language learning objectives and aims (Jeon, 2022), and it is seen as an accepted scale globally for identifying an individual’s language proficiency level (Mousavi, 2009).

The CEFR framework has 6 levels in 3 wider bands of A, B, and C (Cambridge University Press, 2013). These levels start bottom-up from A1 to C2: A1 + A2 are labelled as Basic Users, B1 + B2 are labelled as Independent Users, C1 + C2 are labelled as Proficient Users, and each of the said levels is described in details about what a language learner can do with a language (Council of Europe, 2001).

From CEFR point of view, a successful language learner should possess and use different competences, activities, and strategies in order to interact in a meaningful language communication. These competences are classified as: General Competences, Communicative Language Competences, Communicative Language Activities, and Communicative Language Strategies. The said competences, activities, and strategies of a language learner should be developed via language uses in real life contexts (Council of Europe, 2001). Therefore, as stated by CEFR framework, learner’s language proficiency and outcomes are understood as these below levels and descriptions:

A1 level: a person who can understand everyday simple language and familiar phrases. He/she can introduce himself/herself about personal details. Also, he/she can simply interact with others and talk slowly, which clearly shows that he/she needs help to carry out meaningful communication.

A2 level: someone who can understand sentences and expressions of his/her immediate area of relevance. He/she can communicate in simple language about familiar tasks that need direct exchange of information. Also, he/she can provide descriptions of his immediate needs, environment and background in simple words and terms.

B1 level: someone who is able to deal with most situations, and can comprehend main points of clear language involvement. He/she can provide descriptions of experiences, events, dreams, ambitions, and hopes. Also, he/she can produce simple connected text and can provide reasons and clarifications briefly about views and plans.

B2 level: a person who has the capability of understanding difficult texts, and can interact with native speakers with ease with a suitable degree of fluency. Also, he/she can give related advantages and disadvantages and can produce detailed and clear text on various subjects and areas.

C1 level: someone who uses language easily for academic, professional, and social purposes, also express effortlessly while using a language. He/she can understand long and demanding texts with the capability of recognising implicit meanings. Also, he/ she can produce well-structured texts, with the use of connectors and cohesive tools.

C2 level: a person who has the ability to understand everything heard and read easily. He/she can express himself/herself clearly and fluently in complex circumstances. Also, he/she can summarise both spoken and written language and can restructure arguments as required.

Communicative Language Teaching and Testing

As stated by Jeon (2022), the primary aim of foreign language learning is to help individuals to communicate effectively without difficulty beside their native languages. Accordingly, the CEFR framework views language as communication tool which has several goals and functions and puts extensive attention on communication abilities of language users (Council of Europe, 2001). Therefore, learners’ communicative competences to be promoted through listening, speaking, reading, and writing tasks in an action oriented approach through various strategies and activities (Jeon, 2022).

Here, it is important to mention that, the idea of seen language as communication and to be used as a tool for communication emerged as a result of communicative competence theory by Dell Hymes in the early 1970s (Zulu & Manchishi, 2018). The shift towards communicative approach to language teaching and communicative language testing was a response to the previous language teaching and language testing approaches, which they failed to address learners’ communicative needs in real life situations (Bakhsh, 2016).

Accordingly, Phan (2008) argues that, the following characteristics should be available in any language test in order to consider it as a communicative type of language testing:

Meaningful communication; this means the test should address language learners’ personal needs and help them activate their language use in meaningful way.

Authentic situations; this means language learners’ use of language cannot be separated from real life situations. Therefore, the tests should assess their language skills in an integrated way as it happens in real life contexts in the ordinary daily life.

Unpredictable input and creative output; as in real life, the interaction between a speaker and a listener is not expected, no one knows what the speaker is going to say and how the listener prepares to reply. This feature is essential to be used in any communicative language testing method.

Communicative Language Ability

One of the famous frameworks, which is based on Hymes’ communicative competence theory as a model for communicative language testing, is Bachman’s 1990 framework (Bakhsh, 2016). The said framework is known as Communicative Language Ability Model that contains both knowledge (or competence) and the ability to use that competence properly in the contextualised communicative language use (Bachman, 1990).

Bachman’s (1990) model has three components of: language competence, strategic competence, and psychophysio-logical competence (Congmin, 2023). Language competence refers to the knowledge of language that can be used in an act of communication. This competence is further divided into organisational competence and pragmatic competence (Congmin, 2023). The strategic competence refers to the individual’s mental ability to use his/her language competence in the contextualised act of language use (Bachman, 1990). This competence is also divided into: assessment component, planning component, and execution component (Congmin, 2023). Psychophysio-logical competence refers to the neurological and psychological factors that affect actual practices of language use (Bachman, 1990). The CEFR framework’s model for language, language use and language learning is based on Bachman’s 1990 model of communicative language ability (Congmin, 2023).

In relation to language testing and assessment, the CEFR framework puts focus on language proficiency of the language users, and the test should contain various communicative activities and strategies through different types of discourse (Council of Europe, 2001). Communicative language activities mean those activities that make participants exchange their roles between producers and receivers of a language several turns as much as possible, and the communicative language strategies refer to the capability of the language user in using his/her resources via different skills and procedures to fulfil the demands of meaningful communication successfully (Council of Europe, 2001).

Grade 12 Ministerial Exams

Education structure in the Kurdistan Region of Iraq consists of two main stages; Basic Education stage and Secondary Education stage. The Basic Education stage contains 9 years of studying, and the Secondary Education stage contains 3 years of studying. To graduate from Basic Education stage, students must pass (National Exams) at the end of Grade 9, so as to be able to go further in their studies. Also, to finish Secondary Education stage, students must pass (Ministerial Exams) at the end of Grade 12 in order to graduate and get access into higher education institutes and colleges (Jamil & Bapir, 2017). The researchers of this article are not going to talk about (National Exams) of Grade 9 of the Basic Education stage, because it is not within the scope of this study. Therefore, in the coming sections the researchers try to provide necessary information about Grade 12 (Ministerial Exams) of the Secondary stage.

First of all, it is important to mention that, the ministerial exams are also known as Baccalaureate Exams in the Kurdistan Region, and they are carried out by the Ministry of Education – KRI as a reliable tool for assessing students’ levels at the end of Grade 12 of the secondary education. The results of the ministerial exams show students learning and their readiness to study further in the higher education departments; institutes and colleges. This is according to their results’ averages in the said exams (Jamil & Bapir, 2017).

When we talk about ministerial exams in the Kurdistan Region, there are two types; General Ministerial Exams and Vocational Ministerial Exams. The first type is taken by the Scientific and Literary (=humanities) branches of the academic secondary schools, while the second type is done by the students of the Industry, Commerce, and Agricultural branches of the vocational secondary schools (Mahmud, 2013). Each of the said types has its own curricula and exams, but our focus here is on the General Ministerial Exams of the academic secondary schools, because it is within the scope of the current study.

From educational point of view, the General Ministerial Exams are considered as very high stake type of assessment in the Kurdistan Region, and it is the only reliable tool which is used by the Ministry of Education-KRI every academic year with secondary students to finish Grade 12 and graduate from Secondary Education stage. Accordingly, in order to pass ministerial exams, Grade 12 students must pass (7 subjects) in two attempts; the 7 subjects for the Scientific Branch are: (Biology, Kurdish Language and Literature, Mathematics, Religion Education and Arabic, Chemistry, English Language, Physics), and the 7 subjects for the Literary Branch are: (History, Kurdish Language and Literature, Mathematics, Religion Education and Arabic, Economy, Geography). This is in addition to Turkmanni Language, Christianity, and Syriac language for other students who are not Kurds and who are not Muslims in both of the mentioned branches.

With regards to the contents of Grade 12 ministerial exams, all the topics and contents of Grade 12 textbooks of all of the said subjects (of each of the branches) are included for the exams, and students are responsible to know those contents and answer the questions accordingly in each of the subjects. This means, the same subjects’ materials are included for all the students across the Kurdistan Region, and the same questions are given to the students in each of the subjects. The time period of the exams, which is given to the students to answer the exams’ questions, is different from subject to subject, but it is usually between 3 and 3:30 hr (Mahmud, 2013).

Moreover, regarding the structure of Grade 12 exams, it consists of multiple-choice Only method, in which students have 50 test items with 4 options for each of the test items. This is equal for all the 7 subjects of Grade 12 exams, and students need to choose the correct option from the 4 given options. Alongside, the test paper of each subject, there is an answer sheet (known as SOMA) for the students to indicate their answers on it (Ministry of Education, 2015). The maximum pass in each subject is 100 marks, and the minimum pass is 50 marks, and students get their marks and averages solely from the tests of each subject without considering their school efforts in Grade 12, which they have before taking the ministerial exams (Jamil & Bapir, 2017).

Grade 12 English Language Exam

English language exam (or EFL testing) of Grade 12 in the ministerial exams is an achievement test which is used to assess students’ levels in Sunrise 12 contents and units, which means assessing students’ levels of what has been learnt and taught before (Davies et al., 2002). This achievement test of Grade 12 (for all the subjects, including English language subject) serves as an exit test for Grade 12 students in order to complete their school education, and have access into the higher education colleges and institutes.

Similar to other school subjects, the Grade 12 English language test consists of 50 test items assessing different skills of English language, that follows multiple- choice method in which each item of the 50 items of the test has 4 options (i.e., A, B, C, D) with having (2 marks) for each correct answer, and in total the 50 test items will be out of (100 marks). Students get their marks according to their correct answers, and the minimum mark for passing is 50 marks (Ministry of Education, 2024).

The content of the test is driven from Sunrise 12 contents, topics, and skills such as: reading skills, grammar skills, vocabulary and pronunciation. Also, the language of the test and its instructions are in English, and students need to understand the test in English and choose the correct option for each of the 50 items of the test. This means, the mother tongues of the students are not included in the test and in its instructions. Therefore, it is very necessary for the students to have some level of English in order to enter Grade 12 English test and have suitable level to pass it as well (Ministry of Education, 2024).

Furthermore, the duration of the test is 3 hr and the students need to understand, answer and transfer their answers to the SOMA Paper during the test time. SOMA paper is a special answer sheet, which is exclusively designed by the Ministry of Education-KRI to help Grade 12 students transfer their answers from the question items’ sheet, and later those answer sheets will be scored by the Ministry of Education mechanically to determine students’ marks in the test (Ministry of Education, 2015).

Grade 12 English Language Curriculum

Sunrise 12 is the official textbook for teaching English language in Grade 12 at the KRI public schools (Macmillan Publishers, 2011). Sunrise 12 is part of Sunrise Course textbooks, which is the official course of textbooks for teaching English language from kindergarten level up to Grade 12 level of secondary schools. Each level of Sunrise consists of Student Book, Activity Book, Teacher Book, Audio CDs with flashcards and posters for the its primary levels (Macmillan Publishers, 2011).

Based on the Communicative Approach to language teaching, Sunrise 12 is designed to cover the main four skills of English language; that is, listening, speaking, reading and writing. Also, it enriches students’ vocabulary and grammar skills at their final year in the secondary education stage (Macmillan Publishers, 2011).

Final Level of Sunrise Course in Terms of CEFR

To understand at what linguistic level of performance, will English materials of Sunrise Course conclude at the end of Grade 12? Also, what is the desirable and achievable level of English language performance for Kurdistan students to get when they finish their secondary education? Macmillan Education (n.d., p. 11) documented and informed the Ministry of Education-KRI as below:

From the comparison of the contents, skills and functional and linguistic syllable of the Sunrise English Course, it is clear that the level of language performance anticipated in the materials correlates most closely with the CEFR B1 and Cambridge ESOL PET.

This means, when Kurdistan students study Sunrise Course in full, and successfully complete their ministerial exam in English language at the end of Grade 12 of their secondary education, they should graduate with B1 level according to the CEFR scale in terms of their linguistic performance and their level of English language. Consequently, such kind of English level seems suitable for the students to cope with their studies at the college level, after finishing their secondary stage (Macmillan Education, n.d.).

Previous Studies

Despite the importance of Grade 12 English ministerial exam and its high stake position in the Kurdistan Region’s ELT context, very few studies were conducted to explore or research it in depth. The researchers tried their best, and found the below studies, in which they are briefly reviewed to provide wider view about the topic and the claims that we make in this study.

First of all, a study was carried out by Mahmud and Bostanci (2022) aimed at exploring EFL teachers’ views in the Kurdistan Region of Iraq (KRI) about the current English language testing method for assessing students’ levels of English at Grade 12, and introducing more communicative English ministerial exam at the end of Grade 12 of the secondary stage education. The study’s main finding indicated that the English language test in the ministerial exams in the KRI does not meet students’ communicative needs, and it is crucial to have communicative language testing format for the English ministerial exam with Grade 12 students.

Additionally, another study was conducted by Mustafa et al. (2022) for determining to which extent Grade 12 students are satisfied with the multiple-choice format of the English ministerial exam and to investigate whether textbook activities are reflected in the English tests. The study revealed that, despite the Grade 12 English textbook being based on the communicative approach, the English ministerial exam lacks integration of the four main skills and sub-skills, which affects students’ ability to learn and use English.

Moreover, Omar (2020) conducted a study to identify the negative washback of the English ministerial exam on teaching and learning English in the KRI schools, as well as the reasons behind this type of washback. The study found that EFL teachers at KRI schools reduced the use of English teaching materials from Sunrise 12 textbook, particularly in listening, speaking, and writing. This is because the Ministry of Education-KRI uses multiple-choice format in the Grade 12 English ministerial exam, which assesses only reading, grammar, vocabulary, and phonetic transcriptions, while listening, speaking, and writing are excluded and not tested.

Finally, Sabah (2015) conducted another study to identify the reasons for secondary school students’ failure to learn English and to master English language skills. The study found that the English ministerial exam does not include the core of English language skills to help students enhance their English language proficiency, and it recommends that English ministerial exam should be based on the communicative language testing approach similar to the Sunrise English textbooks.

Methodology

Research Design

In the current study, the researchers followed quantitative research design, this is to answer the research questions effectively. Accordingly, the researchers first provided test items’ analysis of the Grade 12 English ministerial test papers, to determine which language skills and CEFR levels are targeted in the English test, then after frequencies and percentages were used to interpret the data, to get numerical results and give us statistics for the items’ analysis of the tests.

Research Material and Data Collection Tool

The data for the current study was collected from Grade 12 English test papers (N = 2) of the ministerial exams which were carried out in (2023–2024) academic year in the Kurdistan Region of Iraq. This means, the researchers collected data from parallel test forms (i.e., 2 test documents) of the English exam (i.e., Sunrise 12 tests) of both first and second attempts of the (2023–2024) academic year (see Figures A1 and A2). Accordingly, in both of the test documents, 100 test items (i.e., 50 test items for each attempt) were used as a source for our data collection to be examined and analysed for this study.

Data Analysis Tools

In this study, the researchers carried out their data analysis via different steps. First of all, by conducting items’ analysis for Grade 12 English test items, to determine the skills (e.g., grammar, vocabulary, reading abilities) that were targeted in the English ministerial exam. Secondly, via conducting CEFR levels’ analysis for the Grade 12 English test items, by analysing all the test items of the said test papers by using (CEFR Analyser) from (Cathoven AI., 2024), to determine the CEFR levels of the test items one by one. Finally, implementing frequencies and percentages analysis for both of the previous analyses (i.e., skill’s analysis and CEFR level’s analysis) of the test items, to realise precisely the extent of English language skills and English proficiency levels that were targeted in the exam. The said frequencies and percentages analysis was carried out by using the below calculation:

\begin{matrix} Percentage \end{matrix} = \frac{f_{i}}{n} \begin{matrix} \times 100 \end{matrix}

As explained by Mousavi (2009, p. 579) “a percentage is simply a proportion multiplied by 100,” and “in terms of frequency distribution, the proportion can be expressed as:

\begin{matrix} Proportion \end{matrix} = \frac{f_{i}}{n}

Where f_i = the number of observations in a subset of categories, and

n = the total number of observations.”

Here, the researchers view that, it is important to provide necessary information about the AI-CEFR anlayser tool, which was used as one of the data analysis tools in this study, to examine test items’ levels according to the CEFR framework proficiency levels.

The AI-CEFR analyser is part of the Cathoven language hub, an AI platform which was founded by a group of language teachers and AI engineers to support language educators. The AI-CEFR analyser is a smart tool that uses multiple machine learning models via NLP-enabled technology, and provides a detailed analysis of vocabulary, grammar, sentence structure and overall difficulty of any given text to estimate its CEFR level. Additionally, this AI checker tool breaks down each CEFR level into 10 levels to give more precise results. For example, instead of giving only B2 for two different sentences, the AI-CEFR analyser provides one with B2.0 and the other one with B2.3, this is through assessing all the linguistic features and overall difficulty of a given text (or sentence) to estimate its CEFR level from its vocabulary level, verb form level, clause level, and structure level (Cathoven AI, 2024). Finally, many scholarly articles used Cathoven CEFR checker in their studies, such as Pitura (2024), and Hong and Lin (2025) which show its validity and reliability as a powerful AI tool for language related purposes.

The Procedures

Firstly, the researchers started examining and analysing the test items of the test papers, by themselves, to determine the skills and subskills which were targeted in the Grade 12 English ministerial exam. This procedure was performed to answer the first research question of the study. Secondly, the researchers used the AI-CEFR analyser for analysing all of the 100 test items of the said test papers one by one, to discover the CEFR levels of each of the test items, and determine the test items’ levels individually. This is done by putting the test items’ stems (i.e., question parts of the test items) with test items’ keys (i.e., the right options of the test items) together. This procedure was executed to answer the second research question of the current study. Thirdly and finally, the frequencies and percentages analysis was carried out, with both of the motioned analyses, to get a clearer understanding about Grade 12 English ministerial exam through numerical results to build results, findings, and conclusions as required.

Results

The Targeted Skills in the Grade 12 English Ministerial Exam

Regarding answering the first research question, which is (what are the targeted skills which the English ministerial exam measures?), through careful analysis of the test items, the researchers found that, Grade 12 English language exam, in each of the both attempts did not target all the available skills of Grade 12 textbooks. In contrary, the tests targeted reading skills by (36%), grammar skills by (40%), vocabulary by (14%), pronunciation by (10%) only, and with (0%) for assessing listening, speaking, and writing skills. This suggests that, the Grade 12 English ministerial exam is not parallel to the true nature of the Grade 12 English curriculum, accordingly, see (Tables 1 and 2).

Table 1.

Test Items’ Analysis According to the Targeted Skills, First Attempt: (2023–2024) Academic Year.

Skills and subskills of English in the test	Numbers of the test items in the test paper	Frequencies	Percentages
Reading Skills
Reading Comprehension (i.e., unseen passage)	1, 2, 3	3	06
Reading Text based	5, 6, 7, 8, 9, 10, 11	7	14
Literary Reader	12, 13, 14, 15, 16, 17, 18, 19	8	16
Speaking Skills	0	0	0
Listening Skills	0	0	0
Writing Skills	0	0	0
Grammar Skills
Grammar (through functions and use of English)	4, 20, 21, 22, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39	20	40
Vocabulary	23, 45, 46, 47, 48, 49, 50	7	14
Pronunciation (through phonetic transcriptions)	40, 41, 42, 43, 44	5	10
Total	50	50	100

Table 2.

Test Items’ Analysis According to the Targeted Skills, Second Attempt: (2023–2024) Academic Year.

Skills and subskills of English in the test	Numbers of the test items in the test paper	Frequencies	Percentages
Reading Skills
Reading Comprehension (i.e., unseen passage)	1, 2, 3, 4	4	08
Reading Text based	5, 6, 7, 8, 9, 10	6	12
Literary Reader	43, 44, 45, 46, 47, 48, 49, 50	8	16
Speaking Skills	0	0	0
Listening Skills	0	0	0
Writing Skills	0	0	0
Grammar Skills
Grammar (through functions and use of English)	11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 28, 29, 30, 31	20	40
Vocabulary	27, 32, 33, 34, 40, 41, 42	7	14
Pronunciation (through phonetic transcriptions)	35, 36, 37, 38, 39	5	10
Total	50	50	100

Alignment of Grade 12 English Ministerial Exam with the CEFR Framework Levels

Regarding answering the second research question, which is (What are the proficiency levels which the English ministerial exam measures in terms of the CEFR levels?), after analysing the test items of Grade 12 English test papers, of the both attempts, the researchers founded that, the English language exam contains different levels of CEFR scale from A1 to C2+, and the test items are a mixture of different levels of language proficiency levels from elementary to native+ level according to the British Council mapping, and different bands from band 1 up to band 9 of the IELTS results. Accordingly, see (Tables 3 and 4), to realise the CEFR levels of each of the test items with their equivalents according to the British Council and IELTS mappings.

Table 3.

Test Items’ Levels and Mappings, (2023–2024) Academic Year: First Attempt.

Numbers of the test items in the test paper	Frequencies	Items’ levels and mappings
Numbers of the test items in the test paper	Frequencies	CEFR	British Council	IELTS
22, 33	2	A1	Elementary	2
1, 10, 15, 20, 24, 25, 26, 28, 29, 31, 32, 34, 35, 36, 38, 43, 45, 46, 47, 49	20	A2	Pre-intermediate	2, 3, 3.5, 4
2, 3, 4, 5, 6, 7, 13, 14, 17, 19, 21, 27, 30, 39, 40, 44, 48, 50	18	B1	Intermediate	4, 4.5, 5
9, 11, 12, 16, 18, 23, 37, 41, 42	9	B2	Upper-intermediate	5, 5.5, 6
8	1	C1	Advanced	6.5
Total	50

Table 4.

Test Items’ Levels and Mappings, (2023–2024) Academic Year: Second Attempt.

Numbers of the test items in the test paper	Frequencies	Items’ levels and mappings
Numbers of the test items in the test paper	Frequencies	CEFR	British Council	IELTS
16, 22, 24, 25, 28, 33	6	A1	Elementary	1, 1.5, 2
1, 5, 8, 9, 12, 13, 14, 15, 17, 18, 23, 26, 29, 30, 34, 40, 46	17	A2	Pre-intermediate	2, 2.5, 3, 3.5, 4
2, 3, 4, 6, 7, 11, 19, 20, 27, 31, 32, 35, 37, 38, 42, 43, 45, 47	18	B1	Intermediate	4, 4.5
10, 21, 44, 48, 49, 50	6	B2	Upper-intermediate	5.5, 6, 6.5
36	1	C1	Advanced	7
39, 41	2	C2+	Native+	9
Total	50

Based on the analysis of (Table 3), of the first attempt of the English test paper, the researchers got the following percentages for the 50 test items according to the CEFR levels (see Figure 1). Accordingly, there are (2 items = 04%) that carry (A1 level), and there are (20 items = 40%) that carry (A2 level) also, there are (18 items = 36%) that carry (B1 level). Moreover, there are (9 items = 18%) that carry (B2 level), and there is (1 item = 02%) that carries (C1 level).

Figure 1.

Distribution of the first attempt test items according to the CEFR levels by percentage.

Furthermore, based on the analysis of (Table 4), of the second attempt of the English test paper, the researchers got the following percentages of the 50 test items according to the CEFR levels (see Figure 2).Accordingly, there are (6 items = 12%) that carry (A1 level), and there are (17 items = 34%) that carry (A2 level), also, there are (18 items = 36%) that carry (B1 level). Moreover, there are (6 items = 12%) that carry (B2 level). Finally, there is (1 item = 02%) that carries (C1 level), and another item that carries (C2+ level).

Figure 2.

Distribution of the second attempt test items according to the CEFR levels by percentage.

Discussion of the Results

Mismatch Between Grade 12 English Ministerial Exam, and Grade 12 English Curriculum

Nunan (as cited in Al Mamari et al., 2018) states that the link between language teaching and language testing is a crucial principle, and he claims that a communicative language teaching should have communicative language testing. This is to assess and capture learners’ real communicative competence meaningfully as in real contexts of real life (Zulu & Manchishi, 2018). Having said that, from the results of (Tables 1 and 2), the researchers found that there is a clear mismatch between Grade 12 curriculum and Grade 12 English ministerial exam, in terms of targeted skills and structure.

With regards to the targeted skills, Sunrise 12 curriculum targets the main 4 skills of English in addition to Grammar, Vocabulary and Pronunciation subskills through its units via various ways and activities (such as, but not limited, Listen, Do and Speak, Write / writing, Read, Listen and Speak, Read and Write, Listen and Speak, Read, Do and Speak, Think About It, Listen and Understand, Create the adverb and its opposite, Practice your pronunciation, find the opposite of this word in the reference section, …etc.) (Macfarlane, 2011a). Those skills and subskills are there to be taught in the classrooms and they should be practiced by the students as required, but the Grade 12 English ministerial exam does not include all the skills of English language, and it excludes important skills of speaking, listening, and writing to be assessed. These results are in line with findings of Borg and Capstick (2024) who claimed that, the exclusion of oral components of speaking and listening in Grade 12 exam of English is considered as one of the factors that creates obstacle in the process of English language education development in KRI and Iraq as well.

Again, in terms of structure, there is a mismatch between Sunrise 12 curriculum and Grade 12 English ministerial exam; in this regard Sunrise 12 contains several different tasks, activities and techniques (such as, but not limited, Explain in Your Words Discuss the Chart, Answer the Questions; Use Expressions of Purpose, After you read do these tasks, Listen to the Interview and Make brief Notes of the Important parts of the Interview, Listen to Part 1 and 2 and Repeat, Give Full Answers to Write a Paragraph about the Situation after the Captains Death, Use the Letters in the Brackets to produce the Words for the definitions, Label the picture with these groups of words, Before you read do these tasks, Read to find the correct meaning, Match the Fractions in the Box to the Percentages below,…etc.) (Macfarlane, 2011b). These tasks, activities and techniques are there to be used with students in order to involve them in their language learning and promote their communicative skills at the end of their secondary education. In contrary, the Grade 12 English ministerial exam contains only multiple choice technique for assessing students’ levels of English, and the use of this technique solely is far from the tasks, activities and techniques which were used in Sunrise 12 throughout its Student Book and Activity Book units. These results are also in line with Omar (2020) who stated that multiple choice method of testing created negative washback to English language education in the KRI public schools.

Failure to the CEFR Framework Levels

As mentioned earlier, the Grade 12 English ministerial exam contains different levels of CEFR levels, but the dominant levels are A2 + B1 levels such as: 78% of the test items of the first attempt have A2 + B1 levels (40% as A2 level and 38% as B1 levels), and 70% of the test items of the second attempt have A2 + A1 levels (34% as A2 level and 36% as B1 level) see (Tables 3 and 4).

At the first impression those results look promising, because they go in line with final level of Sunrise Course (as mentioned in the Literature Review) of this study. But in reality it is not the case, because the B1 level of Sunrise Course is based on all the skills, topics, contents, functional and linguistic syllable of Sunrise Course as unified topics and skills to be taught in full, and students should be assessed in them in full as they are available in the Course, especially in Grade 12, this is to capture their actual levels in English. Additionally, each level of CEFR levels explains what a language user can do with a language through the use of all the language skills in practice in different situations in real life. This means, any level of CEFR mirrors language user’s level in that level, for example A2 level learner means language learner’s proficiency level in speaking, listening, reading, writing and other related competences for that level.

From Bachman’s communicative language ability perspective, individual’s communicative ability is a combination of both knowing and doing capabilities, and it is made possible through the interaction of those abilities (Taş & Khan, 2020), but in reality this is not the case for Grade 12 students. In Grade 12 English ministerial exam, 3 main skills of English language (i.e., speaking, listening, and writing) are totally excluded in which there are 0% items in the exam for assessing students’ levels in those skills. This means we do not know students’ levels in those skills even when they pass the exam. Consequently, the results which Grade 12 students get from the test cannot be considered as their actual level of English proficiency according to the CEFR framework, but they are their levels in reading, grammar, vocabulary and pronunciation of Sunrise 12 ONLY without considering their levels in speaking, listening and writing skills. More precisely, Grade 12 results in the English ministerial exam show their pass of Sunrise 12 knowledge, not A2 or B1 of English proficiency level.

Lack of Communicative Testing Nature

CEFR framework views language as communication, and it should be taught and assessed as communication as well. Therefore, to assess language user’s proficiency, a language user should be involved in an act of communication through the use of communicative activities and strategies for practicing speaking, listening, reading and writing skills of a language as required to carry out communicative tasks. This means, to have valid and feasible language assessment, any communicative language test should contain (Receptive Activities and Strategies, Productive Activities and Strategies, Interactive Activities and Strategies, Mediating activities and Strategies). Additionally, McNamar (2000) claimed that communicative language tests should assess learners’ levels in both receptive and productive ways in an act of communication, and require from language learners guess and use language as in real life settings.

But, when we analyse Grade 12 English ministerial test papers, we found that the English exam lacks all the specifications which are required for having communicative assessment for English language, and the Grade 12 English exam depends SOLELY on multiple choice technique by (100%) without having any reception, production, interaction, and mediation activities and strategies to engage Grade 12 students in communicative tasks. Moreover, as a result of using this method in testing English language, Grade 12 students are obliged to memorise Sunrise 12 contents’ information and chose correct options in the English ministerial exam without involving in any communicative tasks. This means, when Grade 12 students pass English ministerial exam, they pass what they memorise from Sunrise 12 and answer in the test accordingly. It does not mean that their English results are their assessment of English language through communicative language testing, which involves communicative activities and strategies.

Conclusions

First of all, the study concluded that, there is a clear mismatch between Grade 12 English curriculum and Grade 12 English ministerial testing method. The Grade 12 English curriculum textbooks contain the main 4 skills of listening, speaking, reading and writing in addition to grammar, vocabulary and pronunciation subskills of English, but the Grade 12 English ministerial testing method targets only reading skill with grammar, vocabulary, pronunciation knowledge of Sunrise 12, and excludes listening, speaking, writing skills to be assessed. This means, the English ministerial testing method is considerably far from the true structure of the Grade 12 official textbooks.

Secondly, the Grade 12 English ministerial exam fails to capture the actual language proficiency levels of Grade 12 students according to the CEFR framework levels, and does not go in line with the end level of Sunrise Course which is B1 according to the CEFR framework. Therefore, the results which Grade 12 students get from the English ministerial exam is the results of their pass in Sunrise 12 knowledge only, not their A2 or B1 levels of English proficiency as actual language users of English from CEFR framework’s perspective.

Thirdly, the Grade 12 English ministerial exam cannot be considered as a valid and feasible test. The test lacks communicative language assessment nature and it relies totally on multiple choice technique alone without having any reception, production, interaction, mediation activities and strategies to engage Grade 12 students in an act of communication through communicative tasks.

Implications and Limitations

The findings of the current study contribute significantly to the English language education context in the KRI and provides implications for the Ministry of Education-KRI to start reforming its English language testing system in general and its English language assessment of Grade 12 in particular, this is according to the communicative needs of Kurdistan students, and in line with the communicative specifications and requirements of the CEFR framework and communicative language testing approach as well.

Despite its contributions, the findings of the current study have two major limitations which are necessary to be considered. First, the results of the study based only on English ministerial exam test papers of one academic year, which is the 2023 to 2024 academic year. Therefore, the results might not be generalised, and more test papers of English ministerial exam of previous years are necessary to be examined and analysed to get more comprehensive view about Grade 12 English Ministerial exam. Secondly, the study offers only quantitative analysis about Grade 12 English ministerial exam, further studies are needed to be conducted with the Ministry of Education-KRI officials and with English language testing policy documents to get qualitative data and get more in depth understanding about the current and future method of testing English language in Grade 12 of the secondary education.

Recommendations

Based on the findings of this study, the researchers recommend the followings to the Ministry of Education-KRI:

The Ministry of Education- KRI is recommended to adopt new methods of assessing English language components in the English ministerial exam of Grade 12.

The Ministry of Education- KRI should draw up an action plan (with the help of international partners if needed) to specify the process and timeframe for revising the assessment of English in the ministerial exams of Grade 12. This action plan to be implemented according to the specified timetable.

The revised English ministerial exam should reflect contents, structures, skills, grammar, vocabulary and functions, range of tasks and exercise types of Grade 12 curriculum.

Footnotes

Appendix A

ORCID iDs

Shaswar Kamal Mahmud

Mustafa Kurt

Consent to Participate

This article does not contain any studies with human and animal participants.

Author Contributions

Shaswar Kamal Mahmud: Conceptualisation, Data Curation and Analysis, Investigation, Methodology, Resources, Writing – original draft, reviewing and editing. Mustafa Kurt: Conceptualisation, Project Administration, Supervision, Validation, Writing – reviewing and editing.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data Availability Statement

The datasets used in this study are available from the corresponding author upon a reasonable request.*

References

Abidin

N. Z.

Hashim

(2021). Common European framework of reference (CEFR): A review on teachers’ perception & plurilingualism. Creative Education, 12(04), 727–736. https://doi.org/10.4236/ce.2021.124051

Al Mamari

M. R. H.

Al-Mekhlafi

A. M.

Al-Barwani

T. A.

(2018). Is communicative language teaching being tested communicatively? An analysis of English tests in Oman. English Language Teaching, 11(11), 83–91. http://doi.org/10.5539/elt.v11n11p83

Bachman

L. F.

(1990). Fundamental considerations in language testing. Oxford University Press.

Bakhsh

S. A.

(2016). Testing communicative language skills of the speaking test in an EFL classroom at King Abdulaziz University. International Journal of Educational Investigations, 3(6), 112–120. http://www.ijeionline.com/attachments/article/55/IJEI.Vol.3.No.6.07.pdf

Borg

Capstick

(2024). Iraq – Understanding English language teaching and learning. British Council. https://doi.org/10.57884/P0XB-4S40

Cambridge University Press. (2013). Introductory guide to the common European framework of reference (CEFR) for English language teachers. https://www.englishprofile.org/images/pdf/GuideToCEFR.pdf

Cathoven AI. (2024). Cathoven language hub [Web app]. https://nexthub.cathoven.com

Congmin

(2023). Theorizing the language ability and language assessment. US-China Foreign Language, 21(7), 261–266. https://doi.org/10.17265/1539-8080/2023.07.002

Council of Europe. (2001). Common European framework of reference for languages: Learning, teaching, assessment. Cambridge University Press.

10.

Davies

Elder

(Eds.). (2007). The handbook of applied linguistics. Wiley-Blackwell.

11.

Davies

Brown

Elder

Hill

Lumley

McNamara

(2002). Dictionary of language testing: Studies in language testing (Vol. 7). Cambridge University Press.

12.

Hong

Lin

(2025). Cathoven: Enhancing language acquisition through tailored input and targeted output feedback. RELC Journal. Advance online publication. https://doi.org/10.1177/00336882251332352

13.

Jamil

N. D.

Bapir

B. B.

(2017). Special regulations about the examinations. General Directorate of Examinations-Erbil.

14.

Jeon

(2022). A systematic review of CEFR-related research of English education in South Korea. Journal of Curriculum and Teaching. 11(8), 363. https://doi.org/10.5430/jct.v11n8p363

15.

Macfarlane

(2011a). Sunrise12: Student’s book. Macmillan Publishers Limited.

16.

Macfarlane

(2011b). Sunrise12: Activity book. Macmillan Publishers Limited.

17.

Macmillan Education. (n.d.). Terminal level of sunrise course [Unpublished document]. Report to the Ministry of Education-KRI.

18.

Macmillan Publishers. (2011). Sunrise12: Teacher’s book.

19.

Mahmud

S. K.

(2013). Introducing a new format of baccalaureate exam for Sunrise 12 [Unpublished Master’s thesis, University of Salahaddin- Erbil].

20.

Mahmud

S. K.

Bostanci

H. B.

(2022). English as a foreign language teachers’ perceptions towards the use of the communicative testing method. World Journal of English Language, 12(5), 410. https://doi.org/10.5430/wjel.v12n5p410

21.

McNamara

(2000). Language testing ( Widdowson

H. G.

, Ed.). Oxford University Press.

22.

Ministry of Education. (2015). Educational conference suggestions. Directorate of Media of the Ministry of Education- Erbil.

23.

Ministry of Education. (2022). Mistrial decree with number 138 on 17/03/2022. Ministry of Education- Erbil.

24.

Ministry of Education. (2024). General examination for the preparatory schools: English language test papers of the (2023-2024) academic year. General Directorate of Examinations- Erbil.

25.

Mousavi

S. A.

(2009). An encyclopaedic dictionary of language testing (4th ed.). Rahnama Press.

26.

Mustafa

Azeez

Ahmad

(2022). Assessing the validity of the 12th-grade ministerial exam of English language in Kurdistan. Academic Journal of Nawroz University, 11(1), 93–111. https://doi.org/10.25007/ajnu.v11n1a1259

27.

Omar

J. A.

(2020). Negative washback of 12th-grade baccalaureate English language exam in Kurdistan Region of Iraq. Journal of University of Raparin, 7(3), 672–700. https://doi.org/10.26750/Vol(8).No(2)

28.

Phan

(2008). Communicative language testing. Hawaii Pacific University TESOL Working Paper Series, 6(1), 3–10. https://www.hpu.edu/research-publications/tesol-working-papers/spring-2008/6_1_02Phan.pdf

29.

Pitura

(2024). Enhancing advanced vocabulary in EFL writing: An AI-assisted intervention for English studies students in Poland. Journal of China Computer-Assisted Language Learning, 4(2), 1–26. https://doi.org/10.1515/jccall-2024-0014

30.

Sabah

(2015). Assessing sunrise ministerial exam. In 6th International Visible Conference on Educational Studies and Applied Linguistics (pp. 364–373).

31.

Schmitt

(Ed.). (2002). An introduction to applied linguistics. Arnold.

32.

Taş

Khan

. (2020). On the models of communicative competence. In GLOBETSonline: International Conference on Education, Technology and Science (pp. 86–96).

33.

Weir

C. J.

(1990). Communicative language testing. Prentice Hall International (UK).

34.

Zulu

Manchishi

P. C.

(2018). Teachers’ familiarity with communicative testing techniques. Zambia Journal of Education, 5(1), 81–94. https://journals.unza.zm/index.php/ZAJE/article/view/87/87

The Use of AI-Based Analyser to Examine Language Testing Practices According to the CEFR Framework