Critical language assessment (CLA) has been addressed in numerous studies. However, the majority of the studies have overlooked the need for a practical framework to measure the CLA dimension of teachers’ language assessment literacy (LAL). This gap prompted us to develop and validate a critical language assessment literacy (CLAL) scale to further underscore the role of CLA principles and their practice as an essential part of teachers’ LAL. In the first phase, a pool of items was generated through a comprehensive review of the related studies. In the quantitative phase, the developed scale was administered to 255 English as a foreign language teachers selected through convenience and snowball sampling. The data were analyzed through exploratory factor analysis for construct validity and Cronbach’s alpha for estimating internal consistency. The results showed that the items loaded on five factors: (a) teachers’ knowledge of assessment objectives, scopes, and types; (b) assessment use consequences; (c) fairness; (d) assessment policies; and (e) national policy and ideology. It was found that the scale had a high level of internal consistency and construct validity, which suggests that this scale has the potential to be useful in assessing language teachers’ CLAL and to raise language teachers’ awareness of CLAL constructs.
AlibakhshiG.Rezaei MezajinS. (2013). On the consequences of the violation of critical pedagogy principles from Iranian EFL teacher trainers’ perspectives. Iranian Journal of Applied Language Studies, 5(2), 1–28. https://doi.org/10.22111/IJALS.2015.1875
2.
AriasC.MaturanaL.RestrepoM. (2012). Evaluation in foreign language learning: Towards fair and democratic practices. Lenguaje, 40(1), 99–126. https://doi.org/10.19183/how.26.1.481
3.
BachmanL. F. (1990). Fundamental considerations in language testing. Oxford University Press.
4.
BachmanL. F.PalmerA. (2010). Language assessment in practice. Oxford University Press.
5.
BachmanL. F.PalmerA. S. (1996). Language testing in practice. Oxford University Press.
6.
BakerB. A.RichesC. (2018). The development of EFL examinations in Haiti: Collaboration and language assessment literacy development. Language Testing, 35(4), 557–581. https://doi.org/10.1177/0265532217716732
7.
BernsteinB. (1986). On pedagogical discourse. In RichardsonJ. (Ed.), Handbook for theory and research for the sociology of education (pp. 205–240). Greenwood Press.
8.
BrindleyG. (2001). Language assessment and professional development. In ElderC.BrownA.GroveE.HillK.IwashitaN.LumleyT.McNamaraT.O’LoughlinK. (Eds.), Experimenting with uncertainty: Essays in honour of Alan Davies (pp. 126–136). Cambridge University Press.
ButlerY. G.PengX.LeeJ. (2021). Young learners’ voices: Towards a learner-centered approach to understanding language assessment literacy. Language Testing, 38(3), 429–455. https://doi.org/10.1177/0265532221992274
11.
CanagarajahS. (2005). Critical pedagogy in L2 learning and teaching. In HinkelE. (Ed.), Handbook of research in second language teaching and learning (pp. 931–949). Lawrence Erlbaum Associates.
12.
CoombeC.TroudiS.Al-HamlyM. (2012). Foreign and second language teacher assessment literacy: Issues, challenges, and recommendations. In CoombeC.DavidsonP.O’SullivanB.StoynoffS. (Eds.), The Cambridge guide to second language assessment (pp. 20–29). Cambridge University Press.
13.
CowieB. (2015). Equity, ethics and engagement: Principles for quality formative assessment in primary science classrooms. In MilneC.TobinK.DeGennaroD. (Eds.), Sociocultural studies and implications for science education (pp. 117–133). Springer.
14.
CrawfordL. M. (1978). Paulo Freire’s philosophy: Derivation of curricular principles and their application to second design curriculum design [Doctoral dissertation, University of Minnesota]. ProQuest Dissertations and Theses database.
DeLucaC.LaPointe-McEwanD.LuhangaU. (2016). Teacher assessment literacy: A review of international standards and measures. Educational Assessment, Evaluation and Accountability, 28(3), 251–272. https://doi.org/10.1007/S11092-015-9233-6
17.
DeygersB. (2019). Fairness and social justice in English language assessment. In GaoX. (Ed.), Second handbook of English language teaching (pp. 1–29). Springer.
18.
DeygersB.MaloneM. E. (2019). Language assessment literacy in university admission policies, or the dialogue that isn’t. Language Testing, 36(3), 347–368. https://doi.org/10.1177/0265532219826390
19.
FettermanD. (2019). Empowerment evaluation: A stakeholder involvement approach. Association of Health Promotion Professionals, 30(2), 137–142. https://doi.org/10.1002/hpja.243
20.
FirooziT.RazavipourK.AhmadiA. (2019). The language assessment literacy needs of Iranian EFL teachers with a focus on reformed assessment policies. Language Testing in Asia, 9, Article 2. https://doi.org/10.1186/s40468-019-0078-7
21.
FoucaultM. (1995). Discipline and punish: The birth of the prison (SheridanA. Trans.). Vintage Books. (Original work published in 1979)
22.
FoucaultM. (1982). The subject and power. In DreyfusH.RabinowP. (Eds.), Michel Foucault: Beyond structuralism and hermeneutics (pp. 777–795). University of Chicago Press.
23.
FredriksenJ.CollinsA. (1989). A system approach to educational testing. Educational Researcher, 18(9), 27–32. https://doi.org/10.2307/1176716
24.
FreireP. (1970). Pedagogy of the oppressed. Penguin.
25.
FreireP. (1973). Education: The practice of freedom. Writers and Readers.
26.
FulcherG. (2010). Practical language testing. Hodder Education.
FulcherG. (2015). Re-examining language testing: A philosophical and social inquiry. Routledge.
29.
FulcherG. (2017). The practice of language assessment. In HinkelE. (Ed.), Handbook of research in second language teaching and learning (pp. 463–475). Routledge.
30.
GiraldoF. (2018). Language assessment literacy: Implications for language teachers. Profile: Issues in Teachers’ Professional Development, 20(1), 179–195. https://doi.org/10.15446/profile.v20n1.62089
31.
GirouxH. G. (2020). On critical pedagogy. Bloomsbury.
32.
GreenA. (2007). IELTS washback in context: Preparation for academic writing in higher education. Cambridge University Press.
33.
HardingL.KremmelB. (2016). Language assessment literacy and professional development. In TsagariD.BanerjeeJ. (Eds.), Handbook of second language assessment (pp. 413–428). Mouton de Gruyter.
HickeyD. T.ZuikerS. J.TaasoobshiraziG.SchaferN. J.MichaelM. A. (2006). Balancing varied assessment functions to attain systemic validity: Three is the magic number. Studies in Educational Evaluation, 32(3), 180–201. https://doi.org/10.1016/j.stueduc.2006.08.006
36.
HillK. (2012). Classroom-based assessment in the school foreign language classroom. Peter Lang.
37.
Inbar-LourieO. (2008a). Constructing a language assessment knowledge base: A focus on language assessment courses. Language Testing, 25(3), 385–402. https://doi.org/10.1177/0265532208090158
38.
Inbar-LourieO. (2008b). Language assessment culture. In ShohamyE. (Ed.), Language testing and assessment (Vol. 7, 2nd ed., pp. 285–300). Springer.
39.
Inbar-LourieO. (2013a). Guest editorial to the special issue on language assessment literacy. Language Testing, 30(3), 301–307. https://doi.org/10.1177/0265532213480126
40.
Inbar-LourieO. (2013b, November). Language assessment literacy: What are the ingredients? [Paper presentation] 4th CBLA SIG Symposium Programme, University of Cyprus, Nicosia, Cyprus.
41.
JeongH. (2013). Defining assessment literacy: Is it different for language testers and non-language testers?Language Testing, 30(3), 345–362. https://doi.org/10.1177/0265532213480334
42.
JinY. (2010). The place of language testing and assessment in the professional preparation of foreign language teachers in China. Language Testing, 27(4), 555–584. https://doi.org/10.1177/0265532209351431
43.
KamiyaN.LoewenS. (2014). The influence of academic articles on an ESL teacher’s stated beliefs. Innovation in Language Learning and Teaching, 8(3), 205–218. https://doi.org/10.1080/17501229.2013.800077
KaneM.BridgemanB. (2017). Research on validity theory and practice at ETS. In BennettR. E.von DavierM. (Eds.), Advancing human assessment: The methodological, psychological and policy contributions of ETS (pp. 489–552). Springer Nature.
KremmelB.HardingL. (2020). Towards a comprehensive, empirical model of language assessment literacy across stakeholder groups: Developing the language assessment literacy survey. Language Assessment Quarterly, 17(1), 100–120. https://doi.org/10.1080/15434303.2019.1674855
48.
KunnanA. J. (2004). Test fairness. In MilanovicM.WeirC. (Eds.), European language testing in a global context: Proceedings of the ALTE Barcelona Conference (pp. 27–48). Cambridge University Press.
KunnanA. J. (2013). High stakes language testing. In ChapelleC. A. (Ed.), The encyclopedia of applied linguistics (pp. 1–6). Wiley Blackwell.
51.
KunnanA. J. (2018). Evaluating language assessments. Routledge.
52.
LamR. (2015). Language assessment training in Hong Kong: Implications for language assessment literacy. Language Testing, 32(2), 169–197. https://doi.org/10.1177/0265532214554321
53.
LantolfJ. P. (2009). Dynamic assessment: The dialectic integration of instruction and assessment. Language Teaching, 42(3), 355–368. https://doi.org/10.1017/S0261444808005569
54.
LeeJ. (2019). A training project to develop teachers’ assessment literacy. In WhiteE.DelaneyT. (Eds.), Handbook of research on assessment literacy and teacher made testing in the language classroom (pp. 58–80). IGI Global.
55.
LeeJ.ButlerY. G. (2020). Reconceptualizing language assessment literacy: Where are language learners?TESOL Quarterly, 54(4), 1098–1111. https://doi.org/10.1002/tesq.576
56.
LeenknechtM.WijniaL.KöhlenM.FryerL.RikersR.LoyensS. (2020). Formative assessment as practice: The role of students’ motivation. Assessment & Evaluation in Higher Education, 46(2), 236–255. https://doi.org/10.1080/02602938.2020.1765228
57.
LeviT.Inbar-LourieO. (2020). Assessment literacy or language assessment literacy: Learning from the teachers. Language Assessment Quarterly, 17(2), 168–182. https://doi.org/10.1080/15434303.2019.1692347
58.
LopezA. A.TurkanS.Guzman-OrthD. (2017). Assessing multilingual competence. In ShohamyE.OrI. G.MayS.(Eds.), Language testing and assessment (3rd ed., pp. 91–102). Springer International Publishing.
MaloneM. E. (2013). The essentials of assessment literacy: Contrasts between testers and users. Language Testing, 30(3), 329–344. https://doi.org/10.1177/0265532213480129
McNamaraT.KnochU.FanJ. (2019). Fairness, justice and language assessment. Oxford University Press.
64.
McNamaraT.RoeverK. (2006). Language testing: The social dimension. Blackwell.
65.
MertlerC. A. (2009). Teachers’ assessment knowledge and their perceptions of the impact of classroom assessment professional development. Improving Schools, 12(2), 101–113. https://doi.org/10.1177/1365480209105575
NematiM.AlaviS. M.MohebbiH.MasjedlouA. P. (2017). Teachers’ writing proficiency and assessment ability: The missing link in teachers’ written corrective feedback practice in an Iranian EFL context. Language Testing in Asia, 7, Article 21. https://doi.org/10.1186/s40468-017-0053-0
70.
NevoD. (1996). School based evaluation. Pergamon Press.
71.
PaivandiS. (2012). Education in the Islamic Republic of Iran and perspectives on democratic reforms. Legatum Institute.
72.
PallantJ. (2016). The SPSS survival manual (6th ed.). McGraw-Hill.
73.
PillJ.HardingL. (2013). Defining the language assessment literacy gap: Evidence from a parliamentary inquiry. Language Testing, 30(3), 381–402. https://doi.org/10.1177/0265532213480337
74.
PophamW. J. (2004). Why assessment illiteracy is professional suicide. Educational Leadership, 62(1), 82–83.
RaubenheimerJ. E. (2004). An item selection procedure to maximize scale reliability and validity. South African Journal of Industrial Psychology, 30(4), 59–64. https://doi.org/10.4102/sajip.v30i4.168
77.
RazavipourK.RezagahK. (2018). Language assessment in the new English curriculum in Iran: Managerial, institutional, and professional barriers. Language Testing in Asia, 8(9). https://doi.org/10.1186/s40468-018-0061-8
78.
Rea-DickinsP. (2006). Currents and eddies in the discourse of assessment: A learning-focused interpretation. International Journal of Applied Linguistics, 16(2), 163–188. https://doi.org/10.1111/j.1473-4192.2006.00112.x
79.
SaraniA.AlibakhshiG.MolazehiH. (2014). On application of critical pedagogy principles by ELT instructors and subject teachers: A case of Iranian universities. Research in Applied Linguistics, 5(2), 41–56.
80.
ScarinoA. (2013). Language assessment literacy as self-awareness: Understanding the role of interpretation in assessment and in teacher learning. Language Testing, 30(3), 309–327. https://doi.org/10.1177/0265532213480128
81.
SchisselJ. L. (2019). Social consequences of testing for language-minoritized bilinguals in the United States. Multilingual Matters.
82.
SedovaK.SedlacekM.SvaricekR. (2016). Teacher professional development as a means of transforming student classroom talk. Teaching and Teacher Education, 57, 14–25. https://doi.org/10.1016/j.tate.2016.03.005
ShohamyE. (2017). ELF and critical language testing. In JenkinsJ.DeweyM.BakerW. (Eds.), The Routledge handbook of English as a lingua franca (pp. 583–593). Routledge.
89.
SpolskyB. (1978). Introduction: Linguists and language testers. In SpolskyB. (Ed.), Approaches to language testing (pp. v–x). Center for Applied Linguistics.
90.
SpolskyB. (1981). Some ethical questions about language testing. In Klein-BraleyC.StevensonD. K. (Eds.), Practice and problems in language testing (pp. 5–21). Verlag Peter Lang.
91.
SpolskyB. (1995). Measured words: The development of objective language testing. Oxford University Press.
StigginsR. J. (1999). Evaluating classroom assessment training in teacher education programs. Educational Measurement: Issues and Practice, 18(1), 23–27. https://doi.org/10.1111/j.1745-3992.1999.tb00004.x
94.
SultanaN. (2019). Language assessment literacy: An uncharted area for the English language teachers in Bangladesh. Language Testing in Asia, 9, Article 1. https://doi.org/10.1186/s40468-019-0077-8
95.
TabachnickB. G.FidellL. S. (2013). Using multivariate statistics (6th ed.). Pearson.
96.
TaylorL. (2013). Communicating the theory, practice and principles of language testing to test stakeholders: Some reflections. Language Testing, 30(3), 403–412. https://doi.org/10.1177/0265532213480338
97.
TierneyR. D. (2016). Fairness in educational assessment. In PetersM. A. (Ed.), Encyclopedia of educational philosophy and theory (pp. 793–798). Springer. https://doi.org/10.1007/978-981-287-588-4_400
98.
TsagariD.VogtK. (2017). Assessment literacy of foreign language teachers around Europe: Research, challenges and future prospects. Papers in Language Testing and Assessment, 6(1), 41–63. https://doi.org/10.1080/15434303.2014.960046
99.
VogtK.TsagariD. (2014). Assessment literacy of foreign language teachers: Findings of a European study. Language Assessment Quarterly, 11(4), 374–402. https://doi.org/10.1080/15434303.2014.960046
100.
VogtK.TsagariD.SpanoudisG. (2020). What do teachers think they want? A comparative study of inservice language teachers’ beliefs on LAL training needs. Language Assessment Quarterly, 17(4), 386–409. https://doi.org/10.1080/15434303.2020.1781128
101.
WaltersF. (2010). Cultivating assessment literacy: Standards evaluation through language-test specification reverse engineering. Language Assessment Quarterly, 7(4), 317–342. https://doi.org/10.1080/15434303.2010.516042
YongA. G.PearceS. (2013). A beginner’s guide to factor analysis: Focusing on exploratory factor analysis. Tutorials in Quantitative Methods for Psychology, 9(2), 79–94. https://doi.org/10.20982/tqmp.09.2.p079
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
