Local language tests are an arena where national language standards can be operationalized to create a hub for integrating assessment results and language support. Few studies, however, have examined the operationalization of national standards in local language assessment contexts. In this study, we proposed a model to present the integration of national standards in local contexts to guide assessment, teaching, and learning. Using this model, we conducted a 5-month longitudinal study involving 689 college students to examine (1) the consistency between their internal and external assessment feedback (i.e., standards-based self-assessment ratings and standards-based individualized English as a Foreign Language [EFL] listening test feedback) and (2) the effectiveness of standards-based teaching intervention in enhancing their perceived and actual language development. The results showed that the test feedback generally aligned well with students’ self-assessment and perceptions at the overall listening skill and subskill levels, yet student perceptions outlined needs for feedback refinements. In addition, the use of the standards-based individualized feedback, in conjunction with language support courses and practice materials, facilitated students’ perceived and actual listening achievement. This study makes an important contribution to local language testing by demonstrating the potential of a local instrument to provide a bridge between local instructional goals and national standards.
AjjawiR.BoudD. (2017). Researching feedback dialogue: An interactional analysis approach. Assessment & Evaluation in Higher Education, 42(2), 252–265. https://doi.org/10.1080/02602938.2015.1102863
2.
AldersonC. (2005). Diagnosing foreign language proficiency: The interface between learning and assessment. Continuum.
3.
AllenD.KatayamaA. (2016). Relative second language proficiency and the giving and receiving of written peer feedback. System, 56, 96–106. https://doi.org/10.1016/j.system.2015.12.002
4.
AndradeH.DuY. (2007). Student responses to criteria-referenced self-assessment. Assessment and Evaluation in Higher Education, 32(2), 159–181. https://doi.org/10.1080/02602930600801928
5.
AzeezP. Z.Al BajalaniF. R. H. (2018). Effects of mobile assisted language learning on developing Kurdish EFL students: Listening subskills at Koya University. Koya University Journal of Humanities and Social Sciences, 1(1), 85–95. https://doi.org/10.14500/kujhss.v1n1y2018.pp85-95
6.
BachmanL. F. (1990). Fundamental considerations in language testing. Oxford University Press.
7.
BachmanL. F.PalmerA. S. (1996). Language testing in practice. Oxford University Press.
8.
BachmanL. F.PalmerA. S. (2010). Language assessment in practice. Oxford University Press.
9.
BelgradS. (2013). Portfolios and e-portfolios: Student reflection, self-assessment, and goal setting in the learning process. In McMillanJ. H. (Ed.), SAGE handbook of research on classroom assessment (pp. 331–346). SAGE.
BrownN. A.DeweyD. P.CoxT. L. (2014). Assessing the validity of can-do statements in retrospective (then-now) self-assessment. Foreign Language Annals, 47(2), 261–285. https://doi.org/10.1111/flan.12082
12.
ButlerD.WinneP. H. (1995). Feedback as self-regulated learning: A theoretical synthesis. Review of Educational Research, 65(3), 245–281. https://doi.org/10.3102/00346543065003245
13.
ButlerY. G.LeeJ. (2010). The effects of self-assessment among young learners of English. Language Testing, 27(1), 5–31. https://doi.org/10.1177/0265532209346370
CarlessD.BoudD. (2018). The development of student feedback literacy: Enabling uptake of feedback. Assessment & Evaluation in Higher Education, 43(8), 1315–1325. https://doi.org/10.1080/02602938.2018.1463354
16.
ChristisonM. A. (2018). Learner involvement in assessment. In CoombeC. (Ed.), The TESOL encyclopedia on core issues in language testing and assessment (pp. 1–7). Wiley-Blackwell.
17.
CohenJ. (1998). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum.
18.
Council of Europe. (2001). Common European Framework of Reference for Languages: Learning, teaching, assessment. Cambridge University Press.
19.
CoxT. L.DeweyD. P. (2021). Measuring language development through self-assessment. In WinkeP.BrunfautT. (Eds.), The Routledge handbook of second language acquisition and language testing (pp. 382–390). Routledge.
DimovaS. (2021, June4–5). Integrating testing and learning through formative feedback [Conference presentation]. 17th European Association for Language Testing and Assessment Conference, Budapest, Hungary (Virtual).
22.
DimovaS.YanX.GintherA. (2020). Local language testing: Design, implementation, and development. Routledge.
23.
DweckC. S. (1999). Self-theories: Their role in motivation, personality, and development. Psychology Press.
24.
EdeleA.SeuringJ.KristenC.StanatP. (2015). Why bother with testing? The validity of immigrants’ self-assessed language proficiency. Social Science Research, 52, 99–123. https://doi.org/10.1016/j.ssresearch.2014.12.017
25.
ElderC.ReadJ. (2015a). Post-entry language assessments in Australia. In ReadJ. (Ed.), Assessing English proficiency for university study (pp. 25–46). Palgrave Macmillan.
26.
ElderC.ReadJ. (2015b). Post-entry assessments in other countries. In ReadJ. (Ed.), Assessing English proficiency for university study (pp. 70–92). Palgrave Macmillan.
27.
EvansC. (2013). Making sense of assessment feedback in higher education. Review of Educational Research, 83(1), 70–120. https://doi.org/10.3102/0034654312474350
FieldJ. (2008). Listening in the language classroom. Cambridge University Press.
30.
GibbsG.SimpsonC. (2004). Conditions under which assessment supports students’ learning. Learning and Teaching in Higher Education, 1, 3–31.
31.
GohC. C.AryadoustV. (2015). Examining the notion of listening subskill divisibility and its implications for second language listening. International Journal of Listening, 29(3), 109–133. https://doi.org/10.1080/10904018.2014.936119
JangE. E.DunlopM.ParkG.Van der BoomE. H. (2015). How do young students with different profiles of reading skill mastery, perceived ability, and goal orientation respond to holistic diagnostic feedback?Language Testing, 32(3), 359–383. https://doi.org/10.1177/0265532215570924
35.
JinY.WuZ.AldersonC.SongW. (2017). Developing the China Standards of English: Challenges at macropolitical and micropolitical levels. Language Testing in Asia, 7, Article 1. https://doi.org/10.1186/s40468-017-0032-5
36.
KahnemanD. (2011). Thinking, fast and slow. Farrar, Straus & Giroux.
KunnanA. J.JangE. E. (2009). Diagnostic feedback in language assessment. In LongM. H.DoughtyC. J. (Eds.), The handbook of language teaching (pp. 610–627). Wiley-Blackwell.
39.
LeeY.SawakiY. (2009). Application of three cognitive diagnosis models to ESL reading and listening assessments. Language Assessment Quarterly, 6(3), 239–263. https://doi.org/10.1080/15434300903079562
LeightonJ.GierlM. (2007). Cognitive diagnostic assessment for education: Theory and applications. Cambridge University Press.
42.
LeightonJ. P. (2019). Students’ interpretation of formative assessment feedback: Three claims for why we know so little about something so important. Journal of Educational Measurement, 56(4), 793–814. https://doi.org/10.1111/jedm.12237
43.
LiC. (2019). Using a listening vocabulary levels test to explore the effect of vocabulary knowledge on GEPT listening comprehension performance. Language Assessment Quarterly, 16(3), 328–344. https://doi.org/10.1080/15434303.2019.1648474
44.
LiM.ZhangX. (2021). A meta-analysis of self-assessment and language performance in language testing and assessment. Language Testing, 38(2), 189–208. https://doi.org/10.1177/0265532220932481
45.
LimG. S.GeranpayehA.KhalifaH.BuckendahlC. W. (2013). Standard setting to an International Reference Framework: Implications for theory and practice. International Journal of Testing, 13(1), 32–49. https://doi.org/10.1080/15305058.2012.678526
MaW.WinkeP. (2019). Self-assessment: How reliable is it in assessing oral proficiency over time?Foreign Language Annals, 52(1), 66–86. https://doi.org/10.1111/flan.12379
48.
MaierU.WolfN.RandlerC. (2016). Effects of a computer-assisted formative assessment intervention based on multiple-tier diagnostic items and different feedback types. Computers & Education, 95, 85–98. https://doi.org/10.1016/j.compedu.2015.12.002
49.
MalabongaV.KenyonD. M.CarpenterH. (2005). Self-assessment, preparation and response time on a computerized oral proficiency test. Language Testing, 22(1), 59–92. https://doi.org/10.1191/0265532205lt297oa
50.
MartyniukW. (2010). Aligning tests with the CEFR. Cambridge University Press.
51.
MasonA.SinghC. (2010). Do advanced physics students learn from their mistakes without explicit intervention?American Journal of Physics, 78(7), 760–767. https://doi.org/10.1119/1.3318805
52.
McLeanS.KramerB.BeglarD. (2015). The creation and validation of a listening vocabulary levels test. Language Teaching Research, 19(6), 741–760. https://doi.org/10.1177/1362168814567889
53.
MinS.HeL. (2022). Developing individualized feedback for listening assessment: Combining standard setting and cognitive diagnostic assessment approaches. Language Testing, 39(1), 90–116. https://doi.org/10.1177/0265532221995475
54.
MinS.JiangZ. (2020). 校本听力考试与《中国英语能力等级量表》对接研究 [Aligning an in-house EFL listening test to China’s Standards of English Language Ability]. Foreign Language Education, 41(4), 47–51. http://www.cnki.com.cn/Article/CJFDTotal-TEAC202004009.htm
55.
Ministry of Education of the People’s Republic of China, & National Language Commission of the People’s Republic of China. (2018). China’s Standards of English Language Ability. Higher Education Press & Shanghai Foreign Language Education Press. http://www.neea.edu.cn/res/Home/1908/0c96023675649ac8775ff3422f91a91d.pdf
56.
MoellerA.YuF. (2015). NCSSFL-ACTFL can-do statements: An effective tool for improving language learning within and outside the classroom. In SwansonP. (Ed.), Dimension 2015 (pp. 50–69). SCOLT.
57.
Oral English Proficiency Program. (2015). The Oral English Proficiency Test.3 (OEPT.3). Purdue University.
58.
Oral English Proficiency Program. (2019). OEPT technical manual. Purdue University.
59.
O’SullivanB. O.WuS.LiuJ.DunleaJ. (2020). Linking the Aptis test to China’s Standards of English Language Ability. In DamerowR. M.BaileyK. M. (Eds.), Chinese-speaking learners of English: Research, theory, and practice. Routledge.
60.
PapageorgiouS.TannenbaumR. J.BridgemanB.ChoY. (2015). The association between TOEFL iBT® test scores and the Common European Framework of Reference (CEFR) levels (Research Memorandum No. RM-15-06). Educational Testing Service. https://www.ets.org/Media/Research/pdf/RM-15-06.pdf
61.
PapageorgiouS.WuS.HsiehC.-N.TannenbaumR. J.ChengM. (2019). Mapping the TOEFL iBT® test scores to China’s Standards of English Language Ability: Implications for score interpretation and use (TOEFL Research Report No. 89). Educational Testing Service. https://doi.org/10.1002/ets2.12281
62.
ParkG.-P. (2004). Comparison of L2 listening and reading comprehension by university students learning English in Korea. Foreign Language Annals, 37(3), 448–458. https://doi.org/10.1111/j.1944-9720.2004.tb02702.x
63.
PophamJ. (2008). Transformative assessment. Association for Supervision and Curriculum Development.
64.
PowersD.SchedlM.PapageorgiouS. (2017). Facilitating the interpretation of English language proficiency scores: Combining scale anchoring and test score mapping methodologies. Language Testing, 34(2), 175–195. https://doi.org/10.1177/0265532215623582
65.
ReadJ. (2015). Assessing English proficiency for university study. Palgrave Macmillan.
66.
RichardsL. (2009). Handling qualitative data (2nd ed.). SAGE.
67.
RobertsM. R.GierlM. J. (2010). Developing score reports for cognitive diagnostic assessments. Educational Measurement: Issues and Practice, 29(3), 25–38. https://doi.org/10.1111/j.1745-3992.2010.00181.x
68.
RossS. (1998). Self-assessment in second language testing: A meta-analysis and analysis of experiential factors. Language Testing, 15(1), 1–20. https://doi.org/10.1177/026553229801500101
69.
RostM. (2016). Teaching and researching listening (3rd ed.). Longman.
70.
RunnelsJ. (2016). Self-assessment accuracy: Correlations between Japanese English learners’ self-assessment on the CEFR—Japan’s can do statements and scores on the TOEIC®. Taiwan Journal of TESOL, 13(1), 105–137.
71.
RyanR. M. (1982). Control and information in the intrapersonal sphere: An extension of cognitive evaluation theory. Journal of Personality and Social Psychology, 43(3), 450–461. https://doi.org/10.1037/0022-3514.43.3.450
72.
SadlerR. D. (2010). Indeterminacy in the use of preset criteria for assessment in grading in higher education. Assessment and Evaluation in Higher Education, 34(2), 159–179. https://doi.org/10.1080/02602930801956059
73.
SawakiY.KoizumiR. (2017). Providing test performance feedback that bridges assessment and instruction: The case of two standardized English language tests in Japan. Language Assessment Quarterly, 14(3), 234–256. https://doi.org/10.1080/15434303.2017.1348504
74.
SchunkD. H.ZimmermanB. J. (2007). Influencing children’s self-efficacy and self-regulation of reading and writing through modeling. Reading & Writing Quarterly, 23(1), 7–25. https://doi.org/10.1080/10573560600837578
SongM.-Y. (2008). Do divisible subskills exist in second language (L2) comprehension? A structural equation modeling approach. Language Testing, 25(4), 435–464. https://doi.org/10.1177/0265532208094272
77.
StanovichK. E. (2011). Rationality and the reflective mind. Oxford University Press.
78.
StansfieldC. W.GaoJ.RiversW. P. (2010). A concurrent validity study of self-assessments and the federal Interagency Language Roundtable oral proficiency interview. Russian Language Journal, 60, 301–317. https://www.jstor.org/stable/43669189
79.
SummersM.CoxT. L.McMurryB. L.DeweyD. P. (2019). Investigating the use of the ACTFL can-do statements in a self-assessment for student placement in an intensive English program. System, 80, 269–287. https://doi.org/10.1016/j.system.2018.12.012
80.
SweetG.MackS.Olivero-AgneyA. (2019). Where am I? Where am I going, and how do I get there? Increasing learner agency through large-scale self-assessment in language learning. In WinkeP.GassS. M. (Eds.), Foreign language proficiency in higher education (pp. 175–195). Springer.
81.
TannenbaumR. J.WylieE. C. (2008). Linking English language test scores onto the Common European Framework of Reference: An application of standard-setting methodology (TOEFL iBT Research Report RR-08-34). Educational Testing Service. https://doi.org/10.1002/j.2333-8504.2008.tb02120.x
82.
TigchelaarM. (2019). Exploring the relationship between self-assessment and OPIc ratings of oral proficiency in French. In WinkeP.GassS. M. (Eds.), Foreign language proficiency in higher education (pp. 153–173). Springer.
83.
UnderwoodJ. S.Zapata-RiveraD.Van WinkleW. (2010). An evidence-centered approach to using assessment data for policymakers. Educational Testing Service.
84.
van der BoomE.JangE. E. (2018). The effects of holistic diagnostic feedback intervention on improving struggling readers’ reading skills. Journal of Teaching and Learning, 12(2), 54–69. https://doi.org/10.22329/jtl.v12i2.5105
85.
VandergriftL.GohC. (2012). Teaching and learning second language listening: Metacognition in action. Routledge.
86.
VogtK.TsagariD.CsépesI. (2020). Linking learners’ perspectives on language assessment practices to Teachers’ Assessment Literacy Enhancement (TALE): Insights from four European countries. Language Assessment Quarterly, 17(4), 410–433. https://doi.org/10.1080/15434303.2020.1776714
87.
WallaceM. P. (2020). Individual differences in second language listening: Examining the role of knowledge, metacognitive awareness, memory, and attention. Language Learning. Advance online publication. https://doi.org/10.1111/lang.12424
88.
WangY.Treffers-DallerJ. (2017). Explaining listening comprehension among L2 learners of English: The contribution of general language proficiency, vocabulary knowledge and metacognitive awareness. System, 65, 139–150. https://doi.org/10.1016/j.system.2016.12.013
WinstoneN. E.HepperE. G.NashR. A. (2021). Individual differences in self-reported use of assessment feedback: The mediating role of feedback beliefs. Educational Psychology, 41(7), 844–862. https://doi.org/10.1080/01443410.2019.1693510
91.
WinstoneN. E.NashR. A.ParkerM.RowntreeJ. (2017). Supporting learners’ agentic engagement with feedback: A systematic review and a taxonomy of recipience processes. Educational Psychologist, 52(1), 17–37. https://doi.org/10.1080/00461520.2016.1207538
92.
WuZ. (2019). Lower English proficiency means poorer feedback performance? A mixed-methods study. Assessing Writing, 41, 14–24. https://doi.org/10.1016/j.asw.2019.05.001
93.
YanR. (2012). Improving English listening self-efficacy of Chinese university students—Influences of learning strategy training with feedback on strategy use and performance [Unpublished doctoral dissertation, Durham University].
94.
YanX. (2021, June4–5). Local language testing: Local needs and conditions require context-appropriate responses [Conference presentation]. 17th European Association for Language Testing and Assessment Conference, Budapest, Hungary (Virtual).
95.
YanX.KimH. R.KotnarowskiJ. E. (2021). The development of a profile-based writing scale: How collaboration with teachers enhanced assessment practice in a post-admission ESL writing program in the USA. In LanteigneB.CoombeC.BrownJ. D. (Eds.), Challenges in language testing around the world: Insights for language test users (pp. 529–546). Springer.
96.
YanX.ThirakunkovitS.KauperN.GintherA. (2016). What do test takers say: Test-taker feedback as input for quality control. In ReadJ. (Ed.), Post-Admission Language Assessments of University Students (pp. 113–136). Springer.
97.
YuanJ.SavadattiS.ZhengG. (2021). Self-assessing a test with a possible bonus enhances low performers’ academic performance. Computers & Education, 160, Article 104036. https://doi.org/10.1016/j.compedu.2020.104036
98.
Zapata-RiveraD.ZwickR.VezzuM. (2016). Exploring the effectiveness of a measurement error tutorial in helping teachers understand score report results. Educational Assessment, 21(3), 215–229. https://doi.org/10.1080/10627197.2016.1202110
99.
ZimmermanB. J.SchunkD. H. (Eds.) (2001). Self-regulated learning and academic achievement: Theoretical perspectives. Routledge.
