Given the increasing emphasis of communicative competence in English as a foreign language (EFL) contexts and the lack of validation research on speaking assessments for adolescent EFL learners, in the current study we examined the validity of the TOEFL Junior® speaking test, a relatively new speaking assessment developed by Educational Testing Service. We utilized Messick’s integrative view of construct validity (1996) and Kane’s interpretation argument framework (Kane, 2013), and focused on two sources of validity evidence: internal structure and external relations to other variables (AERA, APA, and NCME, 2014). We used a cross-sectional design with 252 seventh-to-ninth graders in Taiwan, which aligns with the spectrum of the TOEFL Junior®’s target age group. All adolescent participants took the TOEFL Junior® speaking test and two researcher-developed speaking tests. They also completed a survey reporting their history of English learning experiences from kindergarten to present and evaluating their own English language skills. Adolescent participants’ teachers provided evaluations of participants’ English language skills. Results from correlational tests and confirmatory factor analysis showed robust evidence for the test’s internal structure. Correlation and multiple regression models also demonstrated strong positive relationships between TOEFL Junior® test scores and external variables, providing evidence for TOEFL Junior®’s construct validity.
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing.
American Psychological Association.
2.
AugustD.ShanahanT. (Eds.) (2006). Developing literacy in second-language learners.
Lawrence Erlbaum Associates.
3.
BygateM. (1987). Speaking.
Oxford University Press.
4.
BaileyA. L. (2017a).
Theoretical and developmental issues to consider in the assessment of young students’ English language proficiency. In M. K.WolfY. G.Butler (Eds.). English language proficiency assessments for young learners (pp.25–40).
Routledge. https://doi.org/10.4324/9781315674391
5.
BaileyA. L. (2017b).
Progressions of a new language: Characterizing explanation development for assessment with young language learners. Annual Review of Applied Linguistics, 37,241–263. https://doi.org/10.1017/S0267190517000113
6.
BaileyA. L. (2005).
Language analysis of standardized achievement tests: Considerations in the assessment of English language learners. In J.AbediA. L.BaileyF. A.Butler (Eds.). The validity of administering large-scale content assessments to English language learners: An investigation from three perspectives (pp. 79–100).
CSE Tech. Report No. 663. http://cresst.org/wp-content/uploads/R663.pdf.
7.
BaileyA. L.Blackstock-BernsteinA.HeritageM. H. (2015).
At the intersection of mathematics and language: Examining mathematical explanations of English proficient and English language learner students. Journal of Mathematical Behavior,40, 6–28. https://doi.org/10.1016/j.jmathb.2015.03.007
8.
BaileyA. L.HeritageM. (2014).
The role of language learning progressions in improved instruction and assessment of English language learners. TESOL Quarterly,48(3), 480–506. https://doi.org/10.1002/tesq.176
BaileyA. L.HeritageM.ButlerF. A. (2014).
Developmental considerations and curricular contexts in the assessment of young language learners. In A. J.Kunnan (Ed.), The companion to language assessment (pp.423–439).
Wiley. https://doi.org/10.1002/9781118411360.wbcla079
11.
ButlerY. G. (2019).
Assessment of young English learners in instructional settings. In X. Gao (Ed.), Second handbook of English language teaching.
Springer. https://doi.org/10.1007/978-3-030-02899-2_24
12.
ButlerY. G. (2015).
English language education among young learners in East Asia: A review of current research (2004–2014). Language Teaching,
48(3), 303–342. https://doi.org/10.1017/S0261444815000105
13.
ButlerY. G. (2014).
Parental factors in children’s motivation for learning English: A case in China. Research Papers in Education,
30(2), 164–191. https://doi.org/10.1080/02671522.2014.891643
14.
ChenI. W.-L.HsiehJ. J.-C. (2011).
English language in Taiwan: An examination of its use in society and education in schools. In A.Feng (Ed.), English language education across Greater China (pp.70–94).
Multilingual Matters.
15.
ChikA.BesserS. (2011).
International language test taking among young learners: A Hong Kong case study. Language Assessment Quarterly,
8(1), 73–91. https://doi.org/10.1080/15434303.2010.537417
16.
Council of Europe. Council for Cultural Co-operation. Education Committee. Modern Languages Division. (2001). Common European Framework of Reference for Languages: Learning, teaching, assessment.
Cambridge University Press.
17.
Dimitrova-Galaczi, E. (2004). Peer-peer interaction in a paired speaking test: The case of the First Certificate in English [Unpublished doctoral dissertation]. Teachers College, Columbia University, USA.
18.
DucasseA. M.BrownA. (2009).
Assessing paired orals: Raters’ orientation to interaction. Language Testing,
26(3), 423–443. https://doi.org/10.1177/0265532209104669
EllisR. (2008). The study of second language acquisition (2nd ed.).
Oxford University Press.
21.
GoralD.BaileyA. L. (2019).
Student self-assessment of oral explanations: Use of language learning progressions. Language Testing, 36(3), 391–417. https://doi.org/10.1177/0265532219826330
22.
GuL.LockwoodJ.PowersD. E. (2015).
Evaluating the TOEFL Junior® Standard Test as a measure of progress for young English language learners. ETS Research Report Series, 2015(2), 1–13. https://doi.orgC/10.1002/ets2.12064
23.
HarlenW. (2005).
Trusting teachers’ judgement: Research evidence of the reliability and validity of teachers’ assessment used for summative purposes. Research Papers in Education,
20(3), 245–270. https://doi.org/10.1080/02671520500193744
24.
HasselgreenA. (2004). Testing the spoken English of young Norwegians.
Cambridge University Press.
25.
HasselgreenA.CaudwellG. (2016). Assessing the language of young learners.
Equinox.
26.
HsiehC. N.WangY. (2019).
Speaking proficiency of young language students: A discourse-analytic study. Language Testing,
36(1), 27–50. https://doi.org/10.1177/0265532217734240
27.
HuangB. H. (2016).
A synthesis of empirical research on the linguistic outcomes of early foreign language instruction. International Journal of Multilingualism, 13(3), 257–273. https://doi.org/10.1080/14790718.2015.1066792
28.
HuangB. H. (2014).
The effects of age on second language grammar and speech production. Journal of Psycholinguistic Research,
43(4), 397–420. https://doi.org/0.1007/s10936-013-9261-7
29.
HuangB.ChangY. H. S.NiuL.ZhiM. (2018).
Examining the effects of socio-economic status and language input on adolescent English learners’ speech production outcomes. System,
73, 27–36. https://doi.org/10.1016/j.system.2017.07.004
30.
HuangB. H.ChangY. H. S.ZhiM.NiuL. (2020).
The effect of input on bilingual adolescents’ long-term language outcomes in a foreign language instruction context. International Journal of Bilingualism, 24(1), 8–25. https://doi.org/10.1177/1367006918768311
31.
HungY. J.SamuelsonB. L.ChenS. C. (2016).
Relationships between peer- and self-assessment and teacher assessment of young EFL learners’ oral presentations. In M. Nikolov (Ed.), Assessing young learners of English: Global and local perspectives (pp. 317–338).
Springer. https://doi.org/10.1007/978-3-319-22422-0_13
32.
KaneM. T. (2013).
Validating the interpretations and uses of test scores. Journal of Educational Measurement,
50(1), 1–73. https://doi.org/10.1111/jedm.12000
33.
Kondo-BrownK. (2004).
Investigating interviewer‐candidate interactions during oral interviews for child L2 learners. Foreign Language Annals,
37(4), 601–613. https://doi.org/10.1111/j.1944-9720.2004.tb02426.x
34.
Kondo-BrownK. (2002).
A longitudinal evaluative study on child JFL learners’ oral performances. Japanese Language and Literature,
36(2),171–199. https://doi.org/10.2307/3250885
35.
KunnanA. J. (2000).
Fairness and justice for all. In A. J. Kunnan (Ed.), Fairness and validation in language assessment: Selected papers from the 19th language testing research colloquium, Orlando, Florida (pp.1–14).
Cambridge University Press.
36.
LeeS.WinkeP. (2018).
Young learners’ response processes when taking computerized tasks for speaking assessment. Language Testing,
35(2), 239–269. https://doi.org/10.1177/0265532217704009
37.
MayoM. D. P. G. (Ed.). (2017). Learning foreign languages in primary school: Research insights.
Multilingual Matters. https://doi.org/10.21832/GARCIA8101
38.
MessickS. (1989).
Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13– 103).
Macmillan.
MuñozC. (2011).
Input and long-term effects of starting age in foreign language learning. International Review of Applied Linguistics in Language Teaching,
49(2), 113–133. https://doi.org/10.1515/iral.2011.006
41.
MuñozC. (2014).
Contrasting effects of starting age and input on the oral performance of foreign language learners. Applied Linguistics,
35(4), 463–482. https://doi.org/10.1093/applin/amu024
42.
MuthénL. K.MuthénB. (1998). Mplus: The comprehensive modelling program for applied researchers: User’s guide.
Muthén & Muthén.
43.
NikolovM. (2016).
A framework for young EFL learners’ diagnostic assessment: ‘Can do statements’ and task types. In M. Nikolov (Ed.), Assessing young learners of English: Global and local perspectives (pp.65–92).
Springer. https://doi.org/10.1007/978-3-319-22422-0
44.
PapageorgiouS.ChoY. (2014).
An investigation of the use of TOEFL® Junior™ Standard scores for ESL placement decisions in secondary education. Language Testing,
31(2), 223–239. https://doi.org/10.1177/0265532213499750
45.
PappS.RixonS. (2018). Examining young learners: Research and practice in assessing the English of school-age learners (
Vol. 47).
Cambridge University Press.
46.
PappS.WalczakA. (2016).
The development and validation of a computer-based test of English for young learners: Cambridge English young learners. In M. Nikolov (Ed.), Assessing young learners of English: Global and local perspectives (pp.139–190).
Springer. https://doi.org/10.1007/978-3-319-22422-0
47.
RichS. (2014).
Taking stock; where are we now with TEYL? In R. S.Rich (Ed.), International perspectives on teaching English to young learners (pp. 1–22).
Palgrave Macmillan. https://doi.org/10.1057/9781137023230_1
48.
RoeverC.McNamaraT. (2006).
Language testing: The social dimension. International Journal of Applied Linguistics,
16(2), 242–258. https://doi.org/10.1111/j.1473- 4192.2006.00117.x
49.
SchleppegrellM. J. (2004). The language of schooling: A functional linguistics perspective.
Routledge.
50.
SkehanP. (1998). A cognitive approach to language learning.
Oxford University Press.
51.
SingletonD. (2005).
The critical period hypothesis: A coat of many colours. International Review of Applied Linguistics in Language Teaching,
43(4), 269–285. https://doi.org/10.1515/iral.2005.43.4.269
52.
SnowC. E.UccelliP. (2009).
The challenge of academic language. In D. R. Olson & N. Torrance (Eds.), The Cambridge handbook of literacy (pp.112–133).
Cambridge University Press. https://doi.org/10.1017/CBO9780511609664.008
53.
SwainM. (1997).
The output hypothesis, focus on form, and second language learning. In V. Berry, B. Adamson, & W. Littlewood (Eds.), Applying linguistics: Insights into language in education (pp. 1–21).
English Centre, University of Hong Kong.
54.
Szpotowicz, M., & Campfield, D. E. (2016). Developing and piloting proficiency tests for Polish young learners. In Nikolov, M. (Ed.), Assessing young learners of English: Global and local perspectives (pp. 109–137). Springer. https://doi.org/10.1007/978-3-319-22422-0
55.
VygotskyL. (1978).
Interaction between learning and development. In M. Gauvian & M. Cole (Eds.), Readings on the development of children (4th ed., pp. 34–41).
Worth.
56.
Zheng, Y., & Berry, V. (2015). Aptis for teens: Analysis of pilot test data. British Council.
