This study investigated the relationship between latent components of academic English language ability and test takers’ study-abroad and classroom learning experiences through a structural equation modeling approach in the context of TOEFL iBT® testing. Data from the TOEFL iBT public dataset were used. The results showed that test takers’ performance on the test’s four skill sections, namely listening, reading, writing, and speaking, could be accounted for by two correlated latent components: the ability to listen, read, and write, and the ability to speak English. This two-factor model held equivalently across two groups of test takers, with one group having been exposed to an English-speaking environment and the other without such experience. Imposing a mean structure on the factor model led to the finding that the groups did not differ in terms of their standings on the factor means. The relationship between learning contexts and the latent ability components was further examined in structural regression models. The results of this study suggested an alternative characterization of the ability construct of the TOEFL test-taking population, and supported the comparability of the language ability developed in the home-country and the study-abroad groups. The results also shed light on the impact of studying abroad and home-country learning on language ability development.
AldersonJ. C.BanerjeeJ. (2002). Language testing and assessment (Part 2). Language Teaching, 35, 79–113.
2.
BachmanL. F. (1990). Fundamental considerations in language testing. Oxford: Oxford University Press.
3.
BachmanL. F. (2000). Modern language testing at the turn of the century: Assuring that what we count counts. Language Testing, 17(1), 1–42.
4.
BachmanL. F.DavidsonF.RyanK.ChoiI.-C. (1995). An investigation into the comparability of two tests of English as a foreign language: The Cambridge-TOEFL comparability study. New York: Cambridge University Press.
5.
BachmanL. F.PalmerA. S. (1982). The construct validation of some components of communicative proficiency. TESOL Quarterly16(4), 449–465.
6.
BachmanL. F.PalmerA. S. (1983). The construct validation of the FSI oral interview. In OllerJ. W.Jr (Ed.), Issues in language testing research (pp. 154–169). Rowley, MA: Newbury House.
7.
BachmanL. F.PalmerA. S. (1996). Language testing in practice: Designing and developing useful language tests. Oxford: Oxford University Press.
8.
BaeJ.BachmanL. F. (1998). A latent variable approach to listening and reading: Testing factorial invariance across two groups of children in the Korean/English two-way immersion program. Language Testing, 15(3), 380–414.
9.
BandalosD. L. (2002). The effects of item parceling on goodness-of-fit and parameter estimate bias in structural equation modeling. Structural Equation Modeling, 9(1), 78–102.
10.
BandalosD. L.FinneyS. J. (2001). Item parceling issues in structural equation modeling. In MarcoulidesG. A.SchumackerR. E. (Eds.), New developments and techniques in structural equation modeling (pp. 269–296). Mahwah, NJ: Lawrence Erlbaum.
11.
BrowneM. W.CudeckR. (1993). Alternative ways of assessing model fit. In BollonK. A.LongJ. S. (Eds.), Testing structural equation models (pp. 136–162). Newbury Park, CA: Sage.
12.
BuckG. (1992). Listening comprehension: Construct validity and trait characteristics. Language Learning, 42(3), 313–357.
13.
CarrollJ. B. (1965). Fundamental consideration in testing for English language proficiency of foreign students. In AllenH. B. (Ed.), Teaching English as a second language: A book of readings (pp. 364–372). New York: McGraw-Hill.
14.
Chalhoub-DevilleM. (1997). Theoretical models, assessment frameworks and test construction. Language Testing, 14(1), 3–22.
15.
CheungG. W.RensvoldR. B. (2002). Evaluating goodness-of-fit indexes of testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 9(2), 233–255.
16.
CollentineJ. (2004). The effects of learning contexts on morphosyntactic and lexical development. Studies of Second Language Acquisition, 26, 227–248.
17.
CollentineJ.FreedB. (2004). Learning context and its effects on second language acquisition. Studies of Second Language Acquisition, 26, 153–171.
18.
DeweyD. P. (2004). A comparison of reading development by learners of Japanese in intensive domestic immersion and study abroad contexts. Studies of Second Language Acquisition, 26, 303–327.
19.
Díaz-CamposM. (2004). Context of learning in the acquisition of Spanish second language phonology. Studies of Second Language Acquisition, 26, 249–273.
20.
FoulyK. A.BachmanL. F.CzikoG. A. (1990). The divisibility of language competence: A confirmatory approach. Language Learning, 40(1), 1–21.
21.
FreedB. F.SegalowitzN.DeweyD. P. (2004). Context of learning and second language fluency in French: Comparing regular classroom, study abroad, and intensive domestic immersion programs. Studies of Second Language Acquisition, 26, 275–301.
22.
GintherA.StevensJ. (1998). Language background, ethnicity, and the internal construct validity of the Advanced Placement Spanish Language Examination. In KunnanA. J. (Ed.), Validation in language assessment (pp. 169–194). Mahwah, NJ: Lawrence Erlbaum.
23.
HarleyB.CumminsJ.SwainM.AllenP. (1990). The nature of language proficiency. In HarleyB.AllenP.CumminsJ.SwainM. (Eds.), The development of second language proficiency (pp. 7–25). New York: Cambridge University Press.
24.
HuL.-T.BentlerP. M. (1999). Cutoff criteria for fit indices in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55.
25.
KachruB. B. (1984). World Englishes and the teaching of English to non-native speakers: Contexts, attitudes, and concerns. TESOL Newsletter, 18, 25–26.
26.
KlineR. B. (2005). Principles and practice of structural equation modeling (2nd ed.). New York: Guilford.
27.
KunnanA. J. (1992). An investigation of a criterion-referenced test using G-theory, and factor and cluster analyses. Language Testing, 9, 30–49.
28.
KunnanA. J. (1998). Approach to validation in language assessment. In KunnanA. J. (Ed.), Validation in language assessment (pp. 1–16). Mahwah, NJ: Lawrence Erlbaum.
29.
LaffordB. A. (2004). The effect of the context of learning on the use of communication strategies by learners of Spanish as a second language. Studies of Second Language Acquisition, 26, 201–225.
30.
LittleT. D.CunninghamW. A.ShaharG.WidamanK. F. (2002). To parcel or not to parcel: Exploring the question, weighing the merits. Structural Equation Modeling, 9(2), 151–173.
31.
MessickS. (1989). Validity. In LinnR. L. (Ed.), Educational measurement (3rd ed.) (pp. 13–103). New York: American Council on Education and Macmillan.
32.
MorganR.MazzeoJ. (1988). A Comparison of the structural relationships among reading, listening, writing, and speaking components of the AP French Language Examination for AP candidates and college students. (ETS Research Report No. 88–59). Princeton, NJ: Educational Testing Service.
33.
MuthénL. K.MuthénB. O. (2010). Mplus User’s Guide (6th ed.). Los Angeles, CA: Muthén & Muthén.
34.
OllerJ. W.Jr. (1979). The factorial structure of language proficiency: Divisible or not? In OllerJ. W.Jr. Language test at school: A pragmatic approach (pp. 423–458). London: Longman.
35.
RömhildA. (2008). Investigating the invariance of the ECPE factor structure across different proficiency levels. Spaan Fellow Working Papers in Second or Foreign Language Assessment, 6, 29–55. Ann Arbor, MI: University of Michigan English Language Institute. www.isa.umich.edu/eli/research/spaan
36.
SangF.SchmitzB.VollmerH. J.BaumertJ.RoederP. M. (1986). Models of second language competence: A structural equation approach. Language Testing, 3(1), 54–79.
37.
SasakiM. (1993). Relationships among second language proficiency, foreign language aptitude, and intelligence: A structural equation modeling approach. Language Learning, 43(3), 313–344.
38.
SasakiM. (2007). Effects of study-abroad experiences on EFL writers: A multiple-data analysis. The Modern Language Journal, 91(4), 602–620.
39.
SatorraA.BentlerP. M. (1994). Corrections to test statistics and standard errors on covariance structure analysis. In von EyeA.CloggC. C. (Eds.), Latent variables analysis (pp. 399–419). Thousand Oaks, CA: Sage.
40.
SawakiY.StrickerL.OranjeA. (2008). Factor structure of the TOEFL Internet-Based Test (iBT): Exploration in a field trial sample. (TOEFL iBT Research Report No. 04; ETS Research Report No. 08–09). Princeton, NJ: Educational Testing Service.
41.
ShinS.-K. (2005). Did they take the same test? Examinee language proficiency and the structure of language tests. Language Testing, 22(1) 31–57.
42.
SteenkampJ. E. M.BaumgartarH. (1998). Assessing measurement invariance in cross-national consumer research. Journal of Consumer Research, 2, 78–90.
43.
StrickerL. J.RockD. A. (2008). Factor structure of the TOEFL Internet-Based Test across subgroups. (TOEFL iBT Research Report No. 07; ETS Research Report No. 08–66). Princeton, NJ: Educational Testing Service.
44.
StrickerL. J.RockD. A.LeeY.-W. (2005). Factor structure of the LanguEdge™ Test across language groups. (TOEFL Monograph Series Report No. 32). Princeton, NJ: Educational Testing Service.
45.
SwintonS. S.PowersD. E. (1980). Factor analysis of the Test of English as a Foreign Language for several language groups. (TOEFL Research Report No. 06; ETS Research Report No. 80–32). Princeton, NJ: Educational Testing Service.
46.
VandenbergR. J.LanceC. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4–70.
47.
WangS. D. (2006). Validation and invariance of factor structure of the ECPE and MELAB across gender. Spaan Fellow Working Papers in Second or Foreign Language Assessment, 4, 41–56. Ann Arbor, MI: University of Michigan English Language Institute. www.isa.umich.edu/eli/research/spaan
48.
WolfM. K.KaoJ.HermanJ.BachmanL. F.BaileyA.BachmanP. L.FarnsworthT.ChangS. M. (2008). Issues in assessing English language learners: English language proficiency measures and accommodation uses—Literature review (Part 1 of 3). (CRESST Report No. 731). Los Angeles, CA: CRESST/UCLA.
