This paper concerns the divisibility of comprehension subskills measured in L2 listening and reading tests. Motivated by the administration of the new Web-based English as a Second Language Placement Exam (WB-ESLPE) at UCLA, this study addresses the following research questions: first, to what extent do the WB-ESLPE listening and reading items measure different comprehension subskills? Second, to what extent can L2 listening and reading be considered similar or different, with regard to the divisibility of comprehension subskills? Employing a structural equation modeling (SEM) approach, the factor structures of 110 test takers' scores on the WB-ESLPE listening and reading items were examined with regard to three types of subskills, that is, `understanding main/topical ideas', `understanding supporting/specific details', and `making inferences'. The results of the analyses indicated that the WB-ESLPE listening and reading items measure two or three kinds of subskills, and that while L2 listening and reading might share a common comprehension process, they may be distinct in the decoding processes involved due to the difference in mode of presentation. It is suggested that this divisibility of subskills in L2 comprehension tests might depend on the test takers' L2 proficiency as well as on the task characteristics of the test administered to them.
Alderson, J.C., & Lukmani, Y. (1989). Cognition and reading: Cognitive levels as embodied in test questions. Reading in a Foreign Language, 5, 253-270.
5.
Bachman, L.F., & Palmer, A. (1981). The construct validation of the FSI oral interview . Language Learning, 31, 67-86.
6.
Bachman, L.F., & Palmer, A. (1982). The construct validation of some components of communicative proficiency. TESOL Quarterly, 16, 449-465.
7.
Bachman, L.F., Davidson, F.G., & Foulkes, J. (1990). A comparison of the abilities measured by the Cambridge and Educational Testing Service EFL Test Batteries. Issues in Applied Linguistics, 1, 30-55.
8.
Bachman, L.F., Davidson, F.G., Ryan, K., & Choi, I-C. (1995). An investigation into the comparability of two tests of English as a foreign language: The Cambridge-TOEFL comparability study. Cambridge: UCLES.
9.
Bae, J., & Bachman, L.F. (1998). A latent variable approach to listening and reading: Testing factorial invariance across two groups of children in the Korean/English Two-way Imersion program. Language Testing, 15(3), 380-414.
10.
Bandalos, D., & Finney, J.S. (2001). Item parceling issues in structural equation modeling . In A. G. Marcoulides & E. R. Schumacker (Eds.), New developments and techniques in structural equation modeling (pp. 269-295). Hillsdale, NJ: Lawrence Erlbaum.
11.
Bentler, P.M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107, 238-46.
Bentler, P.M., & Chou, C.-P. (1987). Practical issues in SEM. Sociological Methods and Research, 16, 78-117.
14.
Bentler, P.M., & Wu, E. (2002). EQS for Windows: User's guide. Encino, CA: Multivariate Software.
15.
Bernstein, H., & Teng, G. (1989). Factoring items and factoring scales are different: Spurious evidence for multidimensionality due to item categorization. Psychological Bulletin, 105, 467-477.
16.
Britton, B.K., & Glaesser, A.C. (1996). Models of text understanding . Hillsdale, NJ: Lawrence Erlbaum .
17.
Brown, G., & Yule, G. (1983). Discourse analysis. New York : Cambridge University Press.
18.
Brown, M.W., & Cudeck, R. (1992). Alternative ways of assessing model fit. Sociological Methods and Research, 21, 230-258.
19.
Buck, G. (1991). The testing of listening comprehension: An introspective study. Language Testing, 8(1), 67-91.
20.
Buck, G. (1992). Listening comprehension: Construct validity and trait characteristics. Language Learning, 42(3), 313-357.
21.
Buck, G. (2001). Assessing listening. Cambridge , UK: Cambridge University Press.
22.
Buck, G., & Tatsuoka, K. (1998). Application of the rule-space procedure to language testing: Examining attributes of a free response listening test. Language Testing, 15(2), 119-157.
23.
Buck, G., Tatsuoka, K., Kostin, I., & Phelps, M. (1997). The sub-skills of listening: Rule-space analysis of a multiple-choice of second language listening comprehension. In A. Huhta, V. Kohonen , L. Kurki-Suonio & S. Louma (Eds.), Current developments and alternatives in language assessment. Tampere: University of Jyväskylä.
24.
Carroll, J.B. (1972): Defining language comprehension. In R. O. Freedle & J. B. Carroll (Eds.), Language comprehension and the acquisition of knowledge. New York: John Wiley .
25.
Carroll, J.B. (1983). Psychometric theory and language testing. In J. W. Oller Jr. (Ed.), Issues in language testing research (pp. 80-107). Rowley, MA: Newbury House.
26.
Carroll, J.B. (1993). Human cognitive abilities. Cambridge: Cambridge University Press.
27.
Clark, H.H., & Clark, E.V. (1977). Psychology and language: An introduction to psycholinguistics . New York: Harcourt Brace Jovanovich .
28.
Davey, B. (1988). Factors affecting the difficulty of reading comprehension items for successful and unsuccessful readers. Experimental Education, 56, 67-76.
29.
Davis, F.B. (1968). Research in comprehension in reading. Reading Research Quarterly, 3, 499-545.
30.
Drum, P.A., Calfee, R.C., & Cook, L.K. (1981). The effects of surface structure variables on performance in reading comprehension tests. Reading Research Quarterly, 16, 486-514.
31.
Enders, K.C., & Bandalos, L.D. (1999). The effects of heterogeneous item distribution on reliability. Applied Measurement in Education , 12, 133-150.
32.
Freedle, R., & Kostin, I. (1994). Can multiple-choice reading tests be construct valid? Psychological Science, 5, 107-110.
33.
Freedle, R., & Kostin, I. (1999). Does text matter in a multiple-choice test of comprehension? The case for the construct validity of TOEFL's minitalks. Language Testing, 16(1), 2-32.
34.
Hattie, A.J. (1986). Methodology review: Assessing unidimensionality of tests and items. Applied Psychological Measurement, 9, 139-164.
35.
Hayduk, L. (1996). LISREL issues, debates, and strategies. Baltimore, MD: Johns Hopkins University Press.
36.
James, K. (1977). Notetaking in lectures: problems and strategies . In A. P. Cowie & J. B. Heaton (Eds.), English for academic purposes, Reading: BAAL/SELMOUS .
37.
Kishton, J.M., & Widaman, K.F. (1994). Unidimensional versus domain representative parceling of questionnaire items: An empirical example. Educational and Psychological Measurement, 54, 757-765.
38.
Kline, R.B. (1998). Principles and practice of structural equation modeling. London: The Guilford Press .
39.
Larsen, R., & Feder, D. (1940). Common and differential factors in reading and hearing comprehension. Journal of Educational Psychology, 32, 241-252.
40.
Lennon, R.T. (1962). What can be measured? Reading Teacher, 15, 326-337.
41.
Levine, A., & Revers, T. (1988). The FL receptive skills: Same or different? System, 16, 326-336.
42.
Little, T.D., Cunningham, W.A., Shahar, G., & Widaman, K.F. (2002). To parcel or not to parcel: Exploring the question, weighing the merits. Structural Equation Modeling: A Multidisciplinary Journal, 9(2), 151-173.
43.
Lumley, T. (1993). The notion of subskills in reading comprehension tests: An EAP example. Language Testing, 10(3), 211-234.
44.
Lund, R.J. (1991). A comparison of second language listening and reading comprehension. Modern Language Journal, 75, 196-204.
45.
Lunzer, E., & Gardner, K. (Eds.). (1979). The effective use of reading . London: Heinemann Educational Books.
46.
Munby, J. (1978). Communicative syllabus design. Cambridge: Cambridge University Press.
47.
Nasser, F., Takahashi, T., & Benson, J. (1997). The structure of test anxiety in Israeli-Arab high school students: An application of confirmatory analysis with miniscales . Anxiety, Stress, and Coping, 10, 129-151.
48.
Oakeshott-Taylor, A. (1977). Dictation as a test of listening comprehension . In R. Dirven (Ed.), Horverstandnis im Fremdsprachenunterricht. Listening comprehension in foreign language teaching . Kronberg/Ts.: Scriptor.
49.
Olsen, L.A., & Huckin, T.N. (1990). Point-driven understanding in engineering lecture comprehension. English for Specific Purposes, 9, 33-47.
50.
Pearson, P.D., & Johnson, D.D. (1978). Teaching reading comprehension. New York, NJ: Holt, Rinehart and Winston.
51.
Pollitt, A., Hutchinson, C., Entwistle, N., & DeLuca, C. (1985). What makes exam questions difficult? An analysis of `O' grade questions and answers. Edinburgh: Scottish Academic Press.
Rivers, W.M. (1966). Listening comprehension. Modern Language Journal, 50(4), 196-202.
54.
Rost, D. (1993). Assessing the difference components of reading comprehension: fact or fiction? Language Testing, 10(1), 79-92.
55.
Sang, F., Schmitz, B., Vollmer, H., Baumert, J., & Roeder, P. (1986). Models of second language competence: Structural equation approach. Language Testing, 3, 54-79.
56.
Sasaki, M. (1991). Relationships among second language proficiency, foreign language aptitude, and intelligence: A structural equation modeling approach. Unpublished doctoral dissertation, University of California, Los Angeles.
57.
Sasaki, M. (1996). Second language proficiency, foreign language aptitude, and intelligence: Quantitative and qualitative analyses. New York: Peter Lang.
58.
Shin, S. (2005). Did they take the same test? Examinee language proficiency and the structure of language tests. Language Testing, 22(1), 31-57.
59.
Spearitt, D. (1972). Identification of subskills of reading comprehension by maximum likelihood factor analysis. Reading Research Quarterly , 8, 92-111.
60.
SPSS Inc. (2002). SPSS for Windows release 11.5 standard version. Chicago, IL: SPSS Inc.
61.
Thompson, B., & Melancon, J. (1996). Using item `testlets/parcels'in confirmatory factor analysis: An example using the PPDP-78. Paper presented at the annual meeting of the Mid-South Educational Research Association, Tuscaloosa, AL.
62.
Weir, C. (1993). Understanding and developing language tests. New York: Prentice Hall.
