The present study examines the relative importance of attributes within and across items by applying four cognitive diagnostic assessment models. The current study utilizes the function of the models that can indicate inter-attribute relationships that reflect the response behaviors of examinees to analyze scored test-taker responses to four forms of TOEFL reading and listening comprehension sections. The results are interpreted to determine whether the subskills defined in each subtest contribute equally within an item and across items. The study also discusses whether the empirical results support the claims for compensatory processing of L2 comprehension skills and presents practical implications of the findings. The article concludes with the limitations of the study and suggestions for future research.
BrindleyG. (1997). Investigating second language listening ability: Listening skills and item difficulty. In BrindleyG.WigglesworthG. (Eds.), Access: Issues in language test design and delivery (pp. 65–85). Sydney, Australia: National Centre for English Language Teaching and Research, Macquarie University.
BuckG. (1990). The testing of second language listening comprehension. Unpublished doctoral dissertation, University of Lancaster.
12.
BuckG.TatsuokaK. (1998). Application of the rule-space procedure to language testing: Examining attributes of a free response listening test. Language Testing, 15(2), 119–157.
13.
de la TorreJ.DouglasJ. (2004). Higher-order latent trait models for cognitive diagnosis. Psychometrika, 69, 333–353.
14.
DunkelP.HenningG.ChaudronC. (1993). The assessment of an L2 listening comprehension construct: A tentative model for test specification and development. The Modern Language Journal, 77(2), 180–191.
15.
FieldJ. (1998). Skills and strategies: Towards a new methodology for listening. ELT Journal, 52, 110–118.
16.
FieldJ. (2008). Listening in the language classroom. Cambridge: Cambridge University Press
17.
FieldJ. (2013). Cognitive validity. In GeranpayehA.TaylorL. (Eds.), Examining listening (pp. 77–151). Cambridge: Cambridge University Press.
18.
FilpJ. (1975). Relationship among reading subskills: A hierarchical hypothesis. Journal of Literacy Research, 7(3), 229–240.
19.
HartzS. M. (2002). A Bayesian framework for the unified model for assessing cognitive abilities: Blending theory with practicality. Unpublished doctoral dissertation, University of Illinois at Urbana-Champaign, Urbana, IL.
20.
HensonR.TemplinJ. (2007, April). Large-scale language assessment using cognitive diagnosis models. Paper presented at the annual meeting of the National Council for Measurement in Education, Chicago, IL.
21.
HessamyG.SadeghiS. (2013). The relative difficulty and significance of reading skills. International Journal of English Language Education, 1(3), 208–222.
22.
HoribaY. (1996). Comprehension processes in L2 reading: Language competence, textual coherence, and inferences. Studies in Second Language Acquisition, 18, 403–432.
23.
HudsonT. (2007). Teaching second language reading. Oxford: Oxford University Press.
24.
JayS. S. (1975, May). Hierarchical subskills in the reading acquisition process. Paper presented at the Annual Meeting of the International Reading Association, New York. Retrieved from ERIC database. (ED105430).
25.
JunkerB. W.SijtsmaK. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25, 258–272.
26.
KnightC. C.NelsenE. A. (1982, August). Hierarchical relationships among components of reading abilities of beginning readers. Paper presented at the Annual Meeting of the American Psychological Association, Washington, DC.
27.
KodaK. (2007). Reading and language learning: Crosslinguistic constraints on second language reading development. Language Learning, 57(1), 1–44.
28.
KontosS. (1988). Development and interrelationships of reading knowledge and skills during kindergarten and first grade. Reading Research and Instruction, 27(2), 13–28.
29.
LandiN. (2010). An examination of the relationship between reading comprehension, higher-level and lower-level reading sub-skills in adults. Reading & Writing, 23, 701–717.
30.
LeeY.-W.SawakiY. (2009). Application of three cognitive diagnosis models to ESL reading and listening assessments. Language Assessment Quarterly, 6(3), 239–263.
31.
LumleyT. (1993). The notion of sub-skills in reading comprehension tests: An EAP example. Language Testing, 10(3), 211–234.
32.
LundR. J. (1990). A taxonomy for teaching second language listening. Foreign Language Annals, 23(2), 105–115.
33.
McNeilL. (2012). Extending the compensatory model of second language reading. System, 40, 64–76.
34.
MecarttyF. H. (1998). The effects of proficiency level and passage content on reading skills assessment. Foreign Language Annals, 31(4), 517–534.
35.
MunbyJ. (1978). Communicative syllabus design. Cambridge: Cambridge University Press.
36.
NassajiH. (2003). Higher-level and lower-level text processing skills in advanced ESL reading comprehension. The Modern Language Journal, 87(2), 261–276.
OmaggioA. C. (1986). Teaching language in context: Proficiency-oriented instruction. Boston, MA: Heinle and Heinle.
39.
PetersonP.W. (1991). A synthesis of methods for interactive listening. In Celce-MurciaM. (Ed.), Teaching English as a second or foreign language (2nd ed.) (pp. 106–122). New York: Newbury House.
40.
PowersD. E. (1986). Academic demands related to listening skills. Language Testing, 3, 1–38.
RostM. (2011). Teaching and researching listening. Harlow, UK: Pearson.
43.
RuppA.TemplinJ.HensonR. (2010). Diagnostic measurement: Theory, methods, and applications. New York: The Guildford Press.
44.
SawakiY.KimH. J.GentileC. (2009). Q-Matrix construction: Defining the link between constructs and test items in large-scale reading and listening comprehension assessments. Language Assessment Quarterly, 6(3), 190–209.
45.
SongM.-Y. (2008). Do divisible subskills exist in second language (L2) comprehension? A structural equation modeling approach. Language Testing, 25(4), 435–464.
46.
StanovichK. E. (1980). Toward an interactive-compensatory model of individual differences in the development of reading fluency. Reading Research Quarterly, 16(1), 32–71.
