Previous research in cognitive diagnostic assessment (CDA) of L2 reading ability has been frequently conducted using large-scale English proficiency exams (e.g., TOEFL, MELAB). Using CDA, it is possible to analyze individual learners’ strengths and weaknesses in multiple attributes (i.e., knowledge, skill, strategy) measured at the item level. This study explored how a placement test score could be used for diagnosing the L2 reading ability of incoming students to an adult ESL program. Five content experts completed a reading placement test and identified the attributes required for successfully completing each item on the test, while referring to the list of L2 reading attributes. Then, content experts’ codings were analyzed and developed into an item-by-attribute Q-matrix. The Fusion model, a type of cognitive diagnostic model (CDM), was used for refining the Q-matrix and diagnosing 1982 learners’ strengths and weaknesses in L2 reading. Results suggest that 10 major L2 reading attributes were involved in the reading test. In addition, examinees’ strengths and weaknesses were identified for the overall group, three reading proficiency groups (i.e., beginner, intermediate, and advanced), and individual learners. Such information could be provided to ESL program administrator and teachers for enhancing the reading curriculum and developing instructional materials.
AldersonJ. C. (2000). Assessing reading. Cambridge: Cambridge University Press.
2.
AldersonJ. C. (2005). Diagnosing foreign language proficiency. New York: Continuum.
3.
ALTE. (1998). Multilingual glossary of language testing terms. Cambridge: Cambridge University Press.
4.
AmeriksY. S. (2009). Investigating validity across two test forms of the examination for the certificate of proficiency in English (ECPE): A multi-group structural equation modeling approach. Available from ProQuest Dissertations and Theses database. (AAT 3348566)
5.
BachmanL. F. (1990). Fundamental considerations in language testing. Oxford: Oxford University Press.
6.
BachmanL. F.PalmerA. S. (1996). Language testing in practice. Oxford: London.
7.
BirenbaumM.KellyA. E.TatsuokaK. K. (1993). Diagnosing knowledge states in algebra using the rule-space model. Journal of Research in Mathematics Education, 24, 442–459.
8.
BuckG.TatsuokaK.KostinI. (1997). The subskills of reading: Rule-space analysis of a multiple-choice test of second language reading comprehension. Language Learning, 47(3), 423–266.
9.
Celce-MurciaM.Larsen-FreemanD. (1999). The grammar book: An ESL/EFL teacher’s course. Boston, MA: Heinle and Heinle.
10.
ChipmanS. F.NicholsP. D.BrennanR. L. (1995). Introduction. In ChipmanS. F.NicholsP. D.BrennanR. L. (Eds.), Cognitively diagnostic assessment (pp. 1–18). Hillsdale, NJ: Lawrence Erlbaum.
11.
CohenA. D. (2007). Coming to terms with language learner strategies: Surveying the experts. In CohenA. D.MacaroE. (Eds.), Language learner strategies: 30 years of research and practice (pp. 29–45). Oxford, UK: Oxford University Press.
12.
CohenA. D.GlasmanH.Rosenbaum-CohanP. R.FerraraJ.FineJ. (1979). Discourse analysis and the use of student informants. TESOL Quarterly, 13, 551–564.
13.
CohenA. D.UptonT. A. (2006). Strategies in responding to the new TOEFL reading tasks (TOEFL Monograph No. MS-33). Princeton, NJ: ETS.
14.
DakinJ. W. (2010). Investigating the measurement of grammatical knowledge and civics content knowledge in the context of an English-for-specific-purposes program designed for adult learners with low English proficiency. Available from ProQuest Dissertations and Theses database. (AAT 3436226)
15.
de BeaugrandeR. (1980). Text, discourse and process: Toward a multidisciplinary science of texts. London: Longman.
16.
de la TorreJ. (2009). DINA model and parameter estimation: A didactic. Journal of Educational and Behavioral Statistics, 34(1), 115–130.
17.
DiBelloL. V.RoussosL. A.StoutW. (2007). Review of cognitively diagnostic assessment and a summary of psychometric models. RaoC. R.SinharayS. (Eds.), Handbook of statistics, Vol. 26: Psychometrics (pp. 970–1030). Amsterdam: North-Holland Publications.
18.
DiBelloL. V.StoutW. (2008). Arpeggio documentation and analyst manual. St. Paul: Applied Informative Assessment Research Enterprises (AIARE).
19.
FaerchC.KasperG. (1983). Strategies in interlanguage communication. London: Longman.
20.
FalmagneJ. C.DoignonJ. P. (1988). A class of stochastic procedures for assessment of knowledge. British Journal of Mathematical and Statistical Psychology, 41, 1–23.
21.
GrabeW. (2009). Reading in a second language: Moving from theory to practice. Cambridge: Cambridge University Press.
22.
GrabeW.StollerF. L. (2002). Teaching and researching: reading. Harlow: Longman.
23.
GrabowskiK. C. (2009). Investigating the construct validity of a test designed to measure grammatical and pragmatic knowledge in the context of speaking. Available from ProQuest Dissertations and Theses database. (AAT 3368256)
24.
GrayW. S. (1960). The major aspects of reading In RobinsonH. (Ed.), Sequential development of reading abilities (pp. 8–24). Chicago, IL: Chicago University Press.
25.
HabermanS.von DavierM. (2006). Some notes on models for cognitively based skills diagnosis. In RaoC.SinharayS. (Eds.), Handbook of statistics (pp. 1031–1038). Amsterdam: Elsevier.
26.
HaertelE. H.WileyD. E. (1993). Representation of ability structures: Implications for testing. In FrederiksenN.MislevyR.BejarI. (Eds.), Test theory for a new generation of tests (pp. 359–384). Hillsdale, NJ: Lawrence Erlbaum.
27.
HallidayM. A. K.HasanR. (1976). Cohesion in English. London: Longman.
28.
HammadouJ. (1991). Interrelationships among prior knowledge, inference, and language proficiency in foreign language reading. The Modern Language Journal, 75, 27–38.
29.
HartzS. M.RoussosL.StoutW. (2002). Skills diagnosis: Theory and practice. Unpublished manuscript. University of Illinois at Urbana-Champaign.
30.
HerberH. (1978). Teaching reading in the content areas. Englewood Cliffs, NJ: Prentice Hall.
31.
HosenfieldC. (1977). Learning about learning: Discovering our students’ strategies. Foreign Language Annals, 9, 117–129.
32.
HughesA. (1989). Testing for language teachers. Cambridge: Cambridge University Press.
33.
JangE. E. (2005). A validity narrative: Effects of reading skills diagnosis on teaching and learning in the context of NG TOEFL. Available from ProQuest Dissertations and Theses database. (AAT 3182288)
34.
JohnsonJ. F. (2006). Diagnosing skill mastery in the national assessment of educational progress: Applications of the Fusion Model. Available from ProQuest Dissertations and Theses database. (AAT 3223309)
35.
KasaiM. (1997). Application of the rule space model to the reading comprehension section of the test of English as a foreign language (TOEFL). Available from ProQuest Dissertations and Theses database. (AAT 9737153)
36.
LandisJ. R.KochG. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159–174.
37.
LeeY.-W.SawakiY. (2009a). Cognitive diagnosis and Q-matrices in language assessment. Language Assessment Quarterly, 6, 169–171.
38.
LeeY.-W.SawakiY. (2009b). Cognitive diagnosis approaches to language assessment: An overview. Language Assessment Quarterly, 6, 172–189.
39.
LeightonJ. P.GierlM. J. (2007). Cognitive diagnostic assessment for education: Theory and applications. New York: Cambridge University Press.
40.
LiH. (2011). A cognitive diagnostic analysis of the MELAB reading test. Spaan Fellow Working Papers in Second or Foreign Language Assessment, 9, 17–46.
41.
LiaoY.-F. (2009). A construct validation study of the GEPT reading and listening sections: Re-examining the models of L2 reading and listening abilities and their relations to lexico-grammatical knowledge. Available from ProQuest Dissertations and Theses database. (AAT 3348356)
42.
LongD. L.SeelyM. R.OppyB. J.GoldingJ. M. (1996). The role of inferential processing in reading ability. In BrittonB. K.GraesserA. C. (Eds.), Models of understanding text (pp. 189–214). Hillsdale, NJ: Lawrence Erlbaum.
43.
LumleyT. (1993). The notion of subskills in reading comprehension test: An EAP example. Language Testing, 10(3), 211–234.
44.
McGlohenM. K. (2004). The application of cognitive diagnosis and computerized adaptive testing. Available from ProQuest Dissertations and Theses database. (AAT 3145337)
45.
O’MalleyM. J.ChamotA. U. (1990). Learning strategies in second language acquisition. Cambridge: Cambridge University Press.
46.
OxfordR.ChoY.LeungS.KimH.-J. (2004). Effect of the presence and difficulty of task on strategy use: An exploratory study. International Review of Applied Linguistics, 42, 1–27.
47.
PhakitiA. (2007). Strategy competence and EFL reading test performance. New York: Peter Lang.
48.
PressleyM. (2002). Comprehension strategy instruction: A turn-of-the-century status report. In BlockC.PressleyM. (Eds.), Comprehension instruction: Research-based best practices (pp. 11–27). New York: Guilford Press.
49.
PressleyM.AfflerbachP. (1995). Verbal protocols of reading: The nature of constructively responsive reading. Hillsdale, NJ: Lawrence Erlbaum.
50.
PurpuraJ. E. (1999). Learner strategy use and performance on language tests: A structural equation modeling approach. Cambridge: Cambridge University Press.
51.
PurpuraJ. E. (2004). Assessing grammar. Cambridge: Cambridge University Press.
52.
RoussosL. A.DiBelloL.V.StoutW. F.HartzS. M.HensonR. A.TemplinJ. H. (2007). The fusion model skills diagnostic system. In LeightonJ.GierlM. (Eds.), Cognitive diagnostic assessment for education: Theory and applications. New York: Cambridge University Press.
53.
RoussosL.TemplinJ.HensonR. (2007). Skills diagnosis using IRT-based latent class models. Journal of Educational Measurement, 44, 293–311.
54.
SawakiY.KimH.-J.GentileC. (2009). Q-matrix construction: Defining the link between constructs and test items in large-scale reading and listening comprehension assessments. Language Assessment Quarterly, 6, 190–209.
55.
ScottH. S. (1998). Cognitive diagnostic perspectives of second language reading test. Available from ProQuest Dissertations and Theses database. (AAT 39912372)
56.
StanovichK. E. (1980). Toward an interactive-compensatory model of individual differences in the development of reading fluency. Reading Research Quarterly, 16(1), 32–71.
57.
TatsuokaK. K. (1990). Toward an integration of item response theory and cognitive analysis. In FrederiksenN.GlaserR.LesgoldA.ShaftoM. C. (Eds.), Diagnostic monitoring of skill and knowledge acquisition (pp. 543–588). Hillsdale, NJ: Lawrence Erlbaum.
58.
TemplinJ. L. (2004). Generalized linear mixed proficiency models for cognitive diagnosis. Available from ProQuest Dissertations and Theses database. (AAT 3160960)
59.
UrquhartS.WeirC. J. (1998). Reading in a second language: Process, product and practice. New York: Longman.
60.
WeirC. J.HawkeyR.GreenA.DeviS. (2009). The cognitive processes underlying the academic reading construct as measured by IELTS (IELTS Research Reports No. 9, pp. 157–189). Cambridge: IELTS.
61.
WiersmaW. (2000). Research methods in education (7th ed.). Boston, MA: Allyn and Bacon.
