The validation of cognitive attributes required for correct answers on binary test items or tasks has been addressed in previous research through the integration of cognitive psychology and psychometric models using parametric or nonparametric item response theory, latent class modeling, and Bayesian modeling. All previous models, each with their advantages and disadvantages, require item score information and do not focus on conditional validation of cognitive attributes across ability levels and individual test items. This study proposes a method of estimating the probability of correct performance on cognitive attributes across fixed ability levels. The proposed method, referred to here as the least squares distance method (LSDM), is based on the minimization of matrix norms using the Euclidean least squares distance. The LSDM does not require raw or trait scores of examinees as long as IRT estimates of the item parameters are available.
Andersen, E.B. (1973). A goodness of fit test for the Rasch model. Psychometrika, 38, 123-140.
2.
Assessment Systems Corporation. (1995). User's manual for XCALIBRE marginal maximum likelihood estimation program (Windows Version 3.5). St. Paul, MN: Author.
Buck, G., Tatsuoka, K.K., & Kostin, I. (1997). The subskills of reading: Rule-space analysis of a multiple-choice test of second language reading comprehension. Language Learning, 47(3), 423-466.
5.
DiBello, L.V., Stout, W.F., & Roussos, L.A. (1995). Unified cognitive/psychometric diagnostic assessment likelihood-based classification techniques. In P. Nichol, S. Chipman, & R. Brennan (Eds.), Cognitively diagnostic assessment (pp. 361-389). Hillsdale, NJ: Lawrence Erlbaum.
6.
Embretson, S.E. (1984). A general latent trait model for response processes . Psychometrika, 49, 175-186.
7.
Embretson, S.E. (1995). A measurement model for linking individual learning to process and knowledge: Application to mathematical reasoning. Journal of Educational Measurement, 32, 277-294.
Fischer, G.H. (1973). The linear logistic test model as an instrument in educational research. Acta Psychologica, 37, 359-374.
10.
Fischer, G.H. (1995). The linear logistic test model. In G. H. Fischer & I. W. Molenaar (Eds.), Rasch models: Foundations, recent developments, and applications (pp. 131-155). New York: Springer-Verlag.
11.
Fischer, G.H. (2004). Remarks on ``equivalent linear logistic test models'' by Bechger, Verstralen, and Verhlest (2002). Psychometrika , 69(2), 305-315.
12.
Fischer, G.H., & Ponocny-Seliger, E. (1998). Structural Rasch modeling: Handbook of the usage of LPCM-WIN 1.0. Groningen, Netherlands: ProGAMMA.
13.
Gitomer, D.H., & Rock, D. (1993). Addressing process variables in test analysis . In N. Frederiksen, R. J. Mislevy, & I. Bejar (Eds.), Test theory for a new generation of tests (pp. 243-268). Hillsdale, NJ: Lawrence Erlbaum.
14.
Haertel, E.H. (1989). Using restricted latent class models to map skill structure of achievement items. Journal of Educational Measurement , 26, 301-321.
15.
Hartz, S. (2002). A Bayesian framework for the unified model for assessing cognitive abilities: Blending theory with practicality. Unpublished doctoral dissertation, University of Illinois, Urbana-Champaign .
16.
Henning, J.E. (1999). Signs of development in written composition: Identifying thinking in the expository essays of eleventh graders. Unpublished doctoral dissertation, Kent State University, Kent, OH.
17.
Henson, R., & Douglas, J. (2005). Test construction for cognitive diagnosis. Applied Psychological Measurement, 29(4), 262-277.
18.
Irvine, S.H., & Kyllonen, P.C. (2002). Item generation for test development. Mahwah, NJ: Lawrence Erlbaum.
19.
Junker, B.W. (2001). On the interplay between nonparametric and parametric IRT, with some thoughts about the future. In A. Boomsma, A. A. J. Van Duijn, & T. A. B. Snijders (Eds.), Essays on item response theory (pp. 274-276). New York: Springer-Verlag.
20.
Junker, B.W., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25(3), 258-272.
Macready, G.B., & Dayton, C.M. (1977). The use of probabilistic models in the assessment of mastery. Journal of Educational Statistics, 2(2), 99-120.
23.
Maris, E. (1999). Estimating multiple classification latent class models. Pscyhometrika, 64, 187-212.
24.
MathWorks, Inc. (2004). MATLAB. The language of technical computing (Version 7.0). Natick, MA: Author.
25.
Medina-Diaz, M. (1993). Analysis of cognitive structure using the linear logistic test model and quadratic assignment. Applied Psychological Measurement, 17, 117-130.
26.
Mislevy, R.J. (1993). Foundations of a new theory. In N. Frederiksen, R. J. Mislevy , & I. Bejar (Eds.), Test theory for a new generation of tests (pp. 19-39). Hillsdale, NJ : Lawrence Erlbaum.
27.
National Center for Educational Statistics. ( 1996). National educational longitudinal study: 1988-94. Data files and electronic codebook system: Base year through third follow-up ECB/CD-ROM [CD-ROM]. Washington, DC: Office of Educational Research and Improvement, U.S. Department of Education.
Rasch, G. (1960). Probabilistic models for intelligence and attainment tests. Copenhagen, Denmark: Danmarks Paedagogiske Institut .
30.
Riley, M.S., & Greeno, J.G. (1988). Developmental analysis of understanding language about quantities and of solving problems. Cognition and Instruction , 5, 49-101.
31.
Samejima, F. (1995). A cognitive diagnosis method using latent trait models: Competency space approach and its relationship with DiBello and Stout's unified cognitive-psychometric diagnosis model. In P. Nichol, S. Chipman, & R. Brennan (Eds.), Cognitively diagnostic assessment (pp. 391-410). Hillsdale, NJ: Lawrence Erlbaum.
32.
Snow, R.E., & Lohman, D.F. (1989). Implications of cognitive psychology for educational measurement. In R. Linn (Ed.), Educational measurement (3rd ed., pp. 263-331). New York: Macmillan.
33.
Spada, H. (1977). Logistic models of learning and thought. In H. Spada & W. F. Kempf (Eds.), Structural models of thinking and learning . Proceedings of the IPN-Symposium 7. Kiel1975 (pp. 227-262). Bern, Switzerland: Huber.
34.
Spada, H., & Kluwe, R. (1980). Two models of intellectual development and their reference to the theory of Piaget. In R. Kluwe & H. Spada (Eds.), Developmental models of thinking (pp. 1-32). New York: Academic Press.
35.
Spada, H., & McGaw, B. (1985). The assessment of learning effects with linear logistic test models. In S. Embretson (Ed.), Test design: New directions in psychology and psychometrics (pp. 169-193). New York: Academic Press.
36.
Suppes, H. (1969). Stimulus response theory of finite automata. Journal of Mathematical Psychology, 6, 327-355.
37.
Tatsuoka, C., & Ferguson, T. (2003). Sequential classification on partially ordered sets. Journal of Royal Statistical Society, 65(1), 143-157.
38.
Tatsuoka, K.K. (1983). Rule space: An approach for dealing with misconceptions based on item response theory. Journal of Educational Measurement , 20(4), 345-354.
39.
Tatsuoka, K.K. (1984). Caution indices based on item response theory . Psychometrika, 49(1), 95-110.
40.
Tatsuoka, K.K. (1985). A probabilistic model for diagnosing misconceptions by the pattern classification approach. Journal of Educational Statistics, 10(1), 55-73.
41.
Tatsuoka, K.K. (1995). Architecture of knowledge structures and cognitive diagnosis: A statistical pattern classification approach. In P. Nichol, S. Chipman, & R. Brennan (Eds.), Cognitively diagnostic assessment (pp. 327-360). Hillsdale, NJ: Lawrence Erlbaum.
42.
Tatsuoka, K.K., Corter, J.E., & Tatsuoka, C. (2004). Patterns of diagnosed mathematical content and process skills in TIMSS-R across a sample of 20 countries. American Educational Research Journal, 41(4), 901-926.
43.
Tatsuoka, K.K., & Tatsuoka, M.M. (1983). Spotting erroneous rules of operation by the individual consistency index. Journal of Educational Measurement, 20(3), 221-230.
44.
Tatsuoka, K.K., & Tatsuoka, M.M. (1987). Bug distribution and statistical pattern classification. Psychometrika, 52(2), 193-206.
Whitely, S.E., & Schneider, L.M. (1981). Information structure for geometric analogies: A test theory approach. Applied Psychological Measurement, 5(3), 383-397.
