Some Results and Comments pn Using Latent Structure Models to Measure Achievement

Abstract

Under certain circumstances it is possible to apply latent structure models to estimate ζ, the proportion of skills an examinee has acquired among a domain of skills and to estimate β, the probability of guessing the correct response given that the examinee does not know. Models have also been formulated in which ζ represents the proportion of examinees who know a specific skill and again β is the probability of guessing. Macready and Dayton (1976) have used these models to determine the number of items required in deciding whether an examinee has acquired a particular skill and Wilcox (1979) has applied these models to arrive at a formula score which allows guessing to vary over the population of examinees. In this paper we examine a technical problem associated with estimating β and we also indicate situations for which this problem does not occur. It is shown, however, that this problem does not apply when using Wilcox's formula score. In addition, some new, explicit, maxi mum likelihood estimates of ζ and β are derived which may be applied when items are hierarchically related.

Get full access to this article

View all access options for this article.

References

Brownless, V. T. and Keats, J. A. A retest method of studying partial knowledge and other factors influencing item response. Psychometrika, 1958, 23, 67 -3.

Dayton, C. M. and Macready, G. B. A probabilistic model for validation of behavioral hierarchies. Psychometrika, 1976, 41 , 189-204.

Duncan, G. T. An empirical Bayes approach to scoring multiple-choice tests in the misinformation model. Journal of the American Statistical Association, 1974, 69, 50 -57 .

Knapp, T. R. The reliability of a dichotomous test-item: A "correlationless" approach. Journal of Educational Measurement, 1977, 14, 237-252.

Lazarsfeld, P. F. and Henry, N. W. Latent structure analysis. New York: Houghton Mifflin, 1968.

Macready, G. B. and Dayton, C. M. The use of probabilistic models in the assessment of mastery. Journal of Educational Statistics, 1977, 2, 99 -120.

Marks, E. and Noll, G. A. Procedures and criteria for evaluating reading and listening comprehension tests. EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT , 1967, 27, 335-348.

Proctor, C. H. A probabilistic formulation and statistical analysis of Guttman scales. Psychometrika , 35, 73 -78.

Samuel-Cahn, E. Remark on a formula by Fisher , Journal of the American Statistical Association , 1975, 70, 720 .

10.

White, R. T. and Clark, R. M. A test of inclusion which allows for errors of measurement . Psychometrika, 1973, 38 , 77-86.

11.

Wilcox, R. R. and Yeh, J. On latent structure models for measuring achievement on hierarchically related skills. Center for the Study of Evaluation technical report No. 124, Los Angeles: University of California, 1978.

12.

Wilcox, R. R. Achievement tests and latent structure models. British Journal of Mathematical and Statistical Psychology, 1979, 32, 61-71.

13.

Wilcox, R. R. New methods for studying stability . In Harris, C. W. ; Pearlinan, A. ; and Wilcox, R. , Achievement Test Items— Methods of Study. CSE Monograph No. 6, Los Angeles: Center for the Study of Evaluation, University of California, 1977(a).

14.

Wilcox, R. R. New methods for studying equivalence . In Harris, C. W. ; Pearlman, A. ; and Wilcox, R. , Achievement Test Items: Methods of Study. CSE Monograph No. 6, Los Angeles: Center for the Study of Evaluation, University of California, 1977 (b).

15.

Zehna, P. W. Invariance of maximum likelihood estimation. Annals of Mathematical Statistics, 1966, 37, 744 .