A procedure for the sequential optimization of the calibration of an item bank is given. The procedure is based on an empirical Bayesian approach to a refor mulation of the Rasch model as a model for paired comparisons between the difficulties of test items in which ties are allowed to occur. First, it is shown how a paired-comparisons design deals with the usual in completeness of calibration data and how the item pa rameters can be estimated using this design. Next, the procedure for a sequential optimization of the item pa rameter estimators is given, both for individuals re sponding to pairs of items and for item and examinee groups of any size. The paper concludes with a dis cussion of the choice of the first priors in the proce dure and the problems involved in its generalization to other item response models.
Get full access to this article
View all access options for this article.
References
1.
Boekkooi-Timminga, E. (in press). Simultaneous test construction by zero-one programming . Methodika.
Bradley, R.A., & Terry, M.E. (1952). Rank analysis of incomplete block designs. I. The method of paired comparisons. Biometrika, 39, 324-345.
4.
Choppin, B.H.L. (1968). An item bank using sample-free calibration. Nature, 219, 870-872.
5.
Davidson, R.R. (1970). On extending the Bradley-Terry model to accommodate ties in paired comparison experiments. Journal of the American Statistical Association,65, 317-328.
6.
Dijkstra, O. (1960). Rank analysis of incomplete block designs: A method of paired comparisons employing unequal repetition on pairs. Biometrics, 16, 176-188.
7.
Eggen, T.J.H.M., & van der Linden, W.J. (1987). The use of models for paired comparisons with ties. In W. E. Saris & I. N. Gallhofer (Eds.), Sociometric research. Volume 1: Data collection and scaling. London: Macmillan.
8.
Fischer, G.H. (1974). Einführung in die Theorie psychologischer Tests [Introduction to the theory of psychological tests]. Bern : Hans Huber.
9.
Fischer, G.H. (1981). On the existence and uniqueness of maximum-likelihood estimates in the Rasch model. Psychometrika, 46, 59-77.
10.
Ford, L.R., Jr. (1957). Solution of a ranking problem for binary comparisons . American Mathematical Monthly, 64, 28-33.
11.
Glenn, W.A., & David, H.A. (1960). Ties in paired comparisons experiments using a modified Thurstone-Mosteller method. Biometrics, 16, 86-109.
12.
Gridgeman, N.T. (1959). Pair comparison, with and without ties. Biometrics, 15, 382-388.
13.
Gustafsson, J.E. (1980). A solution of the conditional estimation problem for long tests in the Rasch model for dichotomous items. Educational and Psychological Measurement, 40, 377-385.
14.
Harary, F., Norman, R.Z., & Cartwright, D. (1965). Structural models: An introduction to the theory of directed graphs. New York: Wiley.
15.
Kousgaard, N. (1976). Models for paired comparisons with ties. Scandinavian Journal of Statistics, 3, 1-14.
16.
Novick, M.R., & Jackson, P.H. (1974). Statistical methods for educational and psychological research. New York: McGraw-Hill .
17.
Rao, P.V., & Kupper, L.L. (1967). Ties in paired-comparison experiments: A generalization of the Bradley-Terry model. Journal of the American Statistical Association, 62, 194-204, corrigenda 63, 1550.
18.
Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copenhagen: Danmarks Paedagogiske Institut.
19.
Robbins, H. (1956). An empirical Bayes approach to statistics. In Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability (Vol. 1). Berkeley CA: University of California Press.
20.
Robbins, H. (1964). The empirical Bayes approach to statistical decision problems. Annals of Mathematical Statistics, 35, 1-20.
21.
Theunissen, T.J.J.M. (1985). Binary programming and test design. Psychometrika, 50, 411-420.
22.
van der Linden, W.J. (1986, June). Optimal sampling design for parameter estimation in item response models. Paper presented at the annual meeting of the Psychometric Society, Toronto.
23.
van der Linden, W.J., & Boekkooi-Timminga, E. (1986). Algorithmic test design with practical constraints. Enschede, The Netherlands: University of Twente, Department of Education.
24.
van der Linden, W.J., & Eggen, T.J.H.M. (1986). The Rasch model as a model for paired comparisons with ties. Enschede, The Netherlands: University of Twente, Department of Education .
25.
Zermelo, E. (1929). Die Berechnung der Tumierergebnisse als ein Maximumproblem der Warscheinlichkeitsrechnung [The computation of tournament results as a maximization problem in probability calculus]. Mathematische Zeitschrift, 29, 436-460.
