The effect of stratified sampling of items on the estimation of test score distribution parameters by multiple matrix sampling was studied. Item difficulty and/or interitem correlations were the bases of stratification. Various item universes were created by computer simulation and sampled according to several plans. The results indicate that stratification of items does not consistently improve the stability of parameter estimation. The results also show that the variance estimate used in many studies is biased for some item universes when difficulty stratification is used. A variance estimate developed in the current study removes this bias.
Get full access to this article
View all access options for this article.
References
1.
Barr, D. R. and Slezak, N. L.A comparison of multivariate normal generators. Comunications for the Association for Computing Machinery, 1972, 15, 1048-1049.
2.
Cochran, W. G.Sampling Techniques (second edition). New York: John Wiley and Sons, 1963.
3.
Kirk, R. E.Experimental Design: Procedures for the Behavioral Sciences. Belmont, California: Brooks/Cole, 1968.
4.
Kleinke, D. J.The accuracy of estimated total test statistics. (U. S. Department of Health, Education, and Welfare, Project No. IB070, Grant No. OEG-2-710070) Syracuse, New York, 1972.
5.
Lord, F. M.Use of true-score theory to predict moments of univariate and bivariate observed score distributions. Psychometrika, 1960, 25, 325-342.
6.
Lord, F. M.Estimating norms by item sampling . EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT, 1962, 22, 259-267.
7.
Meyer, D. L.Methods of generating random normal numbers. EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT , 1969,29, 193-198.
8.
Plumlee, L. B.Estimating means and standard deviations from partial data—an empirical check of Lord's item sampling technique. EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT, 1964, 24, 623-630.
9.
Searle, S. R.Linear models. New York: John Wiley and Sons, 1971 .
10.
Shoemaker, D. M.Principles and Procedures of Multiple Matrix Sampling. Cambridge, Massachusetts: Ballinger, 1973.
11.
Shoemaker, D. M. and Osburn, H. G.An empirical study of generalizability coefficients from unmatched data. British Journal of Mathematical and Statistical Psychology, 1968, 21 (part 2), 239-246.
12.
Sirotnik, K.An analysis of variance framework for matrix sampling. EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT , 1970, 30, 891-908.
