The purpose of this study was to describe a proce dure based on generalizability theory for assessing the adequacy of item groupings generated by means of a domain-referenced testing system. The procedure is based on an exploration of the extent to which items are grouped into homogeneous sets with similar levels of difficulty, when the sets are defined on the basis of a logical analysis of the perceived skills underlying the items. In addition, an example of the use of the devel oped analytic procedure is presented.
Get full access to this article
View all access options for this article.
References
1.
Brennan, R.L.Extensions of generalizability theory to domain-referenced testing (ACT Technical Bulletin No. 30. Iowa City IA: American College Testing Program, June 1978.
2.
Brennan, R.L.Some applications of generalizability theory to the dependability of domain-referenced tests (ACT Technical Bulletin No. 32. Iowa City IA: American College Testing Program, April 1979.
3.
Brennan, R.L. , & Kane, M.T.An index of dependability for mastery tests. Journal of Educatronal Measurement, 1977, 14, 277-289. (a)
Cardinet, J., Tourneur, Y., & Allal, L.The symmetry of generalizability theory: Applications to educational measurement . Journal of Educational Measurement, 1976 , 13, 119-135.
6.
Cronbach, L.J.Coefficient alpha and the internal structure of tests. Psychometrika , 1951, 16, 297-334.
7.
Cronbach, L.J., & Gleser, G.C.The signal-noise ratio in the comparison of reliability coefficients. Educational and Psychological Measurement, 1964, 23, 467-480.
8.
Cronbach, L.J., Gleser, G.C., Nanda, H., & Rajaratnam, N.The dependability of behavioral measurement: Theory of generalizability, for single and multiple observations. New York. Wiley, 1972.
Harris, C.W.Some technical characteristics of mastery tests. In C. W. Harris, M. C. Alkin, & W. J. Popham (Eds.), Problems in criterion-referenced measurement (CSE Monograph Series in Evaluation No. 3. Los Angeles: University of California, Center for the Study of Evaluation, 1974.
11.
Hartke, A.R.The use of latent partition analysis to identify homogeneity of an item population . Journal of Educational Measurement, 1978 , 15, 43-47.
12.
Hively, W.Domain-referenced achievement testing: Theory and practice. Unpublished manuscript, 1970.
13.
Hively, W.Introduction to domain-referenced testing. In W. Hively (Ed.), Domain-referenced testing. Englewood Cliffs NJ: Educational Technology Publication , 1974.
14.
Hively, W., Patterson, H.L., & Page, L.A "universe-defined" system of arithmetic achievement test forms. Journal of Educational Measurement, 1968, 5, 275-290.
15.
Kane, M.T A sampling model for validity. AppliedPsychological Measurement , 1982, 6, 125-160.
16.
Loevinger, J.A systematic approach to the construction and evaluation of tests of ability . Psychological Monographs, 1947, 61 (4, Whole No. 285).
17.
Lumsden, J.Test theory. Annual Review of Psychology, 1976, 27, 254-280.
18.
Macready, G.B., & Merwin, J.C.Homogeneity within item forms in domain-referenced testing. Educational and Psychological Measurement, 1973, 33, 351-360.
