The performance of two polytomous item response theory models was compared to that of the dichotomous three-parameter logistic model in the context of equating tests composed of testlets. For the polytomous models, testlet scores were used to eliminate the effect of the dependence among within-testlet items. Traditional equating methods were used as criteria for both. The equating methods based on polytomous models were found to produce results that more closely agreed with the results of traditional methods.
Get full access to this article
View all access options for this article.
References
1.
Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika, 43, 561–573.
2.
Bock, R. D. (1972). Estimating item parameters and latent ability when responses are scored in two or more nominal categories. Psychometrika, 37, 29–51.
3.
Carroll, J. B. (1945). The effect of difficulty and chance success on correlation between items or between tests. Psychometrika, 10, 1–19.
4.
Chen, W. , & Thissen, D. (1997). Local dependence indexes for item pairs using item response theory. Journal of Educational and Behavioral Statistics, 22, 265–289.
5.
Cook, L. L. , & Eignor, D. R. (1991). An NCME instructional module on IRT equating methods. Educational Measurement: Issues and Practice, 10, 37–45.
6.
Hanson, B. A. , & Zeng, L. (1995). A computer program for IRT equating (PIE) (Version 1.0) [Computer program]. Iowa City IA: ACT.
7.
Harris, D. J. , & Crouse, J. D. (1993). A study of criteria used in equating. Applied Measurement in Education, 6, 195–240.
8.
Hattie, J. (1985). Methodology review: Assessing unidimensionality of tests and items. Applied Psychological Measurement, 9, 139–164.
9.
Hoover, H. D. , Hieronymous, A. N., Frisbie, D. A.,& Dunbar, S. B. (1994). Iowa tests of basic skills: Interpretive guide for school administrators. Chicago: Riverside Publishing.
Kolen, M. J. (1988). An NCME instructional module on traditional equating methodology. Educational Measurement: Issues and Practice, 7, 29–36.
12.
Kolen, M. J. , & Brennan, R. L. (1995). Test equating: Methods and practices. New York: Springer-Verlag.
13.
Lee, G. (2000). Estimating conditional standard errors of measurement for tests composed of testlets. Applied Measurement in Education, 13, 161–180.
14.
Lee, G. , Dunbar, S. B., & Frisbie, D. A. (1999, April). Measurement models for a testlet-based test. Paper presented at the annual meeting of the National Council on Measurement in Education, Montreal, Canada.
15.
Lee, G. , & Frisbie, D. A. (1999). Estimating reliability under a generalizability theory model for test scores composed of testlets. Applied Measurement in Education, 12, 237–255.
16.
Lord, F. M. , & Wingersky, M. S. (1984). Comparison of IRT true-score and equipercentile observed-score “equatings”. Applied Psychological Measurement, 8, 452–461.
17.
Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149–174.
18.
Mislevy, R. J. , & Bock, R. D. (1990). BILOG 3: Item analysis and test scoring with binary logistic models (2nd ed.) [Computer program]. Mooresville IN: Scientific Software.
19.
Samejima, F. (1969). Estimation of a latent ability using a response pattern of graded scores. Psychometric MonographSupplement No. 17.
20.
Sireci, S. G. , Thissen, D., & Wainer, H. (1991). On the reliability of testlet-based tests. Journal of Educational Measurement, 28, 237–247.
21.
Thissen, D. (1991). MULTILOG: Multiple categorical item analysis and test scoring using item response theory (Version 6.0) [Computer program]. Chicago: Scientific Software.
22.
Thissen, D. , Steinberg, L., & Mooney, J. A. (1989). Trace lines for testlets: A use of multiple-categorical models. Journal of Educational Measurement, 26, 247–260.
23.
Wainer, H. (1995). Precision and differential item functioning on a testlet-based test: The 1991 Law School Admissions Test as an example. Applied Measurement in Education, 8, 157–186.
24.
Wainer, H. , & Kiely, G. L. (1987). Item clusters and computerized adaptive testing: A case for testlets. Journal of Educational Measurement, 24, 185–201.
25.
Wainer, H. , & Lewis, C. (1990). Toward a psycho-metrics for testlets. Journal of Educational Measurement, 27, 1–14.
26.
Wainer, H. , Sireci, S. G., & Thissen, D. (1991). Differential testlet functioning: Definition and detecting. Journal of Educational Measurement, 28, 197–219.
27.
Wainer, H. , & Thissen, D. (1996). How is reliability related to the quality of test scores? What is the effect of local dependence on reliability?Educational Measurement: Issues and Practice, 15, 22–29.
28.
Wang, T. , Kolen, M. J., & Harris, D. J. (1996, April). Conditional standard errors, reliability and decision consistency of performance levels using polytomous IRT. Paper presented at the annual meeting of the American Educational Research Association, New York.
29.
Yen, W. M. (1984). Effects of local item dependence on the fit and equating performance of the three-parameter logistic model. Applied Psychological Measurement, 8, 125–145.
30.
Yen, W. M. (1993). Scaling performance assessments: Strategies for managing local item dependence. Journal of Educational Measurement, 30, 187–213.
31.
Zeng, L. , & Kolen, M. J. (1995). An alternative approach for IRT observed-score equating of number-correct scores. Applied Psychological Measurement, 19, 231–240.
32.
Zeng, L. , Kolen, M. J., & Hanson, B. A. (1995). Random groups equating program (RAGE) (Version 2.0) [Computer program]. Iowa City IA: ACT.
