Previous research on the application of IRT method ology to vertical test equating has demonstrated con flicting results about the degree of invariance shown by these methods with respect to examinee ability. The purpose of this study was to examine IRT equating invariance by simulating the vertical equating of two tests under varying conditions. Rasch, three-parame ter, and equipercentile equating methods were com pared. Six equating cases, using different sets of item parameters, were replicated based on examinee sam ples of low, medium, or high ability or where ability was matched to the difficulty level of the test. The re sults showed that all three methods were reasonably invariant to examinee ability level under all conditions imposed. This suggests that multidimensionality is likely to be the cause of the lack of invariance found in real datasets. Index terms: Examinee ability; In variance in item response theory; Item response the ory, equating; Item response theory, invariance; Test equating; Vertical equating.
Get full access to this article
View all access options for this article.
References
1.
Anderson, D.O. (1985). Scale stability during test equating. Paper presented at the annual meeting of the American Educational Research Association, Chicago.
2.
Angoff, W.H. (1984). Scales, norms, and equivalent scores. Princeton NJ: Educational Testing Service. [Reprint of chapter in R. L. Thomdike (Ed.), Educational measurement (2nd ed.). Washington DC: American Council on Education, 1971.]
3.
Cook, L.L., & Eignor, D.R. (1983). An investigation of the feasibility of applying item response theory to equate achievement tests. Paper presented at the annual meeting of the American Educational Research Association, Montreal.
4.
Cook, L.L., Eignor, D.R., & Taft, H.L. (1984). A comparative study of curriculum effects on the stability of IRT and conventional item parameter estimcates. Paper presented at the annual meeting of the American Educational Research Association, New Orleans.
5.
Cook, L.L., & Petersen, N.S. (1987). Problems related to the use of conventional and item response theory equating methods in less than optimal circumstances. Applied Psychological Measurement, 11, 225-244.
6.
Cureton, E.E., & Tukey, J.W. (1951). Smoothing frequency distributions, equating tests, and preparing norms. (Abstract of presented paper.)American Psychologist, 6, 404.
7.
Curry, A.R., Bashaw, W.L., & Rentz, R.R. (1978). Invariance of Rasch model ability parameter estimates over different collections of items. Paper presented at the annual meeting of the American Educational Research Association, Toronto.
8.
Divgi, D.R. (1981). Does the Rasch model really work? Not if you look closely. Paper presented at the annual meeting of the American Educational Research Association, Los Angeles .
9.
Forsyth, R., Saisangjan, U., & Gilmer, J. (1981). Some empirical results related to the robustness of the Rasch model. Applied Psychological Measurement, 5, 175-186.
10.
Hambleton, R.K., & Swaminathan, H. (1985). Item response theory: Principles and applications. Boston: Kluwer-Nijhoff .
11.
Harris, D.J., & Hoover, H.D. (1987). An application of the three-parameter IRT model to vertical equating. Applied Psychological Measurement, 11, 151-159.
12.
Harris, D.J., & Kolen, M.J. (1986). Effect of examinee group on equating relationships . Applied Psychological Measurement, 10, 35-43.
13.
Holmes, S.E. (1982). Unidimensionality and vertical equating with the Rasch model. Journal of Educational Measurement, 19, 139-147.
14.
Holmes, S.E., & Doody-Bogan, E.N. (1983). An empirical study of vertical equating methods using the three-parameter logistic model. Paper presented at the annual meeting of the National Council on Measurement in Education, Montreal.
15.
Hulin, C.L., Lissak, R.I., & Drasgow, F. (1982). Recovery of two- and three-parameter logistic item characteristic curves: A monte carlo study. Applied Psychological Measurement, 6, 249-260.
16.
IMSL, Inc. (1980). International Mathematical and Statistical Libraries refirence manual. Houston TX: Author.
17.
Jarjoura, D., & Kolen, M.J. (1985). Standard errors of equipercentile equating for the common item non-equivalent populations design. Journal of Educational Statistics, 10, 143-160.
18.
Kolen, M.J. (1981). Comparison of traditional and item response theory methods for equating tests. Journal of Educational Measurement , 18, 1-11.
19.
Kolen, M.J., & Whitney, D.R. (1982). Comparison of four procedures for equating the tests of General Educational Development. Journal of Educational Measurement, 19, 279-293.
20.
Levine, R.S. (1958). Estimated national norms for the Scholastic Aptitude Test (Statistical Report No. 1). Princeton NJ: Educational Testing Service.
21.
Lord, F.M. (1980). Applications of item response theory to practical testing problems. Hillsdale NJ: Eribaum.
22.
Loyd, B.H., & Hoover, H.D. (1980). Vertical equating using the Rasch model. Journal of Educational Measurement, 17, 179-193.
23.
Marco, G.L., Petersen, N.S., & Stewart, E.E. (1983). A test of the adequacy of curvilinear score equating models. In D. J. Weiss (Ed.), New horizons in testing: Latent trait test theory and computerized adaptive testing (pp. 147-177). New York: Academic Press.
24.
Petersen, N.S., Cook, L.L., & Stocking, M.L. (1983). IRT versus conventional equating methods: A comparative study of scale stability. Journal of Educational Statistics, 8, 137-156.
25.
Skaggs, G., & Lissitz, R.W. (1986). An exploration of the robustness of four test equating models. Applied Psychological Measurement, 10, 303-317.
26.
Skaggs, G., & Stevenson, J. (1986). A comparison of ASCAL and LOGIST parameter estimation programs. Paper presented at the annual meeting of the National Council on Measurement in Education, San Francisco.
27.
Slinde, J.A., & Linn, R.L. (1978). An exploration of the adequacy of the Rasch model for the problem of vertical equating. Journal of Educational Measurement, 15, 23-35.
28.
Slinde, J.A., & Linn, R.L. (1979). A note on vertical equating via the Rasch model for groups of quite different ability and tests of quite different difficulty . Journal of Educational Measurement, 16, 159-165.
29.
Wingersky, M.S., Barton, M.A., & Lord, F.M. (1982). LOGIST: A computer program for estimating examinee ability and item characteristic curve parameters (LOGIST 5, version 1). Princeton NJ: Educational Testing Service.
30.
Wright, B.D. (1977). Solving measurement problems with the Rasch model . Journal of Educational Measurement, 14, 97-116.
31.
Wright, B.D., Mead, R.J., & Bell, S.R. (1980). BICAL: Calibrating items with the Rasch model (Research Memorandum No. 23C). Chicago: University of Chicago, Department of Education, Statistical Laboratory.
32.
Wright, B.D., & Stone, M.H. (1979). Best test design. Chicago: Mesa Press.
33.
Yen, W.M. (1986). The choice of scale for educational measurement: An IRT perspective. Journal of Educational Measurement, 23, 299-325.
