Ability and Prior Distribution Mismatch: An Exploration of Common-Item Linking Methods

Abstract

Linking of two forms is an important task when using item response theory, particularly when two forms are administered to nonequivalent groups. When linking with characteristic curve methods, the ability distribution and weights associated with that distribution can be used to weight observations differently. These are commonly specified as equally spaced intervals from −4 to 4, but other options or distributional forms can be specified. The use of these different distributions and weights of the ability distributions will be explored with a Monte Carlo simulation. Primary simulation conditions will include sample size, number of items, number of common items, ability distribution, and randomly varying population transformation constants. Study results show that the linking weights have little impact on the estimation of the linking constants; however, the underlying ability distribution of examinees does have significant impact. Implications for applied researchers will be discussed.

Keywords

calibration linking psychometrics simulation

Get full access to this article

View all access options for this article.

References

Birnbaum

(1968). Some latent trait models and their use in inferring an examinee’s ability. In Lord

F. M.

Novick

M. R.

(Eds.), Statistical theories of mental test scores. (pp. 395-479). Reading, MA: Addison-Wesley.

Camilli

(1994). Teacher’s corner: Origin of the scaling constant d= 1.7 in item response theory. Journal of Educational and Behavioral Statistics, 19, 293-295.

De Ayala

R. J

. (2013). Theory and practice of item response theory. New York, NY: Guilford Press.

Haebara

(1980). Equating logistic ability scales by a weighted least squares method. Japanese Psychological Research, 22, 144-149.

Hanson

B. A.

Béguin

A. A.

(2002). Obtaining a common scale for item response theory item parameters using separate versus concurrent estimation in the common-item equating design. Applied Psychological Measurement, 26, 3-24.

Holland

P. W.

Dorans

N. J.

(2006). Linking and equating test scores. Educational Measurement, 4, 187-220.

Kim

Hanson

B. A.

(2002). Test equating under the multiple-choice model. Applied Psychological Measurement, 26, 255-270.

Kim

Kolen

M. J.

(2003). POLYST: A computer program for polytomous IRT scale transformation. Iowa City: Iowa Testing Programs, The University of Iowa. Retrieved from http://www.education.uiowa.edu/casma

Kim

Kolen

M. J.

(2007). Effects on scale linking of different definitions of criterion functions for the irt characteristic curve methods. Journal of Educational and Behavioral Statistics, 32, 371-397.

10.

Kim

Lee

W.-C.

(2006). An extension of four IRT linking methods for mixed-format tests. Journal of Educational Measurement, 43, 53-76.

11.

Kolen

M. J.

Brennan

R. L.

(2004). Test equating, scaling, and linking. New York, NY: Springer.

12.

Lord

F. M.

(1980). Applications of item response theory to practical testing problems. New York, NY: Routledge.

13.

Mislevy

R. J.

Bock

R. D.

(1990). Bilog 3: Item analysis and test scoring with binary logistic models. Skokie, IL: Scientific Software International.

14.

R Core Team. (2015). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Available from http://www.R-project.org

15.

Skrondal

(2000). Design and analysis of monte carlo experiments: Attacking the conventional wisdom. Multivariate Behavioral Research, 35, 137-167.

16.

Stocking

M. L.

Lord

F. M.

(1983). Developing a common metric in item response theory. Applied Psychological Measurement, 7, 201-210.

17.

Tong

Kolen

M. J.

(2007). Comparisons of methodologies and results in vertical scaling for educational achievement tests. Applied Measurement in Education, 20, 227-253.

18.

Weeks

J. P.

(2010). plink: An R package for linking mixed-format tests using irt-based methods. Journal of Statistical Software, 35(12), 1-33. Retrieved from http://www.jstatsoft.org/v35/i12/

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.14 MB