When tests are made up of testlets, a testlet-based item response theory (IRT) model may be used to account for local dependence among items from a common testlet. This study presents a new test characteristic curve method to link calibrations based on the Bradlow, Wainer, and Wang (1999) testlet model. Procedures for calculating the test characteristic curve linking coefficients are developed. The effectiveness of the method is investigated using simulated data. Results suggest that the linking coefficients are accurately recovered and superior to those obtained when using a traditional IRT model to link calibrations when testlet-based local dependence is present. A real data example illustrates the proposed linking method.
Albert, J. H. (1992). Bayesian estimation of normal ogive response curves using Gibbs sampling. Journal of Educational Statistics, 17, 251-269.
2.
Baker, F. B. (1992). Equating tests under the graded response model. Applied Psychological Measurement, 16, 87-96.
3.
Baker, F. B. (1993a). Equate 2.0: A computer program for the characteristic curve method of IRT equating. Applied Psychological Measurement, 17, 20.
4.
Baker, F. B. (1993b). Equating tests under the nominal response model. Applied Psychological Measurement, 17, 239-251.
5.
Bradlow, E. T., Wainer, H., & Wang, X. (1999). A Bayesian random effects model for testlets. Psychometrika, 64, 153-168.
6.
Davey, T. C., Oshima, T. C., & Lee, K. (1996). Linking multidimensional item calibrations. Applied Psychological Measurement, 20, 405-416.
7.
Dennis, J. E., & Schnabel, R. B. (1996). Numerical methods for unconstrained optimization and nonlinear equations. Englewood Cliffs, NJ: Prentice Hall.
8.
Du, Z. (1998). Modelling conditional item dependencies with a three-parameter logistic testlet model. Unpublished doctoral dissertation, Columbia University.
9.
Forrest, S. (1993). Genetic algorithms: Principles of natural selection applied to computation. Science, 261, 872-878.
10.
Glas, C. A. W., Wainer, H., & Bradlow, E. T. (2000). MML and EAP estimation in testlet-based adaptive testing. In W. J. van der Linden & C. A. W. Glas (Eds.), Computerized adaptive testing: Theory and practice (pp. 271-287). Boston: Kluwer-Nijhoff.
11.
Goldberg, D. E. (1989). Genetic algorithms in search, optimization & machine learning. Reading, MA: Addison-Wesley.
12.
Lee, G., Dunbar, S. B., & Frisbie, D. A. (2001). The relative appropriateness of eight measurement models for analyzing scores from tests composed of testlets. Educational and Psychological Measurement, 61, 958-975.
13.
Lee, G., & Frisbie, D. A. (1999). Estimating reliability under a generalizability theory model for test scores composed of testlets. Applied Measurement in Education, 12, 237-255.
14.
Li, Y., Bolt, D. M., & Fu, J. (2004, April). A test characteristic curve linking method for the testlet model. Paper presented at the annual meeting of the National Council on Measurement in Education, San Diego.
15.
Mislevy, R. J., & Bock, R. D. (1983). BILOG: Item and test scoring with binary logistic models [Computer program]. Mooresville, IN: Scientific Software.
16.
Mitchell, M. (1996). An introduction to genetic algorithms. Cambridge, MA: MIT Press.
17.
R Development Core Team. (2003). R: A language and environment for statistical computing [Computer program]. Vienna, Austria: Author.
18.
Raftery, A. E., & Lewis, S. M. (1996). Implementing MCMC. In W. R. Gilks, S. Richardson, & D. J. Spiegelhalter (Eds.), Markov chain Monte Carloin practice (pp. 115-130). London: Chapman &Hall.
19.
Rosenbaum, P. R. (1988). Item bundles. Psychometrika, 53, 349-359.
20.
Sekhon, J. S., & Mebane, W. R. (1998). Genetic optimization using derivatives: Theory and application to nonlinear models. Political Analysis, 7, 187-210.
21.
Sireci, S. G., Wainer, H., & Thissen, D. (1991). On the reliability of testlet-based tests. Journal of Educational Measurement, 28, 237-247.
22.
Spiegelhalter, D., Thomas, A. & Best, N. (2003). WinBUGS (Version 1.4) [Computer program]. Cambridge, UK: MRC Biostatistics Unit, Institute of Public Health.
23.
Stocking, M. L., & Lord, F. M. (1983). Developing a common metric in item response theory. Applied Psychological Measurement, 7, 201-210.
24.
Stroud, A. H., & Secrest, D. (1966). Gaussian quadrature formulas. Englewood Cliffs, NJ: Prentice Hall.
25.
Wainer, H., Bradlow, E. T., & Du, Z. (2000). Testlet response theory: An analog for the 3PL useful in adaptive testing. In W. J. van der Linden & C. A. W. Glas (Eds.), Computerized adaptive testing: Theory and practice (pp. 245-270). Boston: Kluwer-Nijhoff.
26.
Wainer, H., & Kiely, G. (1987). Item clusters and computerized adaptive testing: A case for test-lets. Journal of Educational Measurement, 24, 185-202.
27.
Wainer, H., & Wang, X. (2000). Using a new statistical model for testlets to score TOEFL. Journal of Educational Measurement, 37, 203-220.
28.
Wang, X., Bradlow, E. T., & Wainer, H. (2002). A general Bayesian model for testlets: Theory and applications. Applied Psychological Measurement, 26, 109-128.
29.
Yen, W. M. (1993). Scaling performance assessments: Strategies for managing local item dependence. Journal of Educational Measurement, 30, 187-213.
