Performance assessments (PA) are used in various contexts of large-scale educational assessment. It is often desirable to compare examinee performance on different forms of an assessment or on the same forms administered at different times. An overview of linking methods applied to PA is presented: major issues and recent developments in linking PAs are discussed, three common linking designs (single group, randomly equivalent groups, and nonequivalent groups with anchor items) are compared, and two major linking methodologies [classical and item response theory (IRT)] are evaluated from the PA perspective. Also described are two classical equating methods (linear and equipercentile) and several IRT equating methods (item response function, vertical, common population, and multiple-group). Areas for future research are identified.
Get full access to this article
View all access options for this article.
References
1.
Ackerman, T. (1996). Graphical representation of multidimensional item response theory analyses. Applied Measurement in Education, 20, 311–329.
2.
Baker, F. B. (1992). Equating under the graded response model. Applied Psychological Measurement, 16, 87–96.
3.
Baker, F. B. (1993). Equating tests under the nominal response model. Applied Psychological Measurement, 17, 239–251.
4.
Bock, R. D. (1972). Estimating item parameters and latent ability when responses are scored in two or more nominal categories. Psychometrika, 37, 29–51.
5.
Bock, R. D., Brennan, R. L., & Muraki, E. (1998). The information in multiple ratings. Unpublished manuscript.
6.
Bock, R. D. , Gibbons, R., & Muraki, E. (1988). Fullinformation item factor analysis. Applied Psychological Measurement, 12, 261–280.
7.
Bock, R. D. , Muraki, E., & Pfeiffenberger, W. (1988). Item pool maintenance in the presence of item parameter drift. Journal of Educational Measurement, 25, 275–285.
8.
Bock, R. D. , & Zimowski, M. F. (1997). Multiple group IRT. In W. J. van der Linden and R. K. Hambleton (Eds.), Handbook of modern item response theory(pp. 433–448). New York: Springer-Verlag.
9.
Bond, L. , Moss, P., & Carr, P. (1996). Fairness in large-scale performance assessment. In G. Phillips (Ed.), Technical issues in large-scaleperformance assessment(pp. 117–140). Washington DC: National Center for Education Statistics.
10.
Burnstein, J., Kukick, K., Wolff, S., Lu, C., Chodorow, M., Braden-Harder, L., & Harris, M. D. (1999). Automated scoring using a hybrid feature identification technique. Unpublished manuscript.
11.
Carey, P. (1996). A review of psychometric and consequential issues related to performance assessment(TOEFL Research Monograph No. MS-3). Princeton NJ: Educational Testing Service.
12.
College Entrance Examination Board (1988). The College Board technical manual for the Advanced Placement program. http://www.collegeboard.org/ap/techman/chap3/score.htm. Princeton NJ: Educational Testing Service.
13.
Crocker, L. ,& Algina, J. (1986). Introduction to classical and modern test theory. FortWorth TX: Harcourt, Brace, & Jovanovich.
14.
DeMauro, G. E. (1992). Investigation of the appropriateness of the TOEFL test as a matching variable to equateTWEtopics(TOEFLResearch ReportNo. 37). Princeton NJ: Educational Testing Service.
15.
Donoghue, J. R. , Isham, S. P., & Worthington L. H. (1996). Data analysis for the reading assessment. In N. L. Allen, D. L. Kline, & C. A. Zelenak (Eds.), The NAEP 1994 technical report(Report No. NCES 97-897, pp. 267–308). Washington DC: National Center for Education Statistics.
16.
Donoghue, J. R. , & Mazzeo, J. (1992, April). Comparing IRT-based equating procedures for trend measurement in a complex test design. Paper presented at the annual meeting of the National Council on Measurement in Education, San Francisco.
17.
Dunbar, S. B. , Koretz, D. M.,& Hoover, H. D. (1991). Quality control in the development and use of performance assessments. Applied Measurement in Education, 4, 289–303.
18.
Feuer, M. J. , Holland, P. W., Green, B. F., Bertenthal, M. W., & Hemphill, F. C. (Eds.). (1999). Uncommon measures: Equivalence and linkage among educational tests. Washington DC: National Academy Press.
19.
Goldstein, H. (1983). Measuring changes in educational attainment over time: Problems and possibilities. Journal of Educational Measurement, 20, 369–377.
20.
Gordon, B. , Engelhard, G., Gabrielson, S., & Bernknopf, S. (1996). Conceptual issues in equating performance assessments: Lessons from writing assessment. Journal of Research and Development in Education, 29, 81–88.
21.
Grabe, W. (1997, March). Developments in reading research and their implications for computeradaptive reading assessment. Paper presented at the 19th Language Testing Research Colloquium Conference, Orlando FL.
22.
Green, B. F. (1995). Comparability of scores from performance assessments. Educational Measurement: Issues and Practice, 14 (4), 13–15.
23.
Haebara, T. (1980). Equating logistic ability scales by a weighted least squares method. Japanese Psychological Research, 22, 144–149.
24.
Haertel, E. H. , & Linn, R. L. (1996). Comparability. In G. W. Phillips (Ed.), Technical issues in large-scale performance assessment(pp. 59–78). Washington DC: National Center for Education Statistics.
25.
Hambleton, R. K. (1989). Principles and selected applications of item response theory. In R. L. Linn (Ed.), Educational measurement(3rd ed., pp. 147–200). New York: Macmillan.
26.
Hambleton, R. K. , & Swaminathan, H. (1985). Item response theory: Principles and applications. Boston: Kluwer-Nijhoff.
27.
Hedges, L. V. ,& Vevea, J. L. (1997). A study of equating in NAEP. Palo Alto CA: NAEP Validity Studies Panel, American Institutes for Research.
28.
Hemker, B. T. , Sijtsma, K., Molenaar, I. W.,& Junker, B. W. (1997). Stochastic ordering using the latent trait and the sum score in polytomous IRT models. Psychometrika, 62, 331–347.
29.
Holland, P. W. , & Thayer, D. T. (1989). The kernel method of equating score distributions(Technical Report No. 89-84). Princeton NJ: Educational Testing Service.
30.
Huynh, H. , & Ferrara, S. (1994). A comparison of equal percentile and partial credit equatings for performance-based assessments composed of free-response items. Journal of Educational Measurement, 31, 125–141.
31.
Kolen, M. J. (1981). Comparison of traditional and item response theory methods for equating tests. Journal of Educational Measurement, 18, 1–11.
32.
Kolen, M. J. , & Brennan, R. L. (1995). Test equating: Methods and practices. New York: Springer-Verlag.
33.
Lee, Y. (1998). Examining the suitability of an IRTbased testlet approach to the construction and analysis of passage-based item sets in an EFL reading comprehension test in the Korean high school contexts. Unpublished doctoral dissertation, Pennsylvania State University, University Park, Dissertation Abstracts International, 59/08-A, 2893.
34.
Li, Y. H. (1997). An evaluation of multidimensional IRT equating methods by assessing the accuracy of transforming parameters onto a target testmetric. Unpublished doctoral dissertation. University of Maryland, College Park, Dissertation Abstracts International, 58/11-A, 4246.
35.
Linacre, M. (1988). Many-faceted Rasch measurement. Chicago: MESA Press.
36.
Linn, R. L. (1993). Linking results of distinct assessments. Applied Measurement in Education, 6, 83–102.
37.
Lord, F. M. , & Wingersky, M. S. (1984). Comparison of IRT true-score and equipercentile observedscore “equatings”. Applied Psychological Measurement, 8, 453–461.
38.
Loyd, B. , Engelhard, G., & Crocker, L. (1996). Achieving form-to-form comparability: Fundamental issues and proposed strategies for equating performance assessments for teachers. Educational Assessment, 3, 99–110.
39.
Messick, S. (1996). Validity of performance assessment. In G. W. Phillips (Ed.), Technical issues in large-scale performance assessment(pp. 1–18). Washington DC: National Center for Education Statistics.
40.
Miller, M. D. & Legg, S. M. (1993). Alternative assessment in a high-stakes environment. Educational Measurement: Issues and Practice, 12 (2), 9–15.
41.
Mislevy, R. J. (1992a). Linking educational assessments: Concepts, issues, methods, and prospects (Policy Issue Perspective). Princeton NJ: Educational Testing Service, Policy Information Center.
42.
Mislevy, R. J. (1992b). Scaling procedures. In E. G.Johnson & N. L. Allen (Eds.), The NAEP 1990 technical report(Report No. 21-TR-20, pp. 199–213). Washington DC: National Center for Education Statistics.
43.
Mislevy, R. J. (1995). Test theory and language learning assessment. Language Testing, 12, 341–369.
44.
Muraki, E. (1992). Ageneralized partial credit model: Application of an EM algorithm. Applied Psychological Measurement, 16, 159–176.
45.
Muraki, E. (1993, April). Variations of polytomous item response models: Raters’ effect model, IF model, and trend model. Paper presented at the annual meeting of the American Educational Research Association, Atlanta GA.
46.
Muraki, E. (1999). Stepwise analysis of differential item functioning based on multiple-group partial credit model. Journal of Educational Measurement, 36, 217–232.
47.
Muraki, E. , & Bock, R. D. (1998). PARSCALE (Version 3.5): IRT item analysis and test scoring for rating-scale data[Computer program]. Lincolnwood IL: Scientific Software.
48.
Muraki, E. , & Carlson, J. E. (1995). Full-information factor analysis for polytomous item responses. Applied Psychological Measurement, 19, 73–90.
49.
Muraki, E. , & Chang, H. (1994). Horizontal and vertical test equating methods based on the generalized-partial credit model. (ETS Internal Report). Princeton NJ: Educational Testing Service.
50.
Muraki, E. , & Engelhard, G. (1989, April). Examining differential item functioning with BIMAIN. Paper presented at the annual meeting of the American Educational Research Association, San Francisco.
51.
Muraki, E., & Hombo, C. (1999). Application of a multiple-group generalized partial credit model to NAEP linking procedures. Unpublished manuscript.
52.
Myford, C. , Marr, D. B., & Linacre, J. M. (1996). Reader calibration and its potential role in equating for the Test of Written English(TOEFL Research Report No. 52). Princeton NJ: Educational Testing Service.
53.
Nitko, A. J. (1996). Educational assessment of students(2nd ed.). Englewood Cliffs NJ: Prentice-Hall.
54.
Oosterhof, A. (1994). Classroom applications of educational measurement(2nd ed.). New York: Macmillan.
55.
Payne, D. A. (1997). Applied educational measurement. Belmont CA: Wadsworth.
56.
Reckase, M. D. (1985). The difficulty of items that measure more than one ability. Applied Psychological Measurement, 9, 401–412.
57.
Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph Supplement, 17.
58.
Sax, G. (1997). Principles of educational and psychological measurement and evaluation(4th ed.). Belmont CA: Wadsworth.
59.
Stahl, J. A. , Lunz, M. E., & Wright, B. D. (1991, April). Equating examinations that include judges (multiple facets). Paper presented at the annual meeting of the National Council of Measurement in Education, Chicago.
60.
Stocking, M. L. , & Lord, F. M. (1983). Developing a common metric in item response theory. Applied Psychological Measurement, 7, 201–210.
61.
Thissen, D. , Steinberg, L., & Wainer, H. (1993). Detection of differential item functioning using the parameters of item response models. In P. W. Holland & H. Wainer (Eds.), Differential item functioning(pp. 67-113). Hillsdale NJ: Erlbaum.
62.
van der Linden, W. J. , & Hambleton, R. K. (Eds). (1997). Handbook of modern item response theory. New York: Springer-Verlag.
63.
Wiggins, G. (1993). Assessment: Authenticity, context, and validity. Phi Delta Kappan, 75, 200–214.
64.
Wright, B. D. , & Linacre, J. M. (1992). BIGSTEPS Rasch analysis[Computer program]. Chicago: MESA Press.
65.
Yen, W. M. (1993). Scaling performance assessments: Strategies for managing local dependence. Journal of Educational Measurement, 30, 187–213.
66.
Yen, W. M. , & Ferrara, S. (1997). The Maryland school performance assessment program: Performance assessment with psychometric quality suitable for high-stakes usage. Educational and Psychological Measurement, 57, 60–84.
67.
Zimowski, M. F. , Muraki, E., Mislevy, R. J., & Bock, R. D. (1996). BILOG-MG: Multiple-group IRT analysis and test maintenance for binary items [Computer program]. Lincolnwood IL: Scientific Software.
68.
Zwick, R. (1991). Effects of item order and context on estimation of NAEP Reading Proficiency. Educational Measurement: Issues and Practice, 10 (3), 10–16.
