Year-to-year changes in average writing performance for a sample of 188 schools were estimated with a group-level version of the generalized partial credit IRT model. School-level IRT-based equating adjustments were required to provide a stable scale over the 2 years and to estimate year-to-year changes in average performance for the sample of schools. The results illustrated that technical equating adjustments can be important even when there have been rigorous efforts to write new prompts comparable in difficulty to the old and to train new rating teams to implement earlier standards of writing sample grading.
Get full access to this article
View all access options for this article.
References
1.
Evaluation Center . (1995). An independent evaluation of the Kentucky Instructional Results Information System (KRIS). Kalamazoo: Western Michigan University.
2.
Koretz, D. , Stecher, B., Klein, S., & McCaffrey, S. (1994). The Vermont portfolio assessment program: Findings and implications. Educational Measurement: Issues and Practices, 13, 5-16.
3.
Mislevy, R. J. (1983). Item response models for grouped data. Journal of Educational Statistics, 8, 271-288.
4.
Mislevy, R. J. , & Bock. R. D. (1993). PC-BILOG 3: Item analysis and test scoring with binary logistic models. Chicago: Scientific Software.
5.
Muraki, E. (1992). A generalized partial credit model: Application of an EM algorithm. Applied Psychological Measurement, 16, 159-176.
6.
Muraki, E. , & Bock, R. D. (1993). PARSCALE: IRT-based test scoring and item analysis for graded open-ended exercises and performance tasks. Chicago: Scientific Software.
7.
Tate, R. L. , & Heidorn, M. (1998). School-level IRT scaling of writing assessment data. Applied Measurement in Education, 11, 369-381.
8.
Tate, R. L. , & King, F. J. (1994). Factors which influence precision of school-level IRT ability estimates. Journal of Educational Measurement, 31, 1-15.
