Partial Credit Analysis of Writing Ability

Abstract

The direct assessment of the writing ability of 2000 randomly selected secondary school students was performed using the partial credit model. The effects on item parameter estimates of rating scale and type of writing sample were investigated. The rating scales used were a sum and an averaging of the individual raters' ratings and the types of items were expository and narrative. Results showed that expository items tended to provide more information for high ability examinees than did narrative items and that the sum holistic rating method yielded greater information than did the average holistic rating scale. Additional analyses of the rating scales used as well as implications for test construction are discussed.

Get full access to this article

View all access options for this article.

References

Ackerman, T. (1986, April). Use of the graded response IRT model to assess the reliability of direct and indirect measures of writing assessment. Paper presented at the Annual Meeting of the American Educational Research Association, San Francisco, CA.

Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika, 43, 561-573.

Bock, R. D. (1972). Estimating item parameters and latent ability when responses are scored in two or more nominal categories. Psychometrika, 37, 29-51.

Breland, H. M. (1983). The direct assessment of writing skill: A measurement review. ETS Research Report (86-9). Princeton, NJ: Educational Testing Service.

Coffman, W. E. (1971). Essay examinations. In R. L. Thorndike (Ed.), Educational Measurement. Washington, D.C.: American Council on Education.

Dodd, B. G. and Koch, W. R. (1987). Effects of variations in item step values on item and test information in the partial credit model. Applied Psychological Measurement, 4, 371-384.

Godshalk, F. I. , Swineford, F. , and Coffman, W. E. (1966). The measurement of writing ability. New York: College Entrance Examination Board.

Masters, G. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149-174.

Masters, G. and Wilson, M. (1988, April). Understanding and using partial credit analysis: An IRT method for ordered response categories. Paper presented at the Annual Meeting of the American Educational Research Association, San Francisco, CA.

10.

Pollitt, A. and Hutchinson, C. (1987). Calibrating graded assessments: Rasch partial credit analysis of performance in writing. Language Testing, 4, 72-92.

11.

Samejima, F. (1977). A use of the information function in tailored testing. Applied Psychological Measurement, 1, 233-247.

12.

Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph Supplement, (No. 17).

13.

Stiggins, R. J. , and Bridgeford, N. J. (1983). An analysis of published tests of writing proficiency. Educational Measurement: Issues and Practice, 2, 6-10 and 26.

14.

Thissen, D. J. (1988). MULTILOG-User's Guide (Version 5.1) Mooresville, IN: Scientific Software.

15.

Thissen, D. J. , and Steinberg, L. (1986). A taxonomy of item response models. Psychometrika, 51, 567-577.