A Comparison of the Reliability and Validity of Ratings of Student Performance on Essay Examinations by Professors of English and by Professors in Other Disciplines

Abstract

For two essay questions—Question 1 given to one sample (N ₁ = 100) and Question 2 to a second group (N ₂ = 100)—ratings of student performance rendered by professors of English and by professors in other disciplines were compared for reliability and concurrent validity. From the data analyses it was concluded that the reliability and validity of the ratings provided by professors outside of English departments and by professors in English departments were nearly comparable. It appeared that differences in the reliability and concurrent validity of ratings assigned to writing samples of students might be more a function of the nature of the question posed or of variations in the average ability level of examinee groups than of differences in level of expertise on the part of readers.

Get full access to this article

View all access options for this article.

References

Breland, H. M. and Gaynor, J. L. A comparison of direct and indirect assessments of writing skill. Journal of Educational Measurement, 1979, 16, 119-128.

Brengelman, F. H. and Frey, J. E. A Diagnostic Test in Written English, Revised Version, Form B. Fresno, Calif.: California State University , c. 1975.

Educational Testing Service. Test of Standard Written English. Princeton, N. J.: Author, 1974- 1978.

Guilford, J. P. Fundamental statistics in psychology and education (4th ed.). New York: McGraw-Hill Book Co., 1965.

Michael, W. B. and Shaffer, P. An evaluation of alternative measures for assessing language proficiency skills of junior-level students in a School of Communications and Professional Studies (CAPS. EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT, 1979, 39, 879-889.