A computer program to be used to assess interrater reliabilities has been written. Given judges' ratings on a set of variables pertaining to subjects or events, the program will produce for each variable, a printed analysis of variance summary table for between subjects/ events, within subjects/events, and between judges sources of variability, a reliability coefficient, an adjusted reliability coefficient and means and standard deviations for each rater. Further, analysis of variance and reliability output for subsets of judges is generated.
Get full access to this article
View all access options for this article.
References
1.
Anastasi, A.Psychological testing. (3rd ed.). New York: The Macmillan Company, 1968.
2.
Ebel, R. L.Estimation of the reliability of ratings. Psychometrika, 1951, 16, 407-424.
3.
Greene, J. F.Scoring creativity tests by computer simulation. Unpublished doctoral dissertation, University of Connecticut, 1970.
4.
Winer, B. J.Statistical principles in experimental design. New York: McGraw-Hill, 1962.
