I describe and evaluate an exercise in which graduate students provide blind reviews of each other's term papers. Besides helping students with their own papers, the exercise seems to help them learn to give constructive, substantive feedback to colleagues. Student ratings of the educational value of peer review are high, but a hoped-for effect of increasing interest in publishing papers has not emerged. A study of reviews from three courses suggested that students' reviews show higher interrater reliability than do professional peer reviews of journal manuscripts. Implications of this finding are discussed in relation to some of the explanations offered in the literature for the low reliability of professional peer reviews.
Get full access to this article
View all access options for this article.
References
1.
AdamsK. M. (1991). Peer review: An unflattering picture. Behavioral and Brain Sciences, 14, 135–136.
2.
BailarJ. C. (1991). Reliability, fairness, objectivity and other inappropriate goals in peer review. Behavioral and Brain Sciences, 14, 137–138.
3.
BeersS. E. (1986). Questioning and peer collaboration as techniques for thinking and writing about personality. Teaching of Psychology, 13, 75–77.
4.
BowersK. S. (1973). Situationism in psychology: An analysis and a critique. Psychological Review, 80, 307–336.
5.
CampleseD. A.MayoJ. A. (1982). How to improve the quality of student writing: The colleague swap. Teaching of Psychology, 9, 122–123.
6.
CicchettiD. V. (1991). The reliability of peer review for manuscript and grant submissions: A cross-disciplinary investigation. Behavioral and Brain Sciences, 14, 119–186.
7.
CrandallR. (1991). What should be done to improve reviewing?Behavioral and Brain Sciences, 14, 143.
8.
DelcomynF. (1991). Peer review: Explicit criteria and training can help. Behavioral and Brain Sciences, 14, 144.
9.
GarciaJ. (1981). Tilting at the paper mills of academe. American Psychologist, 36, 149–158.
10.
HargensL. L. (1991). Referee agreement in context. Behavioral and Brain Sciences, 14, 150–151.
11.
HolmesT. H.RaheR. H. (1967). The social readjustment rating scale. Journal of Psychosomatic Research, 11, 213–218.
12.
HowardG. S. (1982). Improving methodology via research on research methods. Journal of Counseling Psychology, 29, 318–326.
13.
KieslerC. A. (1991). Confusion between reviewer reliability and wise editorial and funding decisions. Behavioral and Brain Sciences, 14, 151–152.
14.
KraemerH. C. (1991). Do we really want more “reliable” reviewers?Behavioral and Brain Sciences, 14, 152–154.
15.
LamingD. (1991). Why is the reliability of peer review so low?Behavioral and Brain Sciences, 14, 154–156.
16.
LykkenD. T. (1968). Statistical significance in psychological research. Psychological Bulletin, 70, 151–159.
17.
MahoneyM. J. (1977). Publication prejudices: An experimental study of confirmatory bias in the peer review system. Cognitive Therapy and Research, 1, 161–175.
18.
MahoneyM. J. (1986). The tyranny of technique. Counseling and Values, 30, 169–174.
19.
MahoneyM. J. (1991). Human change processes: The scientific foundations of psychotherapy. New York: Basic Books.
MarshH. WBallS. (1981). Interjudgmental reliability of reviews for the Journal of Educational Psychology. Journal of Educational Psychology, 73, 872–880.
22.
MarshH. W.BallS. (1989). The peer review process used to evaluate manuscripts submitted to academic journals: Interjudgmental reliability. Journal of Experimental Education, 57, 151–169.
23.
MarshH. W.BallS. (1991). Reflections on the peer review-process. Behavioral and Brain Sciences, 14, 157–158.
24.
MilgramS. (1963). Behavioral study of obedience. Journal of Abnormal and Social Psychology, 67, 371–378.
25.
MillerA. G. (1986). The obedience experiments: A case study of controversy in social science. New York: Praeger.
26.
PetersD. P.CeciS. J. (1982). Peer-review practices of psychological journals: The fate of published articles submitted again. Behavioral and Brain Sciences, 5, 187–255.
27.
PirsigR. M. (1974). Zen and the art of motorcycle maintenance: An inquiry into values. New York: Morrow.
28.
PirsigR. M. (1991). Lila: An inquiry into morals. New York: Bantam.
29.
RoedigerH. L. (1991). Is unreliability in peer review harmful?Behavioral and Brain Sciences, 14, 159–160.
30.
SmelserN. J. (1987). Contemporary classics in the social and behavioral sciences. Philadelphia: ISI Press.
31.
StriekerL. J. (1991). Disagreement among journal reviewers: No cause for undue alarm. Behavioral and Brain Sciences, 14, 163–164.
