Grants peer review is a family of ex ante methods used by federal agencies to select research
proposalsforfunding. This article draws on Chubin and Hackett's 1990 book, Peerless Science:
Peer Review and U.S. Science Policy, to examine both the assumptions and theory underlying
peer review as an evaluation methodology and the issues that arise in the operation of peer-based
systems at the National Science Foundation (NSF) and the National Institutes of Health (NIH).
The article discusses criticisms of peer review, the various criteria that agencies must balance
in interpreting evaluations of expert peers, and outstanding issues to be addressed in the
refinement and reform of peer review systems. The article concludes with suggestions on how
to redress the shortcomings of peer review in decision making, especially the allocation of scarce
public monies.
Get full access to this article
View all access options for this article.
References
1.
Atkinson, R.C., and W.A. Blanpied.1985. Peer review and the public interest . Issues in Science & Technology2:101-14.
2.
Brooks, H.1978. The problem of research priorities. Daedalus107:171-90.
3.
Burd, S.1992. Rift grows between scholars and U.S. officials over way federal funds are awarded. The Chronicle of Higher Education , July 29, A18-A19.
4.
Chubin, D.E.1986. Scientific progress: An interim report. BioScience36:234-35.
5.
—. 1987. Designing research program evaluations: A science studies approach. Science and Public Policy14:82-90.
6.
—. 1990. Doing policy analysis for Congress: The OTA process. The Weaver8:8-9.
7.
Chubin, D.E., and E.J. Hackett.1989. Commentary: On the virtues of self-study . Science, Technology, and Human Values14:96-99.
8.
—. 1990. Peerless science: Peer review and U.S. science policy . Albany, NY: State University of New York Press.
9.
Chubin, D.E., and M.C. LaFollette .1990. Science community needs its conduct rules to be explicit. The Scientist, February 5, 17, 19.
10.
Cole, S., J.R. Cole, and G.A. Simon.1981. Chance and consensus in peer review . Science214:881-86.
11.
Cole, S., L. Rubin, and J.R. Cole.1978. Peer review in the NSF: Phase I ofa study. Washington, DC: National Academy of Sciences.
12.
Kostoff, R.N.1991. A quantitative approach to determining the impact of research . In Proceedings of the 22nd Annual Pittsburgh Conference on Modeling and Simulation
13.
Marshall, E.1992. When does intellectual passion become conflict-of interest ? Science31:620-21.
14.
National Institutes of Health.1976. Grants peer review: Report to the director, NIH phase I. Washington, DC.
15.
National Science Foundation Advisory Committee on Merit Review.1986. Final report. NSF 86-93. Washington, DC.
16.
National Science Foundation.1988. Proposal review at NSF: Perceptions of principal investigators. NSF 88-4. Washington, DC.
17.
—. 1990. Report of the Merit Review Task Force. NSF 90-113. Washington, DC.
18.
Office of Technology Assessment (OTA).1990 . Proposal pressure in the 1980s: An indicator of stress on the federal research system. A staff paper. Washington, DC.
19.
—. 1991. Federally funded research: Decisions for a decade. Washington, DC: Government Printing Office.
20.
Peer review and public policy [Special issue]. 1985. Science, Technology, and Human Values10.
21.
Porter, A.L., D.E. Chubin, and X-Y. Jin.1988. Citations and scientific progress: Comparing bibliographic measures with scientists' judgments. Scientometrics13:103-24.
Roy, R.1985. Funding science: The real defects of peer review and an alternative to it. Science, Technology, and Human Values10:74.
24.
Shapley, D., and R. Roy.1985. Lost at the frontier: U.S. science and technology policy adrift. Philadelphia, PA: ISI Press.
25.
Studer, K.E., and D.E. Chubin.1980. The cancer mission: Social contexts of biomedical research. Beverly Hills, CA: Sage.
26.
U.S. Congress. House Committee on Science, Space, and Technology.1992. Report of the task force on the health of research. 102d Cong. 2d sess. Washington, DC: Government Printing Office.
27.
U.S. General Accounting Office.1987. University funding: Information on the role of peer review at NIH and NSF . Washington, DC: Government Printing Office.
28.
—. 1991. Peer review: Compliance with the Privacy Act and Federal Advisory Committee Act. Washington, DC.
29.
Weinberg, A.M.1966. Reflections on big science. Cambridge, MA: MIT Press.
