Peer Review at the NSF: A Dialectical Policy Analysis

Abstract

The controversy over peer review is viewed as a dialectic. The arguments espoused by advocates and critics of the system wherein research proposals are evaluated by advisors to funding agencies are reviewed, particularly the findings of two recent studies of peer review at the National Science Foundation These findings seem to establish merit as the primary factor m the recommendations of peer reviewers to fund proposals The findings also beg several questions as to 'acceptable' definitions of meritoriousness and innovativeness, the links among belief, perception, and evaluation, and the sanctioned operation of particularistic factors m the review process Future studies, it is suggested, must include psychological variables - especially measurement of applicants' and reviewers' 'cognitive styles' - if data are to narrow gaps in knowledge and inform the debate itself Finally, three models which undergird views of peer review are discussed and related to key social issues m the debate