The evaluation of research is an activity undergoing change. The traditional peer review system with its focus on scientific content and methodology has long been the backbone of research evaluation, but over the last three decades other criteria and considerations have also been integrated into the evaluation of science. This article investigates how recent societal developments - epitomized by the concept of the agora - influence research evaluations and how we perceive them. Are they still grounded in a scientific rationale, or are they more to be understood as a result of a social rationale?
Get full access to this article
View all access options for this article.
References
1.
Albæk, E. (1996) `Why All This Evaluation? Theoretical Notes and Empirical Observations on the Function and Growth of Evaluation, with Denmark as an Illustrative Case' , Canadian Journal of Program Evaluation11 (2): 1-34 .
2.
Barré, R. (2001) `The Agora Model of Innovation Systems, S&T Indicators for a Democratic Knowledge Society' , Research Evaluation10(1): 13-18 .
3.
Beck, U. (1999) World Risk Society. Cambridge: Polity Press .
4.
Beck, U., S. Lash and A. Giddens (1994) Reflexive Modernization. Politics, Tradition and Aesthetics in the Modern Social Order. Cambridge: Polity Press .
5.
Brunsson, N. and B. Jacobsson (2000) A World of Standards. Oxford: Oxford University Press .
6.
Bush, V. (1945) Science The Endless Frontier. A Report to the President. Washington, DC: United States Government Printing Office .
7.
Cave, M., S. Hanney, M. Henkel and M. Kogan (1997) The Use of Performance Indicators in Higher Education: the Challenge of the Quality Movement, 3rd edn. London: Jessica Kingsley Publishers .
8.
Cicchetti, D. (1991) `The Reliability of Peer Review for Manuscript and Grant Submission' , Behavioral and Brain Sciences14(1): 119-186 .
9.
Clark, B. R. (1998) Creating Entrepreneurial Universities: Organizational Pathways of Transformation. Oxford: Pergamon .
10.
Council Document (1999) Declaration by the Danish, Greek, French, Italian and Luxembourg Delegations Concerning the Suspension of New GMO Authorisations. Council Document 9406/99 . 2194th Council meeting, Environment. 24-5 June 1999, Luxembourg.
11.
Dahler-Larsen, P. (2000) `Surviving the Routinization of Evaluation' , Administration and Society32(1): 70-93 .
12.
Elzinga, A. (1995) `Reflections on Research Evaluation' , Science Studies8(1): 5-23 .
13.
Etzkowitz, H. and L. Leydesdorff (1997) Universities and the Global Knowledge Economy: a Triple Helix of University-Industry-Government Relations. London: Pinter .
14.
Etzkowitz, H. and L. Leydesdorff (2000) `The Dynamics of Innovation: From National Systems and “Mode 2” to a Triple Helix of University-Industry-Government Relations' , Research Policy29(2): 109-123 .
15.
Expert Group (EU) (2002a) Benchmarking the Promotion of RTD Culture and Public Understanding of Science. Brussels: European Commission .
16.
Expert Group (EU) (2002b) Benchmarking S&T Productivity. Brussels: European Commission .
17.
Foss Hansen, H. (1989) Academic Autonomy, Bureaucracy and Market: The Development of Public Control of the Universities. The Case of Denmark. Copenhagen: COS .
18.
Foss Hansen, H. and F. Borum (1999) `The Construction and Standardization of Evaluation. The Case of the Danish University Sector' , Evaluation5(3): 303-329 .
19.
Foss Hansen, H. and F. Borum (2000) `The Local Construction and Enactment of Standards for Research Evaluation. The Case of the Copenhagen Business School' , Evaluation6(3): 281-299 .
20.
Frederiksen, L. F. and K. Husted (2002) `Academic Publishing Activities by Danish Companies. A Preliminary Assessment' , Vest15(2-3): 9-25 .
21.
Fujigaki, Y. and L. Leydesdorff (2000) `Quality Control and Validation Boundaries in a Triple Helix of University-Industry-Government: “Mode 2” and the Future of University Research' , Social Science Information39(4): 635-655 .
22.
Gibbons, M. and B. Wittrock (1985) Science as a Commodity. Threats to the Open Community of Scholars Conference, 1983, Stockholm. Harlow: Longman .
23.
Gibbons, M., C. Limoges, H. Nowotny, S. Schwartzman, P. Scott and M. Trow (1994) The New Production of Knowledge. The Dynamics of Science and Research in Contemporary Societies. London: Sage .
24.
Gieryn, T. (1983) `Boundary-work and the Demarcation of Science from Non-science: Strains and Interests in Professional Ideologies of Scientists' , American Sociological Review48(6): 781-795 .
25.
Gieryn, T. (1994) `Boundaries of Science', in S. Jasanoff (ed.) Handbook of Science and Technology Studies, pp. 393-443. Thousand Oaks, CA: Sage .
26.
Hansson, F. (1998) `Evaluation Research and Sociology in Denmark. A Tale of Two Cultures' , New Directions for Evaluation77(Spring): 55-71 .
27.
Hansson, F. (2002) `How to Evaluate and Select New Scientific Knowledge? Taking the Social Dimension Seriously in the Evaluation of Research Quality' , Vest15(2-3): 27-52 .
28.
Hansson, F. and M. Horst (2000) `To Trump the Passions. The Use of Scientific Knowledge in the Introduction of New Gene Technologies in Europe', in J. M. Carvalho de Mello and H. Etzkowitz (eds) The Endless Transition: Relations among Social, Economic and Scientific Development in a Triple Helix of University-Industry-Government Relations. Rio de Janeiro: ITOI, Federal University of Rio de Janeiro .
29.
Hemlin, S. (1996) `Research on Research Evaluation' , Social Epistemology10(2): 209-250 .
30.
House, E. R., C. Haug and N. Norris (1996) `Producing Evaluations in a Large Bureaucracy' , Evaluation2(2): 135-150 .
31.
INRA (Europe) European Coordination Office (1998) Eurobarometer 49 - Food Safety. DG24. Brussels: European Commission .
32.
INRA (Europe) European Coordination Office (2000) Eurobarometer 52.1 - The European and Biotechnology. DG for Research, DG B. Brussels: European Commission .
33.
Kostoff, R. N. (1997) The Handbook of Research Impact Assessment, 7th edn, DTIC Report Number ADA296021. Office of Naval Research. Also available at http://www.dtic.mil/dtic/kostoff/index.html (website consulted 3 March 2003).
34.
Luks, F. (1999) `Post-normal Science and the Rhetoric of Inquiry: Deconstructing Normal Science?' , Futures31(7): 705-719 .
35.
Merton, R. K. (1968a) `The Matthew Effect in Science. The Reward and Communication Systems of Science are Considered' , Science159(3810): 56-63 .
36.
Merton, R. K. (1968b) Social Theory and Social Structure. New York: The Free Press .
37.
Merton, R. K. (1973) Sociology of Science. Theoretical and Empirical Investigations. Chicago, IL: University of Chicago Press .
38.
National Science Foundation (2001) FY 2001 Report on the National Science Foundation Merit Review Process. Washington, DC: NSF .
39.
Nowotny, H., M. Gibbons and P. Scott (2001) Re-thinking Science. Knowledge and the Public in an Age of Uncertainty. Oxford: Polity Press .
40.
OECD (1997a) The Evaluation of Scientific Research: Selected Experiences. OECD/GD (97)194. Paris: OECD .
41.
OECD (1997b) Promoting Public Understanding of Science and Technology. Paris: OECD .
42.
Oppenheim, C. (1997) `The Correlation between Citation Count and the 1992 Research Assessment Exercise Rating for British Research in Genetics, Anatomy and Archaeology' , Journal of Documentation53(5): 477-487 .
43.
Porter, T. M. (1995) Trust in Numbers. The Pursuit of Objectivity in Science and Public Life. Princeton, NJ: Princeton University Press .
44.
Power, M. (1997) The Audit Society. Rituals of Verification. Oxford: Oxford University Press .
45.
Ravetz, J. R. (1999) `What is Post-normal Science?' , Futures31(7): 647-653 .
46.
Stokes, D. E. (1997) Pasteur's Quadrant. Basic Science and Technological Innovation. Washington, DC: The Brookings Institution .
47.
van Raan, A. F. J. (1998) `Assessment of Social Sciences: The Use of Advanced Bibliometric Methods as a Necessary Complement of Peer Review' , Research Evaluation7(1): 2-6 .
48.
Weber, M. (1992) Wissenschaft Als Beruf 1917/1919. Politik Als Beruf 1919 (eds W. Schluchter, B. Morgenbrod and W. J. Mommsen). Tübingen: Mohr .
49.
Weiss, C. H. (1999) `The Interface between Evaluation and Public Policy' , Evaluation5(4): 468-486 .
50.
Weiss, C. H. and M. Bucuvalas (1980) Social Science Research and Decision-making. New York: Columbia University Press .
51.
Wynne, B. (1996) `Misunderstood Misunderstandings: Social Identities and Public Uptake of Science', in B. Wynne and A. Irwin (eds) Misunderstanding Science? The Public Reconstruction of Science and Technology, pp. 19-46. Cambridge: Cambridge University Press .
52.
Yearley, S. (2000) `What Does Science Mean in the “Public Understanding of Science”?', in D. Meinolf and C. von Grote (eds) Between Understanding and Trust, pp. 217-236. Amsterdam: Harwood Academic .
53.
Ziman, J. (1994) Prometheus Bound. Science in a Dynamic Steady State. Cambridge: Cambridge University Press .
54.
Ziman, J. (2000) Real Science. What It Is, and What It Means. Cambridge: Cambridge University Press .
