This study examines the tension between the academic evaluation system and the development of unconventional research agendas. While scholars have studied the evaluation of research that crosses disciplinary boundaries from the perspective of peer reviewers, they have paid comparatively little attention to the experiences of the performers of unconventional science. This study asks how researchers develop unconventional research agendas to address a long-standing health problem and, in the process, make sense of the actions of a site visit committee organized to advise a foundation funding the project. This study develops a process narrative on the development of a specific transdisciplinary team proposing to study premature birth. The findings show that when the performers and the evaluators of unconventional science developed competing understandings of the research agenda, transdisciplinary discovery became limited as a particular research topic became taboo. Yet, the study also reveals how the performers of unconventional science challenged the power of the site visit committee by making it seem as though they followed the committee’s decisions. These findings raise questions about the role of private foundations as funders of academic research and the suitability of traditional evaluation procedures for assessing transdisciplinary discovery.
BarréRemi. 2010. “Towards Socially Robust S&T Indicators: Indicators Are Debatable Devices, Enabling Collective Learning.” Research Evaluation19 (3): 227–31.
2.
BarryDavidCarrollBrigidHansenHans. 2006. “To Text or Context? Endotextual, Exotextual, or Multi-textual Approaches to Narrative and Discursive Organizational Studies.” Organization Studies27 (8): 1091–1110.
3.
BecherTobyTrowlerPaul R.. 2001. Academic Tribes and Territories. Philadelphia, PA: The Society for Research into Higher Education and Open University Press.
Boix MansillaVeronica. 2006. “Interdisciplinary Work at the Frontier: An Empirical Examination of Expert Interdisciplinary Epistemologies.” Issues in Integrative Studies31 (24): 1–31.
CallonMichel. 1986. “Some Element of a Sociology of Translation: Domestication of the Scallops and the Fishermen of St Brieuc Bay.” In Power, Action, and Belief: A New Sociology of Knowledge?, edited by LawJohn, 196–233. London, UK: Routledge & Kegan Paul.
8.
ChubinDaryl E.HackettEdward J.. 1990. Peerless Science. Peer Review and U.S. Science Policy. Albany: State University of New York Press.
9.
CorbinJulietStraussAnselm. 2008. Basics of Qualitative Research. Thousand Oaks, CA: Sage.
10.
DanielHans-DieterMittagSandraBornmannLutz. 2007. “The Potential and Problems of Peer Evaluation in Higher Education and Research.” In Quality Assessment for Higher Education in Europe, edited by CavalliAlessandro, 71–82. London, UK: Portland Press.
11.
EbadiAshkanSchiffauerovaAndrea. 2015. “How to Receive More Funding for Your Research? Get Connected to the Right People!” PLoS One10 (7): e0133061. doi:10.1371/journal.pone.0133061.
12.
FoucaultMichel. 1977. Discipline and Punish. Harmondsworth, UK: Penguin.
13.
FoucaultMichel. 1983. “Afterword: The Subject and Power.” In Michel Foucault: Beyond Structuralism and Hermeneutics, edited by DreyfusHubert L.RabinowPaul, 208–26. Chicago, IL: University of Chicago Press.
14.
GalloStephen A.SullivanJoanne H.GlissonScott R.. 2016. “The Influence of Peer Reviewer Expertise on the Evaluation of Research Funding Applications.” PLoS One11 (10): e0165147. doi:10.1371/journal.pone.0165147.
15.
GillespieGilbert W.ChubinDaryl E.KurzonGeorge M.. 1985. “Experience with NIH Peer Review: Researchers’ Cynicism and Desire for Change.” Science, Technology, & Human Values10 (3): 44–54.
16.
GläserJochenLaudelGrit. 2007. “Evaluation without Evaluators: The Impact of Funding Formulae on Australian University Research.” In The Changing Governance of the Sciences. The Advent of Research Evaluation Systems, edited by WhitleyRichardGläserJochen, 127–51. Dordrecht, the Netherlands: Springer.
17.
HeracleousLoizosHendryJohn. 2000. “Discourse and the Study of Organization: Toward a Structurational Perspective.” Human Relations53 (10): 1251–86.
18.
HuutoniemiKatri. 2012. “Communicating and Compromising on Disciplinary Expertise in the Peer Review of Research Proposals.” Social Studies of Science42 (6): 897–921.
19.
JacobsJerry A.FrickelScott. 2009. “Interdisciplinarity: A Critical Assessment.” Annual Review of Sociology35 (2009): 43–65.
20.
Knorr-CetinaKarin. 1999. Epistemic Cultures: How the Sciences Make Knowledge. Cambridge, MA: Harvard University Press.
21.
KuhnThomas. 1970. The Structure of Scientific Revolutions. Chicago, IL: University of Chicago Press.
22.
LamontMichèle. 2009. How Professors Think. Inside the Curious World of Academic Judgment. Cambridge, MA: Harvard University Press.
23.
LamontMichèle. 2012. “Toward a Comparative Sociology of Valuation and Evaluation.” Annual Review of Sociology38 (21): 201–21.
24.
LamontMichèleHuutoniemiKatri. 2011. “Comparing Customary Rules of Fairness: Evaluative Practices in Various Types of Peer Review Panels.” In Social Knowledge in the Making, edited by CamicCharlesGrossNeilLamontMichèle, 209–32. Chicago, IL: University of Chicago Press.
25.
LangfeldtLiv. 2001. “The Decision-making Constraints and Processes of Grant Peer Review, and Their Effects on the Review Outcome.” Social Studies of Science31 (6): 820–41.
26.
LangfeldtLiv. 2004. “Expert Panels Evaluating Research: Decision-making and Sources of Bias.” Research Evaluation13 (1): 51–62.
27.
LangfeldtLiv. 2006. “The Policy Challenges of Peer Review: Managing Bias, Conflict of Interests and Interdisciplinary Assessments.” Research Evaluation15 (1): 31–41.
28.
LangfeldtLivKyvikSvein. 2010. “Researchers as Evaluators: Tasks, Tensions and Politics.” Higher Education62 (2): 199–212.
29.
LeeCarole J.2015. “Commensuration Bias in Peer Review.” Philosophy of Science82 (5): 1272–83.
30.
LeeCarole J.SugimotoCassidy R.ZhangGuoCroninBlaise. 2013. “Bias in Peer Review.” Journal of the American Society for Information Science and Technology64 (1): 2–17.
31.
LeisyteLiudvikaEndersJürgenBoerHarry de. 2010. “Mediating Problem Choice: Academic Researchers’ Responses to Changes in their Institutional Environment.” In Reconfiguring Knowledge Production: Changing Authority Relationships in the Sciences and Their Consequences for Intellectual Innovation, edited by WhitleyRichardGläserJochenEngwallLars, 266–90. Oxford, UK: Oxford University Press.
32.
LoflandJohnLoflandLyn H.. 1995. Analyzing Social Settings. Belmont, CA: Wadsworth.
33.
LuukkonenTerttu. 2012. “Conservatism and Risk-taking in Peer Review: Emerging ERC Practices.” Research Evaluation21 (1): 48–60.
34.
LuukkonenTerttuThomasDuncan A.. 2016. “The ‘Negotiated Space’ of University Researchers’ Pursuit of Research Agenda.” Minerva54 (1): 99–127.
35.
MaasenSabineLievenOlivier. 2006. “Transdisciplinarity: A New Mode of Governing Science?” Science and Public Policy33 (6): 399–410.
36.
MainesDavid R.1993. “Narrative’s Moment and Sociology’s Phenomena: Toward a Narrative Sociology.” The Sociological Quarterly34 (1): 17–38.
37.
MallardGregoireLamontMichèleGuetzkowJoshua. 2009. “Fairness as Appropriateness. Negotiating Epistemological Differences in Peer Review.” Science, Technology, & Human Values34 (5): 573–606.
38.
McCulloughJim. 1989. “First Comprehensive Survey of NSF Applicants Focuses on their Concerns about Proposal Review.” Science, Technology, & Human Values14 (1): 78–88.
39.
MertonRobert K.1973. “The Normative Structure of Science.” InThe Sociology of Science. Theoretical and Empirical Investigations, edited by StorerNorman W., 267–278. Chicago, IL: University of Chicago Press.
40.
MeskusMiannaMarelliLucaD’AgostinoGiuseppe. 2018. “Research Misconduct in the Age of Open Science. The Case of STAP Stem Cells.” Science as Culture27 (1): 1–23.
NIH (National Institutes of Health). 2013. About NIH: Mission. Accessedhttp://nih.gov/about/mission.htm (accessed May 25, 2014).
43.
OverbekeJohnWagnerElizabeth. 2003. “The State of the Evidence: What We Know and What We Don’t Know about Journal Peer Review.” In Peer Review in Health Sciences, edited by GodleeFionaJeffersonTom, 45–61. London, UK: BMJ Books.
44.
PolkinghorneDonald E.1988. Narrative Knowing and the Human Sciences. Albany: State University of New York Press.
45.
PorterAlan L.RossiniFrederick A.. 1985. “Peer-review of Interdisciplinary Research Proposals.” Science, Technology, & Human Values10 (3): 33–38.
46.
RoelofsJoan. 2015. “How Foundations Exercise Power.” The American Journal of Economics and Sociology74 (4): 654–75.
47.
SchneiderWilliam H.2015. “The Origin of the Medical Research Grant in the United States: The Rockefeller Foundation and the NIH Extramural Funding Program.” Journal of the History of Medicine and Allied Sciences70 (2): 279–311.
48.
ShoreCrisWrightSusan. 2000. “Coercive Accountability. The Rise of Audit Culture in Higher Education.” In Audit Cultures. Anthropological Studies in Accountability, Ethics, and the Academy, edited by StrathernMarilyn, 57–89. London, UK: Routledge.
49.
SmithElta. 2009. “Imaginaries of Development: The Rockefeller Foundation and Rice Research.” Science as Culture18 (4): 461–82.
50.
StraussAnselm. 1987. Qualitative Analysis for Social Scientists. Cambridge, UK: Cambridge University Press.
51.
StraussAnselm L.SchatzmanLeonardEhrlichDanutaBucherRueSabshinMelvin. 1963. “The Hospital and Its Negotiated Order.” In The Hospital in Modern Society, edited by FriedsonEliot, 147–69. Glencoe, NY: Free Press.
52.
TravisG. D. L.CollinsH. M.. 1991. “New Light on Old Boys: Cognitive and Institutional Particularism in the Peer Review System.” Science, Technology, & Human Values16 (3): 322–41.
53.
van der LaanAnna LauraBoeninkMarianna. 2015. “Beyond Bench and Bedside: Disentangling the Concept of Translational Research.” Health Care Analysis23 (1): 32–49.
54.
WellerAnn C.2001. Editorial Peer Review: Its Strengths and Weaknesses. Medford, NJ: Information Today.
55.
WesselySimon. 1998. “Peer Review of Grant Applications: What Do We Know?” Lancet352 (9124): 301–5.
56.
WhitleyRichard. 2010. “Reconfiguring the Public Science: The Impact of Governance Changes on Authority and Innovation in Public Science Systems.” In Reconfiguring Knowledge Production: Changing Authority Relationships in the Sciences and their Consequences for Intellectual Innovation, edited by WhitleyRichardGläserJochenEngwallLars, 3–47. Oxford, UK: Oxford University Press.
57.
ZimanJohn. 1994. Prometheus Bound. Science in a Dynamic Steady State. Cambridge, UK: Cambridge University Press.
