Abstract
How are “interesting” research problems identified and made durable by academic researchers, particularly in situations defined by multiple evaluation principles? Building on two case studies of research groups working on rare diseases in academic biomedicine, we explore how group leaders arrange their groups to encompass research problems that latch onto distinct evaluation principles by dividing and combining work into “basic-oriented” and “clinical-oriented” spheres of inquiry. Following recent developments in the sociology of (e)valuation comparing academics to capitalist entrepreneurs in pursuit of varying kinds of worth, we argue that the metaphor of the portfolio is helpful in analyzing how group leaders manage these different research lines as “alternative investment options” from which they were variously hoping to capitalize. We argue portfolio development is a useful concept for exploring how group leaders fashion “entrepreneurial” practices to manage and exploit tensions between multiple matrices of (e)valuation and conclude with suggestions for how this vocabulary can further extend analysis of epistemic capitalism within science and technology studies.
Introduction
Classic studies of laboratories in science and technology studies (STS) showed how in seeking to study “ripe” intellectual problems, researchers would align what they considered “interesting” problems (based on their interpretation of the research frontier) with a myriad of mundane factors, which, together, would determine whether the research was deemed “doable” or not (Fujimura 1987). In fields like biomedicine, Fujimura showed how laboratory leaders put in place conditions for the construction of doable problems through “articulating alignment” between experimental, laboratory, and social world levels of work organization. Hackett (2005) later added to this perspective, arguing group leaders—henceforth principal investigators—pay extremely close strategic attention to opening “spheres of inquiry,” namely, research problems that they not only deemed “doable” but would likely also help the laboratory to establish an independent and durable identity within a field(s) of research (Hackett 2005, 787). Continuing this tradition, much contemporary research on the organization of scientific research pays close attention to how research governance is shaping the research process. Although cautious to embrace wholeheartedly the idea of epochal transitions toward “mode 2,” “new public management,” or “neoliberal science,” a growing number of researchers in STS have become increasingly interested and concerned about how “technologies of government” like performance indicators, audits, and standards (Miller and Rose 2008) are coming to reshape academic practices and selves (Felt 2009; Stöckelová 2012; Sigl 2016; Hammarfelt, de Rijcke, and Rushforth 2016; Hammarfelt and Rushforth 2017; Kaltenbrunner 2017; Franssen et al. 2018).
At the same time, emerging research on the sociology of (e)valuation has sought to examine the construction and effects of practices for evaluating performance within research and other domains of public policy (Lamont 2012; de Rijcke et al. 2016). Resisting singular notions of “excellence” so often inscribed in technologies for monitoring and evaluating performance, much of the emerging research on (e)valuation posits that as performance is in practice being evaluated through numerous diffuse technologies and standards, academic research is becoming increasingly accountable to multiple—sometimes complementary, sometimes conflicting—evaluative infrastructures and regimes of worth (Stark 2009; Helgesson and Muniesa 2013). Drawing on STS and sociological work on (e)valuation, we seek to consider what this multiplicity means for the long-standing concern of studies of scientific work in laboratories, namely, how do research group leaders strategically construct their inquiries?
To address this puzzle, with others, we return to early insights in STS that compared researchers to capitalists. This analytical connection to capital and capitalism can be traced back to Bourdieu (1975) and, subsequently, Latour and Woolgar’s (1986) credibility cycle and Etzkowitz’s (2003) account of research groups as “quasi-firms.” Prompted partly by the growing specter of (e)valuation, Hackett (1990, 2014), Fochler (2016), and Muniesa et al. (2017) have argued that STS has underutilized the analytic potential of capitalism as a metaphor and model for understanding academic research in recent times. One of the motivations for revisiting earlier analogies is that the rise in performance has brought about the sense that individual researchers are being rendered increasingly self-actualizing (and self-responsible) “entrepreneurial” figures needing to become adept at accumulating worth from their research within “marketplaces of ideas” (Mirowski 2011). In short, the individual researcher is becoming more and more like a capitalist. To investigate the implications for contemporary academic research practices, Fochler (2016) introduces the term epistemic capitalism, broadly conceived as “the accumulation of capital, as worth made durable, through the act of doing research” (p. 924). Fochler goes on to explain that the changing governance of academic research means that faculty and junior researchers are not simply “entrepreneurs” in a more restricted sense of commercializing their research but more broadly that they have become “entrepreneurial managers of their own careers, publications, and grant portfolios” (Fochler 2016, 924).
Importantly, the notion of capitalism here is not limited simply to accumulation of monetary capital but also to generating and accumulating other forms of worth that are durable enough to deploy as capital. In this scheme, while various kinds of worth may be cited by researchers as important and motivating, not all forms can be accumulated and reinvested in subsequent cycles of credit. For instance, producing research that goes “against the grain” or is ambitious may embody forms of epistemic worth with which researchers readily identify, yet in some institutional and disciplinary settings this work may not be capitalized upon by the researchers unless it has also appeared in a journal with, say, a high journal impact factor (JIF) score or goes onto attract a considerable number of citations (Rushforth and de Rijcke 2015). Focusing on capitalistic dynamics in the research process thus entails drawing attention to how multiplicitous forms of worth and the possibilities for accumulating capital from them shape the organization of academic research. This means approaching “capitalistic” or entrepreneurial strategies of accumulating worth not as heroic individual traits, but primarily as constituted through organizational routines and practices, which may be more available to some than others (Stark 2009). Recent sociological theory on (e)valuation also holds that situations defined by conflicting forms of worth need not necessarily be considered debilitating, because, in fact, like other forms of creative inquiry, research may even profit from “the ability to keep multiple principles of evaluation “in play” and to benefit from that productive friction” (Stark 2009, 6). In our view, a broadened account of epistemic capitalism raises important questions for the study of academic research in STS: How is academic work organized to keep different forms of worth in play? How are different forms of worth undergirded by specific organizational routines and infrastructures? How do assumptions about the relative durability of different evaluation practices guide the selection and combination of epistemic spheres of inquiry?
In this paper, we build on these emerging issues by describing strategies employed by principal investigators of two biomedical research groups in an academic medical center in the Netherlands. We argue that the portfolio provides a useful concept for describing how these principal investigators strategically managed their “spheres of inquiry” (Hackett 2005) so as to satisfy a dominant regime of worth, while also keeping alternative forms of worth in play should they become more viable in future. Our account of portfolio strategies is elaborated on principal investigators’ strategies for oscillating between two particular spheres of inquiry that appear ubiquitous in contemporary biomedicine: broader, fundamental (“basic”) research versus narrower, applied (“clinical”) research.
The term portfolio is often used in the world of finance to refer to the complete investments of a company, in art to refer both to a flat folder in which drawings are kept and the artist’s range of works. While this has been used as a metaphor to describe the range of projects invested in by research funders (Wallace and Rafols 2015), to our knowledge, it has not yet been the explicit focus of STS research on academic knowledge production. Similar to these vernaculars, we use portfolios to refer to the range of inquiries being worked on by researchers and research groups at a given moment. Informed by earlier work on laboratories in STS, we consider research portfolio construction efforts to be made possible and limited by a laboratory’s “research technology ensemble,” that is, configurations of materials, techniques, instruments, and ideas and enabling theories drawn upon in producing new research (Hackett et al. 2004, 748). Technology ensembles tend to build up and evolve in ad hoc fashion as local adaptations of international “research systems”: sets of epistemic practices, technologies, and standards used within international fields of research to make particular epistemic objects knowable (Hackett et al. 2016). Rheinberger (1997) describes such ensembles (what he calls “experimental systems”) as important in helping to create stability in the practices and identities of research laboratories (“reproduction”) amid constant turnover in “personnel, results, and environment” (cited in Hackett 2005, 794). He contends that technological ensembles are not only “machines” for extracting or testing particular forms of information but also underpin open-ended inquiries into phenomena (“machines to make a future”; Rheinberger 1997). However, as well as making particular spheres of inquiry possible and durable, over time research technology ensembles generate path dependencies and inertia for groups, narrowing how their work is identified among international colleagues and competitors (“signature”; Jacob 1989) and constraining the research problems members of the group are likely to consider doable (Fujimura 1987). Given that fates of principal investigators and personnel are to varying degrees tied into the affordances and constraints of their technological ensembles, it is not surprising that, much of the time, the thoughts and energies of principal investigators center on negotiating “essential tensions” of reproduction and replenishment (Hackett 2005; Foster, Rzhetsky, and Evans 2015). Building on these perspectives, the concept of “portfolios” offers a timely way of sensitizing how principle investigators develop entrepreneurial strategies for consolidating and refreshing spheres of inquiry against a multiplicity of worth regimes in contemporary biomedical research.
This paper is organized as follows. In the following section, we argue that the different kinds of worth regimes circulating in contemporary biomedical research resemble what Stark (2009) labels “heterarchy.” Accounts of worth in biomedicine have often swung between notions of basic and “applied” research, which, while long-standing in the history of the field, have also found renewed expression and urgency in recent debates about the performance and governance of biomedical sciences in the Netherlands and beyond. We then give an overview of the fieldwork and methods used to study our two cases, before presenting the main findings. In the first section for each case, we discuss how different kinds of research questions that were posed within the groups related to basic or applied clinical forms of epistemic worth. In the second section of each case, we then describe how portfolio arrangements enabled the group leaders to capitalize on research lines from these two spheres of inquiry by drawing on two regimes of worth: excellence and “patient relevance”. We end by discussing the significance of the findings in relation to recent STS interest in academic (e)valuation practices and epistemic capitalism.
Basic and Applied Spheres of Inquiry in Biomedicine
As with previous STS works on basic research, we do not hold there to be any essential distinction between basic and applied clinical knowledge per se, as such terms are well known to serve rhetorical functions as political symbols in various science policy and research communities (Pielke 2012; see also Calvert 2006). However, it would be analytically shortsighted to consider these categories as purely rhetorical or illusory (Hoffman 2015). Historical accounts of biomedicine have shown convincingly how, since the second half of the Twentieth Century, distinctions between basic and clinical research have come to be reified and inscribed within institutions of academic biomedicine and clinical practice (Löwy 1996; Keating and Cambrosio 2003). Taking inspiration from Hoffman’s (2015) recent account of how multiple realities of basic and applied research were enacted in the context of an artificial intelligence research laboratory, our approach is to describe how notions like basic and clinical research (and forms of worth associated with them) fragment and multiply as our informants related to different research problems in the course of their work. To do so, we draw on and develop the argument that basic research and clinical research problems relate to separate yet interacting “regimes of worth”: what our informants categorized as excellence and patient relevance, respectively. Regimes of worth are evaluative principles and associated forms of valuation that become stabilized in institutional settings to the point where they constitute an obligatory frame that individuals and groups accommodate in their research (Fochler, Felt, and Müller 2016). For Fochler (2016), forms of worth become regimes of worth when they are underpinned by durable forms of capital. Whereas some forms of inquiry might matter to researchers, when these do not interlock with established regimes of worth they risk being displaced or downgraded by research that does appeal to these standardized and powerful practices of capital accumulation.
Biomedicine as Heterarchical Work Organization
Stark (2009) introduced the notion of heterarchy to conceptualize organizational settings in which multiple regimes of worth are maintained and deployed by entrepreneurs. Indeed, it is the ability to mobilize and exploit ambiguities and tensions between different worth regimes that defines entrepreneurial activity. Contemporary biomedical research provides a rich and elucidating site in which to study heterarchical work forms and epistemic capitalism, given ongoing concerted efforts to shake up what is considered a dominant regime of worth and make the field altogether more heterarchical. As an institutional context that is subject to a host of different demands often linked to the high levels of funding and expectations and accountability invested by governments and other actors, there are a number of distinct ways in which biomedical researchers demonstrate worth. Arguably, the dominant regime—excellence—is often associated with the rise in bibliometric performance indicators such as the JIF and other citation-based indicators. These measurements have come to play an important constitutive role in shaping reputational dynamics in biomedical research, not only by virtue of their use by “external” actors governing the allocation of funding and career opportunities (Hessels, Van Lente, and Smits 2009) but also by the propensity of researchers themselves to utilize and internalize the logics of the indicators (Rushforth and de Rijcke 2015; Müller and de Rijcke 2017). 1
Although standardized metrics of excellence have become an increasingly stabilized and obligatory regime of worth in recent times (Fochler et al. 2016), there have also been growing regional and international backlashes. Varied sources and movements have criticized standardized excellence regimes in biomedicine for being at odds with patient-focused ideals upon which the vast expenditure on biomedical research is usually mandated (Alberts et al. 2014; Benedictus, Miedema, and Ferguson 2016; Macleod et al. 2014). Critiques of the JIF and bibliometric performance indicators more widely often tap into long-standing concerns that investments in fundamental biomedical research are not being capitalized upon efficiently or effectively enough in the form of tangible new health technologies, medicines, services, or outcomes. In recent times, “translational research” has become an increasing preoccupation for national and regional health research policies in the form of initiatives purporting to bridge “gaps” in the movement of knowledge discoveries from “bench-to-bedside” (Woolf 2008; Rushforth and de Rijcke 2017). In spite of all these policy efforts, one of the major hurdles said to be thwarting such reforms is the persistence of the excellence worth regime that is focused on “high impact factor” journals, around which career structures, funding, and individual reputations still tend to interlock (Ioannidis 2017).
Against this backdrop, the board of the academic medical center in which our cases were embedded was increasingly advertising “patient-relevant research” as a priority even for more basic biomedical research groups and was undertaking new initiatives to provide a “protected space” (Whitley 2014) for forms of work considered important but missing out within the prevailing excellence regime. Building on a national movement, Science in Transition (http://scienceintransition.nl/english), the senior management of the medical center argued publicly that biomedical scientists—including those in their own institution—have become “captured” by artificial metrics (impact factors, citations, individual grants, institutional rankings), which have diverted attention and priorities away from the “true vocation” of biomedical research, namely, to improve the health of patients and populations. Efforts to “recapture” and nurture in-house the “lost” priorities of patient relevance included introducing new evaluation procedures and promotion criteria for staff, albeit at the time of the fieldwork, this was not at a scale or level of development at which the groups we studied could yet capitalize. As patient relevance was underscored only by an embryonic and rather promissory set of evaluative practices at the time of fieldwork, we have termed this a “prospective regime of worth” in our analysis. In some institutional domains, a stable patient-relevant regime may already dominate and offer forms of worth on which researchers can readily capitalize. Yet for these biomedical research groups in this academic medical center, the notion that this prospective regime could erode or even displace the excellence regime was a new reality for “shop floor” researchers.
Empirical Focus and Method
This paper presents material developed through a larger research project about efforts to build capacity for research on rare (or “orphan”) diseases in Europe. The European Union defines rare diseases as conditions that afflict less than 1 in 2,000 of the general population (Orphanet n.d.). Whereas there is a considerable body of literature on the role of patients and patient organizations in shaping research and innovation policy on rare diseases (Rabeharisoa, Moreira, and Akrich 2014), in the lab-oriented research groups, we studied patients were rather remote and present only in the form of distant funders or providers of data for experiments. Instead, rare diseases served mostly as models on which to base contributions to the fields of cellular immunology and human genetics, respectively.
The empirical materials presented were drawn from ethnographic fieldwork carried out by the first author in 2016-2017. Over a period of six weeks per case, he participated as an observer in weekly meetings typically attended by all lab members, one-to-one research meetings, lectures and seminars, Skype and phone meetings with collaborators, and laboratory work. He also met and held conversations with members of the group separately as well as joining them for lunch on a number of occasions. In the genetics case, he was able to shadow the principal investigator for sustained periods by being located in his office. Semistructured interviews were conducted with members of the groups, close collaborators, and departmental colleagues, cutting across a range of career stages, from full, associate, and assistant professors to clinical fellows, postdoctoral researchers, and PhD students (human genetics interviews = 11, immunology interviews = 9). The first author received forwarded e-mails, funding applications, Curricula Vitae (CVs), and departmental vision documents by various informants and accessed publications of principle investigators via PubMed and browsed their institutional webpages.
For each case, our analysis will look first at how members of the two groups defined “interesting” fundamental research problems on rare diseases. This will reveal how notions of what the researchers and their international community find ripe become entangled with other forms of “worth made durable”: “calculative infrastructures” (Kurunmäki and Miller 2013) of citations and JIFs. The second section within each case then addresses how groups were in turn configured with a view to rendering interesting problems “durable” (Hackett 2005). This will shed light on the importance of the portfolio as a concept for describing how principal investigators strategically combine and align inquiries toward multiple worth regimes.
Immunologists
Making Rare Diseases Interesting Research Problems
How were the lines of inquiry pursued within the immunological research group related to different regimes of worth? In analyzing cell activity within blood and serum samples, the broad aim of the immunology group’s work was to open insights into the functioning of the immune system in the whole human body. The group specialized in mechanisms of immune regulation, often, but not always, in the context of childhood autoimmune diseases. Within the international immunology field, the scientific name (or “signature”) of the principal investigator and the group was most strongly associated with the role of T cells in autoimmune regulation. Within the parameters of project funding calls, immunological questions were answered by taking a single disease as a model for analyzing mechanisms active in inflammation processes. As they were primarily interested in more theoretical puzzles, in principle taking common diseases or rare diseases as a model would not necessarily alter the kind of research questions they could pose:
I wondered, the fact that rare disease 1 is a rare disease, does this change the research process at all?
Possibly, well of course, it does but it depends on what you look at. In terms of questions, no not necessarily, I mean the research questions for rare disease 1 are probably the same as for RA [rheumatoid arthritis—a common disease] or for…I mean at least for me, I’m interested in what the role of certain cells is in diseases and the questions I posed here [on rare disease 1] I could also pose in a different disease. (Postdoc 1 interview)
The value of these rare diseases (and their subtypes) as interesting models for exploring broad, general questions about immunology was repeated throughout the fieldwork: What I also believe is that some of these [rare] diseases are very interesting also intellectually so you can have a huge impact because there is a certain mechanism or something that can help in the end a lot of people and ‘a lot of science’. (Principle Investigator interview) The thing is if you stimulate the science for these rare diseases, you will get good papers that also will be cited a lot because there is a lot to discover and also that can be translated to other diseases as well. (PI interview)
One of the things I was thinking about was publishing. So if you’re expected to publish in very high impact factor journals, are rare diseases a risky thing to take on?
Yes. I think there what we’re trying also in the lab is to use cool techniques, new techniques, on these patient samples. So then, in the end, your paper will become very interesting for a broad public as well, since you’re using cool techniques or investigating basic immunological concepts, with the disease as a model system, but not only focusing on that disease. So it’s more like developing, or investigating, new basic concepts in immunology that might apply to many other diseases, or even to a healthy human and doing that using new techniques, cool techniques, that have not been used a lot before. Then in the end, your paper is interesting because other groups can use these same techniques, or can somehow conceptually also use the basic immunological findings that you have. (PhD 1 interview)
Here interesting papers are tightly coupled with having generalizable findings with a broad appeal to an international immunology audience. “Bigger” contributions were also said to derive from working on the relatively well-described and homogenous patient groups that characterized the rare childhood disorders they would use for modeling immunological mechanisms. This meant published findings on mechanisms in these well-defined conditions could have the potential to be extrapolated to more complex and common muscle or chronic rheumatic diseases where the same mechanisms were also known to be important (Clinician Interview). Their findings would thus likely attract more citations when their relevance could be extrapolated beyond the given rare diseases and applied to complex disorders.
Two contrasting research lines are particularly illustrative of how the immunology group related “interesting problems” to the excellence regime. Both were based on the role of T cells in immune regulation and repair, with one focused on variations of a form of juvenile arthritis that is classified as a rare disease but has a steady supply of patient samples obtained by clinical colleagues in the children’s clinic (rare disease 1). Another more up-and-coming line was set up on an ultrarare form of juvenile arthritis (rare disease 2), with fewer patients and samples than rare disease 1. At that moment in time, the rare disease 1 research line was operating at such a scale that it was considered much more likely to deliver broad, high-impact outputs than the newer research line on rare disease 2. Thus, for rare disease 1, journals within their main disciplinary specialty (immunology) with the highest impact factors were coveted as a source of reputation, with the JIF largely thought to signal the novelty and generalizability of research in the journals.
The rare disease 2 research line started life when the principal investigator obtained PhD project funds from a Dutch research charity and hired a medical doctor in training (who had planned to return to training as a specialist once the PhD was complete). The project involved close collaboration with a couple of clinicians in the Children’s Hospital, with the aim of testing a subclinical biomarker within a clinical population. This kind of work was what informants considered “applied clinical research,” that is, testing whether something that was discovered in a lab works or not in a clinical setting. At the time of fieldwork, the rare disease 2 research line was not so amenable to producing high-JIF scoring conceptual studies, however, partly because the disease was ultrarare, meaning patient samples were difficult to access and the disease was less studied in the immunology community overall. As such, this project was likely to have “lower impact” compared to conceptual immunological work, as it would likely reach smaller audiences in fields like rheumatology where readers of journals would be primarily interested in the rare disease itself. According to informants, in the short term, only patient groups were interested in funding this research, which, stereotypically at least, tended to value funding shorter-term clinical research over more abstract general questions. Over time, the principal investigator was hopeful the lab might be able to build a large enough research infrastructure to be able to produce conceptual papers on rare disease 2 that would be eligible for high-JIF immunology journals. However, in the short term, the rare disease 2 PhD project had shoehorned basic research questions into clinically oriented funding calls. The more basic research questions conducted within the rare disease 2 line could thus be conceptualized as standing in promissory relation to the excellence regime, while the applied clinical questions were profitable in the patient relevance regime of worth. How the immunology group was organized in order to exploit accumulation of capital through both excellence and patient-relevant regimes of worth is the topic to which we now turn in further detail.
Accumulating Different Forms of Capital through a Portfolio
Two research lines, separable in terms of disease focus (rare disease 1 and rare disease 2), had emerged and become stabilized within the immunology group. At the time of fieldwork, the two research lines represented two types of epistemic inquiry, both of which tend to persist in academic biomedicine: more general, basic research and more applied, clinical research problem-solving. The two research lines also drew on two distinct regimes of worth. But how did these two regimes of worth interact, and how was the discord exploited (cf. Stark 2009) within the group? While the move of setting up a lower-impact line on rare disease 2 appears to run counter to the excellence regime, in this section, we describe how, in diversifying the groups’ research focus, the principal investigator in fact sought to capitalize from combining basic and clinical research.
rare disease 2 was piloted as part of a doable clinical research project that could be performed by a clinical researcher in the space of four years. The project drew on some immunological techniques similar to other basic research projects on rare disease 1 running in the lab. Therefore, they hoped that a rare disease 2 project might generate some findings that could be of interest to higher-JIF immunology journals, if not in this PhD project then in subsequent follow-up projects. In other words, promising results might lead the principal investigator to try to scale up rare disease 2 into a larger research line, which could appeal to the excellence regime of worth. But if not, it was a low-risk option that would be justifiable in terms of the local patient relevance regime of the academic medical center.
As well as delivering findings that, in the short term at least, would latch onto the emerging patient relevance regime, it was hoped that introducing rare disease 2 into the lab would also bolster the vitality of the rare disease 1 research line in yielding results that could appeal to high impact factor immunology journals. The principal investigator reasoned that adding rare disease 2 would generate inspiration and exchange of ideas between projects and individuals in the lab who worked on different disease models but were united epistemically by a focus on similar immunological mechanisms.
You work with a portfolio of diseases, does there need to be some flexibility in terms of looking at more than one disease and being a specialist on more than one disease in the group?
Yes. That’s also why I try to have meetings where they [members of the group] are all together although they may work on very different diseases but that they will know from each other because they also learn…. So I try to keep it pretty general so they also learn to think not only in the [one] disease…they [group members] have to think in a way that they can not only answer questions on one thing. (PI interview)
A professor who was formerly principal investigator of the immunology group similarly described how answering broad questions relies on diverse conversations and interactions with different kinds of research and clinical communities. One potentially inhibiting factor for a self-identifying basic researcher focusing solely on one rare disease would be a relative lack of inspiration owing to interacting only with members of a single, diminished research community: …there are no more than four or five centers in the world who work on this [rare disease 1]…. If you would stay in this field, you would be—it’s like the one-eyed being, the King of the Blind, it’s like a very small community. Basic research is done, I think, in four centers in the world. So if you are the only ones, so you only get your input from three other centers, which is also why it’s very important to broaden your scope to other diseases, other fields, because otherwise there is nothing of innovation. (Professor interview)
This entrepreneurial ability to oscillate between spheres of inquiry that draw on different regimes of worth is one way in which basic research inquiries are made sustainable in a context in which “social accountability” and patient relevance is being promoted as a prospective regime that could challenge the excellence regime of worth.
Human Genetics
Making Rare Diseases Interesting Fundamental and Clinical Problems
The human genetics group worked on the genetics of rare diseases, many of which were metabolic disorders. Since many rare genetic conditions are caused by a single-gene mutation (compared with more common, complex, multifactorial diseases), the group brought together two core technologies to study this area. First, they used genetic tests and sequencing technologies to identify the causal mutations in many monogenic disorders. Since the mid-2000s, rapid changes in sequencing techniques—especially Next Generation Sequencing (NGS) techniques such as whole-genome sequencing and whole-exome sequencing—have played an important promissory role in genetic science and medicine (Timmermans and Shostak 2016). In the context of the research frontier of the human genetics field where this group worked, NGS developments had led to a “gold rush” in efforts by researchers to discover and link newly identifiable genes to disease phenotypes in patients. Second, the group coupled NGS with the so-called functional work, performing gene knockouts (using zebra fish) to model developmental problems in identified monogenic rare disorders.
The packaging of sequencing and functional techniques had hitherto been interpreted as a research technology ensemble that could, in some instances, yield publications in what were considered high impact factor genetics journals. Aside from the general promise of combining these techniques, there were certain other qualities that might render manuscripts emerging from this pipeline eligible for journals with higher impact factor scores. What several informants considered to be the most important article the principal investigator had produced as senior author was explained as a “good fit” for a high-impact genetics journal because it made a convincing case that the gene they identified belonged to a distinct class of genes and was therefore particularly novel and interesting for the international scientific community. As researchers who are clearly identified with values like novelty and general applicability (and who saw these reflected in JIF scores in their field), in one sense the practice of sorting journals from high to low JIF scores was seen as relatively unproblematic and useful.
Projects might also yield outputs in prestigious journals in genetics by making a convincing argument that the gene they had identified could be a screening candidate for DNA diagnostic tests. In such instances, their fundamental approach would yield what they termed “instantly applicable” clinical implications, as (imagined) clinicians adopting their findings could improve the diagnostic screening and disease management for patients. Informants were able to align certain projects with the two regimes of worth, for instance, by citing examples of results that were published in high-impact journals and that could be mobilized to persuade funders, managers, and clinicians of the patient relevance of their work. What is interesting in the above examples of “high-impact” outputs associated with JIFs is that they do not occlude a priori other pertinent values in biomedical research, such as patient relevance or clinical translation, which is often implied in more outspoken “folk theories” on the influence of citations and the JIF in biomedicine (Rushforth and de Rijcke 2015). While on occasion there could be alignments between these regimes, tensions still tended to occur, more often than not. Informants would acknowledge, for instance, that values associated with patient relevance could be eclipsed by evaluation practices synonymous with excellence regimes. The following statement from the principal investigator explains why he believes a recent manuscript they submitted to one of the top three human genetics journals in terms of JIF was sent out for review rather than desk rejected by the journal’s editor: The reason I think that this story is now in the review [process] while other stories, which are scientifically as sound as this one are not [in review], is that this is the first developmental syndrome caused by a mutation in the [names type of protein]. While other stories might have a bigger impact on patients, because there are slightly more patients [from the disease cohort], but it’s a less exciting gene, so to say. (PI interview)
The manuscript under review at the high-impact genetics journal was contrasted with another project running concurrently, which would not yield forms of novelty that high-impact genetics journals would value, and yet would be highly relevant for the patient group on which the project was modeled: The [higher impact] project is a new link between a gene and a disease, but the disease is more rare than [the lower-impact project’s] syndrome. But the novelty is really playing a big role there, and the type of gene—it’s novel to a class of genes even. [The syndrome in the lower-impact project] is genetically [known]…we already published it in 2012, and now we’re looking to further understanding this orphan [rare] disease. So it will have a lower potential of getting citations, because there are less people working on it and less people interested in it. (PI interview)
In the following section, we describe how lower-impact “patient-relevant” projects were configured alongside high-impact projects in the organization of the human genetics group.
Accumulating Different Forms of Worth through a Portfolio
Having identified how lines of inquiry across the group were related to different regimes of worth, we will now focus on how a “portfolio” of projects focused on the local research technology ensemble was imagined to exploit said regimes of worth. In the five years since the group was established, connecting NGS techniques with functional studies of model organisms to study rare genetic diseases had proved quite profitable in accumulating capital associated with publishing in high-impact human genetics journals, with grants coming in and the group and external collaborations growing.
This profitable practice of combining sequencing and functional approaches had been routinized in the form of a common “pipeline” of experimental procedures, on which most of the different rare disease projects could be based. This relatively stable production line had enabled flexibility and economies of scale around which all kinds of externally funded projects on rare genetic diseases could be configured.
While this had hitherto supported projects that had led to novel outputs in high-impact genetics journals, around the time of fieldwork, the PI had also just recently begun to initiate lower-impact projects around this configuration based on more “applied” disease-oriented research problems and techniques. A functional testing project had been recently set up that experimented with different therapeutic interventions in zebra fish in order to describe their effects on development. This did not include NGS components and as such the novelty of the findings and techniques was seen as insufficient to interest the higher-impact genetics journals. The need to generate “investment alternatives” through a portfolio of more basic and more disease-focused clinical projects was related to a changing trend beginning to emerge in the human genetics field. One important consideration was that the rate of discovery of genetic causes of monogenic rare diseases, which had proliferated in light of international adoption of NGS techniques, was beginning to dry up. A well-cited review article in Nature Reviews Genetics makes the following (bold) claim to this effect: Work over the past 25 years has resulted in the identification of genes responsible for ∼50% of the estimated 7,000 rare monogenic diseases, and it is predicted that most of the remaining disease-causing genes will be identified by the year 2020, and probably sooner. (Boycott et al. 2013, 681) The principal investigator states that once the monogenic disease genes dry up, he will have to drop down the impact factor of the journals in which he can publish. He does not have the expertise or the credibility to move into the really fundamental biology of zebra fish which the life sciences group [with whom they collaborated] focus upon…. He says he will go more toward functional testing in future, where different treatments are given to zebra fish, and their development is then subsequently monitored. They can question for example “does a given treatment reduce the swelling of vessels?” This in turn may allow them to suggest treatments as potential therapeutic options for human patients with the rare diseases. (Field note feedback meeting) I ask whether the medical center and his [the principal investigator’s] department would accept him dropping down the impact factor scale in which he can publish. He states that the future work will have a lot of potential for identifying new treatment options. Therefore if the medical center is serious about making patient-relevant research a priority like it says it is, then it should support him, putting its money where its mouth is. (Field note feedback meeting) For me it is a factor that I don’t need an impact [factor score] of this and this every year. As long as I can explain why I have lower-impact publications, then it is fine. (Field note feedback meeting)
Discussion
In recent years, STS inquiries into the practical work of academic research have become increasingly interested in how growth of research governance—including intensified monitoring and evaluation of performance—is impacting research and academic life in universities more generally.
While policy makers and other actors may privilege particular regimes of worth in their measurement of performance (often at the expense of or in combination with others), we have refocused the need to open up how researchers—an obligatory passage point for research policy—strategically engage in their own kinds practices of accumulating worth. Drawing on emerging research in the sociology of (e)valuation, we contend that there is no singularity to (e)valuation in academia, but rather a multiplicity of (e)valuative practices and infrastructures. This in our view provokes an interesting new avenue of investigation for STS, namely, how are contemporary research practices being configured around multiple, hierarchically ordered regimes of worth? Through retrieving and updating classic STS accounts of research as a capitalist-style process of accumulating worth of various sorts, recent conceptual developments on epistemic capitalism provide a series of helpful insights for approaching this puzzle. In this scheme, the capitalistic or entrepreneurial accumulation of worth is not a voluntaristic property of individuals and groups. Rather, it emerges as a set of contingent outcomes of organizational and infrastructural relations. As such, some forms of worth become more durable and powerful (“capital”) by virtue of being embedded in standardized infrastructures like citation measurements.
Building on these insights, we explored how principal investigators of two biomedical research groups in a Dutch academic medical center navigated and exploited multiple regimes of worth through building portfolios of basic and clinical inquiries within their groups. Our analysis examined how portfolio construction emerged around groups’ research technology ensembles, which both enabled and constrained opportunities either to steer research activities toward novel areas of inquiry or expand into existing ones considered promising vis-à-vis the excellence and/or patient relevance regimes of worth. Based on our findings, we suggest the following insights about the portfolio might be instructive for future STS research on epistemic capitalism.
First, the concept of portfolio can be useful for exploring researchers’ efforts to satisfy coexisting regimes of worth that appear to be in friction with one another. Relating to the notion of heterarchy (Stark 2009), the portfolio helped us make sense of how biomedical principle investigators sought to align their spheres of inquiry in relation to both the excellence and patient-relevant regimes of worth (a dichotomy widely embedded across contemporary biomedical research). For example, in our second case, the human genetics principal investigator used his research technology ensemble to scale a clinical sphere of inquiry, thereby initiating a transition from work framed as relevant to a hitherto dominant regime of worth (excellence) toward work relevant to a prospective regime (patient relevance). In the short term, however, a combined basic and clinical portfolio enabled him to keep both regimes of worth in play. Comparative studies situating “portfolio management” practices in relatively pluralistic versus hierarchical evaluative situations would help further crystallize the scope of the portfolio as a concept for theorizing research problem construction, identity work, and risk management practices in contemporary academic research groups.
Second, accounts of the performative effects of evaluation on academic research practices often suggest that dominant regimes of worth risk overriding the ability to conduct research that is not supported by durable regimes of worth. While this is surely an important concern, our empirical account of portfolio strategies showed how riskier problems underpinned by less stable forms of worth were kept in play by combining them with more “mainstream” problems underpinned by relatively stable measures of worth. Returning to the vocabulary of financial capitalism, portfolio investment may help to “spread risk” and support “niche” areas of inquiry, therefore mitigating—at least partially—potential pressures to conform to dominant, perhaps conservative, hierarchies of worth. In line with Birch’s (2017, 439) recent call for STS scholars to examine technoeconomic assumptions underpinning contemporary forms of research organization and governance, we suggest that notions like “investment alternatives,” “risk management,” and “return on investment” offer a promising set of concepts to tinker with in exploring epistemic capitalist practices of portfolio construction.
Third, portfolio strategies are not only about reacting to emerging regimes of worth; they can also support scaling existing spheres of inquiry in order to latch onto already well-established regimes of worth. For example, the immunology group leader in our first case felt optimistic about opening a new line of inquiry—from an already established research technology ensemble—with which her group might continue to target high-impact factor journals. As such, although portfolio strategies can be useful in making sense of situations where multiple regimes of worth are considered important, it would also be a useful means of accounting for how group leaders seek to latch onto dominant hierarchical regimes of worth by “investing” in different research problems within the group.
Fourth, studies focusing on the changing subjectivities of academic researchers in relation to emerging “capitalist” accumulation practices may wish to explore further how notions of self-worth become attached to portfolio-building practices for exploiting multiplicity. Promising research questions include to what extent are portfolio practices constitutive of new (academic) entrepreneurial selves? to whom is this new normal (not) available? and what forms of academic (self)worth are displaced, damaged, or excluded from such practices? Furthermore, whether portfolio strategies manifest similarly in epistemic domains other than biomedicine—for example, those that utilize “research systems” (Hackett et al. 2004) less amenable toward processes of “rapid discovery science” (Collins 1994)—is an important empirical question our paper raises for future theorizing on portfolios. For instance, portfolio strategies in social science and humanities might crucially include “teaching,” both as an activity and as a regime of worth, that plays an important role in sustaining research activities in disciplines where resources are generally scarce.
Finally, our account of portfolios serves as a reminder that (e)valuative infrastructures are dynamic. This means their durability and importance can change over time, and, perhaps more importantly, can be changed. However, our account suggests that changes are most likely to be rather slow and indeterminate: the ability of academic medical centers to establish heterarchical modes of worth in which patient-relevant research can be capitalized upon sufficiently to sustain research groups is likely to prove a wicked problem going forward.
Footnotes
Authors’ Note
The views expressed are those of the authors and not necessarily those of the NHS, the NIHR, or the Department of Health and Social Care.
Acknowledgments
Many thanks to Max Fochler for helpful comments on an earlier draft of this paper and to colleagues from the KNOWSCIENCE project. Thanks to Edward Hackett and Katie Vann for their careful editorial and manuscript suggestions, the anonymous reviewers, and those who took part in the study.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was conducted as part of the KNOWSCIENCE project, funded by the Riksbankens Jubileumsfond (RJ), Sweden. AR’s work on this paper was part-funded by the National Institute for Health Research (NIHR) Oxford Biomedical Research Centre, grant no. BRC-1215-20008, to the Oxford University Hospitals NHS Foundation Trust and the University of Oxford.
