Abstract
Academia focuses on the interplay of Higher Education Institutions and external stakeholders. In this context, academia is concerned with the societal impact and impact created in interactions with external stakeholders; the latter is often referred to as impact co-creation. There is agreement that the related processes leading to an impact are complex and multi-dimensional. However, academics disagree on how the ultimate, wider impact of research should be measured. This study seeks to conceptualize societal impact through the lens of value co-creation, arguing that societal impact is best conceptualized as the uptake of research. Based on this, we developed a generic research impact assessment framework to facilitate evaluations and enable cross-sector learning. This study contributes to academia by providing an overarching understanding of impact creation, including wider research impact, and offers the perspective that any research project involving stakeholders to a certain extent, also entails co-production.
Introduction
Fuelled by a persistent, even increasing, demand to demonstrate the return on investment of research, research impact assessment is a recurrent issue concerning academics and stakeholders across various disciplines (Louder et al., 2021; Pedersen et al., 2020). The literature covers impact indicators, institutional evaluation guidelines, assessment frameworks, and impact drivers and is characterized by conceptual diversity (Louder et al., 2021; Pedersen et al., 2020). Special focus is directed toward the wider impact of research, often equated with societal impact, where the effects of research evolve mostly out of academics’ influence (Belcher et al., 2020; Louder et al., 2021; Williams, 2020). In marked contrast to scientific impact, which can be measured comparatively straightforwardly (e.g. citations and publications), societal impact assessment is challenging (Bornmann, 2013; Louder et al., 2021; Pedersen et al., 2020). Societal impact often emerges with a time lag and is complex due to multi-dimensional interactions between researchers and societal actors, expressed as non-linear flows of knowledge and relationships (Louder et al., 2021; Milat et al., 2015; Williams, 2020). Researchers have attempted to overcome this challenge by shifting the focus from outcome-based to process-based or integrated impact evaluations (Kieslinger et al., 2018; Upton et al., 2014). Some argue that, frameworks for assessment remain contextual, are not universally applicable, and are thus possibly missing the opportunity to realize synergies (Louder et al., 2021; Pedersen et al., 2020). Others opt for more nuanced and heterogeneous evaluation frameworks depending on the discipline (Greenhalgh et al., 2016b; Pedersen et al., 2020; Smit and Hessels, 2021). The issue at stake might not be a question of right or wrong, but it implies that more conceptual work is needed to clarify the understanding of societal impact and to build a foundation for its measurement (Smit and Hessels, 2021; Williams, 2020).
Acknowledging a driving role of interactions between academics and society in the creation of impact (Greenhalgh et al., 2016a, 2016b; Smit and Hessels, 2021), recent elaborations on research impact have moved the focus to the buzzword “co-creation” often used with “co-production” interchangeably. However, there is some indication that these terms should be considered separately, with co-production being part of the co-creation process (Ranjan and Read, 2016). Particularly in the context of societal impact, research on the role of non-academic actors in the impact creation process is gaining currency, as collaboration is assumed to drive the impact (De Silva et al., 2021; Smit and Hessels, 2021). Much of this research has emerged from the fields of social science and humanities (SSH), health, and education, in which research projects inherently incorporate societal actors. Keywords describing research processes in these fields, depending on the context, are: “participatory research” (SSH), transdisciplinary “research-practice partnerships” (health, SSH), interdisciplinary “research partnerships,” and “citizen-science” (Belcher et al., 2020; Hoekstra et al., 2020; Jagosh et al., 2015). The emerging subdomain lacks common terminology and conceptualization of co-creation while relating diverse academic interactions.
Considering the aforementioned need to prove accountability in the context of societal impact, it is important to conceptualize the role and effects of co-creation in the impact creation process. Societal impact is found to be multi-dimensional and complex, and involves various actors (Pedersen et al., 2020) and attributes that are also related to the creation of value in co-creation contexts (Greenhalgh et al., 2016a; Ranjan and Read, 2016; Smit and Hessels, 2021; Vargo and Lusch, 2004). Academics have called for more conceptual contributions to explain the emergence of societal impacts and enable their assessment (Smit and Hessels, 2021).
The contribution of this article is threefold. First, by synthesizing the concept of value co-creation (VCC) (Ranjan and Read, 2016; Vargo and Lusch, 2004) with theoretical underpinnings of societal impact, the authors conceptualize (societal) impact. The VCC concept comprises two theoretical dimensions: coproduction and value in use (ViU) (Grönroos and Voima, 2013; Ranjan and Read, 2016). These dimensions serve the need to (a) explicate the multi-dimensional pathways of impact creation (co-production), as well as (b) explicate the concept of the societal impact of research (ViU). VCC points to the importance of considering relationships and networks, without which value would not be created (Ranjan and Read, 2016). Second, by integrating societal impact and VCC, this study contributes to the research impact assessment domain in that the new conceptualization of societal impact allows for an overarching understanding of the impact creation process, which includes and explains the wider impact of research. Accordingly, this contribution adds to the extant literature by focusing on networks and relationships as constituent parts of a wider impact. This view suggests refraining from hunting for societal impact measures and analyzing the quality of relationships and nexuses of networks. Third, the modified concept of impact creation that integrates VCC and societal impact enables drawing on theoretical foundations from extant frameworks relevant to the VCC lens of impact creation, hence allowing the formulation of a generic framework for research impact assessment. This framework can be adopted by diverse disciplines, regardless of the extent of the co-production characteristics in the projects. In this way, academic fields can seize learning effects from one another, and research activities and their impacts are made comparable.
The article begins with a detailed literature review on societal impact and co-creation in the context of research impact. In this literature review, first the state-of-the-art conceptualizations and assessment endeavors of societal impact of research are presented and discussed. Following, the article provides an overview of how academia perceives the role of co-creation in the development of research impact. In the third section of the article, the authors outline their choice of methods and theories backing the framework development. Subsequently, the article introduces and appeals for a new understanding of impact co-creation through the lens of the VCC construct. Section “Framework development” then sets up the new framework for research impact assessment by synthesizing the new impact creation construct with common knowledge from the research impact assessment domain. Finally, the main contributions, limitations, and suggestions for future research are outlined.
Societal impact of research and co-creation of research impact: A state-of-the-art literature review
Societal impact: Endeavors to assess and construct
In the early years, investments in research were undoubtedly assumed to impact society (Bornmann, 2013). Well-established impact assessment frameworks accompanying this early development were outcome-based and assumed linear knowledge transfer (Pedersen et al., 2020; Upton et al., 2014; Williams, 2020). Being subject to research for the longest time, scientific impact indicators and measures are well-established and commonly accepted (Williams, 2020). However, over the last few decades, academia has seen a shift in focus toward the societal impact of research (Smit and Hessels, 2021; Williams, 2020). Adding to the condition of accounting for wider (societal) impact, academics’ opinions are more contested, and measures less established (Pedersen et al., 2020; Williams, 2020).
Academia has seen the emergence of multiple labels dealing foremost with societal impact: third-stream activities, public value, and wider impact (Bornmann, 2013; Williams, 2020). Differing in nomenclature, related research activities commonly strive to grasp the extent to which research benefits society (Razmgir et al., 2021). These benefits can be economic, environmental, social, and cultural (Bornmann, 2013; Sørensen et al., 2022). Whereas the categories and conceptualizations of aspired benefits are broadly similar, discord prevails when defining societal impact itself (Pedersen et al., 2020). When clarifying terms, Razmgir et al. (2021) adopt a broad definition of societal impact, where societal impact is seen as “all means by which research can benefit society (. . .)” (Razmgir et al., 2021: 444). Sørensen et al. (2022) are in line with this perspective and adopt a similar, broad definition of societal impact from Reale et al. (2018), where societal impact is seen as “research contributions to addressing current and/or future social, environmental, economic, and other needs outside academia” (Sørensen et al., 2022: 119). The existence of multiple impact definitions, depending on the discipline or domain (Pedersen et al., 2020), might explain the overarching nature of generally applicable definitions for societal impact.
Bornmann (2013) sheds light on the root cause of the problem of universally conceptualizing societal impacts. Following his elaboration, a clear definition is complex for several reasons. First, all areas constituting the ultimate contribution of research to society are broadly conceived, and boundaries are not distinct. Furthermore, societal impact occurs with a time lag, ranging from immediate to intermediary and long-term effects (Bornmann, 2013; Sørensen et al., 2022). In addition, societal impact is characterized by the eventuality of occurring beyond national borders and involving various stakeholders (Bornmann, 2013; Milat et al., 2015). The aforementioned features of societal impact are generally accepted in academia, where researchers recurrently refer to terms such as “multi-dimensionality,” “complexity,” “stakeholder involvement,” and “non-linearity” (Banzi et al., 2011; Louder et al., 2021; Milat et al., 2015; Pedersen et al., 2020). The pluralism of the characteristics of societal impact justifies calls from academics to select impact assessment methods and measures in due consideration of the particular object and the corresponding discipline (Greenhalgh et al., 2016b; Pedersen et al., 2020; Smit and Hessels, 2021). Notwithstanding, the absence of a common understanding prevents academics from elaborating solid measures for assessment (Milat et al., 2015). Consequently, research impact operationalization takes a narrow form (Williams, 2020).
Some researchers have focused on reframing the impact without a suitable conceptualization of societal impact (Smit and Hessels, 2021; Williams, 2020). From a constructivist perspective, Smit and Hessels (2021) highlight the importance of integrating research’s scientific and societal value. According to them, scientific value emerges simultaneously with establishing networks with non-academics; hence, scientific and societal values rest upon similar actor networks (Smit and Hessels, 2021). Williams (2020) adopted a sociological perspective and built a capital-based theoretical framework of impact around the logic of fields of power. Accordingly, impacts can occur along the intersections of academics, politics, applications, media, and economic fields. From her point of view, the weight and importance of research in each related field of power should be the object of evaluation. Put simply, the important considerations in impact evaluations should ask, “What do I seek to achieve in which field of power?” and corresponding measures were selected (Williams, 2020). Whereas Williams’s framework allows for reflexivity in the definition of impact, it tends to take the perspective of academics when evaluating impact. This perspective is somewhat problematic considering the aforementioned relevance of non-academic actors in impact creation. For example, Smit and Hessels (2021) highlight the importance of discourse and the involvement of non-academic actors in impact creation and evaluation processes. Although acknowledging the necessity of reframing impact, even the latest elaborations have severe limitations.
Co-creation in the realms of research impact
Another growing stream of literature is concerned with the keyword “co-creation” in the context of research impact (Greenhalgh et al., 2016a, 2016b; Høydal, 2020). Possibly because it is the nature of the discipline to address societal challenges, the SSH domain acts as a pioneer in this regard. A review of the extant literature elucidates different streams that can be clustered along the high-level categories of science–society, science–business partnerships, and collaborative evaluation. Another important stream focuses on the science–policy interface, which is tightly interwoven with the science–society category, acknowledging a supportive role of co-creation in the achievement of impact on public policies (Šucha and Sienkiewitz, 2020). Although these categories examine different types of research objects, they commonly engage in understanding the joint generation of value. Therefore, literature from any of these categories is suitable to inform the concept of co-creation in research and, by association, the framework developed for the remainder of this study.
From an academic perspective, Greenhalgh et al. (2016a: 393) defined co-creation as “the collaborative generation of knowledge by academics working alongside stakeholders from other sectors” (p. 393) and they relate co-creation to civic engagement, intersectoral collaboration, and university–community partnerships. They argue that science–society partnerships increase the probability of usage and, hence, wider research outcomes. Several academics align with this tenet and have similar understandings of co-creation (Cooper et al., 2021; De Silva et al., 2021; Hoekstra et al., 2020; Muhonen et al., 2020; Olsson et al., 2021). On a related note, Olvera et al. (2021) point to the importance of science–business collaborations in facilitating knowledge transfer. Šucha and Sienkiewitz (2020) understand co-creation as “interlinked collaborative approaches aimed to increase dialogue, trust, understanding of needs and diversity of input” (p. 3). Part of this literature points to the determinants of co-creation and key success principles (De Silva et al., 2021; Greenhalgh et al., 2016a), while other parts address how co-creation leads to impact (Muhonen et al., 2020; Smit and Hessels, 2021). In a recent publication, Muhonen et al. (2020) identified co-creation as one of the four pathways to societal impact. According to them, this pathway comprises collaboration, public engagement, expertise, and mobility. Smit and Hessels (2021) compared the evaluation methods with their developed analytical framework. Following them, in the context of knowledge transfer and utilization, co-creation constitutes a form of interaction in research projects (Smit and Hessels, 2021). Especially in the context of the science–policy interface, boundary organizations and knowledge brokers which act at the intersection of science and policy are discussed. These institutions serve as important co-creation hubs and sources of evidence-based policymaking (MacKillop et al., 2023; Meyer, 2010; Michaels, 2009; Šucha and Sienkiewitz, 2020).
The extant literature suggests that co-creation in research is preeminently interpreted as collaborative research production. Consequently, the impact is co-produced when research projects are jointly undertaken with stakeholders. Considering the importance of stakeholders in the manifestation of societal impact, this interpretation appears too short. The following elaborations will examine the relationship between co-creation and impact in more detail.
Methodological approach
The previous sections outlined the fragmented perspectives of academia with regards to societal impact and co-creation. This study seeks to synthesize the extant literature to overcome this fragmentation, integrate different concepts, and build a solid foundation for a generic research impact assessment framework (MacInnis, 2011). In some instances, the combination of approaches is a good option (Jaakkola, 2020). Therefore, we applied a mixed methods approach comprising a theory synthesis and a model. Theory synthesis, the conceptual integration across multiple theoretical perspectives (Jaakkola, 2020: 22), is concerned with summarizing and integration extant theory to structure fragmented research fields or explicate the theoretical underpinnings of a new phenomenon. In a theory synthesis paper, researchers would typically start with a phenomenon to be studied, proceed by choosing and justifying the domain theories 1 that do already explain parts of the phenomenon, and then identify a suitable theoretical lens together with a method theory (Jaakkola, 2020). We used theory synthesis to structure the theoretical domain around societal impact by analyzing it through the lens of VCC (method theory) in order to prepare for an appropriate assessment (Jaakkola, 2020).
A model is a “theoretical framework that predicts relationships between constructs” (Jaakkola, 2020: 22) and is therefore suited to identify novel connections among constructs or develop propositions explicating relationships. A model paper, in other words, can explain why and how outcomes are achieved (Jaakkola, 2020). We chose to inform our framework with the Outcome Evaluation Approach (OEA) and the Theory of Change (ToC) (Belcher et al., 2019, 2020).
Through the lens of VCC: Transferring VCC from marketing to research impact theory
Greenhalgh et al. (2016a), in their elaborations, found four forms of co-creation relevant to community-based health care. However, only the concept of VCC was chosen as their theoretical lens because (a) it is the only concept involving the entirety of stakeholders in the impact creation process (Greenhalgh et al., 2016a) and (b) with the underlying assumption that value is not created when products are not used, it is able to reflect societal impact (Prahalad and Ramaswamy, 2004; Ranjan and Read, 2016; Vargo and Lusch, 2004).
VCC leads to significant changes in the perceptions of markets and locus of value creation (Prahalad and Ramaswamy, 2004; Vargo and Lusch, 2004). The marketing discipline experienced a shift in its dominant logic from tangible resources (manufactured products) to intangible resources and the co-creation of value and relationships (Vargo and Lusch, 2004). Similarly, Prahalad and Ramaswamy (2004) elaborate on a “next practice” in value creation. According to them, the traditional market is the locus of exchange of created value with consumers; hence, value creation is conducted by firms and separated from the market (and consumers, respectively). Consequently, value creation has traditionally occured inside companies and outside markets. In their view, the emerging concept of a market comes with a radical change in the perception of consumer–company interactions as the locus of value creation (Prahalad and Ramaswamy, 2004). From these underlying assumptions of VCC, one can draw parallels to research impact. Analogously, the research impact discipline has witnessed a substantial shift from Mode 1 to Mode 2 research. In other words, from university-based research to collaboratively created knowledge, from unilateral knowledge translation (with universities equating the firms that produce knowledge and society as the user) to multi-dimensional knowledge transfer pathways, and from separating academics and society to acknowledging the ubiquitous role of non-academic stakeholders (Greenhalgh et al., 2016a; Pedersen et al., 2020).
A foundational assumption of VCC, generally accepted in academia, is that “the customer is always a co-producer of value” (Vargo and Lusch, 2004). This assumption was later revised to “the customer is always a co-creator of value” (Vargo and Lusch, 2008). Although this understanding is nearly axiomatic, Grönroos (2011) critical analysis of VCC in service settings revises this premise. Thus, the previous premise would not hold if the locus of value creation is the value created by a customer’s usage (ViU). Under these circumstances, only the customer creates value, not the firm. The firm then creates value propositions (outputs), which can be converted into real value through usage (Grönroos, 2011; Grönroos and Voima, 2013). Conversely, value creation can also be seen as an all-encompassing process from production to usage, which ViU is part of (Grönroos, 2011). In this regard, the premise is reformulated into “the customer is always a value creator,” implying that the value creation is centered around the customer, with the firm acting as a producer of value propositions, which only turns into value if the customer accepts them and engages with the proposition (Grönroos, 2011). The concrete delineation of value creation and co-creation has been extensively discussed elsewhere (Grönroos, 2011; Grönroos and Voima, 2013). For this study, it should be noted that the interplay between value creation and VCC can be transposed to the creation and co-creation of impact. Besides academic acknowledgment of the collaborative generation of impact, researchers have also highlighted the importance of stakeholders in any kind of research activity and its dissemination (Pedersen et al., 2020). Belcher et al. (2019), in the context of transdisciplinary research, highlighted the shift from linear science-based knowledge production to multi-pathway models centered around stakeholders. Hence, the previous elaborations can be transferred to this domain, where the relevance of non-academic actors is uncontested, and multiple pathways to impact are commonly accepted. Consequently, we make the following foundational premise (FP):
FP: Non-academic stakeholders do not only consume research outputs, rather, they are always part of impact creation.
Framework development
Conceptualizing the impact creation process in research
Building on this premise, in a second step, the “how” in this equation should be analyzed. A conceptualization of VCC, which is suitable for analyzing the processes explaining how external stakeholders create an impact, was made by Ranjan and Read (2016). These authors echo the notion of Prahalad and Ramaswamy (2004) and Vargo and Lusch (2004) that VCC extends beyond the act of co-producing, which will be defined in the course of this chapter. Hence, the current understanding of co-creation outlined in the third section of this article is deficient. However, this notion implies a potential conceptualization of societal impact through the lens of VCC.
Co-production and the impact creation process
According to Ranjan and Read (2016), a VCC encompasses various facets. For instance, market research analyzing customers’ needs does not appear to be co-creation at first sight, yet it represents the least intensive co-creative action. A more obvious form of co-creation is participation in design and development. Notably, the interaction between consumers and firms influences value creation. Finally, value can be generated without any direct contribution by the firm through usage or social relations. By extension, Ranjan and Read (2016) conceptualize VCC as comprising two theoretical dimensions: co-production and ViU.
Co-production, in this view, relates to all indirect and direct collaborations and forms of consumer participation in the design and development process, which can be performed actively or passively. Put simply, the intensity of co-production can vary greatly depending on the research project; however, the simple fact of addressing a challenge from society is a form of co-production. Further, the interaction
2
between the firm and the customer is a characteristic of co-production. Ranjan and Read (2016) depict co-production in a more general way as “a set of activities carried out by economic and social actors within a network” (Ranjan and Read, 2016: 292) which is realized through dialog (interaction), engagement, and collaboration (Prahalad and Ramaswamy, 2004; Ranjan and Read, 2016). Going back to Greenhalgh et al.’s (2016a) definition of co-creation as collaborative knowledge generation among academics and stakeholders, we can see overlapping characteristics of such research with co-production in VCC. Consequently, what academia discusses as co-creation within the research impact literature should be termed co-production. Given the Foundational Premi that external stakeholders always create value, conceptualizing societal impact through the lens of VCC implies that the co-production of impact is relevant not only to SSH or related fields but also to any kind of research undertaken that is supposed to create societal impact. In essence, it is possible to subsume the multiple termini surrounding the topic mentioned in the introduction of this article under the term co-production of impact. Based on the preceding considerations, we propose the following:
P1: Research co-production can occur in any field of research and varies in intensity. P2: All interactions with stakeholders in any research project are primarily forms of co-production.
The preceding propositions have an important implications: the concept of co-production still applies to, for example, research–practice partnerships where the intensity of collaboration is high and diverse outputs are generated, as well as to research assignments that simply generate knowledge without direct interaction with stakeholders.
Furthermore, co-production in VCC encompasses categories of knowledge (sharing), equity, and interaction (Ranjan and Read, 2016). Vargo and Lusch (2004) also noted the exchange of skills and knowledge as being important to the emerging service-dominant logic, which comes along with the already shift in the meaning of markets (Prahalad and Ramaswamy, 2004). At least two of these categories are vital parts of collaborative research and also “non-collaborative” research, Knowledge (sharing), and, interaction in knowledge dissemination and uptake, are both essential goals of research. Based on these considerations, we propose the following:
P3: Research co-production is built upon knowledge exchange and interaction.
ViU and the impact creation process
The second theoretical dimension of the VCC is ViU. ViU is generated only by consumers who use a product or service after (co-)production. ViU can arise from interaction with the company including “simple” consumption without interaction. By definition, ViU extends beyond the co-production process or the market exchange of goods and services and shifts the locus of value creation entirely toward consumers (Grönroos, 2011; Grönroos and Voima, 2013; Prahalad and Ramaswamy, 2004; Ranjan and Read, 2016). Naturally, the consumer is not engaging with a product accidentally; rather, the usage depends on experience, relationship, and personalization, which can be facilitated through the firm (Grönroos and Voima, 2013; Ranjan and Read, 2016). Grönroos and Voima explain the relations of firms, consumers, and value as follows: “activities performed by the provider (i.e. production) result in outputs (potential value) that customers may use in their value creation process.” (Grönroos and Voima, 2013: 141). These activities may also be jointly created (Grönroos and Voima, 2013; Ranjan and Read, 2016). In the second section of this article, we observed terms such as public value, third-stream activities, multi-dimensionality, actor networks, and intersections of academia and stakeholders when we examined the extant literature on societal impact. Discussions on societal impacts overlap to a large extent with those on VCC. Specifically, extant literature, whether in linear or nonlinear depictions of knowledge transfer, locates societal impact (often termed wider impact) at the end of the knowledge transfer chain (Belcher et al., 2020; Buxton and Hanney, 1996; Donovan and Hanney, 2011; Douthwaite et al., 2007). The uptake of research is of interest in this regard (Sørensen et al., 2022). For example, research-backed job training for unemployed people relies on research output. If this training offer is accepted and used by the unemployed, the output would be taken up and the research conducted would have societal impact. Consequently, one can transfer ViU to the ultimate goal of societal impact in research uptake. This leads us to the following proposition:
P4: The uptake of research is societal impact.
Does this proposition hold and vice versa? Is societal impact equatable to research uptake? The answer depends on how impact creation is understood. For those who adopt the view of research impact creation as an all-encompassing process from knowledge production to research uptake, the ViU alone would not suffice to define societal impact, which would need to include co-production. However, one might argue that research uptake is fundamental for impact to emerge (Grönroos and Voima, 2013; Vargo and Lusch, 2004). From this standpoint, co-production ought to be understood as provisioning outputs that can be turned into impacts by uptake. Extant research impact literature suggests that impact is achieved outside researchers’ sphere of control or influence and that outputs and outcomes are stages before real impact occurs (Belcher et al., 2020; Pedersen et al., 2020; Williams, 2020). Smit and Hessels (2021) understand the societal impact as a social construct depending on its usage in policy, social contexts, and science—in other words, research uptake. In line with these assumptions, we propose the following:
P5: Societal impact is the uptake of research outcomes and outputs.
Furthermore, by understanding societal impact creation through the lens of the VCC, the bottom line is as follows:
P6: Societal impact creation has two dimensions, the co-production of research and the impact dimension itself.
To illustrate the main overlaps and differences of VCC and societal impact, Table 1 brings the main characteristics of both concepts face-to-face.
Value co-creation and societal impact—A juxtaposition of the main characteristics and determining the constructs.
Our conceptualization of impact creation is depicted in Figure 1. Because societal impact cannot occur without uptake (P5), the complete process of societal impact creation encompasses both the co-production of research and societal impact in the form of uptake itself. We deliberately stress the
A generic research impact assessment framework (own illustration).
This conceptualization of societal impact and co-creation in the context of research impact accounts for the growing need to explain the wider impact of research and the lack of an appropriate and common understanding of the co-creation of research. Having developed a distinct conceptualization of the interplay of societal impact and co-creation with regard to impact creation, we can work toward a meaningful generic framework for research impact assessment.
A generic research impact assessment framework
Integrating process and outcomes in evaluating societal impact
Following Proposition P3, a research impact assessment should consider exchange and interaction in the impact creation process. This assumption is in line with a recurring call from academics to shift from outcome- to process-based evaluations (Kieslinger et al., 2018; Upton et al., 2014), where the latter is assumed to increase, accelerate, and incentivize research impact (Razmgir et al., 2021; Upton et al., 2014). Extant research has hitherto divided impact assessment approaches into those focusing on outcomes and outputs of research; and those addressing the process of impact generation (Razmgir et al., 2021). However, some research evaluation approaches take account of outcomes and processes but are considered integrative (Razmgir et al., 2021). Process-based approaches to evaluation understand impact generation as a process, which is precisely what the new impact creation framework (Figure 1) implies (Upton et al., 2014). However, measurable outcomes and outputs should not be neglected, as they constitute the results of the research co-production dimension. Therefore, attending to an integrative approach to evaluate research impact is vital for a comprehensive assessment. Consequently, the following proposition:
P7: Research impact assessment frameworks should be integrative.
Acknowledging that all research activities involve co-production (stakeholder engagement to any extent), all types of research are complex and multi-dimensional. If societal impact is to be understood as the uptake of research, complexity is ever-increasing. Academics in the research evaluation domain have already begun compiling evaluation approaches that account for complexity and multi-dimensionality. In this regard, several academics have sought to overcome this complexity by explicating the impact pathways with the help of logic models (Belcher et al., 2020; Douthwaite et al., 2007; Razmgir et al., 2021). Logic models are occasionally criticized for overly linear depictions of the process (Wilkinson et al., 2021). However, they represent a powerful tool for evaluating transdisciplinary (participatory) research projects (Belcher et al., 2020; Douthwaite et al., 2007; Razmgir et al., 2021; Wilkinson et al., 2021). Many of the approaches using logic models stem from the field of SSH, research for development, and sustainability research in particular (Douthwaite et al., 2007; Wilkinson et al., 2021), where projects are mainly interdisciplinary and transdisciplinary, and causal inferences grounded in statistical measures cannot assess impact appropriately (Belcher et al., 2020). Reverting to the first proposition (P1), we can now assume that these approaches can be applied to any research project. This article will draw on extant approaches to impact assessment from the more advanced domains with experience assessing complex, stakeholder-involving projects for further framework development.
Reviewing literature from the domain of framework development, we chose to incorporate the OEA by Belcher et al. (2019, 2020), as we identified some essential parallels of that with the new conceptualization of impact creation. First, the approach adopts a systems perspective, in which any project undertaken is situated in a complex system of interaction (Belcher et al., 2020). This perspective aligns with P3, where we put interaction and exchange at the center of research co-production. Moreover, the main aim of the OEA is to understand how research projects push change processes in society within declining spheres of control and influence without assuming any change will necessarily happen (Belcher et al., 2020). This theoretical underpinning reflects two important findings of impact creation conceptualization. First, by adopting the lens of VCC, we could likewise identify the relevance of different spheres of control (Grönroos and Voima, 2013). Second, referring to Prahalad and Ramaswamy (2004) stating that we cannot predict ViU, we also assume that impact is not a matter of course. As for the spheres of control and influence, Belcher et al. (2020) conceptualize the sphere of control as encompassing research activities and outputs (goods, services, or knowledge to be used by society). These outputs can then affect stakeholders in the sphere of influence, thereby generating outcomes. Ultimately, stakeholders in the sphere of influence can affect further stakeholders, which would then happen in the sphere of interest and create an impact (Belcher et al., 2020). Douthwaite et al. (2007) take a similar perspective of a project’s influence on outputs, outcomes, and impact. We found a consistent conceptualization of value creation in spheres of declining control (Grönroos and Voima, 2013), where value creation covers a provider sphere (production), a joint sphere (value creation in interaction), and a customer sphere (independent value creation). Synthesizing these findings with Propositions P2, P3, and P5, we suggest that research co-production occurs within the sphere of control where outputs are first co-produced. These outputs are later disseminated through interactions with society (the sphere of influence). Impact takes place in the sphere of interest, where ultimate impact creation lies solely in the hands of society in the form of research uptake. Thus, for the new framework, we propose the following:
P8: Any research impact assessment framework should consider the spheres of declining control and influence in which impact is created.
A participatory and up-front approach to evaluate research societal impact
A constituent component of the OEA is the ToC (Belcher et al., 2020). The ToC was the analytical framework for evaluating research projects (Belcher et al., 2020; Wilkinson et al., 2021) and was developed to help model change-processes in complex settings (Belcher et al., 2020; Wilkinson et al., 2021). Basically, the ToC explains how and why certain research project activities lead to specific outputs that ultimately generate impacts. Moreover, it identifies the main stakeholders in the process and attributes their actions to outputs and impacts (Belcher et al., 2020). Academics in the field agree that such an explication of causality is inevitable (Douthwaite et al., 2007; Wilkinson et al., 2021). As the ToC is a powerful tool in explicating causal inferences in complex project settings, we considered the incorporation of a ToC in each research impact assessment. In addition, ToCs can be designed ex ante and ex post and be used for steering purposes (Belcher et al., 2020), a requirement for a research impact assessment framework as postulated in P4. Moreover, ToCs are typically constructed jointly with all the participants in a research project (Belcher et al., 2020; Douthwaite et al., 2007; Wilkinson et al., 2021). Given the growing relevance assigned to collaborative evaluations in the research impact assessment literature (Pedersen et al., 2020; Smit and Hessels, 2021), we expect the integration of a collaboratively designed ToC will generate meaningful impact pathways. When societal impact is the uptake of research (P5), and stakeholders are presumed to inform this uptake, they should, in any case, be part of the ToC design. On a related note, Wilkinson et al. (2021) highlight the importance of involving stakeholders in the evaluation process to diminish the criticized linearity of ToCs and to account for the complexity of programs. Based on the previous argument, we propose the following:
P9: Any research impact assessment framework should incorporate a participatorily designed ToC.
Overall, the ToC is perceived as a stakeholder model of research, attributing an important role to stakeholders in knowledge production processes (Pedersen et al., 2020), which expresses the FP and Proposition P2.
The developed ToC identifies and describes outcomes that can be expected when undertaking a research project (Belcher et al., 2020). Defined ex ante, these outcomes can be utilized to monitor the project. Ex post evaluations build on measurable indicators of the outcomes produced by projects. In this way, the OEA responds to requests for integrative research impact assessment frameworks that pay attention not only to the processes of impact creation but also to measurable outcomes (P7) (Pedersen et al., 2020; Williams, 2020). By jointly the project team and stakeholders jointly determining the measures for outcomes, instead of ex post measurement of overarching normative indicators, the necessary nuancing of impact evaluations demanded by academics is ensured. Consequently, we suggest the following proposition:
P10: Any research impact assessment framework should develop project-specific outcomes and impact measures derived from the ToC assembled upfront.
Although Belcher et al. (2019, 2020) argue that ToCs are capable of reflecting the complexity and multi-dimensionality in collaborative projects, criticism remains that a ToC does not adequately address the interactions of actors involved, their relationship, and the wider context in which the projects operate (Douthwaite et al., 2007; Wilkinson et al., 2021). Now that we have construed societal impact solely around research uptake grounded in relationships and interactions, we must assume that a participatory designed ToC is insufficient for research impact assessment. Consequently, a generic research impact assessment framework should include a component that supports the explication of relationships and interactions. Such integration is useful for overcoming ToC deficiencies (Wilkinson et al., 2021). Wilkinson et al. (2021) recently proposed the integration of system maps in the ToC to understand how projects intervene in complex systems with their dynamics. However, this approach mainly focuses on system dynamics and does not consider personal interactions, relationships, and experiences—attributes we found essential to a new conceptualization of societal impact. Therefore, we used a suitable concept from the participatory research evaluation domain. Douthwaite et al. (2007) developed a participatory impact pathway analysis method. In this framework, the authors suggest the integration of network maps into project evaluation design, in addition to a logic model. Network maps are at a more granular level than system maps and aim to provide information on actors in the ToC representing systems of relationships (Douthwaite et al., 2007). By constructing network maps, the reach of a project can be explained, and interactions and relationships in the sphere of influence become palpable (Douthwaite et al., 2007). Network maps should be constructed at the beginning and end of the project, drawing “now” and “future” networks. The “future” networks, then, are intended to help to depict the dissemination of research, uptake, and usage. Further, the construction of “future” networks at the beginning of the process enables the steering of network building toward the aspired network for the ultimate societal impact (Douthwaite et al., 2007), which benefits the identified need for frameworks to support steering and monitoring (Pedersen et al., 2020). For the reasons stated above, with particular attention to P3–5, we propose the following:
P11: Any research impact assessment framework should integrate a network map in the ToC.
For clarity, we summarized the propositions in Table 2 and integrated them into visualizing the generic research impact assessment framework. Note that the framework does not intend to provide concrete guidance on measures of the constructs in the framework since we propose participatory identification of what should be measured in any specific project. Rather, the suggested framework aims to guide academia toward a new understanding of impact creation, where we acknowledge co-creation as part of any research activity. In doing so, we propose that any research project be appropriately assessed within the framework suggested in Figure 1.
Propositions structured along the two areas of framework development “Conceptualizing the Impact Creation Process in Research” and “A Generic Research Impact Assessment Framework.”
ToC = Theory of Change.
Conclusion
This study originally set out to develop a research impact assessment framework for research projects executed in a co-productive mode. However, at the outset of this study, a necessary shift in the understanding of co-creation in research impact took place. This, in turn, entails a novel understanding of societal impact as solely being the uptake of research.
Theoretical contribution
We hope to make a meaningful contribution to the research impact domain by conceptualizing the interweaving of (co-produced) research projects, impact creation, and societal impact through the synthesis of marketing theory (Ranjan and Read, 2016; Vargo and Lusch, 2004) and research impact theory (Belcher et al., 2019, 2020; Bornmann, 2013; Douthwaite et al., 2007; Louder et al., 2021; Wilkinson et al., 2021). The results have two main outcomes. First, we demonstrate that all the research projects are co-productive. Second, impact creation is a two-dimensional process, one of which is research co-production and the other represents an assemblage of impact categories.
The latter encompasses societal impact, which we argue is equivalent to the ViU. To the best of our knowledge, this perspective is new and requires a change in academic thinking. Adopting the proposed conceptualization would point away from using multiple terms for distinct collaborative projects and turn to universally valid terminology. Most importantly, this terminology refrains from using co-creation in the domain in general, as we acknowledge that we can conceptualize societal impact solely with VCC. Other impact categories might still comprise dimensions independent of usage (i.e. economic impact probably mainly constitutes hard financial facts). In this regard, academia should cohesively utilize the term impact creation. Furthermore, research co-production applies to any research project, encompassing all activities, outputs, and outcomes leading up to impact.
We further contribute a theoretical explanation of impact measurement, specifically for the impact dimensions in co-production, without which the development of quantitative measures is problematic (MacGregor, 2021).
Another contribution derived from this novel conceptualization of impact creation is the possibility of drawing on expertise from participatory research evaluation, which we assume to be valid for any research project. This is a significant step toward developing a generic research impact assessment framework valid for any research project, which reduces confusion and complexity in the fragmented framework landscape, responding to calls from academia (Pedersen et al., 2020; Williams, 2020).
Finally, the impact creation framework proposed benefits the development of an assessment framework. We also add to the extant literature on societal impact and suggest a conceptualization of societal impact that academia has lacked thus far (Bornmann, 2013). With this concept of societal impact in mind, academics are called upon to rethink the necessity of defining measurable indicators for societal impact ex ante, given that neither consumers nor producers can predict customers’ experiences (Prahalad and Ramaswamy, 2004). Rather, we plead for more in-depth involvement with the determinants of societal impact (relationships, networks, interactions) and focus on measuring their quality to push impact creation as much as possible.
Practical implications
The integration of VCC, upon which the generic research impact assessment framework is based, has several practical implications for research evaluation. Most importantly, the generic research impact assessment framework allows practitioners to rely on one framework that is applicable to any type of research project. Therefore, the intricacy of finding an appropriate evaluation tool among a myriad of options (Louder et al., 2021) is eliminated. Meanwhile, the participatory nature of the evaluation process and the development of project indicators allow for reflexivity in the evaluation design. The framework is sufficiently broad in scope (Bornmann, 2013), to ensure that each project’s specificity (Louder et al., 2021; Pedersen et al., 2020) is taken into account. Including stakeholders in developing an evaluation design further benefits the practicability of the evaluation (Milat et al., 2015).
Moreover, practitioners can build and seize synergies across domains, the lack of which has been criticized by academics (Louder et al., 2021). An important function of the generic research impact assessment framework is the steering and monitoring of ongoing projects, which is enabled by the ex ante design of ToCs, anticipated outcomes, and network maps. The research project teams can then always revisit the ToCs to ensure alignment of the ongoing actions and desired outcomes. As we have already argued above, the impact dimension is outside the sphere of influence of the project team; hence, the steering and monitoring function is restricted to the research co-production dimension.
Ultimately, the generic research impact assessment framework enables a comparison of the success of diverse research projects, which we believe has been lacking thus far. Although the evaluation process still includes the development of project-specific indicators of outcomes to measure, the progress of a research project and the extent to which the desired outcomes are achieved can be made explicit by comparing ex ante and ex post ToCs and network maps.
We must of course concede that the execution of the proposed framework demands resources and time. Allowing every project to create its individual ToC, as well as ex ante and ex post comparisons of ToCs and network maps would not only mean shifting funding practices but also increase the duration of evaluations. The resources needed to perform our proposed research assessments might pose a considerable constraint.
Concluding remarks
The overarching aim of this study was to develop a framework for the assessment of research impact that is universally applicable yet allows for project specificities and contributes to the conception of societal impact. The conceptualization of societal impact we propose promises to contribute to these goals. We did not aim to provide a step-by-step guide for evaluators but rather set a new direction in research impact assessment. We suggest that the change in perception of impact creation we propose could lead to a new direction in research impact assessment, where synergies can be exploited, and evaluators’ tasks can be eased. Furthermore, we are strongly convinced that a new conception of societal impact in the sense of usage, networks, and relationships will meaningfully push impact creation.
However, we concede that this study solely addresses the tip of the societal impact iceberg. Future research, therefore, would be well advised to conceptualize relationships and networks relevant to the impact creation process in more detail. This study could relate the VCC construct to the impact creation process, especially for societal impact. Nevertheless, the extent to which the co-production and ViU dimensions and measures apply to the research impact context deserves to be further investigated and adapted.
The study at hand only conceptualizes how societal impact is embedded in the creation of research impact, which is beyond the sphere of influence and ultimately manifests in the uptake of research. Certainly, academia should turn the focus toward the clarification of the societal impact concept—for example clarifying what is meant by research uptake? An interesting starting point when addressing this question could be Carol Weiss’ (1979) elaborations on research utilization. Weiss explores the question of what research use is and identifies different meanings of research utilization that range from conceptual to instrumental, use (Weiss, 1979). Questions future research should address in this regard could be “Can societal impact have many meanings?” or “Can we still apply the proposed farmework if we understand research uptake conceptually, as a change in thinking about and understanding of research consumers?”
Finally, this article merely examined societal impact and co-creation through the lens of marketing theory. It would be worthwhile to put on different lenses from other theoretical perpectives. Understanding societal impact with an eye toward knowledge brokering might be an attractive future research direction which might also generate new perspectives and insights.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research received funding under a grant (03IHS062A) from Innovative Hochschule, Germany.
Data availability statement
Data sharing is not applicable, as no new data were generated or analyzed in this study.