Abstract
Information science researchers are increasingly seeking to understand the utilisation of knowledge generated through scientific research outside of academia. Although the conceptual levels of knowledge utilisation are well established, our understanding of the various information practices for knowledge utilisation employed by researchers remains limited. This study identified such information practices by text-mining 6637 case studies documented under the United Kingdom’s Research Excellence Framework. The results were augmented with expert judgement to develop a framework consisting of nine types based on the theoretical framework of research knowledge utilisation. Three emerging types were identified: deliberation, co-creation and foresighting. They indicate the rise of information practices leveraging social media and analytical capabilities to engage potential beneficiaries in using and realising the value of research.
Keywords
1. Introduction
Information science researchers are increasingly seeking to understand the knowledge utilisation of research outputs beyond academia, which is essential for academic research to have a meaningful impact in practice [1–3]. Knowledge utilisation involves dynamically synthesising, disseminating, exchanging and applying research knowledge beyond academia to generate societal impact [4]. The desire to enhance the utilisation of research knowledge in practice is reflected in the growing number of funding agencies that require publicly funded research to be accountable and measurable [5]. In addition, applying research knowledge to practice serves to increase researchers’ visibility and identity in society and to educate the public on the value of research [6–8].
When scientific research generates useful and reliable knowledge for society, it is more probable to have a societal impact [9]. Research knowledge is often regarded as a kind of public knowledge that should be communicated and disseminated to the audience beyond academia to reap societal benefits at large [10]. This is consistent with the knowledge utilisation framework, which states that the flow of research knowledge to end users in society involves multiple levels, where researchers interact with potential users to make research findings useful and relevant [11]. The framework suggests that researchers should actively disseminate and promote their findings so that research outputs have a societal impact.
While research on knowledge utilisation has identified levels of utilising research knowledge in practice [11], our understanding of information practices enacted by researchers at each level remains anecdotal and limited. Information practices refer to methods and approaches used in a variety of information-related activities, such as information collection, information access and storage and information seeking and sharing [12,13]. It is critical to understand information practices because research knowledge utilisation is inherently information intensive, involving the access, understanding, dissemination, sharing and utilisation of information in practice [14]. Identifying effective information practices also has practical relevance for knowledge brokers (e.g. researchers, decision-makers and other stakeholders), who disseminate and translate information about research outputs into practically useful solutions that benefit society [15]. Examining information practices offers insights into specific informational activities to achieve knowledge utilisation.
This study, therefore, seeks to contribute to this research area by identifying information practices for promoting research knowledge utilisation outside academia. In particular, we analysed a large sample of 6637 actual impact case studies submitted to the Research Excellence Framework (REF) in the United Kingdom. An impact case study summarises the status of research in a certain academic area and presents knowledge that is available for other stakeholders beyond academia to access [2]. More importantly, it provides details of how research outputs are utilised in society to produce societal benefits [16]. Hence, impact case studies may be considered a rich source of data for understanding information practices for research knowledge utilisation. We analysed the case studies using topic modelling with the abductive approach [17], which involves a process of iteratively comparing the topics algorithmically identified with theoretical concepts based on human expert judgement for theoretical development.
The findings of this study potentially extend the existing information science research on knowledge utilisation in three ways. First, our analyses of the impact case studies led to the specification of information practices for each level of knowledge utilisation. This elaborates the knowledge utilisation framework and provides more detailed guidance to researchers and other stakeholders seeking to promote and utilise research knowledge in practice. Second, our findings extend the existing knowledge utilisation framework, first developed in 2001, with three emerging levels of knowledge utilisation: deliberation, co-creation and foresighting. They capture the digitally enabled, participatory and future-oriented nature of impactful research. Third, this study demonstrates an abductive topic modelling approach to theoretical development for information science scholars studying large text corpora.
2. Literature review
2.1. Conceptualisation of knowledge utilisation
Research on knowledge utilisation is not new. This research field evolved in the 1940s, when a core group of scholars from various disciplines, ranging from agriculture to geography, sociology and information science, showed a common interest in how to utilise research knowledge in practice [18]. Since then, research on knowledge utilisation has been expanding rapidly because individual researchers and research institutions are under pressure to promote the utilisation of scientific findings by decision-makers as well as society [19]. However, due to the diversity and abundance of literature in this field, scholars argue that it is challenging to synthesise this field, and more importantly, there is some conceptual disarray regarding what knowledge utilisation is [20,21]. Heinsch et al. [20] identified a list of terms used to describe knowledge utilisation in prior literature, such as transfer, diffusion, transmission and evidence-based practice. While these terms are all related to the process of knowledge utilisation, they may not represent the whole picture of knowledge utilisation. In addition, these terms may be viewed differently by scholars from different fields. Therefore, knowledge utilisation may not be a single, discrete event. Instead, it is a complex process encompassing multiple stages. For example, Hoffmann et al. [21] conceptualised knowledge utilisation as ‘a complex interactive and interactive process in which different forms of knowledge emerge, circulate and are applied in science and practice’ (p. 37).
Research knowledge utilisation plays a crucial role in linking the gap between research outputs and practice [21]. In general, the use of knowledge in practice can be instrumental, conceptual or symbolic [22]. Instrumental use of knowledge indicates that research findings are directly used to help decision-making or product development. Conceptual use is defined as the use of research knowledge in a more general and indirect way. For example, new ideas or new interpretations are provided for the facts and issues surrounding the decision-making or product development process without changing the final decision. Symbolic use refers to the cases where research knowledge is used to legitimate the views of practitioners and policy-makers. Such a typology of knowledge use provides insights into examining how different forms of research knowledge can be utilised outside the academic community.
In recent years, researchers from a variety of disciplines have studied knowledge utilisation in practice. In general, there are two primary streams of this literature. First, a group of studies has focused on the outcomes of knowledge utilisation [23,24]. These outcomes can be tracked based on economic criteria such as patent data, licensing agreements and spin-off creations [25,26]. Compared with other criteria (e.g. culture-related criteria), economic criteria are certainly the easiest to assess, although no reliable indicators have yet been established in the field [1]. This stream revealed that it is apparent that some industries (e.g. engineering) reap more economic benefits from research knowledge utilisation than do others (e.g. history) [27].
Moreover, the second stream of research has examined activities undertaken by researchers to transfer knowledge to potential users in society [19,28]. These studies regard researchers as the source of knowledge and assess their specific proactive behaviours and actions used for knowledge utilisation [29]. Multi-item scales or multistage frameworks have been proposed to explore research knowledge transferred through research institutions to relevant stakeholders such as government agencies, companies and other organisations [28,30,31].
Some authors contend that extant research on knowledge utilisation has mainly focused on the first stream to demonstrate how research outputs bring financial returns in society [32,33]. However, these tangible outcome measures are not able to capture the complete picture of knowledge utilisation [34]. The impact of research outputs on practice might be underestimated if much attention is given to such commercial outcomes of research. A survey study showed that the best way to disseminate research knowledge is through public and personal channels such as nonacademic publications, social media and informal interactions rather than patent data or licensing agreements [35]. In addition, Cosh et al. [36] found that informal contacts facilitate the flow of research knowledge to business activities. Therefore, research on knowledge utilisation needs to shift its focus from outcomes of knowledge utilisation to activities that facilitate knowledge utilisation [30].
Promoting knowledge utilisation is necessary because research knowledge tends to be intangible, tacit, embodied in principles and theories and stored in people’s minds [37]. Scholarly knowledge must be moved to potential users in society through a range of knowledge activities [11]. Knowledge utilisation often involves the use of research findings from multiple research outputs rather than a single research article [28]. For example, policy decision-making is not solely dependent on the research findings from a single study. Instead, it requires knowledge from a series of research outputs that converge in one direction [38]. Different research findings may require different knowledge utilisation activities.
Over the years, scholars have developed various knowledge utilisation frameworks or scales to describe how knowledge is transferred from academia to society [32,39–42]. A review of the literature on knowledge utilisation identified twenty-eight models that describe the whole or part of the knowledge utilisation process [43]. While variations exist in these frameworks or models, they reflect the fact that knowledge utilisation involves an amalgamation of different activities that are linked to researchers’ specific knowledge-related behaviours. However, most frameworks focus too much on instrumental use or a particular use of knowledge (i.e. evaluation) [11]. In addition, some scales pay much attention to the outcomes of knowledge utilisation (e.g. decision-making) while neglecting its utilisation per se [28].
One exception is the knowledge utilisation framework developed by Knott and Wildavsky [31], which includes seven levels of knowledge utilisation. Landry et al. [11] further refined the framework to include six conceptual levels of knowledge utilisation: transmission, cognition, reference, effort, influence and application (see Table 1). They represent the utilisation of research knowledge outside the academic community and demonstrate the different ways science can contribute to the public. This framework has been extensively studied in the literature by Landry et al. [28]. For instance, when assessing how research knowledge is used in the policy process, Webber argued that the framework ‘not only captures the extent to which information is processed cognitively by the policy-makers but also its consequence in the policy process’ [44] (p. 21).
Knowledge utilisation framework [11].
2.2. Information practices for knowledge utilisation
Knowledge utilisation for societal impact is essentially practice-oriented, and it is necessary to go beyond conceptualisation to understand the activities that researchers actually engage in. According to practice theories [45], information practices include a myriad of information activities regarding how information or knowledge is utilised in a specific context. Cox [13] suggested that ‘different practices have different patterns of information-related activities woven through them’ (p. 71). Applying these insights into the context of research knowledge utilisation, information practices refer to those knowledge-related activities that researchers can enact to facilitate the dissemination of research knowledge and its utilisation by the public [14]. Such practices can contribute to satisfying the conceptual, instrumental and symbolic knowledge needs of research users [46]. As such, it is especially pertinent for scholars to identify information practices to promote the transfer of research knowledge into practice.
Several studies have examined information practices adopted for knowledge utilisation in the literature. For example, using a social network perspective, Arza and Carattoli [47] found a positive impact of social ties on bidirectional information practices (e.g. signing contracts with industry and participating in public–private research networks), which further creates knowledge benefits for research institutions. Similarly, Bozeman and Gaughan [48] examined how research grants and contracts affect a range of activities for realising impact (e.g. providing consultant services). Moreover, Landry et al. [28] identified a few information activities related to knowledge transfer (e.g. adopting knowledge in new goods and services) and assessed how four categories of assets (i.e. financial, organisational, relational and personal) influence these activities. In summary, previous studies have identified some information practices in the process of knowledge utilisation based on the study context and research objective. These information practices are summarised in Table 2.
Information practices for knowledge utilisation.
Our review of the literature found that most of the prior studies used survey methods to examine the extent to which researchers adopt information practices when they seek to translate knowledge into society. The information practices identified in the literature are general and can be used across multiple academic disciplines [32,42,49,50] as well as in a specific area (e.g. business, science and engineering) [28,30].
However, it is important to acknowledge that while these information practices facilitate the utilisation of research knowledge among potential users, two major gaps in the literature remain. First, information practices in the existing studies were mostly identified using surveys of or interviews with academic scholars. The coverage of information practices identified is limited to the sample studied. Second, the information practices identified are often presented as a list without a coherent conceptual structure. This study seeks to address the two gaps by using the knowledge utilisation framework as the conceptual foundation for organising information practices identified from a large corpus of impact case studies. Knott and Wildavsky [31] highlighted the importance of keeping the levels of knowledge utilisation distinct and relating relevant information practices. Selecting appropriate information practices allows knowledge brokers to achieve knowledge utilisation effectively.
3. Methods
3.1. Dataset
To understand how information practices are used in actual research to promote the utilisation of research knowledge, we used impact case studies submitted to the REF 2014 in the United Kingdom for analysis. As one of the world’s best-known national evaluation systems [1], the REF 2014 required research institutions to submit impact case studies to one of 36 units of assessment (UoAs). The UoAs were categorised into four main panels with broad areas: Panel A, Life Sciences; Panel B, Engineering and Physical Science; Panel C, Social Sciences; and Panel D, Arts and Humanities. Following the guidelines and a template provided by the REF 2014, the impact case studies aimed to describe the research projects conducted in UK universities and demonstrate evidence of impact (i.e. how the impact of research outputs can be achieved beyond academia). Each case study had five sections: summary of impact, underpinning research, references to that research, details of the impact and sources to corroborate the impact. Experts from the four main panels were assigned to evaluate and rate the case studies, ranging from 1* (lowest) to 4* (highest).
All 6637 available cases were downloaded from the REF 2014 impact case studies website (available at: https://impact.ref.ac.uk/casestudies/). Following prior literature on analysing the REF case studies [51], our analysis focused on the fourth section of the case studies (i.e. details of the impact) in that it provides a detailed description of how research knowledge embedded in the research outputs is utilised outside academia for achieving societal impact. Therefore, the impact cases in the dataset can be a potential source to identify a wide spectrum of information practices for knowledge utilisation.
3.2. Topic modelling
Since the large scale of this dataset prohibited direct human coding of all the documents, we used Latent Dirichlet Allocation (LDA) topic modelling [52] to identify information practices in the REF cases. This unsupervised machine learning approach aims to uncover the hidden semantic structures in a set of documents. It starts with a model to describe the corpus and then adjusts various probability-based parameters to fit the model. It is based on the assumption that there are k topics in the whole corpus of documents, and each document, to some extent, discusses these k topics. As such, by defining a topic as a group of words and word distribution, the LDA represents a document as a mixture of topics [52]. Based on a cluster of words assigned to a topic, the output of LDA topic modelling requires human interpretation from researchers.
4. Data analysis and findings
In this study, we used an abductive approach to analyse the large dataset, consisting of four main steps (see Figure 1). The abductive approach is an iterative comparison between empirical discovery and theoretical justification to expand the understanding of both theory and reality [17]. Different from deductive or inductive approaches, abductive analysis requires researchers to continuously create theoretical inferences based on data and analysis, during which new and emergent concepts might be identified [53]. The four steps in the data analysis process reflect a gradual process of matching empirical data (i.e. information practices in the impact case studies) to theoretical constructs (i.e. knowledge utilisation levels), typical of abductive analysis.
Overview of the data analysis process.
4.1. Step 1: data preprocessing
The textual data were preprocessed using the statistical software R version 3.5.1. First, all sentences in each document were tokenised into sets of singular words. All words were converted to lower case, and punctuation and numbers were removed. Furthermore, using a list of 174 stop words from the ‘tm’ package in R [54], common and uninformative words (e.g. ‘is’, ‘are’, ‘the’ and ‘a’) were excluded from the corpus. In addition, eight specific stop words that were very frequently used in the impact cases were also excluded (i.e. ‘impact’, ‘ref’, ‘date’, ‘new’, ‘case’, ‘study’, ‘page’, ‘research’) because they did not add any insights to our understanding of this corpus [51]. Finally, we performed a stemming step to transform different forms of the same word (e.g. practices, practicing, practised) to its basic form (e.g. practic). With the completion of these preprocessing steps, we created a document-term matrix (DTM) that provided information about word frequencies without considering their order. The DTM consisted of 6637 documents with 150,702 unique terms. The top 100 terms with the highest frequency in the DTM are presented in the word cloud (see Figure 2), which provides a concise snapshot of what these impact case studies are talking about. The font size in each word is proportional to the frequency of each word. These raw word frequencies only indicate the ‘popularity’ of a word within the corpus [51]. The co-occurrence between these singular words will be identified using topic modelling in the following steps.
Top 100 words with highest frequency in the corpus.
4.2. Step 2: algorithmic topic modelling
The second step covered the implementation of topic modelling of the DTM to render topics embedded in the impact case studies. In particular, we applied the LDA algorithm with collapsed Gibbs sampling to the DTM. This method allows iterative steps through configurations to estimate the optimal model fit [55]. We tested a variety of topics between 10 and 100 topics in steps of 5 to identify a reasonable number of topics for the corpus. Default parameter settings in the R package ‘topicmodels’ [56] were used in the topic modelling described here, except that the parameter α was specified at a relatively low value (0.01) to generate topics that were distinctly associated with particular documents [51].
Several data-driven metrics were applied to inform the selection of the topic model, including semantic coherence, information divergence and cosine distance. First, the semantic coherence score measures the internal coherence of words within a topic and is highly consistent with human judgement of topic quality. A higher coherence score indicates a better model fit [57]. Second, information divergence describes how a specific topic differs from others. When the information divergence between pairs of topics reaches a maximum, it is said to be the best model [58]. Third, the model with the minimum average cosine distance of topics would be the optimal model [59]. Figure 3 illustrates how the three metrics changed when the number of topics increased. It was found that the metrics did not change significantly after 55 topics. Therefore, the model with 55 topics might be the most suitable model for our corpus. In addition to these quantitative measures, two researchers who were familiar with the corpus visually examined the outputs of each topic modelling (i.e. top 10 keywords) to determine if they contained any poor topics (e.g. uninterpretable topics, those internally inconsistent, or repetition with other topics). The qualitative analysis also resulted in a decision to run the LDA with 55 topics. Therefore, we ended up with 55 topics for description in the final analysis.
Selecting the optimal number of topics.
4.3. Step 3: human expert analysis
The 55 topics identified through the LDA algorithm were interpreted and labelled by human experts based on the most representative keywords and impact cases [60]. To ensure the validity of the labels, five researchers who were familiar with the topic labelled all topics independently and achieved an intercoder agreement of 94.5%. The discrepancies were resolved through further in-depth discussions of the representative impact cases. Moreover, 11 topics were excluded from further analyses. These topics, such as value significance and direct impact (T9), increase food production (T15) and benefit financial markets (T36), were more related to how research outputs were useful at the end of the pathway to impact or what types of impact that research outputs generated. Although such topics were prevalent and significant in the impact case studies, they did not indicate a clear information practice.
The 44 information practices observed are presented in Table 3, along with the representative keywords. The keywords were ranked based on the beta value, which represents the probability of a word belonging to a given topic. For example, the keywords in topic 2 were ‘polici’, ‘govern’ and ‘influenc’, which appear to describe how research knowledge informs government policies. To confirm the meaning of this topic, we read the representative impact cases and found that they indeed described the influence of research on policy-making. The top five information practices in the dataset are as follows: improve clinical practice (T40; 4.44%); organise exhibitions (T16; 4.10%); develop healthcare interventions (T33; 3.75%); enrich education practices (T41; 3.56%); and propose solutions to environmental conservation (T25; 3.41%). Among the top five practices, two practices (T40 and T33) were related to healthcare because the two topics included keywords such as ‘clinic’, ‘health’, ‘intervent’ and ‘improv’. This indicates that a significant proportion of impact case studies focused on the utilisation of healthcare knowledge. This is in line with the fact that the health and medical fields were among the earliest to emphasise the assessment of research knowledge utilisation for societal impact [1]. As an increasing number of other research fields begin to focus on managing knowledge utilisation in practice, we believe that non-healthcare information practices will become more prominent in topic models in the future.
Topics related to information practices and representative keywords.
4.4. Step 4: abductive analysis for theoretical development
The last step of the analysis focused on analysing the information practices identified with reference to the theoretical framework of knowledge utilisation. The information practices were sorted into categories of the framework as much as relevant. New categories were created for information practices that did not fit into existing categories. The analysis was conducted separately by five researchers, with an intercoder agreement of 93.2%. Disagreements were resolved through an in-depth discussion of the keywords and representative cases. The categorisation is presented in Table 4. Most of the information practices could be categorised according to the knowledge utilisation framework.
Information practices mapped to knowledge utilisation levels.
There are six conceptual levels in the original knowledge utilisation framework. First, we identified seven information practices (e.g. publicise research in the media; T14) that facilitate the transmission of research findings to a wide range of public audiences. Second, two information practices (e.g. strengthen public understanding; T39) could be used at the cognition level so that research findings are understood by various stakeholders. Third, two information practices were relevant at the reference level, as they demonstrate how research findings have been cited or referenced in non-academic reports (e.g. provide reference knowledge for guidelines; T38). Fourth, the effort level comprises information practices capturing the efforts that stakeholders took to make the research more impactful (e.g. promote adoption in various locations; T3). Fifth, half of the information practices (n = 22) were categorised at the influence level. These practices reflect the ways through which research findings can impact our society. Finally, the sixth stage includes practices manifesting what outcomes have been achieved through the application of research findings (e.g. improve clinical practice; T40).
Furthermore, seven of the information practices did not fit well into any existing categories of knowledge utilisation. They indicate newer approaches to research utilisation, and new categories were created to account for them: deliberation, co-creation and foresight. First, ‘Deliberation’ captures the discussion and deliberation of research findings in the general public. Such knowledge utilisation emphasises the fact that research stimulates public discussion on various issues in society (e.g. inform political debates; T45). For example, research conducted by David Paton at the University of Nottingham ‘stimulate[s] public debate [on the issues of adolescent sexual health], notably in the Irish Herald (28th March 2010) …’ [61]. Some case studies also highlighted the role of the Internet in facilitating deliberation. For instance, the Scots Words and Place-names Project (SWAP) conducted by the University of Glasgow led to ‘a wider public discussion forum on the website, to which anyone with an interest in Scots could contribute, as well as reading and commenting on the contributions of others’ [62].
Second, ‘Cocreation’ focuses on the active participation of research users in research activities to develop new knowledge with researchers. Co-creation is an interactive process for building and synthesising new knowledge. It promotes users’ participation in research to define critical problems in society and explore successful research-based solutions (e.g. engaging with communities/social groups; T13). Technological advancements also made such co-creation more achievable. For example, researchers from the Open University developed a social learning community website named ‘iSpot’, which enabled a collaborative learning experience. Participants could actively collect data in the field and obtain feedback from peers to reach scientific conclusions [63].
Third, ‘Foresighting’ is an emerging form of knowledge utilisation enabled by recent advancements in data science and technologies such as the Internet of Things. Information practices for foresighting are different from those for applying knowledge in a traditional way (e.g. envision possible futures; T27). Foresighting focuses more on using knowledge in IT technologies to better inform data prediction and support digital innovation. For instance, research conducted by Birkbeck College provided new methods for combining different potential predictive models to reduce the data uncertainty of the optimal model, which helps the forecast process based on real-time data [64].
5. Discussion and conclusion
5.1. Summary of key findings
This study sought to identify information practices from impact case studies to contribute to research and further development of the concept of research knowledge utilisation. First, we identified specific information practices for the knowledge utilisation framework [11]. These practices could provide guidance when knowledge brokers intend to promote and utilise research knowledge in practice, which in turn helps make research visible and useful. Second, in addition to the six levels in the original knowledge utilisation framework, we identified several information practices that indicate three emerging levels: deliberation, co-creation and foresighting. The three levels represent potential alternative ways of knowledge utilisation and reflect the future nature of impactful research.
‘Deliberation’ describes that the public discusses and debates research topics via multiple channels (e.g. conferences, workshops). More importantly, owing to the prevalence of digital technologies (e.g. social media), such discussions and debates have become much more accessible and convenient among research users [65]. Research users can post comments on online discussion forums and publish opinions about research on social media. Deliberation of research findings by the public builds the capacity of research users to better understand and utilise the findings [66]. Deliberation is of importance in the current era, as the global public health crisis (i.e. COVID-19) has also become a manifestation of the information crisis [67]. Unlimited misinformation and rumours spread fast on the Internet. Sharing and discussion of research problems surrounding the pandemic on social networking sites can inform researchers on emerging public concerns.
‘Cocreation’ emphasises the interaction between researchers and users for producing new knowledge. Information practices (e.g. develop clinical treatments; T24) reflect the participatory design of research, which directly involves users in the research design process to improve research outcomes [68]. Co-creation allows researchers to better understand users’ needs and thus to achieve better knowledge utilisation [69]. Recently, digitisation has enabled users to participate in computer-mediated interactions with research scientists as well as other users [70]. Such participation in knowledge co-creation not only fosters communication between researchers and potential users but also shapes the way new knowledge is generated and interpreted by the public. Therefore, IT-enabled technologies have facilitated knowledge co-creation and strengthened the participatory nature of scientific research.
‘Foresighting’ is concerned with the use of research knowledge to understand possible futures and determine the best course of action to shape the future. Foresighting information practices focus on the role of IT-enabled technologies in identifying new demands, predicting possible outcomes and avoiding undesirable consequences [71]. This is particularly evident in the context of global public crises, such as the COVID-19 pandemic [72]. With the rapid spread of COVID-19, information technologies such as infection prediction models, chatbots and tracking applications are constantly being developed to combat the virus [73,74]. Hence, foresighting describes how researchers, especially those in the information system discipline, could utilise research knowledge in the process of designing various IT artefacts. This emphasises that the utilisation of research knowledge needs to be future-oriented. Information practices for foresighting allow researchers and potential users to actively pursue the best future scenarios and consciously avoid the worst. In summary, foresighting increases people’s awareness of possible futures and informs decisions and actions.
5.2. Implications for research
This study contributes to research on knowledge utilisation in three ways. First, we identified information practices facilitating research knowledge utilisation and enriched the knowledge utilisation framework. They represent a broad range of methods and approaches that knowledge brokers can consider when transferring research knowledge in practice. Compared with several existing studies that applied survey methods to identify information practices [42,48,50], the 44 information practices in our study are concise but comprehensive since they are based on a large sample of actual impact cases from various academic areas. Research on knowledge utilisation has emphasised the need for uncovering complex principles and procedures underlying knowledge utilisation [1,11,31,75]. Our results contribute to this area by detailing how information practices make knowledge utilisation more effective and purposeful.
Second, our study extended the knowledge utilisation framework. While existing knowledge utilisation levels describe a set of potential activities that can be undertaken by knowledge brokers to disseminate and utilise research findings, they have not accounted for the participatory potential of users. ‘Deliberation’ and ‘Cocreation’ identified in our study highlight the fact that research users can contribute to science as well. Information practices may inform scholars on how to enhance the linkage between research and potential target groups in the research design process [76]. Moreover, we found that developments in IT technologies have changed the nature of knowledge utilisation. This is especially true in regard to ‘Foresighting’, where research knowledge is applied in IT-enabled innovations, such as machine learning and artificial intelligence, to predict possible futures. In short, we found that IT technologies have provided new ways of knowledge utilisation and make such utilisation more achievable. Digital information environments should be designed to support information practices for research knowledge utilisation [77].
Third, this study demonstrates an abductive approach to analyse large-scale textual data for theory extension. This approach is different from the traditional inductive approach, in which researchers typically begin with reading a sizeable amount of content to distil first-order codes and may feel lost in the data at an early stage [78]. Specifically, in this study, we first extracted those first-order codes (topics discussed in the case studies) using a computational method (LDA topic modelling). More importantly, while inductive analysis does not logically lead to novel theoretical insights, abductive analysis allows us to iteratively compare our empirical data (information practices) with existing theoretical concepts (knowledge utilisation levels). In this process, empirical data that cannot fit existing concepts open up possibilities for extending the theoretical framework. As a result, additional conceptual levels are further identified. Hence, this abductive topic modelling analysis can be a potentially advantageous approach for information science scholars to develop and extend theories based on textual data in future research [79,80].
5.3. Implications for practice
The extended framework also offers practical implications. First, our framework helps researchers assess their available resources and select relevant information practices to achieve the desired knowledge utilisation. They can also decide what they need to achieve based on the nature of their research. For example, for those who seek to disseminate scientific research to potential audiences, information practices such as publicising research in the media (T14) or presenting findings at conferences/workshops (T51) can be useful. Alternatively, to reach a wider range of audiences in the public, scholars also promote research on websites and social media platforms, such as Twitter and Facebook (T34). Thus, scholars can select information practices based on who the target audience is.
Second, the three emerging levels of knowledge utilisation in our extended framework allow the integration of existing knowledge as well as the generation of new knowledge. In addition, they underline the importance of IT technologies in knowledge utilisation. ‘Deliberation’ and ‘Cocreation’ suggest that researchers need to actively engage with users through formal and informal interactions in a dynamic and functional way [81]. Moreover, ‘Foresighting’ foregrounds the importance of future-oriented research. During the pandemic era, scholars in various academic areas worldwide are collaborating with relevant stakeholders to fight against the invisible enemy [82]. Our extended framework emphasises the need for new forms of knowledge utilisation, including public deliberation, knowledge co-creation and digital foresighting, to alleviate the negative consequences of this pandemic.
5.4. Limitations and future directions
This study should be examined in the context of several limitations. First, we only analysed the REF impact case studies submitted to the United Kingdom. To ensure the generalisability of our results, future research might benefit from analysing impact cases in other national evaluation systems, such as Australia Engagement and Impact Assessment. Second, while impact case studies report information practices to demonstrate evidence of impact, they might include other content such as areas and outcomes of impact. Thus, topics extracted from those case studies might not represent information practices directly, although we attempted to exclude some irrelevant topics in our data analysis process. Interviews and survey studies could be conducted with scholars from various academic areas in the future to validate and complement the current list of information practices in our extended framework. Third, it should be noted that the LDA topic modelling method is not novel for extracting topics from a large corpus. Nevertheless, we combined topic modelling with an abductive approach for theory extension, providing a list of information practices for research knowledge utilisation. Finally, it is possible that new ways of knowledge utilisation might emerge with the ongoing development of digital technologies. For instance, in recent years, developments in the Internet and mobile devices have made it possible for researchers to access diverse stakeholders and share research findings in ways that were not possible before (e.g. online knowledge communities) [83]. Hence, periodic identification of information practices would better inform scholars how to effectively utilise research knowledge in practice.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship and/or publication of this article: This study was funded by Singapore Ministry of Education Academic Research Fund Tier 1 (Grant number: 2017T1-001-095-06).
