Sage Journals: Discover world-class research

Abstract

The purpose of this paper is to investigate data relating to the outcomes of enterprise and entrepreneurship education (EEE) activity in UK higher education institutions (HEIs). This is achieved via the use of data obtained from the Research Excellence Framework (REF), the Teaching Excellence and Student Outcomes Framework (TEF), the Knowledge Excellence Framework (KEF) and the Higher Education Business and Community Interaction (HE-BCI) survey. Overall, the analysis suggests, powerfully, that EEE impacts research, teaching and knowledge exchange in a variety of ways. Firstly, it shows that EEE, in terms of the REF, may be up to 46 times more impactful than other management disciplines. Secondly, with regard to TEF submissions, it highlights a positive relationship between the use of the EEE terms and the award level achieved. Finally, research also demonstrates a link between membership of certain HEI mission groups and improved KEF metrics when compared to the sector averages. There is a clear need to research how to develop successful EEE interventions and demonstrate their impact on the graduate, the university ecosystem and the wider economy. These data sources and methodology have not previously been used to develop a narrative for EEE across a university sector in the UK.

Keywords

Enterprise education entrepreneurship education higher education policy research excellence framework knowledge excellence framework teaching excellence and student outcomes framework

Enterprise and Entrepreneurship Education (EEE) represents a range of intra and extra-curricular activities which are focused on the supporting behaviours, attributes and competencies that are thought to have an impact on individuals’ future careers prospects (Quality Assurance Agency (QQA), 2012; 2018). EEE often forms part of a broader employability approach to learning and development in an institution, albeit a distinct one, with a greater focus on deeper interventions to effect changes in the learner that will enable them to, for example, explore opportunities, investigate and address problems, provide value for others, lead, reflect and remain resilient. The aim is to produce graduates who are able to formulate ideas and act on them. Calls to develop EEE are well established (Lahikainen et al., 2018), and considerable work has already been done to integrate it into higher education institutions (HEIs) (Mulholland and Turner, 2019; Solomon, 2007). However, there is surprisingly little literature that explores the distinct impact of these activities (Nabi et al., 2017).

This is not a new problem: in 2007, Pittaway and Cope noted that evaluations of the outcomes of EEE were rare and in the intervening period subsequent papers have echoed their observation (Bryne et al., 2014; Nabi et al., 2017; Rideout and Gray, 2013; Smith, 2015). A review of the literature by the authors, focusing on this work in the context of UK HEIs concluded that these observations held, and that the limited research that did exist could be categorised into four broad groups. Interestingly, the authors noted that research in all these groups largely ignored the UK’s own Excellence Frameworks, which were created by the Government and HEI funding bodies to capture the status of research, teaching and knowledge exchange activities in UK HEIs (Johnson, 2022). Given that these Framework datasets explore a range of output metrics that might be relevant to an understanding of the outcomes of EEE, this appeared to be an important omission which could be addressed by further investigation.

This paper will, therefore, use the Excellence Framework data as lenses through which to explore the evidence presented in terms of the outcomes of EEE and its associated impacts.¹

EEE outcomes research

In preparing this paper the authors reviewed the academic (Kamovich and Foss, 2017; Nabi et al., 2017; Pittaway and Cope, 2007), stakeholder (Hannon 2007; Rae et al., 2012) and practitioner and policy (APPG, 2018) literatures which explore the outcomes of EEE in UK HEIs. This exercise led to the identification of four broad groups of literature which presented perspectives on the outcomes (and the associated impacts) of EEE: cohort based studies, cross institutional studies, stakeholder reports and regional insight reports. In this section the authors will explore these works to clarify their positions, contributions and their shortcomings.

The first group categorised by the authors included papers that explored enterprising and/or entrepreneurial outcomes through the application of single institution cohort studies. This research typically seeks to connect outcomes – such as business knowledge, attitudes required to run a business, entrepreneurial competencies and entrepreneurial intent – to specific activities that have been used in a programme or module cohort to measure their impact (Bozward and Rogers-Draycott, 2020; Curtis et al., 2020; Sánchez, 2013; Williams, 2011).

The second group included works that explored interventions across institutions. Generally, this was a more diverse literature whose authors sought to explore a range of EEE outcomes. Matlay and Carey (2007), for example, explored entrepreneurship education initiatives over a 10-year period in 40 UK HEIs; unusually, they focused on programme design, delivery and assessment and their impact on how the sector was understood externally. Kitagawa et al. (2015) investigated the differences in entrepreneurial intention for students from two UK HEIs in the same city in an attempt to connect EEE experience (in the institutions) to entrepreneurial economic activity and its impact on the wider city-region. Jones et al. (2017) compared the impact on careers for alumni of two UK universities and found that EEE had long-term positive value in terms of starting a business and career development. Topazly (2018) sought to understand whether EEE activity in three UK universities had a measurable impact on the post-study entrepreneurial activities of Russian graduates on their return to their home country, and what this might mean for the institution with regard to further interventions.

The third group consisted of research reports produced by stakeholder organisations that explored UK-wide approaches. These employed literature reviews, surveys and case study presentations to arrive at broader findings that reflected sectoral trends. The National Council for Graduate Entrepreneurship (NCGE) (Hannon 2007; Rae et al., 2012) explored the development of the entrepreneurial university. Based on its survey of EEE and support activity in English institutions, this work reviewed a wide range of metrics including engagement, provision, support and funding, and the reports that followed used this to suggest institutional improvements and public policy reforms related to EEE. The National Association of University and College Entrepreneurs (NACUE) (NACUE 2010; NACUE, 2014; NACUE, 2021) used surveys and case studies to investigate the role of student-led enterprise societies and their impact on students, intuitions and the wider economy. Reports for the UK Government (Anderson, et al., 2014; Williamson et al., 2013) have variously used case studies and literature views of the efficacy of EEE and its impacts on students, staff, institutions and the economy to argue for policy interventions.

The fourth group consisted of regional insight reports drawn from international surveys, such as the Global Entrepreneurship Monitor (GEM) (GEM, 2021) and the Global University Entrepreneurial Spirit Students’ Survey (GUESSS) (GUESSS, 2021; Saridakis et al., 2016). GEM uses surveys of entrepreneurs, non-entrepreneurs and expert panels combined with economic data to draw conclusions about the worldwide ecosystem for entrepreneurs. GEM’s national reports provide localised insights into the level of entrepreneurial activity in a country and they also compare national conditions to benchmark the country against others and explore its potential as an entrepreneurial economy. GEM has wide-ranging impact, central to which is its role in informing international policy, investment and collaboration. GUESSS confines itself to exploring each participating country’s university-level student population for their intentions and activities in an effort to better understand student entrepreneurship. GUESSS also provides national reports which allow for benchmarking. Its key impact is in helping institutions to better understand their students and the actions they might take to further improve intention and action.

Across the four groups of literature, we observe a diverse range of approaches and measures being applied. These range from a micro-level, exploring specific interventions and their impact on individuals, up to a macro perspective wherein publications have explored broader trends and their impact on economies. Herein, success, or failure, can be measured as anything from a change in mindset to a fluctuation in national GDP.

Furthermore, each group has its own unique challenges: cohort-based and cross-institutional studies tend to be focused on a particular intervention and/or short-term outcomes, which limits their value as general indicators of impact (Nabi et al., 2017); stakeholder reports can provide broader insights into the impact of EEE activities but they are limited by the campaigning positions that underpin their creation; and regional insight reports can provide useful perspectives as to the performance of an economy, or a group of individuals, but their ability to connect this to specific activities is limited, meaning that impacts are hard to identify.

Taken together, these critiques underscore the difficulty in exploring impact in an EEE context which has previously been noted by Nabi et al. (2017). In short, there is a superfluence of impact measures using a range of different data to interpret varied outcomes, some of which may not even be suitable for the intended task. This makes comparison between approaches difficult and serves to obfuscate the true nature of impact in an EEE context.

Crucially, all four of these groups of research largely ignore the UK Excellence Framework data (Smith, 2015) and what it might add to the debate. When taken together, these critiques may explain the limited impact of this work on the development of EEE, and the opportunity this presents to argue for novel approaches.

This paper will, therefore, seek to present an innovative approach by examining the under-utilised information captured by the UK’s Excellence Framework assessments of research, teaching and knowledge exchange in an effort to explore what it shows about EEE in UK HEIs and its impact on research, teaching and students’ entrepreneurial career choices.

UK excellence frameworks

In the UK, HEIs and their activities are governed in part by three Excellence Frameworks against which their research (REF), teaching (TEF) and knowledge exchange (KEF) contributions are judged. These cyclical reviews play a significant role in influencing how the institution is perceived, and how it is funded.

Research excellence framework

The REF is a periodic research impact evaluation for UK universities, which has been running in different formats since 1986 (Arnold, 2018; REF, 2021). The REF is conducted by a process of expert review which draws on subject-based panels of senior academics, international members and research users to assess submissions based on their quality, impact and the institutional environment they represent. The review itself is based on an institutional submission which includes selected research outputs, impact case studies and a narrative element that details the HEI and its research facilities, systems and processes.

The panel evaluates the institutional submission and grades it on a star basis (REF, 2021):

• Four-star: outstanding impacts in terms of their reach and significance.

• Three-star: very considerable impacts in terms of their reach and significance.

• Two-star: considerable impacts in terms of their reach and significance.

• One-star: recognised but modest impacts in terms of their reach and significance

The star grading is used to help stakeholders understand how HEIs are meeting the UK funding councils’ policy aim to ‘secure the continuation of a world-class, dynamic, and responsive research base across the full academic spectrum (UKRI, 2020) within UK higher education’.

According to REF 2021 information (UKRI, 2021), the main use of the REF is to guide the allocation of about £2 billion per year of public funding for universities’ research. The data are also used by UK publishers as a metric in several league-table ranking guides, and will be a key performance indicator for any research-intensive institution.

Teaching excellence and student outcomes framework

The Teaching Excellence and Student Outcomes Framework, commonly known as the TEF, is a voluntary quality mark which measures teaching excellence in HEIs in an effort to help students select the best institution in which to undertake their studies. It was introduced in the Higher Education and Research Act in 2017 (DfE, 2017).

Institutions take part in the TEF to highlight the quality of their teaching as a point of differentiation. The TEF uses a Gold, Silver or Bronze rating to highlight the relative quality of provision in an HEI. It operates on a rolling 3-year submission cycle and, to date, 130 HEIs have undertaken the assessment with 26% achieving a Gold award, 50% Silver and 24% Bronze (Vivian et al., 2019). The TEF is also assessed using an independent expert panel. In this instance, its membership includes students, academics and other teaching and learning experts. The panel assesses the institution by reviewing official data on a range of teaching-related metrics alongside a detailed written submission from the HEI (Office of Students (OFS), 2020).

As Gillard (2018) notes, much like the REF, the process of the TEF is not just data-driven and the narrative overview in the provider submission has a significant effect on the award bestowed. Research on the connection between EEE and the TEF is extremely limited; those papers that do exist have tended to focus on using TEF data as part of a broader conversation about the development of the entrepreneurial university, including the need to promote industry engagement in student programmes (Gray et al., 2020), an internal university culture that is embracing of change and flexibility (McKellar, 2020) and the development of lifelong learning both in and outside the curriculum (Morley and Jamil, 2021).

Knowledge exchange framework

The goal of the KEF is to establish the efficiency, effectiveness and benchmarking of knowledge exchange to businesses (Research England, 2021), with full participation in the KEF likely to be a condition of future Research England funding (Research England, 2020). However, unlike the REF and TEF, the operation of the KEF does not rely on an expert panel to reach conclusions. Instead, the KEF takes data from the annual Higher Education Business and Community Interactions (HE-BCI) survey which is compiled by the Higher Education Statistics Agency (HESA), along with some additional data from Innovate UK and Elsevier, and case study narratives where appropriate. The data are put into a model, with the results presented in interactive dashboards alongside a narrative from the HEI, and these are made available on the KEF website (Research England, 2021) to be interpreted by HEIs and their stakeholders. The dashboard presents a rating against seven perspectives with the perspective of Skills, Enterprise, and Entrepreneurship (SEE) being the most relevant for this paper (Research England, 2020). SEE is underpinned by three metrics: graduate start-up rate using student FTE, CPD-CE learner days delivered normalised by HEI income, CPD-CE income normalised by HEI income. The majority of these ratings are based on data collected through the HE-BCI survey. There are challenges for institutions in collecting and reporting graduate start-up data, and data may well be incomplete (Smith, 2015); however, this remains the best proxy for evaluating entrepreneurship activity over an extended period of time, encouraged and supported by UK universities.

Research approach

The research draws on two approaches to interrogate the data: benchmarking methods and keyword analysis. Both of these have been used previously used to research the impact of interventions in higher education (Epper, 1999; Jackson, 2001; Nazarko et al., 2009; Pojanapunya and Watson Todd, 2021). The paper applies these approaches across the three core data sets – the TEF 2017–2020, the REF from 2014 and the HE-BCI survey, which was used in conjunction with student population data to produce a proxy KEF metric for 2017/2018. These were the most recent accessible datasets for each Framework at the time the analysis was conducted.

In this paper benchmarking is used to compare quantitative data from each university for the HE-BCI submissions, while keyword analysis is used to analyse the research outputs and impact case studies from the REF and TEF submissions.

To arrive at our conclusions, each dataset had to be analysed independently using slightly different approaches to reflect the types of data. The following sections outline the specific approach in each instance and detail its application.

REF methodology

REF submissions are composed of both quantitative (such as number of full-time staff, income, etc.) and qualitative (research outputs and impact case studies) data. The analysis conducted in this paper focuses on the qualitative data: research outputs (identified in the REF 2014 database as REF2); and impact case studies (identified as REF3B) which can all be found in REF, 2014.

The first stage of this analysis involved interrogating data from across the REF 2014 portal (REF, 2014) for all units of assessment using the following five EEE-related terms:

1. enterprise education;

2. entrepreneurship education;

3. entrepreneurial education;

4. business start-up; and

5. entrepreneurship support.

The first three terms were taken from the Quality Assurance Agency (QAA, 2012 and 2018) guidance for enterprise and entrepreneurship education for UK HEIs. Items four and five were used to reflect entrepreneurial outcomes relevant to student and graduate start-up activity, with the search results reviewed to ensure that the outputs explicitly mentioned student and graduate businesses. The documents returned were then blind reviewed in depth by two of the co-authors to ensure that EEE was the focus of the document and not simply an unrelated statement. The search and review of the results produced 23 documents, four of which were impact case studies and 19 of which were research outputs. Finally, to understand the impact of each of the 23 documents, the REF 2014 quality profiles for each institution, showing how much of the submission had met a star rating within the unit of assessment, were linked to the documents to provide an indication of their quality.

TEF methodology

As part of their TEF submission, HEIs again provide both quantitative and qualitative data. The quantitative data focus on a range of metrics which we could not relate to our EEE terms. This meant that we focused on the qualitative data and the accompanying result narratives (Office of Students (OFS), 2020).

The 2017–20 data in this analysis consists of 525 TEF submissions from 399 providers of higher education, including further education colleges and other institutions. For this analysis we focused on the HEI only TEF submission documents and results narratives, which numbered 111. These were selected to position these data on a similar footing to the REF and KEF analyses, which focused only on HEIs and were downloaded and then imported to NVivo (Version 12) to establish a benchmark (Tasopoulou and Tsiotras, 2017). A text search query was conducted using the EEE search terms (1, 2, and 3 above), and this produced a list that identified the frequency of occurrence of the EEE terms in the submissions and/or result narratives. This gave an overview of the total number of documents that contained EEE-related words. The number of EEE words occurring in each award category was explored to understand the use of the term and its relation to the award allocated. The authors recognise that this benchmarking approach lacks sophistication but, as the data will later show, it does indicate that there are clear relationships between the use of the EEE terms and the award made to the institution.

KEF methodology

As noted previously, the KEF is based predominantly on the HE-BCI data which, along with its other component parts, are put into a model. The results of this process are then presented in interactive dashboards on the KEF website, alongside a narrative from the HEI. However, the dashboards themselves indicate only the relative strength of an institution against the cluster average for each element of analysis; they do not facilitate simple comparison or provide any deeper insights as to how a strength has been arrived at. This renders them highly problematic as a unit of analysis. Given that the central element of the KEF is the HE-BCI data, we knew from previous research that those data presented the best available proxy through which this activity could be understood (Smith, 2015). For this project we use the HE-BCI data for 2014–18 relating to university-supported student and graduate start-ups. The HE-BCI data for items 1-6 below were obtained from the Higher Education Statistics Agency (HESA, 2020a). In order to generate a proxy calculation for the KEF, student FTEs for 2018 were also obtained from HESA for item 7 (HESA, 2020b).

1. Number of graduate new start-ups created. These are defined as all new businesses started by recent graduates (within 2 years) regardless of where any intellectual property resides, but only where there has been formal business/enterprise support from the HE provider;

2. Number of graduate start-ups, the number still active which have survived at least 3 years or more.

3. Number of active firms, number of new starts, plus the number of graduate start-ups that have been active at least 3 years or more, plus those companies that have been active for between 1 and 3 years. This is a catch-all category that captures an overview of all registered graduate businesses from start-up to 3 years and beyond.

4. Estimated current employment (EFTE, full time employees);

5. Estimated current turnover (£000);

6. Estimated external investment (£000) from external partners but excluding investment from HEFCE (now the Office for Students)/the Department of Business, Innovation and Skills (now the Department for Department for Business, Energy and Industrial Strategy) third stream funds.

7. Student FTE numbers.

It should be noted that HE-BCI statistics relate to both students and graduates up to 3 years from graduation (HESA, 2020c), and not just students, which might be inferred from the KEF’s use of student FTE. The total student FTE each year also includes non-EU international students who are not able to start up a business whilst a student due to visa restrictions, but may apply as graduates for a visa to start up a business. Further, the FTE year used by UKRI to calculate the KEF ratio relating to EEE is also not clear from the methodology information provided. The total student FTE (total studying at all levels and fee status) for the year of the HE-BCI return has therefore been used as a proxy here (i.e., the 2017/2018 HE-BCI return will be matched with the 2017/2018 total student FTE numbers).

Findings

Research excellence framework review

There were a total of 12,202 research output submissions within the C19 Business and Management Studies subject-based unit of assessment submitted to REF 2014, of which only 16 research outputs were related to our five search terms. A further two relevant outputs were submitted to the C25 Education unit of assessment and one output to D35 Music, Drama, Dance, and the Performing Arts, providing a total of 19 research output submissions relating to EEE. The search also identified four impact case studies from 414 submissions in total within the C19 unit of assessment. The results of these searches are presented in Table 1 (note that one impact case study contained two of the selected search terms).

Table 1.

Research outputs and impact case study searches (REF, 2014).

Term	Research outputs	Impact case studies
Enterprise education	9	2
Entrepreneurship education	5	1
Entrepreneurial education	1	1
Business start-up	4	1
Entrepreneurship support	0	0

The development of a research impact case study starts with the collection of research outputs for the REF submission. From these the HEI selects a paper, or collection of papers, submitted to journals rated 2* or above, from which it will shape a case study that highlights the impact of its research. Impact in this context is defined as an effect on, or change or benefit to the economy, society, culture, public policy or services, health, the environment, or quality of life, beyond academia (UKRI Website).

This means that the ratio of research outputs to impact case studies in a subject theme can be used as a proxy to indicate the level of impact a research area may have achieved. Using the REF web-based database (REF3b) the authors compared research outputs to impact case studies within the Business and Management (C19) unit of assessment. This analysis grouped the research into seven subject categories, and from these it became apparent that Management had three, Economics had two case studies (C18 is Economics and econometrics), Marketing had five, Finance had five, Accounting had nine case studies, and finally Strategy had seven. This suggests that only a small number of outputs are impactful enough to become case studies. To further explore this idea, and relate it directly to EEE research, we examined these ratios of outputs to case studies for the major subjects in the C19 unit of assessment (Table 2). This showed that, when EEE is compared to other subjects, it has a much greater ratio of outputs to impact case studies than other subject categories within the C19 unit of assessment.

Table 2.

Business and Management (C19) research outputs and case studies analysis for major business subject themes.

	Research outputs	Case studies	Ratio
C19 Management	2729	3	0.001
C19 Economics	923	2	0.002
C19 Marketing	829	5	0.006
C19 Finance	727	5	0.007
C19 Accounting	865	9	0.010
C19 Strategy	193	7	0.036
C19 Entrepreneurship (includes EEE)	238	11	0.046
EEE Enterprise and Entrepreneurship Education (across all units of assessment)	19	4	0.211

Finally, although one cannot assess the quality of a solitary case study with the information publicly available, it is possible to infer quality and impact using the star rating awarded for the institution in this unit of assessment. To do this the rating for each of the four institutions that submitted EEE case studies was explored. The overall impact for two of them to the C19 unit of assessment was rated at least 3* (very considerable or outstanding), with a further institution’s impact rated at least 2* (considerable, very considerable, or outstanding). An overall impact rating is not provided for the fourth institution submitting an EEE-related case study as only one case study was submitted to the unit of assessment and REF 2014 does not provide a rating in this situation. Overall, the high level of impact ratings suggests that, although not commonly submitted to assessment, EEE research is impactful, relevant and can contribute well to the REF-related funding that institutions receive as a result of the exercise.

Teaching excellence framework review

The initial benchmarking review, presented in Table 3, demonstrated that 39% of all HEI submissions contained at least one of the EEE-related search terms used: overall, 53% of all Gold awards, 43% of Silver and 20% of Bronze. This suggests a direct, and previously unarticulated relationship between the use of these terms and the award made by the panel.

Table 3.

Percentage of university submissions using enterprise-related words for each award.

Award	Submissions with EEE-related terms	% of total submissions	% total of awards with EEE-related words	No EEE words	% of total Submissions (N = 288)
Gold	41	14	53	36	13
Silver	58	20	43	78	27
Bronze	12	4	20	49	17
Provisional	0	0	0	14	5
Total	111	39		177	61

A review of a sample of submissions suggested that the use of these words ranged from the extremely broad and general (e.g., a mention of being an entrepreneurial university) through to detailed descriptions of particular interventions or activities.

To better understand whether the relationship between the award and the terms was based on the frequency of their usage or on the quality of their application, we looked again at the number of mentions used in each award classification across the dataset (Table 4). This seemed to indicate that the frequency of mentions did not have an explicit effect on the award given, which in turn suggested that something more qualitative in the submissions had to have a bearing on the final award.

Table 4.

Number of mentions per award.

	Enterprise	Entrepreneurship	Entrepreneur(s)	Entrepreneurial
Gold	198	56	28	46
Silver	270	92	49	48
Bronze	44	16	16	18
Provisional	0	0	0	0
Grand total	512	193	93	112

This prompted us to review the submissions, which were analysed to explore how many times each term was used (Table 5). This analysis confirmed that in any single submission the use of EEE terms did, generally, correlate to a higher level of award. We surmise that the reason for this is the differing contexts in which the term enterprise is used in the submissions beyond EEE to describe a range of services and centres.

Table 5.

Number of mentions in individual submissions.

	Enterprise	Entrepreneurship	Entrepreneur(s)	Entrepreneurial
Gold	2.81	0.73	0.60	4.14
Silver	2.08	0.68	0.35	3.25
Bronze	0.71	0.26	0.30	1.54
Provisional	0	0	0	0

The second set of documents which are available with the TEF are the results narratives. These were found to use fewer enterprise-related words than university submissions (see Table 6) with ‘enterprise’ appearing more often than ‘entrepreneurship’ at higher levels of award.

Table 6.

Number of results narratives using entrepreneurial-related words.

	Enterprise	Entrepreneurship	Entrepreneurial
Gold	6	3	0
Silver	3	3	1
Bronze	1	1	0
Total	9	7	1

Initial analysis suggests that the use of the words in results narratives reflects the detail and ‘embeddedness’ (Beech, 2017) of the activity described in the university submissions. Mention of enterprise-related activity appears to be made by reviewers only where there is robust evidence that this is a sustained and strategic part of the university’s educational activity. This provides indicative evidence for the hypothesis that active engagement and explicit articulation of enterprise-related work have an impact in terms of the level of TEF award gained.

Knowledge excellence framework review

One of the simplest ways to measure the impact of EEE in the UK, including extra-curricular start-up support provided by HEIs, is to explore business start-up activity reported through the HE-BCI survey. In the period 2014–18 (following on from the analysis by Smith (2015), who explored 2008–2013) there were 16,202 new student and graduate start-ups, with a total estimate of 91,227 FTE employees, total estimated external investment of £795 million, and total estimated turnover of over £2.5 billion. In total, 21,886 businesses were reported as surviving for at least 3 years over the period (Table 7). It can be seen from Table 7 that the number of new-starts per year was relatively constant for the 4 years presented, at around 4050 businesses each year. This equates to a mean average of 38.3 new starts per returning institution in 2017/2018 (108 HEIs returned data for this item). Active companies reportedly employed an estimated average of 1.9 FTE employees and had an average estimated turnover of £53,506.

Table 7.

HE-BCI graduate spin-out returns and student FTE from 2014/2015 to 2017/2018.

	2014/2015	2015/2016	2016/2017	2017/2018	Total
New starts (N)	4124	3905	4149	4,024	16,202
3-year survival (N)	4474	5421	5810	6,181	21,886
Active (N)	10,978	11,397	12,367	13,314	48,056
Employment (EFTE)	20,886	22,892	22,983	24,466	91,227
Turnover (£000)	574,082	616,732	629,790	750,752	2,571,282
Investment (£000)	302,791	146,252	182,288	164,027	795,358
Student FTE	1,854,610	1,886,855	1,933,875	1,974,575	7,649,915

Figure 1 presents data relating to the numbers of new-start businesses reported by each of the 108 institutions in 2017–18, ranked from the highest number of returns to the lowest. This shows that a relatively small number of institutions were responsible for the majority of the returns, with 10 institutions reporting between 100 and 250 new-starts each, nine institutions returning between 50 and 100 new starts, and the remaining 86 institutions reporting fewer than 50 new starts each. Three of the nine highest ranking institutions were specialist art, music, dance or drama institutions.

Figure 1.

Number of start-ups reported in 2017/2018 by 108 institutions, ranked from the highest number of returns to the lowest.

Specialist universities typically have smaller student numbers and it is therefore important to understand the new start-ups per number of students in each institution (Figure 2), as used by the KEF. Figure 2 shows that the highest-ranking institution returned a number of new starts to HE-BCI that equated to 12.7% of its total student FTE population in 2017/2018. Only six institutions (including the highest ranking) had a KEF proxy figure of over 2% of their total student FTE; 85 institutions reported less than 0.5%. Six of the 10 highest ranking institutions were specialist art, music, dance, or drama institutions, with the KEF proxy used here compared with three using the raw HE-BCI new start numbers.

Figure 2.

Percentage number of new starts by total FTE reported in 2017/2018 by 108 institutions, ranked from the highest percentage of returns to the lowest.

HEIs in the UK have developed a series of groupings, including The Russell Group, The University Alliance, The One Million Group and Guild HE. This allows the groups to represent the particular interests of their member HEIs to Government to inform policy development. The authors have utilised these mission groups (Filippakou and Tapper, 2015) rather than the KEF clusters as the groups are more widely understood and thus this approach presents a more relatable analysis. Therefore, student and graduate start-up data are analysed in Table 8 by mission group.

Table 8.

Data for 2017/2018 based on university groupings, normalized using per one thousand students.

	New starts (N)	3-year survival (N)	Active (N)	Employment (FTE)	Turnover (£000)	Investment (£000)
Sector average	2.11	2.86	6.28	11.93	336.34	104.03
Russell Group	4.01	6.14	13.10	38.70	1423.54	1165.77
University Alliance	9.23	14.80	32.58	75.45	1557.33	184.84
Million Plus	12.99	21.37	45.34	70.55	1376.98	70.68
Guild HE	19.72	10.07	34.67	31.56	548.12	16.76

Note: Some universities are aligned with more than one mission group and for the purpose of analysis they are included in each group in which they are listed.

The Russell Group (2021), consisting of 24 UK research-intensive universities, has the lowest 3-year survival and active businesses. These universities are fundamental to the knowledge economy and therefore the business start-ups from these universities should be high-growth knowledge-based businesses.

The University Alliance (2021) consists of predominantly professional and technical universities. Its mission is to drive local growth through research, teaching and enterprising activity. This group of 12 institutions does well for employment and turnover with around 15% less of the investment that Russell Group students are provided with.

The Million Plus Group (2021) represents the 23 ‘new’ or ‘modern’ institutions across the UK established after 1992. Many of its members are former colleges or polytechnics. Members of this group achieve good results on 3-year survival and active businesses, with 6% less of the investment that Russell Group students are using.

The Guild HE (2021) group consists of the smaller specialist institutions based on art, design, teacher training, agriculture, music and drama. This group has high start-up rates and fairly high numbers of active businesses. That said, it has the lowest figures for employment, turnover and investment, which may be reflective of the applied nature of the businesses that graduates create.

These university groupings based on interest groups allow us to explore the narrative within the UK HEI Excellence Framework data. It is also clear that these groups could be used to address policy issues, but a discussion of this potential are beyond the scope of this paper.

Discussion

Our analysis builds on over a decade of conversation which has explored the challenges of measuring impacts in an EEE context, and called for more research attention to be given to the topic to support its development. Having reviewed this work, we believe that the lack of progress in advancing the impact agenda is the result of three interlinked factors:

• the fluid perceptions of enterprise, entrepreneurship and employability as topics;

• the challenge of arriving at a consensus on which outcomes and impacts to measure; and

• the difficulty in identifying when to measure impacts.

Firstly, although there is broad agreement as to the nature of EEE within the domain community, beyond these boundaries the various edges of the disciplines can appear blurry. Furthermore, the range of practices applied in EEE and the various manners in which these are discussed can make EEE even less distinctive. We feel that the broad nature of EEE means that it can be conflated with other institutional agendas, such as employability, making its particular impact on learning and development harder to unpick, and therefore more difficult to measure. Secondly, the intention of EEE practice is to affect change in the individual or group. However, EEE interventions can take many forms, with the goal of effecting a variety of competencies, attitudes and behaviours. This means that the potential impacts that could be measured are diverse in scope and scale. Educators and practitioners, often with broad EEE briefs, may lack the luxury of setting overly constrained boundaries in an effort to promote inclusivity, encourage engagement, achieve institutional targets and further their own activities. We believe that this situation leads to a scenario in which impact measures are highly dependent on cohort size, and the scope of the intervention; smaller cohorts can be assessed in more depth, while larger groups tend to be explored using more binary metrics.

It is also worth considering whether this focus with regard to EEE affecting change in people has created an unintended bias in the assessment of EEE impacts. Across all the reviews we explored we found no evidence that EEE impacts on teaching, learning, assessment or research had been reviewed in any meaningful detail.

Finally, EEE might affect a student today, tomorrow or several years later and might do so in a variety of ways. This means that deciding on a timescale over which to measure any impact is inherently difficult. Furthermore, linking any impact, especially over longer periods, to any particular intervention(s), while excluding for other external stimuli is challenging. We conclude that this is why there are few national longitudinal studies in any reviews, which is frequently noted as a weakness in the understanding of EEE impacts.

Our work takes a different approach to addressing the issue of EEE impact measures. We have chosen to focus on the three UK excellence frameworks (REF, KEF and TEF) to explore what these previously unreviewed datasets can tell us.

The UK has a unique higher education ecosystem insofar as education policy is, on the one hand, devolved to a national level in terms of structure, regulation and funding (e.g., the Office for Students in England), whilst on the other hand remaining centralised in the context of data collection and assessment (e.g., the Higher Education Statistics Agency). This has led to different approaches to the way data is collected for the REF, KEF and TEF, with some nations and their HEIs not participating fully, especially if the data are not tied directly to funding (e.g., the TEF and Scottish HEIs). However, these datasets still provide one of the most complete national pictures, allowing for a holistic view of higher education performance, which is why it is so odd that they have been largely ignored (with the exception of some critiques of their management and execution) as tools from which useful insights can be gained.

UK HEIs are also quite tribal: as we can see from the data, mission groups are important to understanding the UK higher education landscape, especially with regard to self-employment and venturing activity. The fact that a relatively small number of HEIs are responsible for the majority of graduate start-ups speaks to the features of the universities in these groups. Some with higher levels of activity may be more specialised, with programmes that are highly focused on self-employment or venture creation based on the careers they are preparing students to engage with. Others may be less research-focused, more vocational, or may attract students for whom self-employment is either their only option or part of a portfolio of post-education activity. It may also be that, for some, reporting on EEE activity is simply less important than other metrics. It is certainly a compelling area for further study.

This study is an exploration of the manner in which REF, TEF and KEF interact and what we, as researchers, can learn from this interaction. In an EEE context we know that EEE activity impacts an institution’s TEF score, and we also know that research work focusing on EEE is, arguably, more impactful than other disciplines. We know too that EEE interventions have a direct impact on the KEF through the creation of graduate enterprises.

Although there are no direct linkages between the datasets, there are several important inferences that we can draw. Firstly, the REF and TEF data certainly suggest that impactful research activity likely informs impactful teaching; although they also indicate that the opposite may also be true (i.e., quality teaching may inform quality research). There is certainly scope to research this in more depth. Secondly, it is clear from our review of the TEF narratives that EEE activity that supports the KEF likely also leads to improved TEF outcomes. It is impossible to say whether the reverse is true, although again there is scope here for further research to explore whether excellent teaching learning impacts the KEF. Finally, we know that there is a relationship between the KEF and the REF in so far as KEF activity may lead to research that can form part of the REF submission. What is less apparent is the manner in which REF activity might affect KEF outcomes (research leading to spin-outs, for example) and this may be worth studying in more depth, especially as the KEF will likely be linked to institutional funding in the future.

The literature review conducted for this paper demonstrates that these publicly available data are under-utilised within the higher education research community, especially in relation to the development of education management and UK/national higher education policy. Also, as the REF, KEF and TEF cycles repeat, there will be a longitudinal dataset which can be used to understand the impact of university-level management on the local population, regional GDP and business community.

In the context of EEE, this work could be expanded to create a national benchmark to explore the development of the entrepreneurial university in practice. This would provide an open and transparent methodology for comparing universities in the UK in terms of their EEE activity and its impact. The Excellence Frameworks could also be used to investigate the clustering of universities with a view to creating improved strategic alliances in, or between, mission groups to support regional development.

Conclusions

This paper has reviewed the under-utilised data captured by the UK’s Excellence Framework assessments of research, teaching and knowledge exchange in an effort to ascertain what they might help us to understand about the impact of EEE in UK HEIs.

The REF 2014 results show that EEE may be up to 46 times more impactful than other management disciplines. This implies that, given the link between REF impact and institutional income, more HEIs should focus efforts on developing EEE research activities. An updated review of this conclusion using data drawn from REF 2021 would be an interesting area for research exploration.

The positive relationship between the use of the EEE terms in TEF submissions and the award level achieved by the HEI suggests that EEE activity may play a role in improving the TEF rating of the institution. This analysis should be conducted again, once HEIs have submitted their second application, to understand the strategic differences in these TEF submissions and if the trends identified herein are repeated.

The interpretation of the HE-BCI data as a KEF proxy is more complex because of the range of outputs it addresses. Firstly, we observe that the rate of graduate start-ups has remained relatively flat over the 4-year academic period which was the focus of this analysis. This suggests that any positive or negative changes in EEE activity in HEIs (measurement of which was beyond the scope of this analysis) appear to have had a limited effect on start-ups. Secondly, the analysis shows that a relatively small number of HEIs are responsible for the majority of graduate start-ups. Identification of the reasons behind this requires more research focus to facilitate the sharing of best practice across the sector. Furthermore, the data imply that many of these institutions are small specialist universities, meaning that a review of these institutions is necessary to better understand their function and contribution to the UK economy. Finally, there appears to be a link between membership of an HEI mission group and improved KEF metrics when compared to the sector averages. Further work is also needed to better understand this relationship and the factors underpinning these data.

Taken together, this analysis suggests, powerfully, that EEE impacts research, teaching and knowledge exchange quality in a variety of ways. However, it also shows that the reasons for this are still poorly understood at a micro-level, and that further research activity is necessary to develop a more nuanced understanding of best practice which could lead to improvements across the sector in the UK and beyond.

The combination of these three datasets as presented in this paper offers a unique insight into EEE activity in the UK and its impacts. This has never before been explored in the literature. We offer it here in the hope that it prompts new discussion as to how impact measures can be conceptualised, especially with regard to these national framework datasets. This might help to address some of the critiques noted in the literature review.

In addition to the points above, the connections between REF, TEF and KEF submissions should be further analysed to evaluate the relationships for EEE between research, teaching and knowledge exchange, with a specific focus on how policies need to be shaped to support and develop EEE work.

Limitations

The authors recognise the limitations of this research. Firstly the data are those that HEIs themselves have submitted with no internal or external auditing. Secondly proxies have been used for the KEF data, which means that a fuller picture may not have been realised. And finally, the datasets presented herein contain different numbers of HEIs, and in this current format normalisation is not possible, meaning that comparisons are not like for like.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Enterprise Educators UK (EEUK).

ORCID iDs

David Bozward

Matthew Rogers-Draycott

Vic Curtis

Note

References

Anderson

Culkin

Penaluna

, et al. (2014) An Education System Fit for an Entrepreneur: Fifth Report by the All-Party Parliamentary Group for Micro Businesses.

Arnold

(2018) Review of the research excellence framework: evidence report. Available at: http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-250280 (accessed 16 September 2021).

All-Party Parliamentary Group for Entrepreneurship (APPG) (2018) All-Party Parliamentary Group for Entrepreneurship – Enterprise Education Report.

Beech

(2017) Going for Gold: Lessons from the TEF provider submissions. London: Higher Education Policy Institute.

Bozward

Rogers-Draycott

(2020) Value Creation Programmes: Lessons from an Early Stage Implementation. Entrepreneurship Education 3(3): 287–310.

Byrne

Fayolle

Toutain

(2014) Entrepreneurship education: What we know and what we need to know. In: Chell

Karataş-Özkan (eds). Handbook of Research on Small Business and Entrepreneurship. Cheltenham: Edward Elgar Publishing, 261–288.

Curtis

Moon

Penaluna

(2020) Active entrepreneurship education and the impact on approaches to learning: Mixed methods evidence from a six-year study into one entrepreneurship educator’s classroom. Industry and Higher Education 25(4): 443–453.

Department for Education (DfE) (2017) Higher Education and Research Act 2017. Department for Education. Available at: https://www.legislation.gov.uk/ukpga/2017/29/contents/enacted.

Epper

(1999) Applying benchmarking to higher education: some lessons from experience. Change. The Magazine of Higher Learning 31(6): 24–31.

10.

Filippakou

Tapper

(2015) Mission Groups and the New Politics of British Higher Education. Higher Education Quarterly 69(2): 121–137.

11.

Gillard

(2018) An initial analysis and reflection of the metrics used in the Teaching Excellence Framework in the UK. Perspectives: Policy and Practice in Higher Education 22(2): 49–57.

12.

Global Entrepreneurship Monitor (GEM) (2021). https://www.gemconsortium.org/ (accessed 16 September 2021).

13.

Global University Entrepreneurial Spirit Students’ Survey (2021) GUESSS). Available at: https://www.guesssurvey.org/ (accessed 16 September 2021).

14.

Gray

Adderley

Jordan

(2020) Revisiting the New Entrepreneurial University in Times of Uncertainty. In: Adesola

Datta

(eds) Entrepreneurial Universities. Cham: Palgrave Macmillan, pp. 143–167.

15.

Guild HE (2021). Available at: https://guildhe.ac.uk/ (accessed 16 September 2021.

16.

Hannon

(2007) Enterprise for all? The fragility of enterprise provision across England's HEIs. Journal of Small Business and Enterprise Development 14(2): 183‐210.

17.

Higher Education Statistics Agency (HESA) (2020a) - refers to data obtained from https://www.hesa.ac.uk/data-and-analysis/business-community/ip-and-startups

18.

Higher Education Statistics Agency (HESA) (2020b) - Refers to Data Obtained Student FTEs for 2018. https://www.hesa.ac.uk/data-and-analysis/students/whos-in-he#numbers (accessed 16 September 2021.

19.

Higher Education Statistics Agency (HESA) (2020c). Available at: https://www.hesa.ac.uk/support/definitions/hebci (accessed 16 September 2021.

20.

Jackson

(2001) Benchmarking in UK HE: an overview. Quality Assurance in Education 9(4): 218–235.

21.

Johnson

(2022) The knowledge exchange framework: understanding parameters and the capacity for transformative engagement. Studies in Higher Education 47(1): 194–211.

22.

Jones

Pickernell

Fisher

, et al. (2017) A tale of two universities: graduates perceived value of entrepreneurship education. Education+ Training 59(7/8): 689–705.

23.

Kamovich

Foss

(2017) In search of alignment: A review of impact studies in entrepreneurship education. Education Research International.

24.

Kitagawa

Webber

Plumridge

, et al. (2015) University entrepreneurship education experiences: enhancing the entrepreneurial ecosystems in a UK city-region. In: The 8th International Conference for Entrepreneurship, Innovation and Regional Development, Sheffield, UK, 18–19 June 2015, pp. 794–821.

25.

Lahikainen

Pihkala

Ruskovaara

(2018) European approaches to enterprise education. In: Lahikainen

Pihkala

Ruskovaara

(eds). International Enterprise Education. Milton Park, Abingdon, Oxon: Routledge, 1–22.

26.

Matlay

Carey

(2007) Entrepreneurship education in the UK: a longitudinal perspective. Journal of Small Business and Enterprise Development 14(2): 252–263.

27.

McKellar

(2020) Building a culture of innovation and entrepreneurship in Universities. In: Badran

Baydoun

Hillman

(eds) Higher Education in the Arab World. Cham: Springer, pp. 95–107.

28.

Million Plus (2021). Available at: https://www.millionplus.ac.uk/ (accessed 16 September 2021.

29.

Morley

Jamil

(2021) Introduction: Real World Learning-Recalibrating the Higher Education Response Towards Application to Lifelong Learning and Diverse Career Paths. In: Morley

Jamil

(eds) Applied Pedagogies for Higher Education. Cham: Palgrave Macmillan, pp. 1–17.

30.

Mulholland

Turner

(2019) Enterprise education in the twenty-first century. In: Mulholland

Turner

(eds) Enterprising Education in UK Higher Education. New York, NY: Routledge, pp. 1–16.

31.

Nabi

Liñán

Fayolle

, et al. (2017) The impact of entrepreneurship education in higher education: a systematic review and research agenda. Academy of Management Learning and Education 16(2): 277–299.

32.

National Association of College and University Entrepreneurs (NACUE) (2010) The State of Student Enterprise. London: NACUE.

33.

National Association of College and University Entrepreneurs (NACUE) (2014) Student Enterprise Survey Framework Survey. London: NACUE.

34.

National Association of University and College Entrepreneurs (NACUE) (2021). Available at: www.nacue.com/student-enterprise-survey.html (accessed 16 September 2021).

35.

Nazarko

Anna Kuźmicz

Szubzda-Prutis

, et al. (2009) The general concept of benchmarking and its application in higher education in Europe. Higher Education in Europe 34(3–4): 497–510.

36.

Office of Students (2020) About the TEF, Panel members and assessors. Available at: https://www.officeforstudents.org.uk/advice-and-guidance/teaching/about-the-tef/panel-members-and-assessors/ (accessed 16 September 2021).

37.

Pittaway

Cope

(2007) Entrepreneurship education: A systematic review of the evidence. International Small Business Journal 25(5): 479–510.

38.

Pojanapunya

Watson Todd

(2021) The influence of the benchmark corpus on keyword analysis. Register Studies 3(1): 88–114.

39.

Quality Assurance Agency (QAA) (2012) Enterprise and Entrepreneurship Education. Guidance for UK Higher Education Providers. Gloucester: QAA.

40.

Quality Assurance Agency (QAA) (2018) Enterprise and Entrepreneurship Education. Guidance for UK Higher Education Providers. Gloucester: QAA.

41.

Rae

Martin

Antcliff

, et al. (2012) Enterprise and entrepreneurship in English higher education: 2010 and beyond. Journal of Small Business and Enterprise Development 19(3): 380–401.

42.

Research England (2020) Knowledge Exchange Framework Decisions. https://re.ukri.org/documents/2019/knowledge-exchange-framework-decisions-16-jan-2020/ (accessed 16 September 2021.

43.

Research England (2021) Explore KEF Dashboards. https://kef.ac.uk/dashboard (accessed 16 September 2021).

44.

Research Excellence Framework (REF) (2014). https://results.ref.ac.uk/ (accessed 16 September 2021).

45.

Research Excellence Framework (REF) (2021). https://www.ref.ac.uk/2014/ (accessed 16 September 2021).

46.

Rideout

Gray

(2013) Does entrepreneurship education really work? A review and methodological critique of the empirical literature on the effects of university-based entrepreneurship education. Journal of Small Business Management 51(3): 329–351.

47.

Russell Group (2021). https://russellgroup.ac.uk/ (accessed 16 September 2021).

48.

Sánchez

(2013) The impact of an entrepreneurship education program on entrepreneurial competencies and intention. Journal of Small Business Management 51(3): 447–465.

49.

Saridakis

Iskandarova

Blackburn

(2016) Student Entrepreneurship in Great Britain: Intentions and Activities. London: Kingston University.

50.

Smith

(2015) Measuring the impact of enterprise education and entrepreneurship support in higher education: can routinely collected data be of use? Industry and Higher Education 29(6): 493–503.

51.

Solomon

(2007) An examination of entrepreneurship education in the United States. Journal of Small Business and Enterprise Development 14(2): 168–182.

52.

Tasopoulou

Tsiotras

(2017) Benchmarking towards excellence in higher education. Benchmarking: An International Journal 24(3): 617–634.

53.

Topazly

(2018) A Qualitative Exploration of the Impact of Postgraduate Enterprise and Entrepreneurship Education in Modern UK Universities on Russian Graduates. PhD Thesis. London, UK: University of West.

54.

UK Research and Innovation (UKRI) (2020). Available at: https://www.ukri.org/ (accessed 16 September 2021).

55.

UK Research and

Innovation (UKRI) (2021) Available at: https://www.ukri.org/about-us/research-england/research-excellence/research-excellence-framework/ (accessed 16 September 2021).

56.

University Alliance (2021). Available at: https://www.unialliance.ac.uk/ (accessed 16 September 2021).

57.

Vivian

James

Salamons

, et al. (2019) Evaluation of Provider-Level TEF 2016-17 (Year 2): Measuring the Initial Impact of the TEF on the Higher Education Landscape. UK, January: Report, IFF Research.

58.

Williams

(2011) Impact of business simulation games in enterprise education. In: Paper Presentations of the 2010 University of Huddersfield Annual Learning and Teaching Conference. Huddersfield, UK: University of Huddersfield, 11–20.

59.

Williamson

Beadle

Charalambous

(2013) Enterprise Education Impact in Higher Education and Further Education. Department for Business Innovation and Skills. Final report. London: DBIS.