Abstract
This paper demonstrates how the innovative application of a Collective Intelligence approach enhanced Local Skills Improvement Planning information for employers, education and skills training organisations and regional economic policy organisations. This took place within a Knowledge Transfer Partnership between a Chamber of Commerce and a University. This aimed to develop and deploy regional business intelligence for enhanced policy and decision-making in enterprise and economic development. The project converged knowledge from several research centres including economics, entrepreneurship and innovation, data science, and Artificial Intelligence. The paper presents a project case study which provides two contributions to applied knowledge. Firstly, it demonstrates how a Collective Intelligence (CI) approach can be applied to achieve rapid results in resolving the real-world problem of local skills information availability. Useful real-time data was gathered from employers in three sectors on skills requirements, supply and training. This was analysed using Artificial Intelligence tools, then shared publicly via an automated Internet portal, providing a scalable model for wider use. Secondly, it explores and evaluates how the knowledge exchange (KE) process can function effectively and quickly in applying CI-based innovation in practical ways which create new value, within a Knowledge Transfer Partnership between a University and Chamber of Commerce.environment.
Keywords
Introduction
This paper explains how a Knowledge Transfer Project (KTP) between a University and a regional Chamber of Commerce developed and introduced an innovative approach using Collective Intelligence. This addressed a longstanding gap in sharing knowledge through engagement between businesses as employers, and vocational education and training (VET) providers in skills supply at sub-regional and local levels, alongside funders, professional bodies and advisers as stakeholders.
The paper’s contributions demonstrate firstly how a Collective Intelligence (CI) approach was applied to achieve rapid results in resolving the real-world problem of information availability on skills at local level. In this case useful real-time data was gathered from employers in three sectors on skills requirements, training and supply. This was shared publicly via an automated Internet portal, which provides a scalable model.
Secondly, it explores and evaluates how the knowledge exchange (KE) process can function effectively in applying CI-based innovation in a practical way which creates new value. This was achieved within a time-constrained Knowledge Transfer Partnership between a University and Chamber of Commerce. Environment.
The foreground knowledge areas of Collective Intelligence and use of open data sources; of skills demand and supply and VET; of data science, and Artificial Intelligence (AI); and of Knowledge Exchange and Transfer (KET), were not new, but their integration was. These domains were converged in the innovation process of developing a value-adding new solution for public policy and business requirements to inform regional skills supply and demand. Such innovation can be managed rapidly to develop new solutions by connecting different actors and skillsets within the scope of a University-business network KTP project. The paper creates new understanding of the process of KE and project-based innovation, and provides insights into how this can be applied.
The project included a multidisciplinary approach to organising and managing the skillsets and actors, which used AI tools to allow the scaling up of the gap analysis at regional level, and was readily expandable to national level. The paper uses a Collective Intelligence approach (Mulgan, 2018a) both in its focus on generating innovative solutions from combined activities, and also in the underpinning research into the Knowledge Exchange and Transfer process. Within this, the roles of the partners, and especially the KTP Graduate Associate as ‘active agent’, are important. Hence, the paper is co-authored by the partners in the KTP and consists of the following structure.
The section on summary of prior knowldege provides a summary of the main prior knowledge areas relevant to the case. These include: a. Local skills planning and skills supply in vocational education and training; b. Application of open and economics-based statistical data; c. Development of Collective Intelligence in economic development; d. Application of new data analysis and Artificial Intelligence methods for public policy.
The following sections address the Knowledge Exchange methodology used in the KTP project case, provide a detailed overview of the Local Skills Improvement Plan (LSIP), and the approach to developing the Collective Intelligence Skills Observatory (CISO) in which the knowledge was applied, evaluating its contributions. The final sections offer observations and conclusions on the innovation process which extend beyond the specific and immediate context.
Summary of the main prior knowledge areas
Local skills planning and skills supply in vocational education and training to frame ‘the problem’
The limited availability of skills appropriate for local economies, in this case Leicester and Leicestershire, has been well documented by both the Local Enterprise Partnership for Leicester and Leicestershire (LLEP) and local business representation organisations (LLEP 2022). In a recent survey of businesses, East Midlands Chamber, the Chamber of Commerce for the region, identified that of the two-thirds of businesses that attempted to recruit, 82% struggled to fill vacancies (East Midlands Chamber 2022). When asked at what level difficulties were being experienced, respondents reported this existed equally across skilled, professional, manual and clerical roles. Whilst not unique to Leicester and Leicestershire, certain businesses in the area are believed to be more acutely affected by factors such as the departure of European Union citizens who left the labour market and have not been replaced, along with the high number of skilled workers over 50 who have become economically inactive (LLEP 2022). These include businesses in the traditionally strong local manufacturing and logistic sectors, along with rural businesses where access to work is complicated by poorer transport connections.
Vocational education and training seek to respond to these needs, but their effectiveness have been hindered by a range of factors. These include the lack of a consistent taxonomy when it comes to businesses discussing their people needs and the offerings of vocational curricula, and poor communications between the private sector and educators (East Midlands Chamber 2022). Businesses find forward planning for their future human resources difficult. This is particularly the case when looking at smaller firms, which has a disproportionate impact in Leicester and Leicestershire, where 90% of businesses employ fewer than 10 people. These factors make it difficult to both articulate the ‘demands’ of businesses and then to ensure the understanding of this by the right people who can deliver change in the ‘supply’ of vocational education and training (VET) courses. A further difficulty arises with regards to publicly funded vocation education, where in the past there has been a disconnect between national funding priorities and local needs. This disconnect is in part behind the Government’s Skills and Post-16 Education Act 2022. This introduced legislation to create employer-led Local Skills Improvement Plans which included a requirement for Further Education Colleges to respond to the recommendations in those plans (HM Govt 2022).
The importance of ensuring that the people requirements of local businesses are matched by the supply of skills provided by educators is illustrated by the impact of not getting this right. Having a workforce with the relevant skills levels supports competitiveness and gains in productivity, with access to the relevant skills is also often cited by business representation organisations as being one of the largest determinant factors of inward-investment decisions. Conversely, not having the relevant skills levels not only impacts business productivity and growth, but also limits earnings potential and spending power within the local economy.
Recent NOMIS data for current levels of educational attainment in Leicester and Leicestershire (Jan-Dec 2021) shows 38% of the population being qualified at NVQ Level 4 and above. This compares to 43.6% for Great Britain (GB) as a whole. 59.7% are qualified at NVQ Level 3 and above (compared to 61.5% for GB) and 77.7% are qualified at Level 2 and above (compared to 78.1% for GB). Conversely, 8.1% have no qualifications, compared to 6.6% for Great Britain.
It is likely that these lower qualification figures translate into lower average earnings for people living in the city and county when compared to the rest of the country. The same data shows that in 2022 a full-time worker in Leicester and Leicestershire had a gross weekly pay of £607.90, £34.30 lower than the national average of £642.20. This difference is more pronounced when looking solely at female workers (£43.60) than when looking at male workers (£41.40).
Application of open and economics-based statistical data
Recent decades have seen fast-growing use of social and economic data to transform the way that economics and other social sciences operate (Einav and Levin 2014a, 2014b; Kitchin 2014). As discussed by Einav and Levin (2014a) this is largely due to the growing accumulation and accessibility of datasets as well as increased computing power to analyse those datasets. While economic analysis 40 years ago drew inferences from relatively small-scale surveys, we can now access much larger datasets, that in some cases offer universal population coverage. Such data, combined with advanced econometric and statistical techniques, including machine learning, offers the potential for more powerful research insights. Einav and Levin (2014b) give a range of examples of how the availability of data is transforming our understanding of economic policy. For instance, Choi and Varian (2012) demonstrated how Google Trends can predict economic indicators such as car sales, unemployment claims, and consumer confidence. Jun et al. (2018) provide a review of 657 papers which used Google Trends data since 2006, demonstrating the increasing use of prediction in large-scale data analysis.
While data is transforming economic and social science research, there is a challenge of data availability. To illustrate the size of the challenge, Einav and Levin (2014a) documented the proportion of papers published in the flagship journal, the American Economic Review, that asked for a journal exemption to use non-publicly available data. In 2006 only 8% of papers used non-publicly available data. By 2014 this had risen to 46% of papers, of which 26% used administrative data and 20% private data. This dramatic rise in non-publicly available data is alarming. It implies privileged access to data by a small subset of researchers, which at best limits economic insight and at worse can lead to corrupt practices. Christensen and Miguel (2018) summarise the case for a different approach to economic research. One strand of this approach is to encourage open data.
Open data can be defined as data that is non-confidential and non-privacy restricted and made publicly available for wider use with no (or very limited) restrictions on use. Open data offers exciting potential to improve economic and social science research and policy through its greater availability. Given these potential benefits an increasing number of organisations, including national and local government, are making their datasets openly available. As an example, Leicester City Council launched an Open Leicester web portal that provides datasets on a range of topics, such as planning, transport and public safety. This data can be used to inform our understanding of the local economy. For example, the study by Ferm et al. (2021) uses data from five local authorities, including Leicester, to assess the impact of deregulation of planning control. The data used for this paper was obtained by asking local planners. Comprehensive open data would, hopefully, facilitate such research by reducing the costs of data collection.
Janssen et al. (2012) summarize the many benefits of open data, including transparency, accountability, improved decision making, ‘wisdom of the crowd’, and re-use of data. Advantageous though it can be, there are several non-trivial challenges in effective open data. These challenges include concern over privacy and confidentiality (Christensen and Miguel 2018) as well as the quality and usability of data (Einav and Levin 2014b). In terms of quality and usability the key point to keep in mind is that open data is typically not collected and published with the researcher or analyst in mind. It is instead data that has been collected for some administrative purpose and then released to the public. Such data may miss key questions of interest to the researcher. It may also be difficult to combine and analyse data sets published in widely differing formats. Such challenges mean that there are substantive barriers in the use of open data. In short, the existence of open data is no guarantee it will be used. Moreover, organisations may be reluctant to release data that could be of most use. These barriers and more are also summarized by Janssen et al. (2012).
To date it seems clear that the potential for open data has yet to be fully harnessed. This is not only about the availability, but also the quality of data. The most effective form of data collection and analysis involves a two-way process in which the researcher can help shape the data collection (Einav and Levin 2014a). Researcher involvement in data collection maximizes the chances the data will have the desired quality and content to allow powerful research analysis and insight. Such involvement, however, imposes costs on all parties involved. In particular, it involves researcher time to work with the organisation and it means the organisation may need to implement new data collection processes. Having incurred such costs, the researcher and organisation may be reluctant to make the data freely available.
Atapour-Abarghouei et al. (2020) discussed, in the context of cyber-security, this data-sharing paradox. They argue that, because all parties have differing incentives, it is difficult to obtain effective data sharing. The solution ultimately needs to come from all parties ‘co-owning’ the problem. This logic applies not only to the cyber security challenge but other pressing social and economic problems, such as addressing skill needs in a local economy. Effective open data thus requires aligning incentives and enabling an eco-system in which all parties can see the benefits of open data collection and sharing, pointing towards the value of collective intelligence. This is a basic design feature of the Regional Business Intelligence Unit.
Using collective intelligence in economic development
Atlee and Por (2007) defined Collective Intelligence (CI) to include (but not restricted to) human intelligence, as it also relates to the intelligence of ecosystems. Socially, they described it as “the capacity of human communities to evolve towards higher-order complexity and harmony, through such innovation mechanisms as differentiation and integration, competition and collaboration”. This points towards forms of CI which may occur in entrepreneurial and innovative activities.
CI increasingly finds points of convergence with Artificial Intelligence (AI), aiming to inform and supplement human judgement rather than replacing it (Shadbolt and Hampson, 2018). Mignenan (2021) hypothesised CI as the connections between Human, Relational and Intellectual capital. Human collective and Artificial intelligence should be seen as complementary, but the interfaces between them are complex and involve both high-level skills and technology applications, which increasingly need to be shared to inform business practices and decision-making. This process is demonstrated in the case study.
The concept of Collective Intelligence as a field of knowledge stems partly from growing interest in sociated and distributed organising, evident in research from the mid-2000s. For example, in ‘Democratizing Innovation’ (Von Hippel, 2006); by Surowiecki, in “The Wisdom of Crowds (2004); Howe (2008) on crowdsourcing; Sunstein (2006) on Infotopia; through ‘Swarm Intelligence’ (Bonabeau and Meyer, 2001); and in an influential Sloan Management Review article (Bonabeau, 2009). This interest also arose from curiosity about new forms of organising, and their relation to the growing power and access to distributed forms of computing and access to online and digital knowledge resources. There are credible prior works in multiple disciplines on (for example); social cognition (Bandura, 1986); organisational social psychology and sensemaking (Weick 1979, 1995); collective memory (Freud) and social learning (Wenger, 1998).
Mulgan (2018a) brought the topic to professional and public attention with the notion of CI as a ‘Big Mind’. This connected with works on collectivism in sociology (Tarde via Latour, 2015) on group mind and collectives; and with the sociology of networks (Granovetter, 1973); “actor network theory” (Latour, 2015) seeing humans and machines as inseparable, all playing roles in networks. CI forms a convergence between these and other streams of psychological, sociological and philosophical awareness of collective thinking; with the burgeoning fields of both digital access to knowledge resources; and the growth of ‘information commons’ and Open, freely shared resources. The ascent of Artificial Intelligence (AI) and Machine Learning as influential cyberscience movements opened possibilities of intersections between AI and CI (Shadbolt and Hampson, 2018), as well as the role of AI in education (Seldon, 2018).
The relationships between CI, cyberscience and data science have also seen CI become a subdomain of data analytics, with lesser attention paid to the human and social interactions (eg Nguyen and Nguyen, 2018; Reia et al., 2018; Suran et al., 2020). However, CI represents more than a system or a knowledge platform; the interaction with information systems increasingly enhances human and collaborative capabilities in novel ways, whilst the relationships with ‘the information commons’ and the field of Open resources are influential. Creative and Learning Commons have developed for 20 years as means of freely sharing ideas, resources and thinking (Meyer, 2020). Whilst open access to scientific data developed from the 1957 advent of World Data Centres, the Internet made scientific, governmental, organisational and private knowledge publicly available and enabled their widespread use.
Open data is an essential contribution within the range of Open resources and processes (Rae, 2020), which enables their application through, for example, Open-prefixed Government, Science, Innovation and Entrepreneurship. Advocates of Open Data have also progressed to champion CI, including Shadbolt, Mulgan and others. Mulgan (2018b) argued that the convergence of AI and CI represent a fruitful direction for integration of human, collaborative and machine intelligences.
The uncontrolled growth in the availability of data resources meant that analysis by human agency alone is increasingly unrealistic and machine-assistance via AI has become essential. However, there are growing concerns that dependence on digital determinism alone could have adverse consequences for humanity, as well as interest in how the limitations on the effectiveness of individual decision-making can be moderated by shared intelligence systems. In May 2023, the Center for AI Safety issued a statement on AI Risk, signed by many AI scientists and public figures. This called for “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war” 1 .
The CI movement was marked by the emergence of research and innovation centres, created to recognise opportunities and develop data- and intelligence-led innovations in this field. The NESTA Centre for Collective Intelligence Design enables uses of CI in social innovation. Open Innovations in Leeds (UK) developed problem-led innovation from Open Data systems. Such organisations and leading practitioners advance innovation, animating communities of collective intelligence in their fields.
Collective Intelligence methods have informed collaborations by groups of business owners to address market disadvantages (eg by Wang and Tan in Canadian telco’s, 2019) and in collective action in small-scale fisheries (Child, 2018). Rouse et al. (2021) charted collective action by micro-entrepreneurs excluded from income protection during COVID lockdown in the UK, highlighting the informational asymmetry encountered by individual entrepreneurs and small networks in confronting data-rich public organisations. There is advanced analysis on path-creating new entrants in the Artificial intelligence industry in the Montreal city-region by mapping analysis of the AI industry policy network, such as by Doloreux and Turkina (2021).
Application of new data analysis and artificial intelligence methods for public policy
Few previous studies address the application of AI methods to local labour market analysis, but interestingly two were found in the Russian context. Belov et al. (2022) demonstrated the use of methods and algorithms in an analytical platform for automated monitoring and analysis of the regional and national labour market in the Russian Federation, and linked analysis of the compliance of the higher education system’s specialist training provision in meeting current labour market needs. The study involved natural language processing methods and Big Data technologies, using semantic analysis, automated monitoring and intellectual analysis on open-source datasets. This study developed and applied the prior work of Belov et al. (2017) and collection of secondary data gathering has continued from 2015. This demonstrated analytical capability, but the input from employers was limited to analysis of job vacancy data and applicant details.
An additional challenge when analysing gaps, barriers or disparities between skills requirements and supply, is the amount of data to be considered. Although the amount of data available in this project is not quite in the realm of Big Data, it is still too large to be analysed manually whilst too unstructured and vague to be easily organised and manipulated by standard statistical/automated techniques. The project used AI, mainly Natural Language Processing (NLP), techniques to help analyse the data at large scale (regional economy, and scalable even further to national level with no extra effort) and provide a tool for quick correlation of the data.
A similar approach was reported previously by Belov et al. (2017 and 2022) but subsequently followed by few studies, concentrated on the Russian job market. In recent years the NLP models based in deep learning have become much more advanced than word2vec used by Belov. We compared the GPT-3 and BERT models, finding the latter the most efficient and reliable for analysis rather than generation of text.
Although no novel AI techniques have been developed, we applied in a novel and much more generalised way the available AI based language models than had been used previously. The results were then fed into the CI process, for use by the Insight Unlocked portal.
The knowledge exchange methodology used in the KTP project and the case
To explain the purpose of Knowledge Transfer projects, KT is defined as a dynamic and iterative process that takes place within a complex system of interactions between researchers and knowledge users, which may vary in intensity, complexity and level of engagement. (Canadian Institutes of Health Research) 2 .
KT operates within the broader field of Knowledge exchange (KE), with multi-directional, dynamic flows of knowledge between contexts, such as Higher Education and industry, creating insights together. Quantitative analysis illustrated that five broadly-defined components of knowledge exchange (problem, context, knowledge, activities, use) can all be in play at any one time and do not occur in a set order. Qualitative analysis revealed a number of distinct themes which better described the nature of knowledge exchange (Ward et al., 2012) as: problem identification and communication; analysis of context; knowledge development and selection; knowledge exchange activities/interventions; knowledge use.
The Knowledge Transfer Partnership (KTP) approach is longstanding, being funded by Innovate UK and involving projects between employers (mainly businesses) and knowledge-intensive organisations, mainly universities Abreu et al (2008). The model requires a project with a duration of between 6 and 36 months in which a strategic opportunity for business development or improvement is identified and addressed by exchange of research, knowledge or practices with the partner organisation. The means is via a graduate Associate being appointed to mediate between the organisations and to conduct a planned programme of knowledge transfer, supervised by expert personnel from each organisation.
In this case, the identified need was how to collect, analyse and use business and economic data systematically, to inform and enhance regional economic development and enterprise more effectively in the East Midlands (EM) region. The business partner was the East Midlands Chamber of Commerce (EMC), which covers the core regional counties of Nottinghamshire, Leicestershire and Derbyshire through its membership of 4500 businesses and other organisations. The EM region has a diverse economic and business base, but no single organisation is charged with economic and business development policy and implementation. This differs from the situation in regions such as the West Midlands, North-West and Leeds, where single unitary authorities have been created with the power and resources to bring together an economic development partnership. Since 2011, this has been the responsibility of Local Enterprise Partnerships (LEPs) and local authorities. The limitation in the East Midlands is that no single public organisation takes a strategic overview, and the capability to undertake data analysis to develop economic development strategies in these organisations is limited. EMC, together with academics at De Montfort University, recognised the opportunity to develop this capability and to provide it as a service through forming a Regional Business Intelligence Unit, partly funded by a KTP and by other external funding.
The KTP proposal was written and submitted during 2020, requiring significant work to show how the project would create additional revenues required to gain approval. This achieved, the supervision team was formed from the Director of Policy at EMC, Professor of Economics and Associate Professor for Artificial Intelligence at DMU, and as lead academic the Professor of Enterprise. The next stage was the recruitment of the KTP Associate which took place in late 2020. Framing this post with the combination of knowledge and skills in business intelligence, data and computing was an essential step, requiring expertise which is in growing demand. Fortunately, there was a positive response to the open selection competition, and the successful candidate was a graduate from India with a first degree in Computer Engineering, IT industry work experience and had recently completed a Masters’ degree in Business Intelligence Systems and Data Mining at De Montfort University. He was appointed, enabling the project to start on 1 April 2021.
The KTP project led to developing the LSIP Collective Intelligence Skills Observatory
The Associate initially worked on enhancing the Quarterly Economic Survey which all Chambers of Commerce are required to produce in their area, by automating processes in this and personalising it for respondents as an added value service. Contacts were developed with the Local Enterprise partnerships on the scope for enhancing business and economic intelligence in their areas. This meant that when the Department for Education introduced a ‘trailblazer’ call for proposals to enhance Local Skills Improvement Plans in Autumn 2021, the Chamber was ready to respond with an innovative proposal, based on applying a Collective Intelligence approach suggested by the project.
The Local Skills Improvement Plan (LSIP) for Leicester and Leicestershire presented an opportunity to think differently around how education and training offerings align with the skills needs of business. The project plan was created in collaboration with a broad range of policy, educational and business stakeholders and adopted a new approach to understanding and framing the requirements of employers.
The LSIP had three objectives, and focused on identifying the Knowledge, Skills and Behaviour areas (KSBs) that businesses identify as priorities to meet their growth aspirations: • To identify the KSBs required to meet employers’ growth aspirations, their relative Importance, and how current VET provision meets these needs. • To automate the process of evidence gathering with a user-led approach to its presentation. • To identify barriers in the LSIP geography to ensuring VET meets employers’ needs, and to recommend an action plan for addressing these.
The approach to developing recommendations and actions emphasizes Collective Intelligence: shared or group intelligence arrived at via collective effort. Collaboration across stakeholders has been a central tenet behind the production of the LSIP as has the democratisation of this process, using technology to make engagement as accessible as possible and automating the process of displaying the results. The LSIP project is detailed in the published reports and the Collective Intelligence Skills Observatory (CISO) is shown on the website https://www.insight-unlocked.co.uk/. This section explains how the CISO project informed and shaped this process.
The Collective Intelligence Skills Observatory is an innovative tool that brings multiple skills supply and skills demand datasets into one place, supported by business users’ opinions gathered by short e-surveys on their perceptions of the current skills market. It is built using automation to provide an efficient and sustainable approach. The automated processes ensure that there is minimal human effort involved in project development leading to increased efficiency in terms of time, costs and resources. The sustainable approach keeps the project running in future and processes are easy to transfer across different domains. This means the model can readily be adapted for other areas and applications beyond the first LLEP project. These concepts are supported by the underlying technology which produces the outputs in the form of reports and dashboards.
CISO datasets.
The data on skills demand was scraped from a third-party organisation through an API (application program interface). The business opinions were collected through daily surveys which in itself is an innovative process. In the first project, panels of small-medium sized businesses were invited in three targeted industry sectors: Manufacturing, Logistics and Sport and Health. To deliver the outcomes, the process from data collection or extraction to data reporting is supported by an automated infrastructure. However, even before the process of data extraction a lot of effort is put into understanding the metadata to convey the right information in an understandable way to the audience. To do this, we follow a principle of informing users on three main questions before actually looking at the dashboards: • What does the dashboard show? • How is it useful? • Where does the data come from?
These questions help users to understand how the dashboards can be useful to them and what is the smallest (most local) level of geography available, along with latest timelines.
The top-level automation architecture of CISO is supported by virtual machines, databases, dashboards and a web interface. All these components rely on cloud technology to avoid any dependency in future of CISO development. Each component in the cloud supports critical processes of the data lifecycle. It starts with running the scripts run on virtual machines. These scripts scrape data from sources and check for any redundancies or anomalies in the data. The processed data is stored in databases. The databases feed data into dashboards where the process of Extract-Transform-Load (ETL) is again performed. Finally, the dashboards are integrated into a website. It is important to note that the only manual intervention occurs during the embedding stage of the project. Figure 1 illustrates the architecture of the Regional Business Intelligence Unit, incorporating the CISO project under the Skills theme and as a process. Architecture of the regional business intelligence unit.
The approach to collecting data from business engagement used is different from filling normal lengthy surveys that consume a lot of time and energy from respondents. An LSIP Business Panel was convened with 42 manufacturing, 52 sport and health, and 28 logistics businesses (East Midlands Chamber, 2022). This group of businesses was asked to engage with daily surveys through answers requiring not more than 30 s to complete the survey. Moreover, the surveys were directly shared with businesses through a mobile app, eliminating the effort needed for businesses to access a website link and enhancing respondents’ experiences in the journey. This innovative approach worked successfully for the development of CISO.
It also helped to explore the trends of respondents’ interactions with the application, such as their responses to types of questions. Figure 2 shows the response rates from 83 businesses (68%) and trends on user interactions from each of the three sectors with survey forms for each day spanning the complete 8-week period of business engagement surveys. Responses by sector, theme and weekday.
This graph shows that the number of responses fell after week 6. This provides an insight into the type of formats used for survey. Before week 6, only questions using types of sliders, checkboxes, radio buttons were asked. After week 6, the focus was more on open-ended questions. The reason for the fall in numbers might not only result from the open-ended questions. After discussions with several businesses, it was found that they started filling the survey and lost interest before completion due to workplace-based tasks. This reason is anecdotal and once the novelty of completion had worn off may be another explanation.
The major benefit of developing CISO is its flexibility, scalability, ease of use and support for users in accessing the dashboards in form of user journeys. Flexibility implies that the concept of understanding supply, demand and business validation can be applied to any other domain or research areas. As the data scraped from sources covers a national geographical landscape, the CISO can be scaled for any geography and across any sector. The simplest yet most important feature of CISO is its ease of use and accessibility feature. The factor that makes CISO unique is its support for users through the concept of user journeys. User journeys help different user groups to access dashboards by following different paths and to avoid getting lost in the sea of data reports.
Recommendations
The LSIP report made 10 recommendations, detailed in the report. Coordinated work is needed with third-party suppliers to maximise their ability to support the Future Skills Insight Unit. This includes Government ensuring it is asking the ‘right’ questions, determined through meaningful engagement with businesses and providers to provide insights relevant to the needs of a local area and not assumptions made nationally.
There is much that third-party suppliers can offer to a system that makes best use of data to understand skills supply and demand-side issues, however, at present there is little consistency in quality or provision. Current provision is often based on what providers are easily able to provide and not necessarily what would support better outcomes.
It is essential that the ‘right’ questions are asked at the outset and third-party providers are challenged to come forward with solutions to these questions as opposed to their own pre-existing products. While recognising that in many instances these are commercial businesses, they all demonstrated a willingness to support, and with coordinated direction can play an important role in providing the solutions required.
The CISO design was seen by experts at DfEE and in the Local Enterprise Partnerships and Chambers of Commerce networks to be an innovative and effective model. As a result, the CISO model is being adopted by four areas in the East and West Midlands, with further potential for development. This includes CI applications for different clients beyond the LEP and Skills areas. The region includes a major freight airport, international logistics businesses and a Freeport which present future opportunities. The ability to engage employers through gathering regular narrative inputs from them is a distinctive feature, which takes the project beyond the earlier work by Belov et al. (2022). This capability has value in its own right as a longitudinal survey method for use with businesses, and an App is being developed to exploit this.
The KTP innovation journey and reflections on its effectiveness
At the final project management meeting towards the end of the 2-year KTP, the Associate drew the picture in Figure 3 to visualise his narrative of the project. The innovation journey.
This shows the project team at the start, the Associate (Harsh), significant innovation opportunities on directional signs, personal achievements and awards, on the uphill journey towards the peaks of Vision, Innovation, Success and Regional Prosperity.
The project is recognised as successful and has generated effective innovations, notably including the CISO model and the ability to gather and share regular contributions from participants automatically. This capability advances on the earlier work by Belov et al. (2022). There were also process innovations to the EMC Quarterly Economic Survey (QES) which all Chambers are required to complete. This provided personalised data and comparisons for respondents. These are novel contributions in the Collective Intelligence arena for smaller businesses, business engagement with policy, support and VET providers.
So why was this innovation process effective, and what transferable lessons can be learned and be useful for other innovation and KE projects? These are our reflections.
Create a shared vision
The case for the project was developed through dialogue and shared diagnosis by actors within EMC and DMU of a need for enhanced regional economic and business intelligence data, which would inform better decision-making. This recognised need was explored and shared with others, and during the gestation period for the project proposal developed a scope embracing enterprise policy; economic development; applications of open data; data analytics and science, which then extended to include Artificial Intelligence as the enabling technology. This developed an interdisciplinary project and strengthened the innovative potential.
Build on trust
The relationship between EMC and DMU was not new and there was a productive history of collaboration. Writing the proposal required significant investments of time from both organisations which was facilitated by the KTP Officer in the University who was recently appointed and committed to securing an approved project. The first iteration was reviewed by the KTP Advisor, whose feedback informed a revised proposal with a more detailed project plan and stronger business case. This gained approval in the formal Innovate UK process. This shared experience enhanced the trust and collaboration within the project team and strengthened the relationship with the KTP Advisor.
Hire the best person
Project approval and contracting was followed by a recruitment process for the Associate. This involved both partners fully in specifying the role and, after advertisement, shortlisting and appointing to the role. It was evident in the selection interviews that just one candidate met all the requirements and gained the approval of the full selection panel. He (Harsh) had the advantages of Indian software industry experience and a recent MSc from DMU, but the slight disadvantage of requiring a work visa for the UK as an Indian citizen. The project start was delayed for 2 months whilst this was obtained, but the lesson is very clear: appoint the person best fitted for the role, and give the Industry Partner the final selection decision as they are, in effect, the employer (contractually the University is the employer who deals with the legal and administrative issues).
Effective partnership and collaboration require structured, regular engagement
KTPs have a series of four-monthly formal Local Management Committees. These were supplemented by weekly half-hour Teams-based stakeholder meetings. As the project started during the COVID lockdown, these were essential for updating, communication and co-ordination.
Let the Associate drive the project
The proposal requires detailed project and workplans which are difficult to prepare in advance for a project which is emergent, inter-organisational and interdisciplinary. The plans were accepted, and the Associate and supervisory team confirmed them. But fairly quickly the plans started to change. The Associate brought new ideas and techniques, the external context was evolving during the COVID lockdown with new policy and economic factors bringing new opportunities. Trying to adhere to or revise the original detailed plans would have constrained the project’s effectiveness.
Open collaboration enables innovation
The project did transfer existing knowledge from DMU to EMC via the staff and Associate, but the innovation was based on a much broader knowledge and expertise base. Both organisations and the team members had networks of expertise and contacts with other knowledge-based organisations, including public sector agencies, businesses and not-for-profits. Being able to sample and select experience, data and services from a wide palette resulted in experimentation and the optimal combinations being found. An example was the interaction with Open Innovations, a data innovation hub with a public mission and The Data City, its commercial services arm, both of which were among the organisations whose services and expertise were harvested.
Follow the opportunity
There was a commercial revenue expectation embedded in the project, but at the outset it was not clear who the target market or the services provided might be to generate this. The call for ‘Trailblazer’ proposals for LSIPs was well-timed and being able to connect the KTP to develop the innovative CISO concept, gave the project a clear direction and the scope to create and deliver an AI and CI-based innovative approach. Achieving this created the platform for EMC to be recognised as an effective innovator, to exceed the revenue targets, and to develop an economically sustainable business intelligence unit which can permanently employ the Associate as well as two new posts and future intern or project opportunities.
Conclusions: evaluating the contributions of the case beyond the LSIP context
The concepts and working model developed by the EMC in development of the skills observatory gained early momentum at local and national level. This is reflected in new contracts won by the EMC from LLEP and three UK Chambers. The Alan Turing Institute was in contact with the technical lead of the project to understand the methodology and collaborate on new innovation through Data Skills Taskforce, an organisation that promotes collaboration and drive action in the national response to complex and evolving data skills landscape. The Tony Blair Institute was interested in understanding the methodology of the skills observatory.
The skills observatory has been included in conversations on developing the UKC3 cyber cluster where the CISO methodology can be used. Overall, the CISO has gained profile and pace in its development, demonstrating its potential to change the social scenarios of how organisations utilise combinations of open-source and proprietary datasets which can then be verified through business engagement. The survey method also demonstrated the practical effectiveness of engaging a group of business respondents to contribute their perceptions on a longitudinal basis over a short period.
From an academic perspective, the KTP benefited from an interdisciplinary academic team combining entrepreneurship, economic data analysis and artificial intelligence methods in an innovative way. The project was able to utilise and apply a range of academic knowledge and expertise to deliver practical and policy outputs. The partnership with EMC provided a regional business and policy scope, benefiting multiple organisations and areas. This has opened the way for future knowledge exchange projects.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
