Abstract
There is increasing recognition regarding the potential of Higher Education Institutions (HEIs) to be significant Anchor institutions capable of impacting the wellbeing of the local communities where they are situated. Subsequently, more universities are prioritising their social and civic responsibility by embedding a research ethos based on closer engagement with their locality, within their key institutional processes. However, demonstrating the social value of community engagement in a meaningful way is difficult, particularly the type of intangible benefits such as relationship building and co-production of knowledge that are forged through collaboration. We present results of a pilot study which applied a Socially Modified Economic Valuation (SMEV) approach to three very different case study projects within a University-Community partnership initiative situated in Belfast, Northern Ireland, to help evaluate how well the university’s civic and social obligations are being fulfilled. Two sets of potential social weights were derived. The first was based on the official Deprivation Index and the second on a combination of desirable ‘social’ and ‘strategy’ goals. Findings showed that the social valuation element of SMEV may potentially be crucially important in assisting with nuanced interpretation of social value across different communities in different places.
Introduction
Changes in the global economy such as labour market restructuring and the growing significance of knowledge intensive sectors, have created new forms of spatial disconnectedness which have deepened the poverty, exclusion and otherisation of the most deprived communities and the children and young people who live there (Enright and Rossi, 2018). For example, the rationalisation of many large industries that would once have been major employers in local areas, has resulted in significant job losses, with subsequent damage to local economies. These deeply structural processes mean that more stable local assets, often referred to as ‘Anchor institutions’, play an increasingly significant role in the economic and social development of the communities where they are based, especially in more economically vulnerable places.
There is increasing recognition regarding the potential of Higher Education Institutions (HEIs) such as universities to be significant Anchor institutions capable of impacting the wellbeing of the local community by helping to address societal challenges through locally focussed research involving collaboration, partnership and co-creation with the public (UUP Foundation Civic University, 2019). Moving beyond the kind of civic engagement that all universities should and can do, a growing number of universities have prioritised their social and civic responsibility by embedding a collaborative research ethos based on building partnerships locally, nationally and internationally, within their key institutional strategies and processes.
The significant prominence of collaboration, engagement and co-production within universities (and the wider private and public spheres) is driven by a quest to make knowledge production more democratic – creating the conditions where historic inequalities can be addressed, ultimately leading to more relevant and higher quality research (Facer and Enright, 2016). At the same time as an increase in levels of engagement between universities and communities, there has been a discernible growth in processes of accountability, with a rise in the use of terms such as ‘peer review’, ‘risk assessment’ and assessing the ‘impact’ the university’s research is having (May, 2022).
Promoting a positive economic and social impact on society through research and education is in keeping with the UN’s Sustainable Development Goals and the goals of local and national government, as well as major funding organisations. However, the challenges facing universities to communicate the positive social impact they have on their localities, in addition to their global role and the importance of collaboration and partnership, is well recognised (UUP Foundation Civic University, 2018). Better insight into the value generated by universities’ societal engagement, particularly the type of intangible benefits that are difficult to capture through traditional economic measures, has been the focus of much work, nationally and internationally. Some key examples are discussed in the following section.
This article reports on a pilot study which applied a ‘Socially Modified Economic Valuation’ (SMEV) approach to selected examples of projects within a university-community partnership initiative working to address pressing social and economic challenges – Queen’s University of Belfast’s Queen’s Communities and Place (QCAP). The aim was to illustrate the potential for SMEV to give insights into (1) the broader social value of the work undertaken by QCAP and (2) the contribution by the university to the wider community. The work focusses on projects carried out with our anchor pilot community based in inner South Belfast. The longer-term aim is to contribute to existing efforts to help legitimise ways of knowing that are often marginalised by conventional evaluations and practices based on the application of a traditional and unimaginative output-oriented concept of ‘impact’.
Anchor institutions
Anchor institutions are generally described as large organisations with significant assets, geographically situated in specific places and are stable and unlikely to relocate – hence the term ‘anchor’ (Bergan et al., 2019). Some definitions focus more on the role of anchor institutions as international connectors for a region (e.g. universities, airports and ports), while others focus on economic priorities around development or regeneration (Goddard et al., 2014). While definitions vary in different contexts, all definitions highlight the scale of their local impact on the economic, environmental and cultural wellbeing of the places where they are rooted.
Policy makers and legislators are increasingly placing priority on collaborative working between institutions who provide services and the society who use them to generate innovative ways to improve quality of life, and the economy to achieve better outcomes. For example, countries such as the USA, Australia, the UK and Ireland have developed national policy frameworks and/or support mechanisms to increase community engagement in higher education (Farnell, 2020; Maassen et al., 2019). This is particularly relevant in a time when other local anchor institutions are declining or relocating.
There is growing consensus regarding the civic and social role of universities in addressing issues like poverty and inequality through collaboration and partnership with local communities, businesses and government agencies, where addressing regional issues collectively, is said to reinforce a university’s commitment to the community they serve (Council of Europe, 2020). The Council of Europe has been leading a consortium of organisations seeking to establish an anchor institutions network across Europe (Bergan et al., 2019), with the experience of the COVID pandemic underpinning the importance of this initiative (Bergan et al., 2021).
The Anchor Institutions Task Force (AITF) in the US was created in 2009 and has contributed significantly to promoting the concept more widely. According to the approach of the AITF, HEIs are engaged anchor institutions when they operate as genuine partners with their communities, helping to improve their local place. In this way they emphasise ‘social responsibility, including values of collaboration and partnership, equity and social justice, democracy and democratic practice, and commitment to place and community’ (Guarasci and Maurrasse, 2015: 102).
However, the history and practice of university-community partnership research differs across national contexts, with different traditions of collaborative research being defined and shaped by various factors including, for example, uneven economic and educational resources and diverse knowledge and capacities in relation to different topics and amongst different sectors and populations (Facer and Enright, 2016). Successful collaboration can often be the result of universities having well established roots in collaborative partnership working and some communities being more actively ‘research ready’ than others. Nevertheless, the propensity for bringing together expert and public knowledges through collaboration – what has sometimes been referred to as the ‘participatory turn’ in the processes of knowledge production (Facer and Enright, 2016: 1) – is now a well-established practice in both universities and the wider policy and public spheres.
It should be noted that, despite the widespread acceptance of creating new knowledge through expert and public collaboration and the potential for positive outcomes, co-production has limitations as well as strengths and is not devoid of controversy (Goodson and Phillimore, 2012). This is reflected in the growing body of work critiquing the conceptual confusion or ‘messiness’ that surrounds co-production. A key example being the loose range of associated terms such as ‘co-creation’, ‘co-design’ and ‘engaged research’ which are often used interchangeably in the academic literature and wider use, and potentially weakening conceptual coherence (Durose et al., 2022; Flinders et al., 2016; Voorberg et al., 2015).
Alongside the shift towards closer collaboration with the local, is the growing consensus that research should have an impact beyond academia and that we can establish metrics to assess this impact. While there has been support for the idea that research should have impact, there have been concerns around how ‘impact’ is defined and then measured (Smith et al., 2020). Research has evidenced how the pressure on universities to achieve demonstrable economic or social impacts, can hinder the creation of the type of living knowledge that could be of most benefit to community partners and to the broader knowledge base. For example, many researchers report how the value attributed to co-production by funders reflect a narrow understanding of the impact of co-production, continuing to favour traditional measures of success such as academic outputs and quantifiable and measurable impacts (Perry, 2022). As noted by Munck (2022), the emphasis on outcomes, visible in the current impact agenda, is less than ideal for evaluating the valuable process of public engagement itself, that is ‘the debates and discussions, the messy co-creation of knowledge process and the relationships built during that process.’ (Munck, 2022: 9).
International tools do exist to assess community engagement in higher education. Key examples include the EU-funded project Towards a European Framework for Community Engagement in Higher Education (TEFCE) is one of the key frameworks being used by the Council of Europe group on the Local Mission of Higher Education for promoting community engagement by HEIs. TEFCE, and its follow-on project ‘Steering Higher Education for Community Engagement’ (SHEFCE), seeks to build the capacities of universities, policymakers and stakeholders in Europe for mainstreaming community engagement in higher education. The TEFCE/SHEFCE toolbox to support community engagement uses qualitative methods of critical self-reflection and peer-learning, rather than a measurement process through the collection of quantitative data (Farnell et al., 2020). Other examples are the UK’s National Coordinating Centre for Public Engagement (NCCPE) Edge Tool which acts as a self-assessment matrix to assess institutional support and progress on public engagement and the development of action plans to facilitate future work. Also, the methodological outputs from the Cross-Research programme AHRC led ‘Connected Communities’ initiative’ which looked at the measure of the success of partnerships between HEI’s and their local communities, drawing on the methods and theories of the arts and humanities.
The pilot study which this paper reports on was initiated to illustrate the potential for a SMEV approach to give insights into the broader economic and social value of the work being undertaken by QCAP and the contribution by the University to the wider community. It was also hoped that finding ways to objectively demonstrate value of engagement could contribute to strengthening university-community partnerships, as well as add to existing efforts to legitimise ways of knowing that are often sidelined by the application of an output-oriented concept of ‘impact’.
In this pilot, the different social weightings are an attempt to reflect the different sets of expertise and experience that university and community partners offer, and in so doing, help to address the challenge of a democratic deficit that universities share with public bodies who struggle to devise participation methods capable of overcoming it. Connected to the democratisation aim is the desire to see university (and other anchor institutions) resources mobilised to support vulnerable communities.
These aims cannot be pursued through a single, generally applicable method. Rather, it requires the development of a comprehensive methodology consisting of a suite of mutually complementary methods. It is hoped this work contributes to the creation of a space where universities and communities can collaborate to create new ways of understanding, as well as potentially breaking new ground in interpreting the nuances of value across different communities in different places.
Socially modified economic evaluation (SMEV)
Socially modified economic evaluation (SMEV) is an approach that has its roots in fundamental welfare economics. It applies techniques derived from social cost-benefit analysis to make estimates, expressed in monetary terms, of the broader economic and social value of a range of activities, such as university community engagement, that may not be captured in more traditional market based or financial analysis (Kelly et al., 2008; Kelly and McNicoll, 2009).
At the SMEV core is the development of a ‘Theory of Change’ or Logic model, which delineates and tracks the path between the resources and inputs used in a programme, the activities carried out, the deliverables produced because of those activities through to the eventual desired outcomes – to which the outputs are intended to contribute. However, the key difference between the SMEV approach and many other evaluation methodologies (such as Social Return on Investment (SROI) or Outcome-Based-Accountability (OBA)) is that analysis is focussed on the outputs or deliverables of any given project rather than the more general desired outcomes. As many researchers and commentators have highlighted, a sole focus on outcomes is not entirely useful for assessing value delivered at programme level, not least because of considerable attribution difficulties (Gray and Birrell, 2017). A programme may contribute towards an outcome but most often the desired outcome is dependent on a wide range of other factors as well (Northern Ireland Assembly, 2021).
Focussing on the outputs or deliverables enables assessment of the real economic value delivered by the project or programme. It also provides a way to compare value generated by very different types of programme within a common framework – helping to overcome the difficulty of comparing ‘apples and pears’. Once an economic value has been estimated, it is then possible to apply sets of social weights to the economic value to derive an estimate, in monetary terms, of additional ‘social’ value generated, above and beyond the economic value (Kelly and McNicoll, 2011).
In summary, it takes a two-stage approach to valuation:
1. Estimating the economic value delivered by the activity under consideration.
This is achieved by identifying, quantifying and, finally, pricing the outputs delivered. It is important to note here that the economic value may not always be the same as the financial value. It is the estimated economic efficiency price that equates to the economic value. Take, for example, a service that is provided free of charge. With no money changing hands, the service would have no apparent financial value. However, it still has economic value. In this situation, a process of shadow-pricing is required, to estimate the value of the outputs as if they had been delivered within a free, competitive and market.
2. Development and application of appropriate social qeights.
The application of social weights helps reflect how the activity under consideration is achieving agreed ‘socially desirable’ goals, above and beyond its economic value. Social weights are not ‘set in stone’ but need to be transparently devised and appropriate for the nature, rationale and broader intended goals of the activity being analysed. For example, they could be ‘distributional weights’ based on the income levels of people affected by the activity, as presented in HM Treasury Green Book. In that case, if an activity is intended to benefit lower income or disadvantaged groups, then its delivery to lower income groups attracts a higher weighting than its delivery to those in better off groups.
Other forms of weighting may also be devised based on, for example, how far the activity aligns with the University’s own institutional strategic goals, on the priorities of the relevant funder of the activity, on the priorities expressed by the ‘participant’ community groups and so on.
The application of social weights to the economic valuation observed in (1) will modify the final valuation accordingly, to help reflect the overall ‘social value’ of the activity, expressed in monetary terms. Programmes could then, for example, be compared in terms of which may generate more ‘social’ or ‘strategic’ value than others for any given amount of actual investment.
The pilot study: Applying SMEV to case study examples of the Queen’s Communities and Place initiative
Queen’s Communities and Place (QCAP)
QCAP is a ‘place-based’ community-academic partnership initiative working to strengthen the connection between universities and their places. QCAP is supporting communities through applied research and a diverse range of activities driven by local community needs. QCAP is strongly aligned to Queen’s University’s Strategy 2030 pillar ‘Social and Civic Responsibility and Economic Prosperity; Public Engagement; and Impact’ and responds to the University’s concern for a stronger commitment to working with socially excluded communities, making research and teaching more locally relevant.
There are a number of inter-disciplinary research strands within the initiative, including:
The genesis of this work lies in the authors’ extensive engagement with a range of communities over past decades, as well as QCAP’s rapid response to approaches for help during COVID. The initiative was first launched with our anchor pilot in the Market community situated in inner South Belfast. The core community partner is the Market Development Association (MDA), with whom we have co-designed the first phase of work. The initiative will eventually expand into other urban, small town and rural settings across Northern Ireland to explore the transferability, sensitivity and suitability of the QCAP approach in different geographical and socio-economic contexts.
QCAP takes an innovative, collaborative, approach to addressing key societal challenges by adopting a ‘place-based’ approach. This focusses on working together with local communities and policy makers to co-create solutions that address that particular community’s needs, while also informing wider research and policy development.
A key challenge for QCAP is to translate findings in ways which can demonstrate the value of ‘non-commercial’ university activity and open up an effective dialogue and understanding between academics, policy makers and the public. Funding was received from the ESRC’s Impact Acceleration Account (IAA) in 2022 to undertake a pilot study exploring a potential way to indicate the broader economic and social value of different types of projects being delivered within the QCAP framework. The pilot study was a desk-based exercise, and no ethical approval was sought for this. However, the QCAP initiative has overall ethical approval from Queen’s University Belfast’s School of Social Sciences, Education and Social Work.
QCAP case study examples for the pilot study
The pilot study applied SMEV to three quite different case study examples of QCAP work.
QCAP and university support for the refurbishment and development of the local Primary School Community Garden.
QCAP development of an innovative Community Data Profile tool designed in conjunction with the MDA and intended to support the MDA in their ongoing work.
The role of the QCAP community liaison officer in working with the MDA and acting as a knowledge broker between the university and the wider community.
Each of these activities are very different in their nature and scale. They were chosen to help test the flexibility of a SMEV approach to estimation of social value. It is important to note that this pilot assessment is focussed on the value of the university contribution to the project because the focus of this pilot analysis was to show how QCAP contribution to the wider community can be assessed. The numbers presented are illustrative and reliant on desk-based consideration of possible approaches to social weighting. It is also important not to confuse the economic value and social value figures as being related to ‘hard cash’, which relates only to financial value. The economic and social values are those that can be imputed to the project using the SMEV principles outlined in this paper.
Case study 1: School Garden Project
(Anon) Primary School is located in central Belfast. In 1998, work began to create a wildlife garden in the school grounds. This was an innovative development, intended to give the children access to green space, the opportunity to learn about the environment as well as increasing biodiversity in the area. However, the garden and its pond area became a challenge for the school to maintain; it fell into decline and become difficult for children and teachers to access safely. Following discussions between the QCAP team, the MDA, the School’s Principal and teachers, it was agreed that the garden still represented a significant potential resource for supporting both the schoolchildren’s educational development and environment benefit in the local area. An accessible garden would provide an onsite resource for projects related to the school curriculum – in particular to support delivery of ‘The World Around Us’ thematic learning, which includes elements of Geography, History, Science and Technology. The refurbishment of the garden and developing its educational use potential was adopted as a first stage project within the QCAP Communities and the Knowledge Economy strand of activity.
A summary Theory of Change model for the School Garden Project was developed, indicating how the project is aligned with and contributes to a range of broader desired outcome goals, including the strategic aims of both the University (e.g. contributes to Strategy 2030 goals) and the Partner community organisation (e.g. supports improved educational environment). A 12-month period was used across all three projects so that the projects can be compared within a similar time frame.
Economic value generation
All the work carried out by QCAP was undertaken at no charge to the school or wider community, so there were no financial flows between the University and the School. However, there is still an economic value because the University invested both staff time and some financial resource into supporting and developing the project. As set out above, the first step was to undertake an economic valuation of the outputs delivered by the project. The valuation of outputs can be derived by directly pricing /shadow-pricing the outputs themselves or, alternatively, by pricing/shadow-pricing the inputs to the project. The mixed nature of the School Garden Project outputs, involving both gardening and landscaping as well as broader educational advisory input and delivery of specific educational activities, meant that an approach to valuation of outputs was adopted using both labour and non-labour inputs as well as including an element of economic ‘normal profit’. Fuller discussion of approaches to economic pricing can be found in (Kelly and McNicholl, 2022). The economic value of outputs over a 12-month period was estimated as £7654.
Social value generation
Two sets of examples of potential weights were derived. The first is purely ‘social’, using the Northern Ireland Index of Multiple Deprivation (NIMD). The Market area is defined as comprising five neighbour area data zones (Northern Ireland Statistics and Research Agency, 2021). It was possible to then analyse the proportion of the Market population living in areas affected by multiple deprivation, using the NIMD relative rankings. Weightings can then be applied to impute relatively greater value to programmes affecting areas of greater deprivation.
For this SMEV assessment the economic value always carries a baseline weighting of 1. The economic value is the ‘bottom line.’ Only where a project outputs are perceived to have additional ‘social’ characteristics – in this case that they are delivered within an area that is relatively more deprived – do they attract an additional weighting (greater than 1.)
Applying the suggested weightings to the proportion of the Market population affected by multiple deprivation led to an overall ‘social’ weighting from programmes affecting the neighbourhood of 1.54 (see Table 1).
Distributional social weighting using the N.I. deprivation index.
The second set of weights are based on a combination of desirable ‘social’ and ‘strategy’ goals for QCAP projects, taken from the University perspective. Firstly, several broader desirable characteristics or desired outcomes for QCAP projects was selected following a review of university, community and government documentation and strategies. The QCAP team assigned a relative importance score to this selection (see Table 2).
Relative importance of impact of any individual QCAP project in each of these areas.
Secondly, adjustments were made to one of the features in the selection ‘Benefits or potentially benefits people living within a deprived or disadvantaged area’ to accommodate the NIMD weighting, which takes account of the relative deprivation level of the Market area. As these are the relative priorities expressed from the University side of the partnership, these are described as ‘University Preference Weights’ (see Table 3).
University preference weights.
For each of the case study projects, a decision was made based on the project’s contribution to each of the desired priorities. A score of ‘one point’ was given for each of the priorities to which the project contributed. The individual university preference weights were then applied to the outcome of this assessment, enabling an overall ‘social weighting’ to be devised for each project, reflecting the project’s contribution towards the desired goals. Once analysed, the preference weights for the School Garden Project resulted in an overall social weighting of 1.73 (see Table 4).
Social weight generation for the School Garden Project.
The estimated economic value generated by the School Garden Project comes to £7654 over the first 12 months of the project. Once additional social and strategy factors are taken into consideration, that is from the standpoint of the project benefitting a disadvantaged area (NIMD weighting of 1.54), the overall social modified economic value comes to £11,787. When other additional social and strategic factors considered important by the University were considered (University Preference weighting of 1.73), the social value increased, in this case to £13,241.
Case study 2: Development of a community data profile tool
The community data profile (CDP) dashboard tool was designed in conjunction with MDA staff and is intended to support the MDA across their activities. Formulated as part of the Social Economy and Community Wealth Building strand of QCAP activity, it is an operational tool and interactive resource that can help with a range of MDA operational and strategic information needs including research, reporting and MDA essential internal recording keeping. The CDP tool enables a statistical profile to be created of the local area, by aggregating publicly available core small area data on demographics and socio-economic characteristics from relevant NI Government departments and other sources. The core tool is currently hosted by QCAP but is designed to be low technical maintenance to require a minimum of ongoing resource for upkeep. The tool addresses a need encountered by many smaller community groups and social enterprises, who tend to lack capacity and sometimes technical skills in data collation and record keeping. It enables easy access to data required for both funding applications and for reporting to funders. The QCAP team worked with the MDA to integrate the tool into their existing processes and trained staff in its use.
As in Case study 1, a summary Theory of Change model for the CDP project was developed, indicating how the project is aligned with and contributes to a range of broader desired outcome goals.
Economic value generation
The CDP tool was developed and made available at no charge. Given that there were no financial transactions involved between the University and its community partner, the project has no obvious financial value. However, it still has an economic value, and the first step was to undertake an economic valuation of the outputs delivered by the project, in this case the CDP tool itself.
One way to do this is to estimate what provision of such a tool would be likely to cost, if it were to be retailed within a free, competitive, market. However, it proved very difficult to identify any reliable pricing source. Similar tools that were identified tended to be either public information or research tools, which were not openly available for purchase (hence no prices available), or subscription based commercial dashboard tools for which full pricing was also difficult to obtain as commercial firms tend not to be entirely transparent in their pricing models. Therefore, a valid alternative approach of shadow pricing the university inputs to the project was carried out. The main inputs were skilled staff time, including in statistical software model design and usage. As in case study 1, a 12-month period was used so the projects can be compared within a similar time frame. The estimated economic value of outputs was calculated to be £15,264.
Social value generation
Additional value was imputed to the project based on the proportion of the population in the Market area living in relative deprivation (the NIMD weighting of 1.54). The second set of weights are based on a combination of desirable ‘social’ and ‘strategy’ goals for QCAP projects, taken from the University perspective. Once analysed, the preference weights applicable to the CPD project resulted in an overall composite social weighting of 1.93 (see Table 5).
Social weight generation for the community data profile project.
The estimated economic value generated by the CPD project was £15,264 over the first 12 months of the project. Once additional social factors of benefitting a disadvantaged areas are taken into consideration (NIMD weighting of 1.54), the overall social modified economic value comes to £23,507. When other additional social and strategic factors considered important by the University were considered (University preference weighting of 1.93), the social value was nearly double that of the economic value increasing in this case to £29,460.
Case study 3: Community liaison role
A core strand of the QCAP initiative is a 3-year study Growing Up in the Market (GUiM), which aims to follow the experiences of Market residents – children, young people and their families, at key educational transition points. The purpose of the study is to better understand challenges around health, education and employment and explore the persistent nature of place-based disadvantage.
To support this work a local liaison officer and administrator were appointed, funded by the Northern Ireland Department for Communities, to embed, build and maintain the in depth links and interaction needed between the community and university researchers. The coordination role is seen as an essential bridge and knowledge broker, building trust and making connections initially for the GUiM project but extending across other facets of the QCAP team’s work with the Market Community.
As in case studies 1 and 2, a summary Theory of Change model for the GUiM project was developed, indicating how the project is aligned with and contributes to a range of broader desired outcome goals.
Economic value generation
The Theory of Change summary showed that many of the outputs of the Community liaison role may be seen as intangible – key examples include building relationships and networks, developing mutual respect and recognition – providing the essential ‘glue’ required to facilitate successful interaction and build knowledge capacity. In this example, shadow-pricing the inputs to the role (Community Liaison Officer, Administrative Support, non-labour operating costs e.g. equipment and materials) was considered likely to be an easier way to estimate its economic value than trying to pin down the myriad different outputs (given that, as explained in Kelly and McNicoll (2022) the shadow priced value of inputs plus normal profit is equal to the shadow priced value of outputs)). It should be noted here that the coordination functions (Coordinator and Administrative support) are the sole focus of consideration. The economic value generated was estimated to be in the region of £102,465 over 12 months.
Social value generation
As with the other case studies, overall social weights derived from two alternative sets of weights were applied. The first weighting is purely ‘social’, using the NIMD weighting (1.54) for projects operating in the specific local community area. The second set of weights are based on the ‘University Preference Weights’. Once analysed, the preference weights applicable to the Community Liaison project resulted in an overall composite social weighting of 2.08 (see Table 6).
Social value estimated for the community liaison role.
The economic value of the Community Liaison role is estimated to be in the region of £102,465 over the first 12 months. When the additional social factor is considered of the project benefitting a relatively disadvantaged area (NIMD weighting of 1.54), the overall social modified economic value comes to £157,796. When other additional social and strategic factors considered important by the University were considered (University Preference weighting of 2.08), the social value was more than double that of the economic value increasing in this case to £213,127.
In all three case study examples, application of the sets of social weights to the economic valuation considerably increased the value of the projects in terms of their social value. In other words, the social weights helped to demonstrate, in a quantitative manner, how each project was achieving agreed ‘socially desirable’ goals, above and beyond its economic value.
Discussion
It is increasingly recognised that HEIs need to be anchored in their local communities and that engaging with the local is fully compatible with global engagement (Bergan et al., 2019). Hence, a growing number of universities have now prioritised their social and civic responsibility by embedding a collaborative research ethos based on building partnerships locally, nationally and internationally, within their key institutional strategies and processes. Consequently, HEIs are now under increasing pressure to demonstrate how well they are fulfilling their responsibilities and measure the impact of civic and social engagement activities. However, this is a difficult task because most conventional approaches to evaluation tend to focus on what is quantifiable and measurable. Intangible impacts such as relationship building and mutual respect are not immediately amenable to evaluation in terms of a linear model of research where inputs lead to outputs, in turn leading to outcomes that may promote social change (Johnson, 2020: 11).
This study has shown how application of the appropriate social weights within SMEV to the economic valuation gives a quantitative indicator (expressed in monetary terms) of the additional social value to a community of a particular initiative. The SMEV approach shares some common characteristics with other approaches to ‘social valuation’ (e.g. with SROI) and also uses a commonly applied tool in the form of a Theory of Change to tease out the relevant project elements and how these work in the context of contributing to overall desired outcomes. However, the fundamental difference of SMEV is that: (a) the first element of SMEV (economic valuation) focusses on estimating the economic value of the project and programme outputs or deliverables, rather than straining to capture value from the, sometimes more nebulous, outcomes. Desired outcomes often involve additional factors beyond the boundaries of the project, which makes valuation of outcomes difficult to relate back to the contribution of the particular project or programme; (b) the second element (social modification of value) takes into consideration other project impacts, including elements of the overall desired outcomes, in the development of a series of ‘social weightings’ which can be applied to the economic value estimates to derive a ‘Socially Modified Economic Value’. In this way, a SMEV combines an objective core to a valuation along with the capacity for interpretation in the light of socially desired objectives above and beyond economic value.
Limitations
This was a small-scale desk-based pilot project, and the results should be considered as illustrative rather than absolute. The second set of social weights (the University Preference Weights) is based on a combination of desirable ‘social’ and ‘strategy’ goals for QCAP projects, taken from the University perspective. These are the relative priorities expressed from the University side of the partnership. It is possible that other QCAP partners’ priorities may differ. However, the pilot analysis has been able to give more insight into how a SMEV approach can be implemented and what can be learned from it. We know, for example that a more in-depth analysis using SMEV would permit more refined and disaggregated sets of weights to be developed, allowing comparison of similar projects operating in different communities which may have different prevailing priorities. The nature of the pilot study meant there was little opportunity to capture qualitative testimony but, going forward, qualitative data from key stakeholders would be an asset to the analysis.
Conclusion
The purpose of this pilot study was to test the potential for a SMEV approach to give insights into the broader economic and social value of the work being undertaken by QCAP and the contribution by the University to the wider community. It was also hoped that finding ways to demonstrate value of university engagement could contribute to strengthening university-community partnerships, as well as potentially breaking new ground in interpreting the nuances of value across different communities in different places.
Findings indicate that it is possible to apply a SMEV approach to three quite different kinds of projects and has laid the groundwork for similar analysis of other strands of QCAP work. The social valuation element of SMEV may potentially be crucially important to QCAP in assisting with nuanced interpretation of social value within a place-based framework. The development of sets of social weights appropriate to the circumstances and environment (geographical, political, social, cultural) will enable a flexible social policy and preference overlay to the baseline economic value. For example, despite the peace process, Northern Ireland is still a deeply divided society with different national identities and emotional responses to representations of culture (Muldoon et al., 2020). Consultation and discussion with communities and key stakeholders to identify their specific priorities and social preferences would allow the derivation of sets of ‘social preference weights’ from each community/key stakeholder perspective. These can then both standalone and be combined to derive a composite weight incorporating the interest of all parties. This would mean that any given QCAP strand of activity could potentially be evaluated, and results presented in terms of the value they represent to each Stakeholder separately and in terms of the overall value incorporating all of the Stakeholder preferences. This could also be done with future major new partners or funders, to show the value generated by QCAP in their terms. Developing the appropriate sets of weights would most likely initially be relatively resource intensive. However, the more refined the weights, the more useful insights may be given.
The flexibility of a SMEV approach in taking account of the different social preferences in different circumstances and different communities could support analysis of the impact of similar initiatives taking place in very different places. This could potentially provide opportunity for working with other partners and other universities in comparing the differing social value generation associated with projects in different cultural and political settings.
Finally, being able to deliver estimates of both the economic value and, where appropriate, the relevant social value of an initiative helps give strong evidence in making cases for funding. This is particularly likely to be the case when the ‘social value’ indicators applied are specifically in line with the goals, missions and desired outcomes of the potential funders. Furthermore, the approach also enables comparison between different types of projects within the same framework. There will always be choices to be made between deserving projects. Overall, analysis of the QCAP activity strands within the same analytical framework would help inform judgements on which projects are likely to deliver the most economic and social benefit for any given amount of investment. Any project where the economic benefit clearly exceeds the financial cost is worthy of support. However, the situation may arise where two projects may deliver broadly the same economic benefit per £1 of investment but when the social weighting (or SMEV measurement) is taken into account, one of those may be seen as delivering higher social value than the other. Overall, the findings will benefit researchers across disciplines, who are committed to making knowledge production more democratic, promoting a positive economic and social impact on society, locally and globally, and working to address challenges associated with judging project value that too often rely on conventional measures of research impact.
Footnotes
Declaration of conflicting interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Ursula Kelly received a consultancy fee for providing the expertise and advice required for the development of the more advanced economic and social weights used in this pilot project.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Economic and Social Research Council [Impact Acceleration Account, Leading Impact Award)] in 2022.
Data Access Statement
All data is provided in full in the section of this paper entitled ‘The Pilot Study: Applying SMEV to Case Study Examples of the Queen’s Communities and Place Initiative’.
