Abstract
Coproduction is seen as a mechanism for generating public value. Research has focused on coproduction’s conceptualisation and implementation, whereas less is known about the benefits to individuals of involvement in coproduction activities. This paper adds to an emerging body of work examining the outcomes of participation in coproduction for individuals. It draws on the conceptual ideas of value-in-production, value-in-use and value-in-context (Osborne et al., 2022) to examine value creation at the individual-level. Longitudinal quantitative data on young people involved in the coproduction of the Talent Match employment programme in England is used to test the hypothesis that service users involved in coproduction achieve better outcomes than those who are not involved. The findings suggest individuals gained value from coproduction, by expanding their skill set and improving their wellbeing, although this did not translate into improved employment outcomes compared to their peers. In viewing coproduction within a wider ecosystem of value-creation processes, claims made of the transformative effects of coproduction for services and service users are tempered.
Introduction
A substantial body of literature on coproduction in public services has developed over time largely focused on conceptualising coproduction (Brandsen and Honingh, 2016; Nabatchi et al., 2017), its potential to create value (Loeffler and Bovaird, 2021), and the implementation of coproduction processes (Brudney, 2020; Farr, 2018; Sicilia et al., 2019). The involvement of citizens in the design and delivery of services has routinely been promoted by researchers and policymakers as making services more efficient and effective, producing better outcomes for users (McMullin, 2023). Yet, it is uncommon to find accounts of why it is that coproduction is expected to produce its espoused benefits (Durose et al., 2017).
Public administration and management (PAM) scholars, however, have incorporated coproduction into work theorising value creation. Osborne et al. (2022) situate coproduction within wider value-creation processes occurring at varying levels within a public service ecosystem, with users central to value creation. This paper draws on this framework and the conceptual ideas of value-in-production, value-in-use and value-in-context to examine value creation at the individual-level. Specifically, the paper tests the hypothesis that service users involved in the coproduction of services achieve better outcomes than service users who are not involved because they experience enhanced value creation.
Using data on over 13,000 young people engaged in the Talent Match (TM) employment programme in England, the paper explores how young people who were, and were not, involved in a defined set of coproduction activities, experienced value creation. While scholars have begun to examine individual-level effects of participation in coproduction (e.g. Cicatiello et al., 2023; Jo and Nabatchi 2019), the assumption that coproduction leads to better outcomes for users remains largely untested. Empirical work testing conceptual ideas related to value creation has, however, started to emerge (Cui and Osborne, 2022; Dudau et al., 2023; Go Jefferies et al., 2021; Jaspers and Steen, 2021). This paper adds to these emerging enquiries, by taking the conceptual ideas of value-in-production, value-in-use and value-in-context and applying these empirically to examine the relationship between involvement in coproduction activities and outcomes achieved by individuals.
In addition, the paper assesses a second hypothesis: involvement in different types of coproduction activities leads to differences in outcomes achieved. There has been limited empirical work testing the relative efficacy of different methods of direct public participation (Jo and Nabatchi (2021) provides a rare example). The analysis here builds on emerging work in this area to test further the theory that involvement in different types of participation (coproduction activities in this instance) leads to differences in outcomes achieved.
Longitudinal programme evaluation data provides the evidence for this paper. The dataset utilised is the largest known dataset tracking participants of a youth employment programme. It thus provides a robust counter-factual, enabling comparison of outcomes achieved by participants who were, and were not, involved in defined coproduction activities. With no known comparable studies, the evidence in this paper therefore provides a unique empirical contribution to the literature on the outcomes of participation in coproduction for individuals and to wider work on the nature of value creation in public services.
The evidence presented indicates that engagement in coproduction activities was associated with achieving all soft outcomes explored and some hard outcomes, suggesting service users did gain value from involvement in TM coproduction activities (value-in-production). Most notably, they experienced improvements in wellbeing and skills beyond those experienced by their peers. Engagement in coproduction was, however not associated with the key programme outcome of securing sustainable employment. Coproduction was not a silver-bullet for tackling the problems faced by the young people engaged in TM, and in situating coproduction within a wider public service ecosystem the findings here help moderate the claims often made of the transformative effects of coproduction. Finally, no clear pattern was found in terms of which forms of coproduction were most strongly associated with outcomes, so no firm conclusions can be drawn here.
The remainder of this paper provides a brief background to coproduction and the emerging evidence base on outcomes, a discussion on value creation in public services and formulates hypotheses to be tested. A consideration of the TM programme in England as the case setting then follows before the methodology and methods are discussed. The results and analysis are then presented, and a discussion and conclusion then follow examining both the implications of the evidence presented for our understanding of the benefits to individuals of involvement in coproduction activities and an assessment of the usefulness of the conceptual ideas utilised.
What is coproduction?
The concept of coproduction was originally introduced in the US in the late 1970s/early 1980s by Ostrom (1972) and colleagues (Parks et al., 1981) as a way of explaining the role of citizens in the production of public services. It was developed at a time of fiscal restraint when the involvement of citizens in the design and delivery of public services was seen as a potential means of increasing the effectiveness and efficiency of service provision. Following a lull in interest during the 1990s, when the market was viewed as key to driving improvement in public services, there has been a resurgence of interest. This has been associated with the development of new pluralistic models of public service, the global financial crisis and associated fiscal constraints, and the desire of scholars and practitioners to reinvigorate the role of citizens in communities (Nabatchi et al., 2017).
There remains, however, a lack of consensus on precisely what coproduction means (Brandsen and Honingh, 2016) with the concept proving highly generalisable across a variety of contexts. Nabatchi, Sancino, and Sicilia, for example, describe coproduction as ‘an umbrella concept that captures a wide variety of activities that can occur in any phase of the public service cycle and in which state actors and lay actors work together to produce benefits’ (2017: 769). Bovaird and Loeffler also adopt a broad understanding, defining coproduction as ‘professionals and citizens making better use of each other’s assets, resources and contributions to achieve better outcomes or improved efficiency’ (2013: 4).
Brandsen and Honingh (2016), however, argue rather than trying to determine one encompassing definition, several different types of coproduction can be distinguished based on the extent to which citizens are actively involved in service design. Indeed, numerous typologies and frameworks have been developed demonstrating how involvement ranges along a continuum. Nabatchi et al.'s (2017) typology, for example, considered who is involved in production and at what level (individual, group, or collective), when in the service cycle coproduction occurs, and what is generated in the process, with Steiner et al. (2023) latterly proposing the addition of where (context) and why (reasons behind coproduction) to this typology. Osborne et al. (2016) conceptualise coproduction as intrinsic to the process of public service delivery, where service user involvement can be both an involuntary or voluntary effort.
The use of conceptual tools by scholars, such as Arnstein’s ladder of participation (1969), to help examine the extent to which service users are meaningfully engaged in coproduction processes has also been highlighted (Mazzei et al., 2020). Relatedly, Brudney (2020) has proposed a new framework, the ‘Coproduction Amplification Model’, which focuses on the expansion of citizen involvement in, and the effectiveness of, coproduction.
This paper focuses on a case of the public management perspective of coproduction, the foundations of which originated with the work of Ostrom (1972), where involvement is active and voluntary and not just an inherent aspect of public service delivery, and where the aim is to deliver public services ‘in an equal and reciprocal relationship between professionals, people using services, their families and their neighbours’ (Boyle and Harris, 2009: 11). TM as an example of this type of coproduction is discussed below.
The emerging evidence-base on the outcomes of coproduction
The involvement of citizens in coproduction has been promoted as aligning services with user preferences, producing more efficient services (McMullin, 2023), and in turn improving ‘public value’ and outcomes for citizens. Based on this assumption, impressive claims have been made of the transformative effects of coproduction. It has been argued coproduction can break the top-down alignment of policymaking and service delivery and provide a source of revitalisation of democracy (Vamstad, 2012). Jo and Nabatchi (2019) highlight theorised benefits in terms of increases in efficacy, empowerment, knowledge and self-esteem.
Indeed, the potential for coproduction to improve service quality, empower, democratise and generate outcomes for individuals have been routinely emphasised, yet the assumption that coproduction leads to better outcome for users has largely been untested, with ‘the actual and potential impact of coproduction [is] as yet only sketchily researched’ (Loeffler and Bovaird, 2016: 1013), with its conceptual ambiguity and elasticity contributing to this lack of evidence on impacts (Durose et al., 2017). Certainly, there has been a lack of critique about coproduction in general. In recognition, Dudau, Glennon, and Verschuere have called for ‘constructive disenchantment with the magic that surrounds co-design, co-production and value co-creation in public services’ (2019: 1577). This includes drawing on more empirical evidence about the potential positive and negative effects.
In terms of existing evidence, a 2015 systematic review (Voorberg, Bekkers, and Tummers), found a dominance of studies aimed at understanding the coproduction process and a dearth examining outcomes. In over half of the studies, no specific objective as to why coproduction is important was mentioned. Most studies were also single or comparative case studies and qualitative in their approach. More recent work examining coproduction in healthcare (Marsilio et al., 2021), also found outcomes still mainly investigated through qualitative methods, while a review into the outcomes of public service co-creation (Acar et al., 2023) concluded the claim outcomes of co-creation are only scarcely studied remains valid but is a topic increasingly receiving attention.
Certainly, a body of work utilising quantitative approaches to examine individual-level effects of participation in coproduction has started to emerge. Jo and Nabatchi (2019), for example, found evidence of positive increases in issue awareness, empowerment and trust among those involved in collective coproduction, while Lindenmeier et al. (2021) demonstrated significant indirect effects of coproduction in public service organisations on client satisfaction. Cicatiello et al. (2023) recently used survey data to demonstrate a positive correlation between parents’ coproduction in online schooling and their satisfaction with schooling services, while Kang and Van Ryzin (2019) used survey experiments to examine the effects of coproduction on trust in government, although they found little evidence that coproduction leads to greater trust.
More critical accounts, examining the potential negative effects of coproduction have also begun to materialise, such as Williams et al.'s (2016) and Steen et al.'s (2018) examinations of the ‘dark side’ of coproduction. The notion of value co-destruction has also begun to be utilised in the context of public service delivery to help understand how value can be diminished by coproduction processes (Cui and Osborne, 2022; Engen et al., 2021; Osborne et al., 2016). A lack of transparency, mistakes, a lack of bureaucratic competence, and an inability to serve have, for instance, been identified as causes of value co-destruction (Engen et al., 2021). The potential dysfunctional aspects of coproduction have also been highlighted by Loeffler and Bovaird (2020), who argue the case for coproduction cannot be judged fully while research on potential dysfunctions remains minimal.
In practice, coproduced services have included the following: user-run and user-led organisations, peer support projects and services which involve users in designing and delivering services to other service users and professionals (Egdell and Graham, 2017). Evidence of coproduction activities between young people beyond school age and professional staff, however, is relatively rare (Bashir et al., 2014). What evidence there is, suggests the most common methods utilised are group discussions, forums and councils and conferences. The case of the TM programme, therefore, helps fill a gap in our knowledge of the involvement of young people in coproduction and more marginalised groups where there is limited understanding about how coproduction works in practice (Cullingworth et al., 2024 provides an exception). It also provides a valuable contribution to the literature on coproduction in employment services and programmes, where there are also minimal studies (Lindsay et al., 2018 provides a rare example).
The prevailing view of the literature is that coproduction leads to improved outcomes for service users. Yet, the theoretical basis for coproduction remains under-developed (Mayer and McKenzie, 2017). Wider literature examining the nature of value creation in public services, has, however incorporated coproduction into its conceptual thinking which the next section explores.
Value creation in public services
Value is a fuzzy, multidimensional and subjective concept (Dudau et al., 2023; Strokosch and Osborne, 2020). Within the PAM literature, the public service logic (PLS), drawing on service management theory, sees value as encompassing the benefits, both private and public, that individuals and society derive from public services. The intersection between public and private value creation has been discussed (e,g. Alford, 2016; Dudau et al., 2023), with PSL scholars tending to focus on the creation of value at the level of the individual as the foundation of public service delivery (Cui and Osborne, 2022). Scholars have also argued that to understand value creation, we should focus not just on the production of public services, but also their use/consumption (Alford, 2016; Osborne et al., 2021).
Indeed, how value is created and the extent to which value is co-created is debated. Trischler et al. (2023) point to the influence of service research on the PSL, specifically the service logic (SL) and service-dominant logic (SDL), and how both define value as something that is created during use, yet differ on the concept of value co-creation: SDL sees value as always co-created, while SL affirms direct interactions are needed for co-creation to happen. In PSL there has been a tendency to adopt the latter focus: co-creation of value occurs within the production processes of public service delivery through the interaction of service users, providers and other stakeholders in co-design and coproduction, while service users can create value in their own right through their use of a service (Osborne et al., 2021). However, as Ongaro et al. (2025) emphasise, there is no clear consensus on the definition of co-creation of value.
PSL theorists have further drawn on service-research, specifically SDL, in adopting an ecosystems approach (e.g. Engen et al., 2021; Osborne et al., 2022; Petrescu, 2019). In SDL, the service ecosystem lens was introduced by Vargo and Lusch (2016), who argued that value is not created through the activities of a single actor or between a firm and its customers but among a whole host of actors at various levels of aggregation. This service ecosystem approach has since been adopted by PSL scholars to examine the multi-actor and multi-level nature of value creation in public services.
Osborne et al. adopted an ecosystems approach to develop an overarching framework for understanding value creation (2022). In this framework, value creation is understood as taking place at varying levels within a public service ecosystem (macro, meso, micro and sub-micro), with coproduction situated within several other value-creation processes. Specifically related to coproduction, they see value created at the meso-level for public service users, other stakeholders, and public service officials by their engagement in the processes of co-design/coproduction of public services – value-in-production. Here co-design is ‘the collaborative engagement of the key stakeholders (such as public service users, their carers and family, and public service staff) in the design and/or re-design of a public service’ (2022: 637), while coproduction is ‘the collaborative involvement of these stakeholders in the management and delivery of a public service’ (2022: 637). Value created may take the form of an accrual of skills and/or gains in confidence through involvement in coproduction processes. For example, through involvement in a steering group or a panel and working together with others, users may develop interpersonal skills and build their self-esteem and confidence, which may also positively impact their wellbeing. However, as highlighted in the preceding section, value can also be diminished by coproduction processes, i.e. members of a panel may not believe they are being listened to, which could have negative impacts on their confidence and willingness to engage in future coproduction processes or with services more widely.
More generally at the micro-level, for Osborne et al. (2022) value is also created as a direct result of the use of a public service, with users and other stakeholders integrating resources created in the production process (i.e. public services) with their own needs, experiences, and expectations to create value in their lives - value-in-use. Again, this concept can be traced to service research and SL which considers value as created by the user, or emerging for the user, during the usage of resources and processes: value-in-use evolves for the user as the usage process proceeds and it may also take negative, destructive turns (Grönroos, 2019). As Engen et al. (2021) highlight, when users do not possess the necessary resources, and/or when providers fail to make service offerings that their users can make sense of and employ to create value, value will be destroyed. The focus here is on the experience of a public service, rather than outcomes, but Osborne et al. (2022) point to growing evidence these are inextricably linked. Positive service experiences and satisfaction are important not only in their own right, but also because they drive public service outcomes (Osborne, 2020).
Value may also be added or destroyed at the micro-level from how a public service impacts upon the needs of a service user, in the context of their life experiences/expectations – value-in-context. The concept, once more adapted from service research, denotes a constellation of sub-processes in the user’s sphere. Osborne et al. (2021) have detailed these as follows: the individual/societal values and personal life experiences a user brings to a public service encounter; how a public service addresses the social and economic needs of the service user; how a user’s previous experiences of a service frames future expectations of that service; and the impacts of a service on the whole-life of a user, beyond the intended impacts of the service. Value may be created here in the form of both short and long-term service outcomes.
Hypotheses
Taking the conceptual ideas of value-in-production, value-in-use and value-in-context, it seems feasible to theorise that service users involved in coproduction activities may achieve better outcomes than users of the same service not actively involved in its coproduction. Those involved may experience value-in-production (e.g. by accruing skills and personal confidence), value-in-use (through their experience of the service) and value-in-context (having needs met in the context of their life experiences/expectations). Those not involved in coproduction activities may experience value-in-use and value-in-context but not value-in-production. Osborne et al.’s (2022) example of a school as an ecosystem helps illustrate how those involved in coproduction might gain additional value. Learning processes/experiences create (or destroy) value for all pupils (value-in-use), while learning outcomes will be contextualised, dependent upon pupils’ needs, expectations of the learning experience, their life experiences, and their learning skills (value-in-context). Some pupils, in addition, may act as mentors for younger pupils, which will add value-in-production to their lives by the social skills and personal confidence that this role will engender.
The accrual of skills and confidence gained through value-in-production is understood in this paper as the development of soft outcomes, that is outcomes that are subjectively assessed, such as improved life satisfaction and skills, e.g., communication, managing feelings, confidence etc. Given previous evidence on the association between soft skills and labour market outcomes (Forrest and Swanton, 2021), it is presumed the development of soft outcomes may lead to hard outcomes. Hard outcomes are defined as outcomes which can be definitively and objectively measured, such as securing employment, work placements or volunteering. The link between experience of a public service and outcomes is also assumed correct, with a user’s interaction with a service having the potential to produce both soft and hard outcomes (value-in-use). Outcomes may also be generated by meeting user needs in the context of their life experiences/expectations (value-in-context).
Of course, users may also experience value destruction through value-in-production, value-in-use and value-in-context, and outcomes may be limited or negative, for example wellbeing may decrease. Yet, the assumption in much of the literature is that coproduction leads to improved outcomes for service users and this paper offers an opportunity to examine the conceptual ideas of value-in-production, value-in-use and value-in-context empirically by testing the following hypothesis:
Service users involved in the coproduction of services achieve better outcomes than service users who are not involved.
Given previous evidence on the considerable variation in the degrees of meaningful representation of citizens in coproduction processes (Mazzei et al., 2020), and earlier work by Jo and Nabatchi (2021) which found outcomes varied by different methods of direct public participation, the following hypothesis is also proposed:
Involvement in different types of coproduction activities leads to differences in outcomes achieved.
Methodology and methods
The case setting: The Talent Match programme
To test the hypotheses outlined, the TM programme has been selected as an example of a service where a sub-set of users were involved in coproduction. TM was a £108 million National Lottery Community Fund 1 (NLCF) strategic programme addressing unemployment amongst 18-24-year-olds. It was delivered between 2014 and 2018 through voluntary and community sector led partnerships in 21 geographical areas in England. The programme sought to support young people furthest from the labour market through personalised, non-standardised provision addressing participant needs and aspirations. It recognised the cohort were not a single group but may have faced complex mental and physical health barriers, or physical and practical barriers (Damm et al., 2020). The programme aimed to facilitate pathways into secure, meaningful, sustainable employment or enterprise and had a target of assisting one-fifth into sustained employment.
TM had innovative features including the long-term duration and flexibility of the programme, its non-mandatory nature, and the acceptance that some innovative aspects of the programme may be tested even though they may fail. Most notable, however, was the commitment to placing young people at the heart of the programme’s design and delivery, setting it apart from previous youth employment initiatives and government employment programmes. This can be seen in the words of the England Chair of the NLCF when the TM programme was launched: Youth unemployment is an issue that threatens the hopes of an entire generation as well as our economic wellbeing. Talent Match promises to help the very hardest to reach young people to make the most of their skills and ambitions, not just as an end goal, but by giving them a say in how the programme itself is designed and put into practice.
As discussed, coproduction remains a contested concept but can be understood as a continuum. Given the person-centred approach of TM, arguably all participants, to a greater or lesser extent, coproduced their own employment journeys through close engagement with key workers. In this paper, however, the focus is on the involvement of young people in a specific and defined set of activities focused on programme design and delivery, which aimed to inform, influence or lead the activities of TM partnerships. The results and analysis section details the specific types of activities participants were involved in; however, involvement was to be active and voluntary with services designed and delivered in partnership with young people, fitting with the public management perspective of coproduction outlined above.
Regarding why young people were to be involved in coproduction, an evidence review produced for the NLCF ahead of the implementation of the programme found broadly three rationales for involving young people in policymaking and planning for service delivery: ‘practical benefits to services and service delivery; benefits related to citizenship and social inclusion; and benefits related to the personal and social development of the young people involved, as well as of the staff involved in promoting young people’s involvement’ (Bashir et al., 2014: 3). TM partnerships thus devoted considerable time, resources and energy to the involvement of young people in coproduction and saw it as a central aspect of provision (Damm et al., 2020).
The focus in this paper is on exploring the rationale that coproduction results in benefits (or value) related to the personal and social development of those involved, here measured in terms of achieving soft and hard outcomes. It seems reasonable, theoretically, to assume that those most actively involved in coproduction would experience the most benefit, as the relationship between coproduction and outcomes would be strongest here. This paper therefore focuses on the involvement of a sub-set of TM participants in a specific set of coproduction activities. The availability of unique longitudinal programme evaluation data, able to track outcomes achieved both by this sub-set of users and the wider programme population, was also central to this focus and to the selection of TM as the case setting and its ability to address the hypotheses outlined.
Data sources and study design
Data from the evaluation of TM forms the evidence for this paper. Self-reported measures collected through a Common Data Framework (CDF) which all young people engaged on TM were expected to complete, and data collected by TM partnerships detailing which young people were involved in coproduction activities, and the scope of this involvement, have both been utilised. Ethical approval for the evaluation and associated data collection was granted via university ethics review. Data was collected on young people only after they had started to work with a key worker or mentor in a delivery partner organisation. The purpose and use of the data collection was explained to them, and their consent obtained via completion of questionnaires also detailing how their data would be utilised.
The CDF was designed in the form of an online questionnaire. Data were collected at a baseline stage (on entry to the programme) and then at three, six, 12, and 18 months. The participant data allowed monitoring of who had participated in TM, what they had done, what difference it made to them, and the impact it had on their labour market outcomes. The analysis focused on responses received to the CDF questionnaires up until the end of December 2018 when the main period of programme delivery ended, and CDF data collection ceased.
The data collected by partnerships focused on those engaged by the programme up to the end of December 2017. Partnerships were provided with lists of young people engaged by this point and recorded on the CDF and asked to identify who had been involved in coproduction and in what ways 2 . One partnership was unable to provide any information, so their participants do not feature in the analysis. There were also five partnerships that indicated only where young people had been most heavily involved, for example through membership of a youth panel or as a ‘youth ambassador’. For the purposes of analysis those not indicated as being heavily involved have presumed to have not been involved in coproduction at all. The descriptive analysis was also run with these cases excluded for reference and the results were similar. In addition, two partnerships provided information on a section of young people involved in coproduction rather than every young person listed. Where no information was provided, cases have been excluded from the analysis.
Data collated by partnerships has been linked to CDF responses to assess whether involvement in coproduction of the programme led to better outcomes for those involved compared to those not involved (H1), and if there was any relationship between types of coproduction activities and outcomes (H2). Initial CDF baseline data was collected on 25,885 participants. Of these, 67% completed a follow-up response at 3 months, 56% at 6 months, 42% at 12 months and 30% at 18 months. The analysis presented is based on follow-up CDF returns from 6 months onwards and where cases were able to be linked to partnership data. Descriptive analysis on the characteristics of those involved or not is based on a total of 20,190 responses (1497 involved in coproduction, vs 18,693 not involved). Analysis examining outcomes is based on if these were recorded at any of the six-, 12- or 18-month follow-up time points (3-month responses have not been included to allow outcomes to emerge/be sustained). In total, 13,233 responses have been used in the analysis examining outcomes (1382 involved in coproduction, vs 11,851 not involved). Information beneath individual Tables details responses included in specific analyses.
The analysis presented updates an earlier assessment published elsewhere (Bashir et al., 2018), utilising the final CDF dataset to provide a more complete appraisal of outcomes achieved. In situating the analysis within an emerging body of empirically informed critical reflections on coproduction and value creation, this analysis also offers new insights into the relationship between coproduction and outcomes through the example of the TM programme.
Methodological limitations
It is important to note that the data collected by partnerships is unable to show when young people were involved in coproduction and for how long. For future research in this area, it is recommended data is collected on the duration and intensity of participant engagement in coproduction activities, for example the number of youth panel sessions participated in, or the length of time served as a peer mentor. This would, however, need to avoid being overburdensome e.g. if, as in this study, service providers are required to collect data, their capacity to do this would need to be considered. As indicated above, there were also some inconsistencies in how partnerships recorded involvement in coproduction activities. There are additional other limitations. There will, for example, be young people excluded from the sample as their follow-up data is incomplete. The CDF will also suffer from time-aggregation bias, with transitions occurring between different survey stages not picked up. The dataset also focuses on a particular group engaged on a particular programme and cannot tell us about the experiences of people involved in coproduction more widely. Data collection was also designed for the purposes of the programme evaluation and not to test the hypotheses and conceptual ideas outlined in this paper.
The methodological approach taken is also unable to establish causality between involvement in coproduction and outcomes. The focus here has been an exploration of the relationships between coproduction activities and outcomes for individuals. Future studies could look to isolate the impact of involvement in coproduction and seek to ascertain causal relationships, for example via quasi-experimental methods such as propensity score matching. Nevertheless, the data provides a rare opportunity to explore the relationship between coproduction and individual-level outcomes and to test conceptual ideas around value-creation for service users.
Results and analysis
This section tests the hypotheses outlined above.
Extent and type of involvement in coproduction
Types of coproduction activities.
Base: 1497.
We can take the activities in Table 1 and attempt to link them to existing typologies of coproduction in the literature (e.g. Nabatchi et al., 2017; Steiner et al., 2023). When considering ‘who’ was involved and at what level, participants were engaged both as individuals and as groups as lay actors, and this varied across and within activities. For example, both individuals and groups of participants had a remit to engage young people in TM and deliver peer support and training. Regarding the ‘when’ - activities occurred across the service cycle. TM supported a ‘test and learn’ approach, with partnerships encouraged to adapt the ways they delivered activities or services. Participants were engaged in the design and re-design of programme activities, the commissioning of new services, programme delivery and assessment. In terms of ‘what’ coproduction involved, activities were voluntary and intended to inform, influence or lead the activities of partnerships. ‘Where’ activities took place was within lead partner and delivery partners organisations in the context of an employment programme, and ‘why’ was the commitment to placing young people at the heart of the programme’s design and delivery.
Clusters of types of coproduction activities.
Characteristics of young people involved in coproduction
Personal characteristics of young people involved in coproduction.
Base: 1303–1497 (involved); 16,743–18,693 (not involved).
Results are based on two-sided tests with significance level .05. For each significant pair, the key of the category with the smaller column proportion appears in the category with the larger column proportion.
Descriptive assessment of outcomes secured
Outcomes achieved.
Base: 1349–1382 (involved); 11,616–11,851 (not involved). Base is lower for Satisfaction with life no longer a ‘low score’: 434 (involved); 3198 (not involved).
Results are based on two-sided tests with significance level .05. For each significant pair, the key of the category with the smaller column proportion appears in the category with the larger column proportion.
A z-test for proportions was used to test for differences between the groups in the percentage achieving outcomes. No statistically significant differences were identified in the proportions securing employment or sustained employment. A statistically greater proportion of those involved in coproduction had, however: secured an apprenticeship or formal education; secured a work placement or volunteering; recorded a higher wellbeing score (of those who recorded a ‘low score’ at baseline); and recorded higher scores across all six My Journey measures.
An initial assessment of outcomes achieved by type of coproduction activities (project management, frontline delivery and evaluation and communications) identified some differences, however, most young people were involved in coproduction in more than one way 6 . Therefore, logistic regression analysis was used to attempt to unpick the importance of type of involvement in coproduction on outcomes.
Modelling
Results of logistic regression – importance of involvement in coproduction in outcomes achieved.
Base: 12,005-12,126. Base is lower for Satisfaction with life no longer a ‘low score’: 3305.
The analysis found no statistically significant association between being involved in coproduction and securing sustained employment, but found statistically significant associations between being involved in coproduction and the following, after controlling for all other variables: • securing employment: on average young people involved were 1.15 times more likely to have secured employment than those who had not been involved. • securing an apprenticeship or formal education and securing a work placement or volunteering: the strongest association was with securing a work placement or volunteering. On average young people involved were 3.07 times more likely to have secured this outcome than those who had not been involved. • an improved wellbeing score: on average young people who had initially recorded a ‘low score’ on the ONS satisfaction with life measure were 1.47 times more likely to have recorded a higher score at a later stage than those who had not been involved. • improved scores across all six My Journey measures: on average young people involved were between 1.2 and 1.42 times more likely to have reported a higher score than those who had not been involved.
Another series of models were run examining the importance of involvement in the different types of coproduction activity and their association with outcomes achieved. Involvement in each type of activity was compared to not being involved this way, either through involvement in a different way or not being involved at all. The same demographic variables were included, along with TM partnership and the proximity to the labour market measure at baseline. Analysis did not find a clear pattern in terms of which forms of coproduction activities were most associated with outcomes, and the analysis has therefore not been presented, although a greater number of statistically significant associations were found between being involved in frontline delivery and evaluation and communications compared to project management (there were, however, no associations found between any of the three types of involvement and securing employment or sustained employment).
In summary, the findings presented partially support the hypothesis that service users involved in the coproduction of services achieve better outcomes than service users who are not involved (H1). Analysis found no statistical association between involvement in coproduction and sustained employment, and the association identified between involvement and securing employment was relatively weak. Stronger associations were, however, found between involvement in coproduction activities and securing an apprenticeship or formal education, and most notably, between involvement in coproduction and securing a work placement or volunteering. It is worth noting, however, that young people’s involvement in coproduction itself might have been considered as volunteering or work experience by partnerships. Being involved was also associated with improved wellbeing and skills. In considering the second hypothesis that involvement in different types of coproduction activities leads to differences in outcomes achieved (H2), no firm conclusion can be drawn from the evidence presented, with no clear pattern found in terms of which forms of coproduction activities were most associated with outcomes.
Discussion and conclusions
The empirical evidence presented suggests that those engaged in the coproduction of the TM programme gained value from their involvement, most notably by expanding their skill set and improving their wellbeing, although this did not translate into improved employment outcomes compared to their peers.
This group could potentially experience more positive long-term outcomes. Analysis in this paper focuses on data collected between six to 18 months after initial engagement with the TM programme, so a relatively short period. Benefits could emerge over a longer period which have not been captured. Future studies might consider tracking participants for longer, although attrition in the CDF responses points to the challenges with this, particularly with those from more marginalised groups.
Personal characteristics might also provide an explanation for the absence of a link between involvement in coproduction and sustained employment. Those involved were more likely to have a limiting disability, and further analysis finds those involved in coproduction were less likely to have applied for jobs or gained employment before engaging with the TM programme, suggesting there may be significant barriers to employment faced by this group. There may also be an element of selection bias, with partnerships not engaging those likely to quickly achieve employment outcomes in coproduction activities. This finding supports the notion of value-in-context – user needs, their backgrounds, prior experiences and expectations are essential to how a user interacts with a service and gains value.
If we return to the overarching framework conceived by Osborne et al. (2022), coproduction is situated within a wider ecosystem of value-creation processes. It is possible some users not involved in coproduction gained more value from their interaction with TM (value-in-use) than those who were involved. Other forms of support provided by the programme outside of coproduction activities, such as information, advice and guidance about careers and support in addressing practical barriers, might have been more influential in producing employment outcomes. As has been highlighted, coproduction also has the potential to destroy value (Cui and Osborne, 2022) so it is possible some young people involved in coproduction activities may have had a negative experience with this process.
While Osborne et al. (2022) discuss value-in-production for service users in terms of their engagement in coproduction activities, they also describe value being added to a public service at this meso-level through service improvement. Therefore, while those not involved in coproduction may not gain value at this level, they may subsequently experience value-in-use at the micro-level due to an improved service offer. Specifically, improvements made to the design and production of the TM programme, including through the involvement of TM participants in coproduction, may have led to value for the wider TM cohort at the point of use. While the analysis here cannot determine if this is the case, it does help illustrate the interaction between the different levels of Osborne et al.’s framework (2022), something they have called for further work to focus on.
Indeed, in its current conception, the framework has been criticised for being too user-focused and therefore too narrow to fully understand value creation in public services (Kinder and Stenvall, 2023; Trischler et al., 2023). As discussed earlier, the extent to which value is co-created has been subject for debate. Trischler et al. (2023), see Osborne et al.’s framework as being limited by an emphasis on the user as the ‘value creator’, and the provider as the ‘value facilitator’, arguing it should be open to the opportunity of both taking on different roles in the co-creation of value. This study has seen value creation through value-in-use and value-in-context as primarily in the user’s sphere through using/consuming a service, and the co-creation of value occurring within the production processes of public service delivery (value-in-production). However, as Osborne and colleagues have themselves indicated elsewhere (Osborne et al., 2021), the production and consumption of service delivery are dynamic processes, warning against creating an unnecessary dichotomy. They argue that conceptualising both sets of processes allows for them to be explored and evaluated, with the disaggregation of concepts an essential precursor to their integration. The possibility that the involvement of TM participants in coproduction may have led to value for their peers, points to this dynamism.
In addition, while the focus here is on the meso and micro-levels of a public service ecosystem, and specifically value creation for service users, the wider societal context is also critical. Scholars have emphasised the importance of context in how we evaluate and understand the impacts of coproduction and the relationships between initiatives and outcomes (Brix et al., 2020; Loeffler, 2020). For Trischler et al. (2023), levels are not fixed in a service ecosystem approach and can be changed depending on the phenomenon of interest. Thus, to get a complete picture, one must zoom in to investigate micro-level phenomena and zoom out to investigate macro-level phenomena. In this paper the focus has primarily been on value creation for individual service users through service production and use, but zooming out, the wider labour market context was also important to the extent value was created at a user-level but also at the programme and societal levels.
TM primarily focused on pre-employment support, and while almost all partnerships performed some form of job brokerage, there was less consistency in terms of job creation activities and demand-side interventions. The TM model, therefore relied on a labour market that was working reasonably well to take up the supply of work-ready young people. Regardless of whether young people had been involved in coproduction activities or not, if the local labour market was weak, arguably there was only so much coproduction could achieve. Indeed, the majority of both those involved in coproduction and not involved did not secure employment. While individuals do appear to have gained value from their involvement, coproduction was clearly not a panacea for the individuals involved or the wider programme cohort in terms of tackling the problems of unemployment faced by these young people. In addition to the focus on coproduction, the programme might have benefited from paying more attention to the demand-side of the labour market. This could have included fuller engagement with employers in partnerships and delivery, and more extensive provision of in-work support, recognising the risks of participants not sustaining employment. That relatively little attention was paid to demand-side considerations may have limited employment outcomes for participants.
The TM partnerships themselves and any differences in how they approached coproduction with young people, and their structures and management, are also likely to have played a role. This paper has focused on the relationship between coproduction and outcomes after controlling for other factors. Which partnership a young person had worked with was included in the models run to account for different labour market contexts, however the importance of the TM partnerships and their structures and local labour market contexts, has not been fully explored. While some lead partners were youth organisations with extensive experience of involving vulnerable young people, others had more limited experience having more of a function leading and managing contracts. There was therefore an inevitable diversity of approaches towards involving young people. An initial re-assessment of the models discussed above finds a range of significant associations between working with different partnerships and achieving outcomes. This points again to the importance of value-in-context and how a public service addresses the social and economic needs of the service user (Osborne et al., 2021). That some lead partners were more experienced in engaging vulnerable groups likely had an effect, not only the nature of coproduction activities, but programme delivery more widely. A more in-depth assessment of the contextual factors affecting coproduction than is possible here would be useful in developing our understanding of the effects of coproduction.
To conclude, the concepts of value-in-production, value-in-use and value-in-context have proved useful tools for examining the relationship between involvement in coproduction activities in the TM programme and outcomes achieved by individuals. However, the dynamic nature of these concepts and the processes they represent, need to be considered when applying them. This paper theorised that service users involved in the coproduction of services achieve better outcomes than service users who are not involved, due to the additional value they experience through value-in-production. The evidence presented suggests, in the case of the TM programme, this appears to be true, although there were limits to this. The concepts of value-in-use and value-in-context have also been beneficial in exploring why there were limits, and the interaction between value at different stages and levels e.g. how value created through production can feed through to value-in-use. This also points to the interdependencies of value creation across different levels of an ecosystem and the multiple actors involved.
Critically, viewing coproduction within a wider ecosystem of value-creation processes, has provided an important lens for tempering the claims often made of the transformative effects of coproduction. In the context of a literature presenting many theorised but untested benefits (Jo and Nabatchi, 2019), more realistic, detached, evidence-based assessments of the actual and potential benefits of coproduced approaches are required. This paper has provided a rare opportunity to test the assumption that involvement in coproduction activities leads to improved individual outcomes and in doing so, contributes to emerging empirically informed critical reflections. Although the dataset utilised cannot tell us about the experiences of people involved in coproduction more widely, it has provided a robust and unique counter-factual assessment of the outcomes achieved by participants who were, and were not, involved.
With only limited empirical work undertaken to test conceptual ideas around value creation, the findings of this paper will also be of interest to PAM scholars examining the nature of value creation for and by public service users, and those looking to operationalise theoretical work in this area. The focus here has been value creation for service users of an employment programme at the meso and micro-levels of a public service ecosystem. As indicated above, more work on the contextual factors affecting coproduction would be useful but also the role of wider actors. As alluded to above, there has been criticism that Osborne et al. (2022) and other PSL scholars’ focus on the user dismisses the value creating role of public service professionals and staff (Kinder and Stenvall, 2023). Future work examining the role these individuals play within the public service ecosystem empirically would be useful to test the concepts explored here further and their real-world application. This paper has shown coproduction can create value for service users, but that other interactive processes within an ecosystem can be more important for outcomes e.g. local labour market conditions. Policymakers and researchers should thus prioritise the importance of context in the implementation and assessment of coproduced initiatives.
Footnotes
Acknowledgements
Thanks are due to the National Lottery Community Fund for funding the Talent Match evaluation. The views expressed here are those of the author alone and do not necessarily reflect those of the National Lottery Community Fund. Thanks are also due to the 21 Talent Match partnerships collecting data on participants and to the members of the evaluation team, in particular: Chris Damm, Sarah Pearson, Peter Wells and Ian Wilson at the Centre for Regional Economic and Social Research at Sheffield Hallam University and Ryan Powell at the University of Sheffield (formally at Sheffield Hallam University).
Ethics considerations
Ethical approval for the evaluation and associated data collection was granted via Sheffield Hallam University ethics review. Data was collected on young people only after they had started to work with a key worker or mentor in a delivery partner organisation. The purpose and use of the data collection was explained to them, and their consent obtained via completion of questionnaires also detailing how their data would be utilised. This included being used for other research purposes including in journal articles.
Funding
The author disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The author is part of the consortium which undertook the national evaluation of the Talent Match programme funded by the National Lottery Community Fund.
Declaration of conflicting interests
The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Data Availability Statement
The participants of this study did not give written consent for their data to be shared publicly, so due to the sensitive nature of the research supporting data is not available.