Abstract
As part of a large pan-European project on co-creating public services we supported the design of a programme in England that attempted to operationalise research on desistance, through a model of co-created, strengths-based working. We then evaluated its implementation and impact. The programme was implemented in a Community Rehabilitation Company. It was delivered in the context of rapid organisational change, often in response to rapidly changing external events and a turbulent policy environment. These factors impeded implementation. An impact evaluation did not identify a statistically significant difference in re-offending rates between the intervention group and a comparator group. However, in-depth qualitative evaluation identified positive examples of co-production and co-creation, with individual case managers and service users supportive and noting positive change. Taken as a whole our findings suggest that a co-created, strengths-based model of probation case management is promising but needs to be accompanied by wider systems change if it is to be embedded successfully.
Introduction
As part of a large pan-European project on co-creating public services and building on previous piloting work in the UK (Fox et al., 2018), we supported the design of My Direction and then evaluated its implementation and impact. 1 My Direction sought to implement personalised, co-produced and co-created models of case management in a probation setting in England. Underpinning this research was an interest in exploring ways to ‘operationalise’ desistance theory, within which personalisation and co-production are strongly implied. In this paper we first set out the key concepts and theory that framed the project before going on to describe My Direction and our evaluation methodology. We then set out key findings from the evaluation before moving on to discuss these findings and their implications for operationalising desistance theory in criminal justice settings as well as for wider public service reform.
Desistance, personalisation and co-creation
My Direction was an attempt to operationalise desistance, drawing on concepts of personalisation, co-production, co-creation, and strengths-based working. Research and theory about why and how people desist from offending has become increasingly influential in the English criminal justice system (McNeill et al., 2012; Ward and Maruna, 2007). McNeill (2009: 28) argues that desistance thinking suggests that, when it comes to rehabilitation “One-size-fits-all processes and interventions will not work”. However, a challenge posed by desistance research is that it is “not readily translated into straightforward prescriptions for practice” (Weaver and McNeill, 2010: 6) although this is not necessarily problematic, because developing a prescriptive model of practise would undermine the personalised approaches it implies.
Desistance implies personalised approaches are required where tailored life plans that recognise an offender's assets as well as their criminogenic risk factors are central (McNeill, 2009). This requires new approaches to assessment and sentence planning, new training for staff and rethinking the language of practise (McNeill et al., 2012). In the UK, personalisation is most developed within the health and social care sectors. Fox et al. (2013) have argued that the criminal justice sector could learn from social care when considering the challenge of reforming the criminal justice system and developing innovative approaches to offender rehabilitation.
Personalisation is closely linked to strengths or asset-based ways of working. The starting point for many public services is that they try to fix things for people in the short-term or encourage them to take action that fits the service's priorities, not their own (Wilson et al., 2018). This is a deficit-based approach that: [L]eaves people without clarity about the changes they want to make or the knowledge, confidence or support to get there. It often only addresses a single (and often most visible) aspect of people's lives, without taking account of what else is going on. (Wilson et al., 2018: 5)
Wilson et al., characterise this as ‘bad help’. This can be ineffective in a number of ways including failing to identify the underlying issue that led to the person accessing the service, and failing to share power and responsibility with the result that people feel disempowered (or ‘done to’) reinforcing inaction and dependency. Instead, public services need to adopt asset or strengths-based approaches, something that is often assumed in the for-profit sector. In contrast, strengths-based approaches explore, in a collaborative way, the entire individual's abilities and their circumstances rather than making the immediate issue that brought them to the service the sole focus of the intervention (Baron et al., 2019). Strengths-based approaches do not impose the same structure on diverse communities. Instead, they support citizens’ development of their capacity and their opportunities to exercise agency in undertaking small acts that build meaningful relations (ibid.). These can make huge differences in people's lives. This implies that services should be personalised and contextualised by community, asking questions such as ‘what matters to people?’ and not ‘what is the matter with them?’ (Prandini, 2018).
Along with personalised and strengths-based approaches there has been an associated shift towards the idea of public services that are co-produced between citizens and front-line staff (Needham, 2008). Indeed, co-production can be said to have “come to redefine contemporary social and public policy” (Bevir et al., 2019: 179). Co-production has travelled across service domains beyond its strongholds in the care and health sectors although negotiating meaningful co-production in the criminal justice system presents many challenges (Weaver, 2011, 2012). Co-production overlaps with the more recent term ‘co-creation’, defined as the “active involvement of end-users in various stages of the production process” (Voorberg et al., 2015: 1335). Osborne and Strokosch (2013) and SCIE (2015) regard co-production as limited to implementation without requiring user involvement in the shaping and planning of services, as co-creation does. Despite widespread enthusiasm and support for co-production and co-creation as humane and inclusive, some critical voices warn of tokenism and failure to fully recognise imbalances of status and power (Bevir et al., 2019). Crompton (2018) observes that idealised narratives of collaboration and empowerment may not be borne out in the experience of less powerful stakeholders. Dudau et al. (2019) suspect that the so called ‘co-paradigm’ lacks coherence and substance. Co-creation has antecedents in the private sector seeking to harness the expertise of customers for commercial advantage (Brandsen and Honingh, 2018). However, such models may be less appropriate in the public sector. For example, retention of customers is typically important for profitability but ‘repeat business’ is likely to be a sign of failure in public services (Osborne, 2018). Moreover, the reality of unwilling or coerced ‘customers’ is unfamiliar to the for-profit sector but quite common in public services (Ibid.). The My Direction pilot presented an opportunity to explore how co-production and co-creation would be received when service users were mandated.
My direction
My Direction was piloted in a Community Rehabilitation Company (CRC) managed by Interserve. CRCs were created as part of the UK government's ‘Transforming Rehabilitation: A Strategy for Reform’ policy initiative (Ministry of Justice 2013, see also Albertson and Fox (2019). The pilot took place in one large city centre probation office. Prior to the pilot Interserve had developed its ‘Interchange’ model of case management that all case managers (a mix of probation officers and probation support officers) were already using, this had a strong emphasis on personalised and strength-based working (Interserve, 2016). The Interchange Model had three phases of case management: Interact, covering strengths based assessment and initial sentence plan; Intervene, covering interventions to deliver the sentence of the court and relationship building and Integrate, covering support to service users to help them develop personal and community networks for community integration.
My Direction built on the learning from personalisation in social care (Fox and Marsh, 2016) and earlier small-scale ‘proof of concept’ pilots that tested different elements of person-centred practise and co-produced working (Fox et al., 2018). Findings from the ‘proof of concept’ were incorporated into a larger scale and more holistic model that covered all aspects of sentence planning and supervision. An important element in the development of My Direction that supported co-creation of the service was the use of Community Reporting undertaken by People's Voice Media. Community Reporters were trained in storytelling. Eleven members of staff and peer mentors captured approximately 40 stories from staff and service users about their experience of probation services. Community Reporters were supported to analyse the storeys and deliver participatory workshops in which findings were showcased to CRC managers, front-line staff. Community Reporters and CRC staff then determined how storeys could be utilised within the CRC to support personalised service delivery.
My Direction encouraged Case Managers to work with service users to co-produce and execute a support plan to help service users achieve their goals. Staff were trained in person-centred practise and tools such as the Three Conversations Model 2 were adapted to provide a structure for person-centred working. Case Managers were to utilise person centred practise to co-produce a rehabilitation plan using professional discretion to tailor assessments, planning and supervision to the holistic needs of the service user. The rehabilitation plan also needed to ensure that the sentence of the court was delivered in line with requirements. Case Managers received training and had access to ongoing support throughout implementation.
Increasing volunteering by service users in the CRC was an element of the programme and seen as a mechanism for encouraging co-production and co-creation. As part of the pilot, eligible service users had access to a range of interventions outside of conventional services. Interventions were designed and identified to facilitate co-production and personalised working. They included resources aimed at increasing accessibility and promoting integration into the community through the creation of pro-social networks, such as an Enabling Fund, a sum of money provided by the CRC for service users to purchase good and services, Life Links, a volunteer support network for individual service users, Life Coaching and Time Banking. Some interventions were developed ‘in house’, whereas others were pre-existing services delivered by charities and community groups and identified as complementary to personalised working. The programme also intended to use a mobile app to compliment service provision and enhance the service user experience.
My Direction was launched in May, 2018 and ran for 9 months. Ten case managers were trained and 84 service users started the programme.
Methodology
The evaluation team worked closely with Interserve during the intervention design making use of empirical findings from earlier pilots (Fox et al., 2018), knowledge of relevant theory on desistance and a review of previous research on co-production and co-creation (Voorberg et al., 2015). Evaluation findings were reported and presented periodically to project leader and senior management throughout implementation. Case Managers who took part in the evaluation were invited to attend sessions aimed at generating awareness of the evaluation, sharing interim findings and to encourage research participation. Reporting of fieldwork findings acted as a feedback loop that helped guide implementation and provide programme managers with greater oversight of programme activities.
An implementation evaluation of the programme took place between May 2018 and May 2019 during which time one of the research team routinely spent 2–3 days per week at the programme site. Fieldwork was undertaken in two phases, phase one ran from May 2018 to September 2018 and phase two from February 2019 to May 2019. Each phase was followed by an intensive period of data analysis and write up. Table 1 provides a summary count of those interviewed during each phase. Of the sixteen service users interviewed during phase one just five participated in follow up interviews. The high attrition rate was partly due to order non-compliance, a circumstance whereby service users were either in breach, had a warrant out for their arrest, were in the process of being resentenced or had returned to prison. Many service users led chaotic lives with periods of homelessness.
Sample overview.
Three observations of one-to-one appointments between case managers and service users were conducted. Informal observations were undertaken throughout the duration of implementation whilst the researcher was present in the probation office, these were recorded in a fieldwork diary. Qualitative data was transcribed and analysed in NVIVO. Offender manager case records were analysed for twenty offenders selected to be broadly representative of the intervention group based on gender and age. This included data from 235 scheduled face-to-face appointments between case managers and offenders case records was extracted from paper anonymised prints outs in the CRC office. Data extracted included information on appointments (e.g. outcome, duration, location and format), frequency and type of needs discussed, case manager actions, access to services, referrals and evidence of personalised or co-produced working.
An impact evaluation drew on casefile data. The evaluators analysed the characteristics of the intervention group and investigated the effect of My Direction on reoffending outcomes. These were defined as additional events (for something other than a breach) following the current one within a year of the cohort start date – when an individual started the community part of their sentence. To do this we constructed a counterfactual group comprised of individuals supervised in the same probation office with disposal dates with the same range as those in the intervention group. This was possible because not all case managers in the office were trained to work on the My Direction pilot. However, assignment of case managers to My Direction was not random (see below). Once case managers joined the My Direction programme, they took on new cases as they had capacity. As assignment to My Direction was not randomised, it was necessary to estimate its effect using the observational (caseload) data, conditioning on covariates to make the treatment and outcome independent. We estimated the ATET – the Average Treatment Effect amongst the Treated subjects (calculated by including only those who were in the intervention group and using the actual and potential outcomes of the intervention group). Caseload data were missing for some covariates (deprivation 15% missing, and ethnicity 23% missing) so two models were employed in the analysis, in order to maximise the number of observations, and the number of covariates included respectively.
Implementation evaluation
Case managers
Case managers were approached by local managers and programme leads to participate in the programme. The aim was to recruit a mixture of case managers with varying experience. All ten case managers recruited were female. Four were senior case managers and six case managers. Four of the team had been involved in the pilot stage the previous year. Case managers had a mean average of 5.8 years’ experience in their current role. Managers had the most experience having typically worked in the sector for a minimum of 10 years. Most staff who worked in probation had done so for most of their careers. Roles prior to joining probation had mostly been in people-facing support roles in the wider criminal justice or social work sectors.
The main motivation to working in probation, as described by several staff, was to work with people and to support them to move forward with their lives. Most case managers reported that they wished to develop their careers within the CRC, although they often felt that doing so was not straightforward. Case Managers reported that administrative and desk-based tasks limited their ability to engage in face-to-face work with service users and this contributed to some job dissatisfaction. Overall, we assessed the team of case managers as experienced and motivated.
Cohort
The intervention group consisted of eighty-four medium and low risk people with convictions (n = 84), of which 57% were commencing a new Community Order, 27% were on supervision in the community following a custodial sentence and 14% were serving a Suspended Sentence Order. On average, service users had 10 previous offending related events. The highest number of previous events was 68 and the mean average length of a community sentence was 11 months. The most common index offence category was violence (35%) followed by theft (non-motor) (15%) and public order (12%). Table 2 provides further breakdown of demographic and background characteristics and a comparison between treated and untreated groups. It shows that service users in the treated group were mostly male and on average 34 years old. The majority were white British (90%) meaning the cohort was not ethnically diverse.
Demographic and background characteristics based on case records.
When we compared the intervention group to the counterfactual group (individuals supervised in the same probation office with disposal dates with the same range as those in the intervention group) they were mostly balanced, however we observed some differences between the intervention and counterfactual groups (see Table 2). These included a higher proportion of the intervention group being female (35% compared to 21% in the counterfactual group); a correspondingly lower average OGRS score (40% compared to 45%); and a higher proportion being subject to Alcohol Treatment Requirements (64% compared to 48%). These differences were probably due to some case managers on the pilot only working with women.
Qualitative data provided additional insights in the lives of the service users. Many had experienced adverse childhood experiences (Felitti et al., 1998). Such experiences led to them responding to challenges with anger, aggression, self-harm and many were seeking or receiving counselling and psychological support. Many had experienced a breakdown in family relationships because of their behaviour although, for some, family provided a vital support network and were fundamental in motivating change. Some service users had lost custody of their children. Misuse of drugs and alcohol was prevalent amongst almost all service users interviewed. A significant number of service users reported that their reasons for using drugs was to self-medicate and block out traumatic experiences.
Although insecure accommodation was not an issue prominent in analysis of case records, many service users were in temporary accommodation including hostels, sofa surfing and shared housing. Younger service users sometimes lived with parents. Some people were street homeless. It was common for service users to have had disruptive education experiences. Many had functional skills deficits and learning difficulties. Mental health issues were mentioned by almost all service users interviewed. The majority of income for those we interviewed came from welfare payments and many faced financial difficulties and debt. Some had caring responsibilities that limited their opportunities for work. Almost all work that service users mentioned was low paid, hourly and temporary. When asked the majority of service users held a negative perception of the area they lived in, often identifying it as an obstacle to change, particularly in relation to drug use. Some described feeling trapped and unable to leave because of limited financial means. Those with poor relationships with others in the community reported feeling scared, worried or threatened.
Implementation in the context of transforming rehabilitation
The programme was implemented at a time of rapid change within the CRC and in the wider sector in response to the unfolding Transforming Rehabilitation policy (Ministry of Justice 2013). It was common for CRC staff encountered during the evaluation to express their disapproval of the Transforming Rehabilitation policy and its implications, including private ownership of CRCs. Such attitudes were disproportionately expressed amongst longer serving members of staff, particularly those who had experienced the split from the National Probation Service. A minority of staff were resistant to change. Those that showed resistance talked about leaving the organisation, others reflected on previous ways of working and wished for their return. The majority of staff found the pace and scale of change challenging and overwhelming. Although morale across the CRC appeared low, staff showed camaraderie and, provided practical and emotional support at times of need. Many had relationships that transcended the professional and knew one another on a personal level.
The CRC experienced high staff turnover as many left the organisation. There were difficulties in recruiting new staff and an over reliance on temporary staff. Many new staff appeared to be overwhelmed and found adapting to work conditions difficult. Problems with resourcing appeared to be exacerbated by a higher than normal rate of staff sickness. Part way through programme implementation the CRC underwent an internal restructure, a move that affected staff on all levels and had a profound impact on case managers. The restructure reduced the number of case managers on the pilot as many took up new roles or had their roles modified. Although Senior Management made efforts to minimise the disturbance to My Direction, the changes were substantial. The disruption of the restructure caused widespread reallocation of cases, including those in the intervention group to case managers who had not been trained at the start of the programme.
Although not directly linked to the restructure, over a six-month period of project implementation, the Director-level role in the CRC with overall responsibility for the programme was filled by three separate people. This was suggested by programme leaders to have had a negative effect on pilot awareness, momentum and engagement a local level. For several months key members of the local delivery team were taken off the programme to support preparation for an inspection by Her Majesty's Inspectorate of Probation. This had a negative impact on engagement from all staff involved in the pilot and was especially problematic for pilot leaders, some of whom were unable to dedicate sufficient time to activating interventions: an important objective at the time. There can be no doubt that the combined effect of these changes had a significant, negative impact on effective project implementation.
Implementing a co-produced and strengths-based approach to case management
Based on detailed analysis of 20 cases we found that, except for women who were seen at the Together Women's Project (TWP), almost all appointments took between 20–45 min and were conducted at the CRC office in private rooms. The majority of appointments were one-to-one. However, three-way meetings with a keyworker from another agency, often a drug and alcohol worker, were common. In a minority of cases, family members attended. On average, service users attended 70% of appointments and were recorded absent in 30%. Analysis of case notes showed evidence of a discussion of a wide range of offender needs. Drug and alcohol needs were discussed in 35% of appointments, accommodation needs in 34% and relationship needs in 22% of appointments.
Documented evidence of personalised working, including the use of person-centred practise tools or reference to My Direction interventions appeared in less than 10% of appointments. However, while case managers did not tend to explicitly document personalised work, this came into clearer focus through the interview and observation programme. We saw evidence of case managers using the person centred practise tools introduced to them during training sessions.
“I have made good use of the good day/bad day tool. I usually do the writing so as to let the service user talk” (Fieldnotes)
Case managers noted that service users responded to these tools differently and felt that the way the tools were deployed was important. Case managers used their discretion to adapt the tools. For example, some service users liked task-based exercises as it left them with a sense of accomplishment, whereas others preferred a more open, goals-focussed conversation.
“I don't always sit and do the worksheet, we will do a flipchart paper, with others they would pick words that SU could relate to, we would pick colours and work differently” (Finley, Case Manager)
“I have used good day and bad day fluidly, you don't need a worksheet for this I don't think. I would write it in a structure before the session without the prepared sheet to do the exercise” (Janet, Case Manager)
Person centred practise was widely regarded as offering structure to sessions and maintaining focus. This resulted in a better use of time and for sessions to be more constructive.
“I think I get more out of my sessions now as they are more structured. Before you would just check in and now I have a better idea of what I am doing. I think SUs benefit from this a bit more as well as there is more purpose” (Joe, Case Manager)
Case managers often described their practise in terms of relational and strengths-based approaches, which they saw as a key mechanism in enabling case managers to gain a better understanding of a service user's life and circumstances and was often a pre-cursor to co-produced, strengths-based work that might follow. As case managers built empathy and insight, strengths-based approaches became more viable and could lead to better structured and more purposeful one-to-one sessions with service users:
“Then we go to strengths, this can take a while as a lot of them struggle. I rephrase the questions, I will ask well what would you friends say about you, I will always look at the positive things, you have to find something that they consider to be positive even if it is just them getting there on time” (Fieldnotes)
A key objective identified by most case managers was to promote agency, autonomy and responsibility and holding service users to account for their actions, particularly those they had previously agreed to, was seen as vital. Case managers described how they often reflected upon progress by reinforcing good behaviour and challenging negative behaviour. They also noted the need to be consistent. However, creating the balance between strengths-based, positive language, genuine concern, and interest in the lives of service users and promoting a greater sense of agency was a fine balance that could easily tip over into paternalistic and dependent relationships:
“Some of those I work with at probation I really look up to, they offer me support are like a mother figure” (Alex, Service User)
Almost all case managers regarded their personal style as significant in working in co-produced or strengths-based ways. This often started with the importance of making service users feel at ease and striking a balance between remaining professional yet appearing relatable. The ability to remain positive during interactions and to highlight that which was going well was regarded as important. Being non-judgemental, accepting and sympathetic and responsive to service user's emotional state was also mentioned. Effective exercise of professional discretion was seen as important by case managers. Several case managers described their approach as being tailored and adaptive. Such an approach was based on identifying areas of interest or techniques that helped them feel at ease with the hope maximising engagement. An example of this cited by a case manager was the offer to meet a service user in the community and/or with a family member present.
“This time it is different, she tells me I can ring up when I can't make it or I have a problem. She does house visits with me which I prefer sometimes as I find it hard to leave the house” (Zac, Service User)
One advantage of person-centred practise was its ability to uncover aspects of a service user's life that were otherwise unknown. Possessing a deeper understanding of a service user's life improved the accuracy of risk assessments, mirroring a similar finding in the earlier piloting work (Fox et al., 2018). One case manager suggested that obtaining information about risk was possible in a way that was less intrusive when using person-centred ways of working.
An important element of strengths-based, co-produced working was to focus on the service users positive relationships and in several cases, service users mentioned that they approved of their case manager making links with family members. In multiple cases, service users felt that sharing rehabilitation plans and having family members and close friends involved increased their viability. The inclusion of family was most common with young service users and women service users.
“My current CM (case manager) is the best I have had, and I have been on probation for over 10 times. She listens to me unlike the others, they never cared. I know she knows I want good and that she believes in me. She knows I am doing well; she has been to my work, she has been to my home and met with my mum” (Jamie, Service User)
We found evidence that co-produced and strengths-based approaches were recognised and valued by service users. Service users were more likely to value probation services when they believed their case manager was committed to supporting them.
“I have never trusted probation in the past because of the way they have treated me – this time around my CM is sound, she listens and is there for me” (Zac, Service User)
For service users, key to establishing trust in the early stages of a relationship with a case manager was evidence that engagement with probation would help them meet practical and immediate needs such as health and housing. Service users tended to remember instances where probation had helped them access food, money, documentation or access to essential services. Being responsive in times of need or present when experiencing a crisis helped to establish a bond between service users and case managers. Case managers who made time and took the effort to get to know service users personally were regarded well. Key indicators of this were when a case manager remembered key aspects of a service user's life, such as the name of their child or important events such as anniversaries or memorials.
“But with my first CM (case manager) we would sit down and talk for a while – get her sheets out. She would ask about my daughter, she came into my life to find out about what I had been doing, how I had been getting on. She would try and get involved she was different” (Sara, Service User)
Continuity of case manager was seen as key by service users. When discussing both past and present orders service users disliked sessions that containing inconsistent and/or temporary case managers. When asked why, service users explained that this meant they had to repeat information and cover old ground to which they found frustrating. Moreover, service users felt key conversations had been forgotten and that temporary case manager failed to acknowledge the progress they had made and/or concentrated on negatives. While the approach implemented incorporated elements of general good probation practise, it was distinctive in providing more customised and tailored support to service users using specially designed person-centred practise tools. Additionally, there was greater focus on the wider determinants of a positive lifestyle, as defined by the service user, and co-production of a plan to address these, giving more choice and control to the service user around support beyond mandated interventions. A framework of innovative enabling interventions was designed and relevant services established.
Uptake of interventions
The programme struggled to implement some interventions, to resource others and to generate sufficient referrals to those that were implemented. In November 2018, it was agreed that some interventions including time banking and Life Links would be discontinued and available resources be focussed on person centred practise. There were various reasons interventions were not implemented. Lack of resources was a key issue. As discussed above, the programme was implemented at a time of considerable change and pressure on resources. We found that overcommitted senior staff were often unable to give sufficient time to developing and then promoting interventions. For example, at its inception, Life Links targeted and successfully recruited volunteers to facilitate supportive ‘circles’ with service users. Almost all early volunteers were university students because of their perceived suitability for the role. In response to lack of uptake and volunteer turnover the recruitment criteria were later adapted to include ex-service users. However, no Life Link circles took place. There appeared to be a lack of demand among case managers for the service and no successful referrals were made. Limited resources and lack of input from senior leaders meant the intervention was not promoted sufficiently to case managers.
Case Managers were often stretched and unable to allocate sufficient time to properly engage with interventions resulting in a lack of awareness of interventions. Where there was some engagement with the service, case managers often had only a limited of understanding as to how and why the interventions were intended to support service user rehabilitation. In some instances case managers raised questions about the coherence of the theory of change or logic model that underpinned the intervention but lack of time meant that instances of case managers helping to co-create interventions that they considered under-developed were rare.
Service users were often judged by their case manager as unsuitable for some interventions, in which cases common reasons given were concerns over managing risk and the timing of the intervention in the service user's sentence. One example was the use of the Enabling Fund. A form of personal budget, the enabling fund was designed to support service users to achieve rehabilitative goals that could not be progressed through existing services such as welfare payments or referrals to other agencies. However, uptake of the Fund was very low. There was some evidence that case managers resisted the loss of professional control (as they saw it) entailed in giving service users more autonomy to pursue personal rehabilitation goals. Programme managers felt that case managers were either uncomfortable about exercising discretion or were struggling to embrace a more co-produced and strengths-based approach to their work, issues that were also identified in the earlier, small scale pilots (Fox et al., 2018). Programme managers initially hoped to adopt a technical solution in the form of issuing service users with a pre-paid card linked to a phone app with spend restricted to certain goods and services providers. However, this idea was eventually dropped because of difficulties in setting up appropriate governance arrangements. Financial issues facing the CRC resulted in the organisation reducing the size of the Enabling Fund and it was withdrawn completely in January 2019. While it was available there were only ten instances of its use by nine different service users amounting to £453.40 of funding in total. It is noticeable that when used, amounts spent were relatively small. The most expensive purchase was £84. The majority of purchases were linked to basic needs such as accommodation, food and clothing. This pattern of usage was consistent with the earlier pilot (Fox et al., 2018).
Another example of a mismatch between the ethos behind an intervention and the way of working within the CRC was time banking. Locally, time banking was a run by a small community-based organisation where people identified skills/assets they could offer to others in the scheme and undertook tasks to earn time credits that they could spend on different goods and services offered by other members. Examples of these were gardening, painting, dressmaking, yoga teaching and piano teaching. A representative of the Time Banking organisation introduced the intervention to the programme team. However, some members of the programme team noted that case managers were wary of the additional work it might entail in terms of reporting and monitoring and were concerned about managing risk. A member of the time banking organisation highlighted the difficulty of aligning their strengths-based approach with the approach articulated by case managers:
“I had to do a lot of persuasion to get them on board - a lot of this included providing reassurance… But we come from a place where we look at people's advantages. We have to trust people - we can't look at people as offenders - we have to look at them as people” (Partner agency)
These two examples illustrate the difficulty of interworking across agency, professional and sector boundaries. They also suggest that, in some cases, strengths-based and co-produced working was less developed than intended, with case managers struggling to build secure, trusting relationships with service users and work in a co-produced manner. In the face of externally focussed, innovative interventions some case managers found issues of risk overwhelming: “It sounds interesting, but I just think that there are so many different risk elements to take account of, especially when this is being done outside of the organisation - perhaps it should be done internally. You see, we know all of the risks that a are attached to different service users (Sus)” (Fieldnotes)
Some of the interventions could not be implemented due to regulatory or wider systems issues. For example, a mobile app was intended to be implemented to support person-centred case management by allowing case managers and service users to access individualised content remotely, allowing case managers and service users to schedule appointments, send messages to each other and register attendance at services. The app was based on an ‘off-the-shelf’ application developed by an external company that could be tailored to meet the specific needs of the pilot. Despite effort and enthusiasm from both the developer and Interserve, concerns around data security and regulatory concerns could not be overcome. It was recognised that data governance issues would require formal approvals from the Ministry of Justice that could not be achieved within the lifetime of the project and further work on the app was discontinued.
Volunteering
Volunteer service users were important within the My Direction. Only service users who displayed pro-social attitudes were considered for the role of peer mentor. Service users discussed their volunteer readiness with their case manager. Although volunteering didn't appeal to all, some of those with ambitions to volunteer saw it as a long-term goal. Recognising that ex-service users excelled at communicating and bridging the gap between professionals and service users, the CRC tended to place volunteers in service user-facing roles. This included facilitating inductions, recovery groups and delivering offence focussed work. Supporting group sessions would entail booking service users in at reception, sending reminder texts, arranging transport, room set up and being on hand to aid comprehension of tasks or remove barriers to participation (e.g. though reading and writing support). Ex-service users provided visible proof that change could be achieved in a way that CRC staff could not. This helped to motivate and inspire service users by instilling a sense of hope that they too could turn their life around. We observed ex-service users speaking with passion and galvanising service users into action by sharing key events in their lives and low points which they had since overcome. The value of lived experience in understanding the desistance process and inspiring change was articulated, for example, by a senior manager:
“It makes people believe it is possible. Service users would say [to CRC staff] ‘how do you know what it is like?. And that's a fair challenge. Unless you have a conviction, you don't know how it feels or how you are perceived – these experiences are powerful and shape how you perceive others” (Sam, Senior Manager)
Service users discussed their volunteering in terms of giving back. They recognised the negative impact they had had on those close to them. For example, one ex-service user spoke of how they felt a sense of guilt about their drug using past It became increasingly clear as the pilot progressed that despite LifeLinks’ failure, volunteers were key participants in and indeed champions of co-creation in the CRC. This role potentially represents some redrawing of the boundaries between ‘professional’ and ‘user’ in ways that go to the very heart of co-creation.
Impact evaluation
Simple observation of outcomes found 13.1% of the intervention group reoffended, compared with 15.1% of the counterfactual group. However, as assignment to the intervention group was not randomised and the groups were imbalanced across a number of covariates we estimated the Average Treatment Effect on the Treated (ATET) using regression adjustment to condition on a range of covariates.
The first regression model was conditioned on gender, age and risk level. It estimated that if those in the intervention group had not been part of My Direction, their probability of reoffending would have been 14.7%. The model gave a treatment effect (ATET) point estimate of 1.6% - i.e. that the probability of the intervention group reoffending was in fact 1.6% lower (13.1%) than had they not been treated. It is important to note however that this ATET is not statistically significant at the 0.05 level, and its associated 95% confidence interval suggests that the it could fall within a range of values from positive to negative. Based on the first model it is therefore not possible to reject the null hypothesis that My Direction had no effect on reoffending outcomes for those in the intervention group.
The second regression model included deprivation as a covariate, in addition to gender, age and risk. This model also found a small ATET (0.4%) which was not statistically significant, therefore leading to the same overall conclusion as the first model. In short, while we observed small reductions in reoffending, these were not statistically significant and therefore the treatment effect estimate for My Direction does not provide evidence to suggest that participation in My Direction affected the probability of reoffending for those in the intervention group.
Discussion and conclusion
MyDirection was delivered in the context of rapid organisational change, often in response to rapidly changing external events and a turbulent policy environment. These factors, particularly the re-assignment of staff during the pilot, clearly impacted upon implementation and so we conclude that MyDirection was subject to a degree of implementation failure and the project demonstrates the difficulties of developing and implementing innovative approaches to rehabilitation when the policy context is volatile. That the impact evaluation did not find a statistically significant difference in reoffending rates between the intervention group and a comparator group is unsurprising given the challenges around implementation. However, it is important to note that there were limitations in the design of the impact evaluation including in the limited choice of outcome measures available and relatively short follow-up period. Reoffending is an outcome which was relatively distal to the MyDirection intervention – the intervention was not specifically designed and implemented to directly target recidivism, but rather to build an environment more conducive to the desistance process and it would be expected that progress towards this outcome might require a longer follow-up period.
The challenges with implementation notwithstanding the approach that was piloted did contain promising elements. In-depth qualitative evaluation identified positive examples of personalisation with individual case managers and service users supportive and noting positive change. Desistance research and understandings of strengths-based approaches suggest more outward looking approaches to case management that support people to build individual and social capacity, but our findings illustrate how challenging this can be for probation services. Resistance to change is not uncommon particularly in public service organisations where professions, such as probation staff have a high level of technical and procedural knowledge and are repositories of a set of standardised knowledge that they apply to each individual case. They operate following what has been defined as ‘inward look’ (Boyle and Harris, 2009), whereas a move towards strengths-based and co-created ways of working, in which ‘knowledge by experience’ is valued alongside professional knowledge requires staff to operate an ‘outward look’ to deliver complex interventions that are social and not technical (Mortensen et al., 2020). Staff in this study sometimes struggled with these issues in a pressured, time and resource-constrained environment. Observing the different ways in which staff responded to MyDirection, adopting some aspects of the model, but either adapting or side-stepping others, we are reminded of Lipsky’s (2010) concept of ‘street-level bureaucrats’ whereby public service workers must operate in an environment in which their work is often highly scripted to achieve policy objectives, but also requires them to use discretion to meet the particular needs of individual clients. Motivations of front-line staff are therefore complex, and their decisions effectively become the public policies they carry out (Lipsky, 2010). At the individual level our findings support the proposition that ‘resistance to change’ by front-line staff should be understood, rather than seen as a ‘problem’ to overcome. Co-creation highlights the importance of investing in greater use of reflective practise and a need to re-evaluate the relative value of experiential and professional expertise.
A particular challenge to co-creation in probation was how to manage risk and this is reflective of challenges within the wider probation sector around implementing desistance-based practise and deciding what constitutes appropriate evidence to support risk assessments (Maruna and Mann, 2019). These challenges were particularly evident in tensions perceived by front-line staff in how to approach risk management and led some front-line staff to avoid the use of some interventions such as the enabling fund or the time-banking and to limit the extent to which they embraced co-production. However, in so far as desistance focuses on positive human change in which people seek primary goods to achieve wellbeing, practitioners need to strike a balance between promoting goods while managing risk, aware that:
Taken as a whole our findings suggest that, a co-created, strengths-based model of case management is promising at the level of individual case managers and service users, as a strategy for operationalising desistance. But, while the evaluation has shown that tools developed to support person-centred practise in a social care context can be used effectively in a criminal justice context, it also shows that they are not sufficient in and of themselves to support the development of co-produced, strengths-based practise consistent with a desistance approach. More needs to be done at a system level. In this case, Transforming Rehabilitation posed particular challenges, but, even when the policy context is more favourable, where interventions are implemented in complex systems where change is the norm we should not expect change to be linear and outcomes to be discreet and predictable but instead contingent and emergent (Eppel and Rhodes, 2018). Implementation of co-creation needs to be supported by complexity-informed management practises that recognise that ‘interventions’ are not delivered in organisations, but within complex systems (Lowe et al., 2020). Delivering desistance-focussed services through co-created and strengths-based working implies a new generation of probation services that put people at the heart of service design and delivery, create the conditions for learning and are open to wider systems change (Lowe et al., 2020). Such approaches have potential to address the complexity that is inherent within modern criminal justice systems.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project has received funding from the European Union's Horizon 2020 research and innovation programme under grant. agreement No. 770492.
