Abstract
Background
The program “Réalise Tes Rêves” (Make your dreams come true; RtR) was a program of action research aimed at trialing new approaches to address the job seeking needs of people not in employment, education or training (NEETs), and unemployed women in disadvantaged neighborhoods.
Objective
This study examines the use of social return on investment (SROI), a social valuation methodology, to examine and capture the broad socio-economic value of the employment program RtR.
Methods
Qualitative data was collected from 54 program participants and 21 program staff. The analysis was also informed by program data such as program components engaged in by participants, retention and exit outcomes.
Results
Positive outcomes were identified for the program participants, program staff, and the community. The most important direct outcomes were the increase in personal assets, enhanced career prospects, and health improvements. The SROI ratio was €16.62 for every €1 invested.
Conclusions
This study illustrates the program's tangible and intangible program outcomes by applying SROI methodology and demonstrates the social impact of supporting employment and self-employment for disadvantaged job seekers.
Keywords
Introduction
Persistent unemployment is associated with high costs to the individuals affected, their families, and their communities (Duell et al., 2016). People who are long-term unemployed are at risk of knowledge and skills obsolescence and depleted job networks (Aaronson et al., 2010), which in turn further reduces employment prospects (Duell et al., 2016). Access to the labor market after being out of work for six months or more is influenced by a range of factors, with younger people being especially disadvantaged. Young adults not in employment, education, or training (NEET) face particular barriers including lack of work experience and poor qualifications (Mawn et al., 2017), and thus have more difficulties returning to the labor market after a prolonged period of inactivity than experienced workers (Lyshol et al., 2021). As such, public employment programs have a long history of providing much-needed help for the challenges faced by this group, and more needs to be done to effectively meet the growing needs of the NEET population impacted by COVID-19 (Lambovska et al., 2021).
A systematic review and meta-analysis of re-engagement interventions aimed at NEET population found a 4% improvement in employment outcomes between intervention groups and control groups (Mawn et al., 2017). The authors identified that more intensive programs (identified as daily contact for at least 2.5 months) not only increased employment outcomes but also longer-term wages, and that the provision of financial support improved both recruitment and retention rates (Mawn et al., 2017). A study conducted in South Korea, identified that the risk of NEET decreased by 6.6% for intervention participants compared to non-participants over 3 years (Park et al., 2020). Notably employment leads to valued social outcomes as well as financial outcomes. A systematic review of programs targeting NEETs identified the achievement of a range of social outcomes such as increased quality of life and well-being, improved social functioning and a reduction in drug taking behaviors (Stea et al., 2024).
To date, we could find no evidence of attempts to capture and quantify the financial and social impacts of employment programs for NEET populations, though a number of studies have been published on the social impact of employment programs for other disadvantaged populations, including for job seekers with disabilities. People with disabilities typically experience higher level of unemployment than people without disabilities (OECD, 2022) and can become reliant on welfare long-term. A study in the USA adopted a social return on investment (SROI) approach to evaluate a program providing training and job coaching for participants with development disabilities (Owen et al., 2015). The authors identified several social outcomes including increased quality of life, independence, increased social participation, and better emotional, physical and material well-being (Owen et al., 2015). In addition, a number of outcomes were also identified for families including a reduction in care giving activities, reduced stress and increased income due to having more time for paid work. The SROI analysis identified that $1.77 of social value was created for every $1 invested in the program. An Indian employment program aimed at youth and unemployed women, identified a similar ratio determining 1.9 rupees of social value for every 1 rupee invested with outcomes including personal development, increased confidence and independence as well as increased income levels (Gambhir et al., 2017). An Australian study forecasting the social value of a new social enterprise to support the integrated employment of people with disabilities in commercial catering, identified a significantly higher SROI ratio of $8.48 of social value for every $1 invested (Hutchinson et al., 2024). This was largely attributed to the study including government outcomes such as decreased welfare payments and increases in taxation (Hutchinson et al., 2024). We identified no social impact studies conducted in the French national context.
The RtR Program
In France, the Ministry of Labor launched a call for projects called “Competencies Investment Plan – 100% Inclusion” in 2019 to benefit young job seekers with few or no qualifications living in disadvantaged neighborhoods (Ministère du Travail, 2020). According to the French National Institute of Statistics and Economic Studies, while the unemployment rate in Metropolitan France was 8.4% in the first quarter of 2019, it was of 12.4% among the young NEET (aged between 15 and 29). This rate is three times higher in disadvantaged neighborhoods that suffer from financial insecurity, social exclusion and inadequate housing. Furthermore, 23% of women in disadvantaged neighborhoods are unemployed, compared to 9.5% for women in the surrounding areas.
One of the winners of the call for projects was the program “Réalise Tes Rêves” (Make your dreams come true; RtR), a two-year intervention in three regions (a part of the northern region, the Paris region and a part of the Southern region). This program consists of conducting action research aimed at trialing new approaches to address the job seeking needs of NEETs and unemployed women in disadvantaged neighborhoods. Led by a consortium of five agencies, this program consists of offering personalized support for remobilization to access employment or entrepreneurial activity and included workshops, outdoor activities, coaching and mentorship. In addition, monetary and in-kind support are offered to the participants.
RtR aimed to identify potential, to develop participants’ capacities and skills, and to remove the obstacles to accessing employment. The program focusses on preparing participants for the current labor market with its expansion of digital technology and artificial intelligence, the need to adapt to ecological upheavals, and new opportunities in entrepreneurship. The consortium worked together to implement this project. These agencies were known for their contribution in a range of fields including technological and social innovation, research, education and pedagogy, entrepreneurship, providing support to low-skilled workers, collaborative project management, and digital technology. The intervention consisted of sourcing participants, designing the training course, operational follow-up, project coordination, and the reporting of the project's results.
First, program participants are guided by their expectations and needs either toward the “employment path” that intended to support their professional integration as employees, or the “entrepreneurial path” that is designed to help them become self-employed. During this phase, participants attended coaching sessions with certified job coaches that involved completing a range of measures to identify participants’ needs, aspirations, skills, interests and motivation, as well as their key personality traits, cognition and technical skills. These assessments were followed by discussions between program participants and job coaches to identify participants’ objectives, preferences and strengths. From this data, an algorithm was established to make recommendations regarding what types of professional activities would be most suitable. In parallel, advisors are assigned to assist the program participants and to orient them towards relevant workshops conducted by workshop leaders. For instance, some of the workshops to support program participants on the “employment path” included: overcoming fears in job search, identifying talents and aspirations, linking passions and skills to the professional project, writing/updating a CV and a cover letter, practicing for a job interview, developing a network, and job discovery workshops (e.g., florist, technologies, urban agriculture, food, new technologies, services). Program participants on the “entrepreneurial path” were oriented towards workshops for creating and developing their entrepreneurial activity by their advisors. These workshops covered business planning (competitive analysis, commercial strategy, business model), logistics, collaborating with stakeholders, communication (logo creation, pitching the project, and using social media), developing a marketing strategy, funding, and addressing legal and administrative aspects of running a business. Workshops that supported both pathways included soft skills development (public speaking, time management, stress management, etc.).
Job coaches also worked one on one with program participants to discuss professions that aligned with participants’ preferences and competencies and worked with them to assess the feasibility of the identified professions for them. To support informed decision making, meetings are organized between program participants and experts in identified fields of interest thanks to partnership agreements between the RtR program and multiple companies and associations. Program participants on the “entrepreneurial path” worked to develop, test, and improve their ideas at an early stage through a prototyping process in collaboration with business incubators in partnership with RtR projects. Finally, program participants worked with their adviser and job coach to put an action plan in place. To reduce withdrawal from the program due to a total lack of resources, program participants benefit from financial aid as the program had a dedicated budget of 300,000 € (about US$322,000) for the payment of allowances, debt relief, travel expenses, training fees, etc. Following the initial phase of funding, the program benefited from extended funding for a further year. This allows the program staff to refine participant recruitment, identify the right facilitators and refine the content of workshops. Furthermore, the platform for enrolling participants was developed which streamlined the process of commencements.
In total, the program had 2032 enrolled participants, and 1381 completed the program (67.9%). Participants ranged in age for 18 to 59 years (mean 29.8 years), 55% were female and 64.2% percent had less than or equivalent to a high school education. Almost one third (31.5%) attended the entrepreneurship path, with the remainder enrolled in the employment path. Reasons for dropping out of the program included health-related disruptions, lack of legal status to remain in France, securing employment shortly after enrollment, enrollment in other programs similar to RtR and, for some female participants, a personal decision to give up job seeking to take on home-related duties. It should also be acknowledged that the program was in operation during the COVID-19 global pandemic, with France experiencing three national lockdowns between March 2020 and April 2021. During these lockdowns, the RtR program shifted from in-person workshops to virtual formats and transitioned participant interactions to video calls.
The funder of the RtR program required the program organizers to allocate a proportion of the budget for the purposes of independent evaluation using SROI methodology to evaluate the impact of the program on participants, the staff members delivering the program and the wider community.
Social Return on Investment Methodology
We aim to contribute to the literature by applying the SROI methodology to estimate the social value that the RtR program created in the lives of the program participants and program staff (advisors, coaches and workshop leaders), and to demonstrate the program's social impact to the funder, the French Labor Ministry. SROI offers a framework for understanding and measuring the social, economic and environmental value of an investment or an organization's activities in a monetary sense (Nicholls et al., 2012). A cost-benefit ratio is created by comparing the total monetized benefits with the costs/investment in an intervention or program (Wilson & Bull, 2013).
SROI was developed in the USA in 2000 but has since been further developed by the New Economics Foundation in the UK, a government agency that has championed the use of SROI methodology across a wide range of public policy settings (Aeron-Thomas et al., 2004; Millar & Hall, 2013). Today the methodology remains most extensively used in the UK but interest in the methodology is gaining momentum. A recent systematic review identified 284 peer-reviewed SROI studies across Anglo-Saxon nations (UK, USA, Canada, Australia) as well as Europe, Asia and Africa. Furthermore, 80% of these articles expressed optimistic evaluations of the approach and its usefulness (Corvo et al., 2022).
The guide to SROI developed by the Office of the Third Sector in the UK Cabinet Office is the most cited resource guiding the use of the methodology (Nicholls et al., 2012). This guide outlines six stages to perform a rigorous SROI. These are: 1) establishing scope and identifying stakeholders, 2) mapping outcomes, 3) evidencing outcomes and giving them value, 4) establishing impact, 5) calculating the SROI ratio, and 6) reporting findings (Nicholls et al., 2012).
The Current Study
The purpose of this study was to apply the six stages of SROI analysis to explore the social value of the RtR program. Specifically, we sought to address the following research questions:
What personal and professional outcomes are experienced by program participants? Did any outcomes occur for program staff (job coaches, advisors, workshop leaders) and the wider community when people from disadvantaged backgrounds participated in the RtR program? What is the social value of the RtR program relative to the investment made by the French Ministry of Labor? (the SROI ratio)
For this analysis we draw on program data as well as the perspectives of program participants, and advisors, job coaches and workshop leaders (collectively known as ‘program staff’ throughout unless referring to one particular group of staff members). In addition, we draw on secondary data from existing literature to support the development of financial proxies (to estimate the value of outcomes experienced) and appropriate discounts to avoid overclaiming the social value created.
Methods
Ethics Statement
This research received ethics approved from the “Laboratoire d’Intelligence Collective et Artificielle (LICA)”. Informed consent was obtained from all participants (program participants and program staff) prior to interview and survey data collection. Interviews were recorded with the permission of participants and then transcribed in full by a professional transcription service. The interviewer (first author) had no involvement in the design or delivery of the RtR program, only in its evaluation.
Participants
RtR program participants and program staff were identified as the key informant stakeholders for this SROI analysis. Program participants attended the program for between 2 weeks and 16 months. To be invited to be interviewed participants had to have completed a minimum of six months in the program. Length of participation was tracked on an Excel spreadsheet and 283 program participants met this criterion. Participants were contacted six to eight months after their graduation. Of these, 76 did not answer phone calls, 31 were approached but did not wish to be interviewed, and 176 agreed to be interviewed. The first author randomly selected program participants from the list of those who had agreed to participate, until 30% had been interviewed (n = 54). It was not possible to interview all of those who had agreed to participate due to resource limitations.
Of those that did not wish to be interviewed some declined as they reported they had not experienced any outcomes from their participation. Reasons for this were reported to be due to the perception that the personal development phase of 6 months being too long, concerns regarding financial support and how financial support from the program might impact other welfare payments, some activities being considered too repetitive or irrelevant (especially for those with clear entrepreneurial goals) and a mismatch between the program and their personal needs.
In total, 61 program staff members supported program participants across the three regions, consisting of n = 51 advisors and workshop leaders and 10 job coaches. All staff were also invited to participate in the study via semi-structured interviews and, of these, 21 consented and were interviewed.
Semi-structured interview guides were developed for program participants and program staff and focused on the impact that involvement with the RtR program had on their lives. For example, the program participants were asked about outcomes experienced and observed since starting at RtR, the activities and workshops they engaged in during the program, and the degree to which the intervention was responsible for these outcomes. Interviewees were offered the option of an in-person interview or an interview via video conferencing as per their preference and availability.
Procedure – The Six Stages of the SROI
The six stages of performing a rigorous SROI were followed (Nicholls et al., 2012). The SROI method emphasizes the involvement of the individuals or groups of people that are targeted by the intervention to identify pertinent indicators that reflect their values (Moody et al., 2015). As identified above, in this first stage of the SROI analysis, the stakeholders identified to inform the analysis were program participants and program staff delivering the intervention. In total, 54 program participants and 21 program staff were interviewed. In addition to the qualitative data, other data that informed the SROI included program data collected by the consortium.
The second stage consists of the development of a theory of change that illustrates the pathway of how value was created. The theory of change aims to illustrate the ‘story’ of the intervention and its expected social impact, this includes resources (inputs), activities (outputs), and short- and long-term outcomes for stakeholders (Classens, 2015; Klemelä, 2016; Walk et al., 2015). Mapping outcomes included the creation of “chain of events” to illustrative how outcomes were achieved from the outputs of the program. A theory of change was developed in consultation with key stakeholders and the research team and is shown in Figure 1 below.
Theory of Change.
The third stage of the SROI analysis involves evidencing outcomes and giving them value. At this stage, the research team identified the indicators for the outcomes (how we would know the outcome had been achieved), the evidenced outcomes from interviews, and discussed possibilities for the best financial proxy for each outcome.
There are a variety of techniques and methods to monetize outcomes (Nicholls, 2017; Office of the Third Sector, 2009; Perrini et al., 2021). We employed the market comparison method to monetize outcomes by analyzing the results of interviews and similar studies (Owen et al., 2015; Scholten et al., 2006). Together with all stakeholders, we agreed on how long outcomes were expected to last, known as the ‘benefit period’. We set this at seven years, although we acknowledge that the effects of being employed versus unemployed can last for the entire lifespan of an individual. This decision represents a compromise, a way to account for long-term effects while still taking a cautious approach.
The fourth stage was to identify and apply discounts to the values to avoid over claiming. There are four discounts that need to be considered, though not all may be relevant to every analysis. The discount types are: displacement (did the intervention displace other valuable outcomes), deadweight (would the outcomes have been achieved without any intervention given time), attribution (how much of the outcome was due to the intervention in question versus other interventions, services and supports the participant may be using), and drop-off (does experience of outcomes drop-off over time). These discounts were informed by the interview data and were discussed and agreed between the research team.
For example, to assess the duration of an outcome, interviewees were asked how long they expected the experienced change to last. In order to assess deadweight, they were asked whether they expected any of the changes identified to have happened if they had not been involved in the program. To assess displacement, interviewees were asked how much of the outcome has displaced other outcomes. As for the attribution, interviewees were asked if any other service or supports may have contributed to the changes (outcomes) described, and to estimate the percentage/proportion RtR contributed to the change.
Qualitative Data Analysis
For qualitative data analysis, we used a thematic analysis approach that emphasizes identifying, analyzing, and reporting patterns within data (Braun & Clarke, 2006). Particularly, analytic induction and constant comparison strategies were used to elicit common themes in the transcripts and further explore the responses (Braun & Clarke, 2006). The coding and theme generation were directed by the content of the data and were organized to address the following themes: the outcomes of the program, the chain of events underlying the outcomes, the indicators and duration of the outcomes, and the outcomes’ discounts. In the first round of coding, the first author created parent nodes for identified program outcomes which were discussed and confirmed with the second author. In the second round of coding, the first author created child nodes for each outcome to capture causal links with the program and data to estimate appropriate discounts, which was checked and verified by the second author. We used qualitative research software NVivo10 to track the coding and manage the data. This tool helped in organizing relevant code arrangements and provided visualizations of categories and subcategories.
Results
Identifying Inputs
The inputs refer to what stakeholders contributed in order to make RtR program's activities possible, calculating the investments made at each stage of the project over 3 periods of time: 1) program participants sourcing and hiring of program staff, 2) program implementation, and 3) capitalization and learning activities for similar future projects for the consortium. The overall inputs for the job and skills training program was nine million eight hundred and eighty thousand five hundred and twenty-five euros (9,880,525 €) (see Table 1).
Valuing Inputs to Conduct the RtR Intervention.
Defining and Mapping Outcomes
In this section, we outline qualitative data from program participant interviews in relation to the outcomes identified in the theory of change.
Outcomes for the Program's Participants
Increase in Personal Assets
Program participants particularly valued three personal assets developed whilst attending RtR, these were self-confidence, self-esteem, and motivation. During the interviews, the majority of program participants confirmed an increase in self-confidence and were able to acknowledge their abilities to succeed. As a consequence, program participants became active and engaged in the job search process and started to take actions to launch their entrepreneurial projects. It's a huge confidence boost, because now I know that I have the capabilities to make it, that I am legitimate to do this project. I can express my point of view, my opinion and define what I want. (Program Participant 1, age 25, female, entrepreneurial path) I feel more comfortable in speaking, in making decisions, a while back I wouldn't have been able to do that. (Program Participant 2, age 25, male, employment path)
The program participants also discussed improving self-esteem, as they have a more positive and rewarding self-image. Throughout their RtR program, the program participants were referred to as “talents” and were helped to gain an awareness of their inherent worth and the value of their skills and qualities. We have been accepted and received without judgement. No matter our origin, culture or background, we arrive with our dreams and our dreams are not judged. (Program Participant 3, age 26, female, entrepreneurial path) We needed to feel valorized and RtR brought us that by providing us with group work, discussion and exchange. (Program Participant 4, age 26, female, entrepreneurial path)
In our interviews, we identified several instances where program participants were motivated to take control over their lives and achieve their goals. Program participants went beyond the stage of reflection, to create concrete actions and became more productive in the realization of their projects. Without RtR, I would not have insisted because I had already applied for this training, I had been interviewed and unfortunately it was negative. So, this time, I insisted, I searched for two months, and finally, I was able to find another session. I contacted them, I took the necessary steps and in January I was accepted. (Program Participant 5, age 25, female, employment path) Every day, I think about my business, I look for an office via real estate agencies. I have put a questionnaire online to better understand the job of a caregiver. I talk about my project to as many people around me as possible, I see the competitors, their rates, how they work. I identify the streets that are close to my business and place flyers in those places. (Program Participant 6, age 26, female, entrepreneurial path)
Development of Entrepreneurial Skills and Creation of Entrepreneurial Activity
The majority of the program participants in the entrepreneurial path acquired entrepreneurial skills. This covered three main aspects: 1) the theoretical and practical knowledge of the various modules of business creation (communication, legal, human resources, accounting, etc.), 2) the refinement of the project idea and the construction of a business plan, and 3) the public presentation of the project to convince an interlocutor. We do modules, for example in management, HR and communication, and there are challenges at the end of each module that allow you to challenge yourself and to put on paper and implement what you have seen during the week. That's a confirmation that the information has been mastered. (Program Participant 7, age 26, female, entrepreneurial path)
A number of these program participants finalized the steps to creating their business up to its registration and started to generate income. They created companies in various fields such as beautician, home help, communication and marketing consultant, board game creation, marketing of childcare accessories, sports consulting, and film-making, etc. I actually started working and billing as a communication consultant in June 2020. (Program Participant 8, age 26, female, entrepreneurial path) For this project, the short film project, I invoice up to 60 euros per hour. And for comics, it's a little cheaper, it would be around 40 euros per hour. (Program Participant 9, age 25, female, entrepreneurial path)
Enhanced Career Prospects
Enhanced career prospects were described, including choosing a professional path, job readiness and employability, and finding employment. First, program participants were able to deepen their values, identity and overcome obstacles, build motivations, and identify their aspirations. Then, they were supported to choose a target job that aligned with their skills, potential, and values. As a result, they chose and validated their professional path. It is about diagnosing, fixing, deciding on the upcoming professional project. I had an idea; it was the field of professional integration, and I was able to confirm my vision through the workshops. (Program Participant 10, age 31, male, employment path) I had an idea in mind to become a chef in the restaurant industry. And thanks to the coaching sessions, I was able to do a lot of exercises, which confirmed this choice. (Program Participant 11, age 28, male, employment path) I have received explanations about the difference between the jobs I was considering. I didn't really know the disadvantages of certain jobs and the advisors explained it to me. I was able to tell the difference and find the job that suits me. (Program Participant 12, age 23, female, employment path)
Some of them acquired skills related to employment (writing a CV and a cover letter, mastering the professional social networks, and passing job interviews) following a training course or started an internship in relation to the targeted job. Thus, they experienced a so-called “job readiness and employability” outcome. There were especially the two workshops that were very important to write the CV and the cover letter. And what was very important and helped me a lot was the workshop on how to express myself in a job interview. We were taught how to behave, how to speak, and how to try to convince an employer during an interview. (Program Participant 13, age 22, female, employment path) All this resulted in the fact that the job that best suited me in terms of my values, my skills and what I can bring to the table, was the job of countryside ranger. So, this year, I focused on taking courses for this field, in order to start a 6-month training course. And, in November, I will pass the competitive examination for this job. (Program Participant 14, age 22, female, employment path) I started an internship as a technician in computer science, systems, and networks. I will finish in November 2021 which should allow me to find a job more easily. (Program Participant 15, age 22, female, employment path)
Other program participants completed these steps and signed a promise of employment or an employment contract. The fields in which they have found employment are varied: sales in a clothing store, catering, job placement consulting, web development, the army, the police, social work, marketing communication, security guard, project management, computing, teaching, fiber optic assembly, and sports coaching, etc. I started my job as a mobile web developer, and it was important for me to give meaning to what I do. For example, in my company, they are building a mailbox that is going to be adapted to visually impaired people and dyslexics and it is going to be more ecologically adapted. (Program Participant 16, age 26, female, employment path) My advisor found a training that corresponded to what I wanted to do, she contacted me and helped me to start this training. Later I was able to get my diploma, and during the training, there are companies that come to see us to hire potential new recruits, and I have been scouted. (Program Participant 17, age 25, male, employment path)
Increase in Social and Professional Networks
Program participants also acknowledged that they were able to expand their networks as a result of the many encounters they had in the RtR program. The network for them represents contacts and connections with other program participants from the same or previous cohorts, advisors, as well as external speakers, and professionals (i.e., experts in identified fields of interest thanks to partnership agreements between the RtR program and multiple companies and associations). This gave them access to constructive feedback and to exchange advice and ideas, which allowed them to reflect on how to improve their projects. Some program participants associated the network with a sense of belonging to a group where they find solidarity and comfort. Others consider these contacts to represent potential professional partners or clients. The collaborations in my new entrepreneurial project are completely based on the network I made in RtR. (Program Participant 18, age 20, male, entrepreneurial path) I met a girl in a workshop who is interested in computers, and she is going to do the website for my company. (Program Participant 19, age 21 female, entrepreneurial path) We'll keep in touch with the participants of the projects, and if someone didn't make it, I'd be willing to go and say, ‘Well, what's wrong? What do you need? Look, I have an event here, don't you want to come with me? You're going to meet such and such a person who is in your field of activity. (Program Participant 20, age 24, female, entrepreneurial path)
In addition to the network expansion, the program participants have developed social skills that allow them to develop their network outside the program. They explained that they now had a better understanding of how to communicate and interact with different types of potential partners, such as clients, colleagues, and competitors to find new collaborations.
Improvements in Physical Health and Mental Well-Being
Program participants claimed that their mental and physical health had improved considerably as a result of attending the RtR program. Before joining the RtR program, some program participants reported suffering from anxiety, sleep and mood disorders as well as spasms and back pain.
Improvement in psychological health was related to the selection of a suitable profession, getting started in working in that profession, made them feel secure about their future. This was also the case for the program participants on the entrepreneurial pathway who created their business. Some program participants highlighted the encounters and exchanges that they had which made they feel listened to and understood, especially given that many have been isolated due to health restrictions related to the COVID-19 pandemic. Other program participants associated the improvement in their psychological health with the opportunity of receiving financial aid as RtR participants, which allowed them to improve their living conditions. These include financial support for starting their business, housing assistance, debt repayment assistance, telephone bill payments, mobility assistance, food assistance, etc. This financial support resulted in improvements to their living conditions and reduced their anxiety. The RTR project also provided me with a salary. So RTR had a partnership agreement with this region, which means that you have a remuneration during your training called the ASP (Allocation de sécurisation professionnelle) [Job security allowance]. It was a contract where you do 30 h a week for 6 months. So, there is a remuneration that is put in place and I benefited from it. (Program Participant 21, age 25, female, entrepreneurial path) So, thanks to RtR, I got a new apartment. I had been waiting for that for five years. (Program Participant 22, age 30, female, entrepreneurial path) When I introduced myself that I don't have an income, I was able to have an income while I was doing RtR. For me, it was a very important boost because I was able to keep my family safe for six months and it gave me six months of breathing room to further develop my business in a more comfortable way. (Program Participant 23, age 38, female, entrepreneurial path)
As for the improvement in physical health, for some, it came as a result of the improvement of the mental health: When you’re anxious, you usually have pain in your body. In my case, for example, I have less back pain and fewer physical contractions. (Program Participant 24, age 27, male, employment path)
For others, the improvement in physical health was the consequence of having left a previous physically challenging job to a more comfortable one. The back problem was due to the four years of work I had done as an order picker. If there is a link with the RtR project, it is that it prevented me from continuing in that direction and from moving towards a less physical job. (Program Participant 25, age 25, male, employment path)
Development of Socially Responsible Behavior
Some of the program participants mentioned that they had been inspired by the workshops on the subject of sustainable development, as well as by the ecological tools used during these workshops. They now consistently prioritize ecological alternatives in their daily lives and in their entrepreneurial projects.
Program participants also acknowledged that they felt grateful for the opportunities that they had and wanted to give back after receiving help from the program. Consequently, they became committed to their community through volunteering and association work. In my association, I'm going to create opportunities for people who want to find a way to be useful, even if they don't have an excellent CV. Basically, if I hadn't had this experience with RtR, I wouldn't have put myself in their shoes and I'd have stuck to an ideology based on CVs and experience. (Program Participant 26) I chose to work as a volunteer in the field of computer maintenance because I want to share my knowledge and pass on my skills. (Program Participant 27) Before RtR I used to volunteer as a basketball coach, but now I have increased my volunteer hours from 3 h a month to 6. (Program Participant 28)
None of the program participants identified any negative outcomes from their participation in RtR. However, program data on 716 participants identified that 27 reported no change as a result of their RtR attendance (3.8%).
Outcomes for Program Staff
Our research team conducted interviews with 21 program staff. The results revealed that the program staff identified 10 outcomes since joining the RtR program: Improvement in work-life balance, increased emotional resilience, improved teamwork skills, improved organizational and time management skills, increased self-confidence, enhanced relational and communication skills, career development and job security, improved coaching skills, improved work and job satisfaction, and improved skills in delivering workshop. We will illustrate 4 of these changes in detail below. We report where outcomes relate more to one particular group of program staff (job coaches, advisors or workshop leaders) rather than all program staff.
Improvements in Work-Life Balance
Some of the advisors monitored a number of program participants in difficult situations. They considered it their duty to help them resolve such problems, even if it meant working outside their scheduled hours. As they became overwhelmed by these challenges, they started realizing the importance of focusing on their needs to continue to support program participants’ needs and establish a better balance between private and professional life. To do so, they have restricted unpaid extra hours to emergencies and disconnected from electronic communications. I work with my participants, I do my job, but now I know how to say no in a certain way and that my shift is over at 6pm. I detach myself a little so I can approach things calmly, because I have other projects in my life too, which mean I can't put a mental load on myself. (Advisor 1) I'm only available to help when things get difficult. For example, if a participant were to become homeless. Then, of course, I mobilize my time, even if it means working overtime, but I only do it because it's an emergency. (Advisor 2)
Increased Emotional Resilience
In our interviews, we identified several instances where program staff learned how to cope with situations that generate negative emotions and maintained a state of calm. The advisors reported that their work has exposed them to inequalities in society and challenges that they were unable to resolve. In such situations, they were likely to feel frustrated and angry. Over time, they realized the importance of not allowing themselves to be overwhelmed by these negative emotions and have learned to accept situations that they cannot change and to minimize their negative impact on themselves. A year ago, it would have consumed me to think that I hadn't gone far enough, that I hadn't helped her, and now I've said to myself you can't do any more. (Advisor 3)
Therefore, when it is no longer possible to make further efforts to find a solution, our respondents point out that they now know when to stop, take a step back and focus on what they can learn from their experience. You have to be willing to move on. You have to say to yourself, OK, I've tried, I've learned this, this and that. Too bad, it didn’t work, I'm doing something else and taking what I've learned and moving on. My goal is to recover the positive whenever a negative event happens. (Advisor 4)
Improved Teamwork Skills
The program staff members, more specifically workshop leaders, discussed improving teamwork skills. This change was achieved through the process of making mistakes and learning from them, leading to becoming aware of the limitations of individual work. For instance, one workshop leader worked alone on a workshop designed for a large number of participants. He then found it difficult to convince his colleague to run the workshop, or even to persuade the program participants to attend. After recognizing that not involving his colleagues had negative consequences, he changed his behavior and learned how to work effectively with others.
For others, this change was the result of an appreciation of the positive results achieved by bringing together people with diverse and complementary skills on a single mission. With several of us, using different skills and linking competencies, we were able to reach a wider audience and offer more comprehensive, high-quality content. (Advisor and workshop leader 5)
As a result, workshop leaders are more likely to work as a team in preparing workshops, and to ask their colleagues for help on matters that fall within their area of expertise.
Improved Organizational and Time Management Skills
This skill covers both the organization of workshop creation and the planning of everyday activities. In terms of workshop organization, workshop leaders have a better mastery of the steps required to successfully complete a workshop (finding resources and materials, gathering information on a subject, designing a visual aid, attending team meetings to define the scope of work, calling on guest speakers, etc.). As a result, they are better able to anticipate these steps and successfully carry out a workshop on time.
Organizational skills, on the other hand, concern the planning of daily work and the provision of adequate resources to perform different tasks within the required timeframes. In this context, advisors and workshop leaders had an intense work rhythm and were overwhelmed and dispersed. Therefore, they found it challenging to track the progress of program participants or appointments and lacked efficiency in work. This led them to identify habits that help them to be productive and to better plan their week. For example, they would create schedules, dashboards and Excel tools on their computers, where they collated their objectives, reports, conversations with talent and other observations.
These tools, used on a daily basis, helped our respondents to better plan their week and avoid procrastination. Organization means anticipation, so in the end, I felt less worried about appointments, because it was much more formal, there was no repetition, it was more settled. (Advisor and workshop leader 6)
Valuing Outcomes
Twenty-two outcomes were identified in the theory of change, eleven of which were experienced by RtR program participants, and they fall into five categories, ten by RtR team members and one by community members (Figure 1).
For some outcomes, it was possible to determine a direct valuation. For example, the financial proxy for finding employment was based on the increased income of the program participants. When no direct valuation was possible, we employed the market comparison method, also known as replacement valuation and comparison cost. These financial proxies reflected an estimate of what it would cost to achieve the same outcome on the market of traded goods and services.
SROI methodology is based on the involvement of stakeholders who experience changes due to the intervention under investigation (Perrini et al., 2021). To make the process of assigning value more robust, authors suggest that the financial proxies described by beneficiaries should be tested through further research for appropriateness and relevance (Banke-Thomas et al., 2015).
This strategy was employed to value the results. We first asked respondents to estimate an alternative service that would have produced a similar result and have a market price, and then we selected the answer given by the majority of respondents to value an outcome. For example, the cost of paying for professional coaching served as a proxy for choosing a professional career path (€166 per hour×4 session/month×10 months). In cases where respondents provide inconsistent responses about the value of the same outcome, the research team identified an appropriate financial proxy from existing research. The agreed financial proxy value of each outcome was then multiplied by how many of the participants experienced that particular outcome. An estimate for how many participants experienced each outcome was based on interview data. For example, the proxy for choosing a professional career path is the cost of career coaching sessions, with a unit value of 166 € per hour×4 session/month×10 months = 6,640 €. Out of the 54 interviews with program participants, 12 participants experienced this result, which is the equivalent of 22.22%. In this instance, the unit value of the outcome was multiplied by the number of participants who have completed the program and then by the percentage of those who identified the outcome in question. For the example of choosing a professional career path, this is represented as follow: 6,640 €×1,381 × 22.22% = 2,037,742 €.
Table 2 describes the outcomes for each category of stakeholder, the outcome indicators and the financial proxies we used to monetize outcomes.
Outcomes, Indicators, and Financial Proxies.
Applying Discounts
It was identified from program data on 716 program participants that no positive outcomes were displaced by participation in the program, therefore, displacement values were set to 100% (preservation value i.e., 0% discount). Furthermore, it was identified that drop-off was 100% (presentation value) as the program participants said that they did not expect their experience of the outcomes to drop off. Longitudinal research would be needed to verify this. However, values were applied for deadweight and attribution. The discount information was provided by the interview data. For example, program participants identified that the intervention was largely responsible for them experiencing self-esteem, development of entrepreneurial skills, improved physical health, and job readiness and employability, resulting in between 90% and 100% (preservation values) of these outcomes being attributed to the intervention. Program participants did not believe that they would have experienced self-confidence, improve mental health and reduce anxiety, and choosing a professional career path without the intervention, resulting in deadweight of between 82% and 85.5% (See Table 3).
Applying Discounts.
Calculating the Social Return on Investment Ratio
To calculate the denominator of our SROI ratio, we determined the monetary value of the various inputs used in all activities relating to the RtR project. This led to a total input value of 9,880,525 €. To calculate the numerator, we first calculated the value of the outcomes, and multiplied those values by the proportion of people expected to experience each of the outcomes. Following this, the discounts for attribution and deadweight are applied; this produced the social value for each outcome for Year 1. Finally, social value is projected across the benefit period of 7 years using a further discount of 2.8% to calculate net present value (NPV) of the outcomes in today's Euros. This discount reflects the inflation rate in France which was estimated in December 2022 to be 2.8% (French National Institute for Statistics and Economic Studies (INSEE).
Taking the outcome of improved physical health as an example, the financial proxy for this outcome was € 3,701,080. Based on data from the program participants interviews, a weighting of 7.41% was applied (to reflect the proportion of participants expected to experience this outcome), then discounts of 23.75% were used for deadweight and 10% for attribution, reducing the social value of this outcome to € 188,138 for Year 1 (that is, € 3,701,080 × 0.074 × 0.7625×0.9 = €188,138). The accumulated social value of this outcome over the 7-year benefit period, minus a further discount from years 2 and 7, was €1,181,040.57. Following this process for all the outcomes, the accumulated social value over the benefit period was:
- €163,884,086.27 for program participants - €339,016.62 for program staff - €30,433.39 for community members.
The total social value over the benefit period for all stakeholders is €164 253 536,29. See Table 3.
Based on these results, we therefore calculated the SROI ratio = present value of outcomes / value of inputs:
This means for every euro of investment in the RtR program, €16,62 of social value was created.
Sensitivity Analysis
A sensitivity analysis was carried out to examine differences in the SROI ratio when different estimates are used (e.g., optimistic/mid-range/pessimistic). The SROI analysis was calculated based on mean responses provided in the interviews for the discounting rates and the most frequently occurring responses for the duration. Therefore, we adopted a mid-range assumption, that we also consider a more likely conservative estimation. The sensitivity analysis was based on the upper and lower boundaries of the suggested ranges and showed that the ratio can fluctuate from €2.64 to €24.40 for every €1 invested. Table 4 presents the most noteworthy findings from sensitivity analysis.
Sensitivity Analysis.
Discussion
Social accounting is a general term of accounting, which goes beyond traditional economic evaluation tools when evaluating investments (Bellucci et al., 2019). This study has demonstrated the utility of SROI methodology in assessing the social impact of a job and skills training program, thereby contributing to social impact measurement research. To our knowledge, this is the first time that SROI methodology has been used to determine the social impact of a French National program for employment. In this study, interviews were preferred over standardized questionnaires, to encourage open discussion with participants and illustrate the theory of change that explains how the intervention activities led to these positive outcomes.
Some of the changes identified among program participants and program staff are consistent with a number of SROI studies of job entry or employment and skills training programs for NEETs and people with disabilities. For instance, our findings align with those of other program evaluations and assessments of social impact in that employment programs deliver broader outcomes than employment, enabling participants to improve their skills and health, including emotional and physical well-being, personal development, and interpersonal relations (Gambhir et al., 2017; Hutchinson et al., 2024; Owen et al., 2015; Walk et al., 2015). Similar to Owen et al. (2015), program staff in our study also experienced increased satisfaction, interpersonal skills in interacting with a disadvantaged audience, and improved career options and job opportunities. Our findings reveal that, while the changes experienced by the program participants are a direct result of the program's activity, most of the changes stem from challenging experiences, followed by a personal effort to overcome difficulties and improve their skills and wellbeing.
Our data analysis suggests that most of the results are likely to be long-term in nature, knowing that we have adopted conservative benefit period assumptions to avoid over-claiming. This aligns with the findings of Walk et al. (2015) for whom tangible and intangible outcomes resulted from employment and skills training programs for NEETs were expected to endure beyond program participation.
SROI widely engages with stakeholders yet calculating a monetary value of outcomes remains a challenging task, particularly when some program participants give conflicting responses to valuing the same outcome. These challenges can be overcome by reviewing similar studies and consulting with the research team to determine an appropriate financial proxy.
The reported SROI ratios varied from 2.64:1 to 24.4:1,and represent positive ratios for funders’ investment. This analysis demonstrates that RtR program has generated significant social and economic outcomes for the participants, as well for other stakeholders who are affected by the program. The achievement of these outcomes is reliant upon the valuable work that the program undertook with the job seeking NEETs and unemployed women in disadvantaged neighborhoods. Overall, the intervention had a positive social value on the program's participants of both employment path and entrepreneurial path. None of the program participants reported a negative outcome, but some of them declared that the RtR did not bring any change for them (3.8%).
One of the key reasons for initiating a SROI study is to demonstrate social value and to communicate the generated social impact to funders and other supporters. Another relevant aspect is to provide opportunities for similar organizations and stakeholders to learn from the RtR's experiences. In the RtR program, participants benefitted from regular one-to-support and coaching, being matched to workshops that addressed their personal employment or entrepreneurial goals, personal development opportunities, and financial support. The length of the program allowed people to explore employment interests or business ideas, set goals and develop and embed skills over a period of several months (up to 16 months) rather than expecting quick employment outcomes which may not be sustainable. Furthermore, the later changes to the program when funding was extended made access to the program more streamlined thereby removing potential barriers to participation. By using the RtR's SROI results, other organizations can gain a better understanding of how social impact is created, and identify the activities and processes that generate the greatest social impact.
We identify several limitations to our study that need to be considered when interpreting these findings. Firstly, the qualitative analysis might be limited insofar as we spoke to a specific group of program participants who completed the program and were contactable, and their opinions may not accurately reflect the perspectives of the entire cohort. Given limited demographic data on participants, it was not possible to determine if the participants that completed the program differed from the group that dropped out of the program. Secondly, our SROI study does not reach all the affected stakeholders. The RtR program could have affected the State, in terms of reductions in welfare payments and tax increases. It could also have affected the families of the participants and their immediate circle, as well as the companies that hired them. Including these stakeholders in our analysis would have resulted in a more complete picture of the impact of the RtR project. Future research is needed to examine longitudinal data to determine the specific magnitude of these changes to verify the persistence of impacts described in the interviews. Thirdly, the benefit period was set at seven years which may have been inadequate to capture all the longer-term benefits of the program. Fourthly, though we captured the extent to which other factors may have contributed to the successful outcomes of RtR in the SROI equation (attribution), we acknowledge that each person has different personal resources (such as for example, family support) which may influence their ability to stay in the program as well as to secure and maintain employment or a business. Finally, as with other employment programs internationally during this timeframe, the impact of the COVID-19 global pandemic must be acknowledged. Given lock downs in France and the curtailment of usual economic activities, this likely negatively impacted on the outcomes of the program. Furthermore, the shift to virtual workshops and one-to-one support via video calls instead of in-person may have impacted on the programs ability to meet personal needs. Given the global impact on mental health of the pandemic (Kola, 2020), it may be that participants faced additional barriers to job seeking during this period.
Conclusion
This significant investment by the French Labor Ministry has been shown to deliver substantial social value to program participants, program staff and the community. Though this SROI ratio is quite large, it is likely to be conservative given that sustained employment over time often leads to higher salaries/profits and a compounding of many of the other positive benefits identified here. At the societal level, a reduction in welfare payments and an increase in taxation would likely have increased the SROI substantially. This analysis shows the efficacy of diverse supports and one on one coaching for disadvantaged job seekers.
Footnotes
Acknowledgements
The authors gratefully acknowledge the invaluable assistance of Dr. Loïc Aubrée, research director of Lille Catholic University. We are indebted to Catherine Dervaux and Cécile Meurillon for their help in our data collection.
Ethical Considerations
This research received ethical approval from the “Laboratoire d’Intelligence Collective et Artificielle (LICA)”. The authors declare no ethics number was given.
Funding
This work was funded by the French Labor Ministry (grant number 20190088).
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Informed Consent
Informed consent was obtained from all research participants.
