Abstract
Keywords
Poor outcomes for out-of-home care (OHC) experienced youth have been shown consistently in research for decades across contexts, comparison groups, and outcomes (Brännström et al., 2020; Gypen et al., 2017; Maclean et al., 2016; Sariaslan et al., 2022; Wiberg, 1976). Previous research on Swedish care leavers has found an elevated risk for experiencing lower general well-being (Olsson & Skoog, 2025), poor physical, mental, dental, and reproductive health outcomes (Berlin et al., 2018; Vinnerljung et al., 2006, 2007; Vinnerljung & Ribe, 2001), as well as poor educational attainment (Brännström et al., 2017; Forsman, 2020), substance use (Vinnerljung & Hjern, 2014; von Borczyskowski et al., 2013), public welfare dependence (Brännström et al., 2017; Vinnerljung & Hjern, 2011), and low income and/or detachment from the labor market (Brännström et al., 2017; Österberg et al., 2016). Some of these negative outcomes persist throughout the life course (e.g., Brännström et al., 2017; Brännström et al., 2020; Seker et al., 2022). Moreover, care leavers report a lack of sufficient preparation for and participation in their own care-leaving process (Bengtsson, 2022; Höjer & Sjöblom, 2014a, 2014b).
As child welfare services (CWS) assume responsibility for children by placing them in OHC, they bear the responsibility of preparing these young individuals for their transition from OHC (Mendes et al., 2014). Provision of interventions (Sundell & Olsson, 2017) that support youth in preparing for the transition has been highlighted as one way to mitigate negative outcomes and prevent future adversities for this group (Beauchamp, 2016; Stein, 2019). In this study, we assess a priori fit of two independent living services (ILS) that are currently under development in Sweden. ILS are interventions that aim to “provide young people leaving care with skills that will limit their disadvantage and aid in their progress to adulthood” (Montgomery et al., 2006, p. 1436). Examples of independent living skills include: financial management skills, study- or work skills, and psychosocial skills such as emotional regulation and conflict resolution (Taylor et al., 2021). Methods employed by ILS to help youth develop such skills vary (Cruz et al., 2025).
Several systematic reviews conducted during the past 10 years have attempted to establish evidence of transition services’ benefits (Greeson et al., 2020; Gunawardena & Stich, 2021; Montgomery et al., 2006; O’Donnell et al., 2020; SBU, 2020; Taylor et al., 2024; Woodgate et al., 2017). Although intervention outcomes identified in these reviews were mixed (e.g., Gunawardena & Stich, 2021; Taylor et al., 2024), some promising results are presented, indicating that youth benefit from certain services, in areas such as housing, education, employment, and life skills (Gunawardena & Stich, 2021; O’Donnell et al., 2020; Yelick, 2017). Less common is evidence of transition services’ effect in supporting youth mental, physical, and social health (O’Donnell et al., 2020). Additionally, systematic reviews shed light on the paucity of studies on ILS developed in countries other than the United States (Evans et al., 2023; O’Donnell et al., 2020). This is important since the relevance of interventions under study may not be comparable across contexts (Olsson, von Thiele Schwarz, et al., 2024) due to national differences in policy, legislation, and practice (O’Donnell et al., 2020; Yelick, 2017), as well as differences in target group characteristics (e.g., Olsson et al., 2020).
Directed services to support youth in their transition from OHC to independent living are not commonly provided by the municipal social services in Sweden (Bergström et al., 2023; Höjer & Sjöblom, 2011; SBU, (2017). Unlike other countries, Sweden has no legislation requiring transition service provision for youth in OHC (Stein, 2019; Storø et al., 2019). In response to this, two initiatives have recently emerged, both aiming to develop novel (Olsson, von Thiele Schwarz, et al., 2024) support services for OHC leavers. My Choice—My Way! (MCMW, swe. Mitt val—Min väg!; Olsson, Bergström, et al., 2024) is being developed collaboratively between university researchers and social workers and is provided by the municipal CWS. The Youth Programme (YP) is being developed and provided by an international nongovernmental organization.
When new interventions are introduced into organizations it is not uncommon that they fail to achieve the intended change. For an intervention to be successful, it is important that it fits both the needs of the target population (i.e., person–intervention fit) and the context in which it is to be delivered (i.e., environment–intervention fit; Randall & Nielsen, 2012). If the newly developed ILS has poor person–intervention or environment–intervention fit, it is unlikely to lead to any meaningful changes for the young person transitioning from OHC. Good person–intervention fit is achieved when the program theory matches the actual needs of the target group. Environment–intervention fit is achieved when the intervention is seen as feasible and acceptable within the context of delivery (i.e., by individual service providers, organizational prerequisites, or by the society at large).
Evans et al. (2023) call for the integration of several different focus areas in program evaluation for a stronger evidence base, proposing increased attention to program theory, implementation, and process evaluation in addition to outcome evaluation. The information gained through such deeper investigation is essential for understanding how an intervention may operate and interact with the context and systems in which it will be implemented (Evans et al., 2023; Hawe et al., 2009). This is information that could be useful already during intervention development, to minimize the likelihood of developing interventions with poor fit.
In the process of developing novel interventions, different strategies can be used when specifying the program theory as well as the format and content of the intervention (O'Cathain et al., 2019). Implicitly, such strategies are largely aimed at ensuring intervention fit. Investigating intervention fit prior to scale up or dissemination therefore constitute an important step in the intervention development process and can provide a more in-depth understanding of the issues relevant to both service users and practitioners (Fraser & Galinsky, 2010). Furthermore, results from such studies can uncover and support efforts to mitigate possible implementation barriers early on, thereby avoiding resource waste by introducing a large-scale program, which is likely to fail (Damschroder et al., 2009; Donald, 2018; Powell et al., 2015).
Ethical practice in the practice of intervention development also brings to light a tension between the need to design studies robust enough to capture intervention effectiveness through, for example, large sample sizes and comparative designs while also protecting youth from being exposed to harmful interventions prior to an understanding of the intervention's effects. Even well-intentioned interventions can be harmful (Williams et al., 2021). A conservative, and arguably ethical, approach to intervention development would be to carefully and transparently investigate important aspects of the intervention, such as intervention fit, from multiple perspectives prior to large-scale testing so that any identified weaknesses in design can be ameliorated when warranted (Eldridge et al., 2016; Fraser et al., 2009; Whitehead et al., 2014). In addition to identifying intervention components that may require revision, the assessment of intervention fit in a small-scale study can assist in the development process by providing useful information on how interventions or implementation strategies could be modified (Bowen et al., 2009).
As both programs reported on in this paper were still under development, the two studies can be viewed as preimplementation studies. In order for an ILS to fit the individual youth and the context, it is important that it is perceived as acceptable, appropriate, and feasible by youth and the social workers providing the service (e.g., Proctor et al., 2011). An ILS is acceptable when it is perceived by youth and service providers as agreeable, palatable, or satisfactory. Appropriateness concerns the extent to which the program fits the needs of the youth (person–intervention fit) and the extent to which social workers perceive the program as fitting the needs of the youth. The feasibility of an ILS refers to the extent to which it can be successfully used or carried out in the intended context (Proctor et al., 2011). Issues of feasibility could arise due to intervention characteristics (innovation domain; e.g., complexity, adaptability, and design), the immediate organizational context (inner setting; e.g., staff turnover and available resources), or the external context (outer setting; e.g., sociocultural values, policies and laws, and financing; Damschroder et al., 2022b).
Proctor et al. (2011) conceptualize acceptability, appropriateness, and feasibility as implementation outcomes that are of particular importance in early stages of implementation. Other authors have described the same three constructs as potential predictors of implementation outcomes (Damschroder et al., 2022a). Thus, we believe these three constructs to be valuable as a priori indicators of fit, together with fidelity (i.e., the degree to which the intervention is implemented as intended; Proctor et al., 2011).
Despite being conceptually distinct, fidelity, acceptability, appropriateness, and feasibility may interact and reinforce each other (Fehlberg et al., 2024). If, for example, social workers perceive the intervention as too complex and the intervention materials as difficult to use (i.e., issues of feasibility), they may not see it as matching the needs of the youth (Damschroder et al., 2022b). This could result in social workers not following the basic plan for service delivery, thus compromising fidelity (Olsson, von Thiele Schwarz, et al., 2024).
Study Objectives
In this article, we report on two parallel studies, one on MCMW (Study I) and one on YP (Study II). The goal of both studies was to increase the understanding of how we can develop ILS with good intervention fit, by identifying areas of importance for further development and revision to increase the fit of MCMW and YP. The specific aims of both studies were to (1) assess the extent to which youth and social workers perceive the program as fitting to their unique circumstances, (2) contribute to an increased understanding of possible future barriers to program uptake, and (3) assess the extent to which further development and testing of the programs is warranted.
We ask the following research questions:
To what extent did MCMW and YP follow the basic plan for service delivery? To what extent did social workers adapt program content to individual circumstances? To what extent were MCMW and YP perceived as acceptable, appropriate, and feasible from the perspective of youth and social workers?
The aim and methodology were the same for Studies I and II. Although the aim is not comparative, we report on both studies in one paper. We believe that the ability to discuss the results of both studies conjointly elevates and benefits the discussion and conclusions.
Study I
Study Design, Study I
We used longitudinal survey design with pretest–posttest design for the youth participants and repeated measure design for the social workers. Quantitative and qualitative data were collected through questionnaires administered to social workers and youth. We gave equal weight to both data types during analysis. This means that the different data types had a converging function and were merged to answer the same research questions from different perspectives (c.f. Palinkas et al., 2011).
Participants and Procedure, Study I
All youth who were participating in MCMW and the social workers who were providing direct services to youth participants were eligible for participation in the research study.
Recruitment Procedure, Study I
Youth. Youth participants were recruited to MCMW through the CWS’ standard procedure between June 2021 and September 2022. Social workers informed youth about the research study and provided written information about the research and their rights as participants. If interested, youth were referred to the research team who contacted youth to provide further verbal information and obtain informed consent. All youth participants were offered a gift card voucher worth 99 Swedish crowns (approx. 10 €) each time they completed a questionnaire.
Social Workers. Social workers tasked with providing MCMW to youth were contacted via email by a researcher who provided written information about the study in conjunction with an invitation to participate. Social workers were included in the study upon giving their informed consent to participate.
Participants, Study I
Youth. Fourteen youth were referred to the study and eleven were recruited into the study. Nine youth (82%) completed both the T1 and T2 questionnaire (Figure 1). The mean age of the participants was 17.7 years, with 64% being female and 36% male. The majority (70%) had a first language other than Swedish and were born outside of Sweden (54%). Although 91% had at least one living parent at baseline, only half (55%) had a parent currently residing in Sweden. Most participants (91%) attended school at baseline (T1). Youth characteristics are summarized in Table 1.
Flowchart Detailing Youth Participant Flow Through Study I.
Youth Background Characteristics at Baseline, Study I (N = 11).
Social Workers. Four social workers were recruited and two (50%) completed at least one of the weekly questionnaires. Participating social workers worked with direct service provision via the municipal social CWS in a large urban area. Background information was not collected on the social workers.
MCMW Program Description
MCMW (Olsson, Bergström, et al., 2024) aims to offer support and develop skills to facilitate transition from OHC to independent living. MCMW is based on a clear theory of change (Olsson et al., 2023) and targets the primary outcome areas of education, employment, positive routines, and positive help-seeking behavior. MCMW is individually provided by trained social workers to youth placed in OHC aged 15 + years. Deliverers of MCMW were trained during 2 days in a remote (digital) group of 15 participants by one of the program developers (professor in developmental psychology). The training included theoretical, didactic, and interactive components as well as individual and group reflection exercises.
The MCMW program theory draws upon principles from self-determination theory (Deci & Ryan, 2000), social cognitive theory (Bandura, 1986), and the Capability, Opportunity, and Motivation for Behavior Change (COM-B) Model (Michie et al., 2011, 2014), integrating insights from prior research on interventions supporting the transition from OHC to independent living (Blakeslee et al., 2020; Geenen et al., 2015; Olsson et al., 2020; Phillips et al., 2015). Further details of the MCMW program theory can be found in Figure 2.
My Choice—My Way! Program Theory.
Comprising 16 chapters, the MCMW pilot manual includes instructional materials for deliverers and resources as well as optional material for the youth participants (Olsson et al., 2022a, 2022b). The manual assists the social worker in guiding the youth participants through sessions oriented toward their future based on the youth's self-identified goals. One chapter may require more than one session to complete depending on the needs of individual youth. The first block consists of sessions designed to orient the youth toward the future (e.g., goal-setting and prioritizing); the second block consists of a range of knowledge, skill and ability topics, and exercises (coping with stress, communication, building and maintain social relationships and relationships with professionals, applying for education or jobs); and the third block consists of sessions designed to reorient the youth toward the future as well as apply new knowledge, skills, and abilities gained through MCMW (e.g., network meeting and celebrating completing the program).
MCMW spans 4–8 months, and deliverers are encouraged to tailor sessions to best meet the individual youth's needs, planning sessions around issues most pertinent to youths’ self-identified goals. The sessions are carried out in the residence of the youth, staff facilities, or public places.
Data Collection Youth, Study I
Youth were asked to complete self-report measures at two time points: baseline (T1) and follow-up (T2). T2 occurred 8 months after T1. The digital questionnaire was administrated by the research team. All youth were given the option to receive the questionnaire to their email address and complete the questionnaire digitally by themselves or to answer the questions with a researcher by telephone or in person. Questionnaires were distributed via Qualtrics software (2025 Qualtrics).
Youth Measures at T1 and T2, Study I
The following measures were used to assess program appropriateness, meaning how well program components correspond to the needs of the target group (i.e., person–intervention fit). The measures were chosen based on constructs that the intervention aims to improve as defined in the program theory.
Sustainability of Living Inventory (SOLI). SOLI (Hou et al., 2019) is a self-report measure assessing perceived regularity of daily routines anchored in the drive to thrive theory that proposes that stress resilience is determined by sustaining routines in daily life. The scale consists of 42 items encompassing eight routine dimensions: hygiene, eating, sleep, duties at home, leisure at home, exercising, social activities, and work/study involvement. Items are measured on a 6-point Likert-scale ranging from not often to often. Internal consistency on this scale was adequate in Study I (α=.94).
The Multidimensional Scale of Perceived Social Support (MSPSS). MSPSS is a 12-item instrument developed by Zimet et al. (1988). The scale measures perceived social support using three subscales: family, friends, significant others, and a total scale. Items are rated on a 7-point Likert scale ranging from very strongly disagree to very strongly agree. The MSPSS psychometric properties have demonstrated good support for the three-factor model and the scale has been translated into Swedish (Ekbäck et al., 2013). Internal consistency on this scale was adequate in Study I (α=.87).
My Life Self-Efficacy Scale (MLSES). Self-efficacy is measured with the 17-item MLSES (Blakeslee et al., 2020). The scale was specifically developed to assess youths’ beliefs that they can carry out skills targeted by diverse leaving care programs (e.g., problem solving, self-monitoring, and working with adults; Blakeslee et al., 2020) and includes three subscales “self-regulation,” “managing others,” and “achievement” (i.e., actions towards goals), as well as a total self-efficacy score. Items are measured on a 5-point Likert scale ranging from strongly disagree to strongly agree. The MLSES's psychometric properties including factor structure, convergent validity, and reliability have been tested in several studies previously (Olsson et al., 2020; Turner et al., 2023). Internal consistency on this scale was adequate in Study I (α=.86).
Resilience Scale (RS-14). The RS (Wagnild & Young, 1993) was developed to assess the successful psychological adaptation to adversity or stress in the general population. It is one of the most widely used resilience measures globally and has been used with a variety of study populations, including youth. The short version RS-14 was used in the current study. Items are rated on a 7-point Likert scale ranging from strongly disagree to strongly agree. The measure has been used in at least two prior studies of youth in OHC (Blakeslee et al., 2020; Olsson et al., 2020). License to use the Swedish version of the RS-14 was granted by The Resilience Center (https://www.resiliencecenter.com) on October 27, 2021 and May 8, 2024. Internal consistency on this scale was adequate in Study I (α=.90).
American Institutes for Research (AIR) self-determination scale. Self-determination was measured using the 18-item AIR self-determination scale (Wolman et al., 1994). The scale assesses individual aptitude for and opportunity to exercise self-determination, as these are distinct dimensions of self-determination and particularly of interest in relation to youth in OHC, who may experience limited opportunities to demonstrate self-determination. Three of the instrument's subscales were included: “Things I do,” “How I feel,” and “What happens at school” as well as an overall self-determination score. Items are measured on a 5-point Likert scale ranging from never to always (e.g., Garrels & Granlund, 2018; Turner et al., 2023). Reliability and validity have been tested in several prior studies (Geenen et al., 2013; Olsson et al., 2020; Wolman et al., 1994). Internal consistency on this scale was adequate in Study I (α=.93).
Youth Measures at T2 Only, Study I
The following measurements were included in the follow-up questionnaire only and were used to assess fidelity and acceptability:
Program Participation. Youth were asked to assess the extent to which they participated in the program by answering two questions. (1) How long did you participate in the program? and (2) Approximately how many times did you meet the person that was helping you in the program? We used this measure to assess program fidelity.
The Client Satisfaction Questionnaire (CSQ-8). Client satisfaction was measured at T2 using the Swedish version of the eight-item CSQ (Attkisson & Greenfield, 2004; Larsen et al., 1979). The items are measured on a 4-point scale. Total scores range from 8 to 32, with higher values indicating greater client satisfaction. The CSQ-8 has been validated among several populations and has been found to have good psychometric properties (e.g., Attkisson & Greenfield, 1994; Pedersen et al., 2022; Roberts et al., 1984). License to use to CSQ-8 (TMS2001442) in this study was provided by Tamalpais Matrix Systems, LLC on October 28, 2020, and April 1, 2021.
Open-Ended Questions at T2. Open-ended questions were used to generate qualitative answers that were used to assess acceptability, appropriateness, and feasibility. Open-ended questions at T2 included: (1) Reason for non-completion of the program and (2) “Please write in your own words what you thought about participating in the program.”
Data Collection Social Workers, Study I
Social workers received a weekly email between May 2021 and July 2022 with a link to a questionnaire about their perception of their ongoing work with the program. The questionnaire was the same each week and very brief as to minimize time consumption. Questionnaires were distributed via Qualtrics software (2025 Qualtrics).
Social Worker Measures, Study I
The weekly questionnaire included the following instruments. The scales were translated from English to Swedish using consensus method (Douglas & Craig, 2007).
Acceptability. We used the four-item acceptability of intervention measure (AIM; Weiner et al., 2017) to assess social workers’ perceived acceptability of discrete components of each program. Items were rated on a 5-point scale ranging from completely disagree to completely agree.
Appropriateness. We used the four-item intervention appropriateness measure (IAM; Weiner et al., 2017) to assess social workers’ perceived appropriateness of discrete components in each program. Items were rated on a 5-point scale ranging from completely disagree to completely agree.
Feasibility. We used the four-item feasibility of intervention measure (FIM; Weiner et al., 2017) to assess social workers’ perceived feasibility of discrete components in each program. Items were rated on a 5-point scale ranging from completely disagree to completely agree.
Open-Ended Questions. The weekly questionnaire included open-ended questions which were used to generate qualitative survey responses. Social workers were asked which program component they had been working with during the past week and were encouraged to detail their experience of working with this specific component in their own words through open-ended questions. Social workers were asked to describe the extent to which they adapted (i.e., changed) the planned component during the given week. They were also asked to describe how and why they deviated from the component content as described in program materials. We used the results from the self-reported adaptations to assess fidelity.
Quantitative Data Analysis, Study I
We used IBM SPSS Statistics software version 29.0.0.0 and Microsoft Excel to conduct all statistical analyses. Group-level differences between T1 and T2 on youth measures were analyzed using descriptive statistics (e.g., percentages) and paired samples t tests. Due to the low sample size, we did not hypothesize for these tests to achieve statistical significance.
To gain a more nuanced understanding of program appropriateness, we used the Leeds Reliable Change Calculator (Leeds Reliable Change Indicator [LRCI]; Moreley & Dowzer, 2014) to assess differences between T1 and T2 measures at the individual level. The LRCI calculates whether an observed difference between an individual's pretest and posttest score is of a magnitude that cannot be explained by measurement error (Moreley & Dowzer, 2014). This method allowed us to follow individuals across outcomes over time and assess whether they had made an improvement, deteriorated, or exhibited no reliable change.
The reliable change index (RCI) method has been used in previous studies with relatively low sample sizes (e.g., Chemtob et al., 2019; Ghahramanlou-Holloway et al., 2020). Despite originating from a medical context, the RCI method can be useful for early detection of systematic deterioration as a result of participation in psychosocial interventions (e.g., Rozental et al., 2017). Systematic deterioration can be an indicator of iatrogenic effects (poor fit). However, when relying solely on group-level analysis, systematic deterioration in individuals’ may be overlooked. We included only those youth who completed the questionnaire at both T1 and T2 in these analyses. A response rate below 80% on individual items was treated as missing for the total scale score. We assessed the robustness of results by removing the individuals who responded not remembering participating in any program and reanalyzing the data.
For the social worker data, we averaged scores of the AIM, IAM, and FIM (both total scores and at the item level) for each program component. A mean value below three on single items was interpreted as negative, a mean of three was interpreted as neutral, and a mean above three was interpreted as positive. This interpretation is in line with the wording of the response alternatives. We present results in a heat map, a method particularly useful for visualizing patterns in data (Gehlenborg & Wong, 2012; Salvati et al., 2023).
Qualitative Data Analysis, Study I
We assessed responses to open-ended questions using qualitative content analysis (Graneheim et al., 2017; Graneheim & Lundman, 2004). Four overarching themes: fidelity (adaptations), acceptability, appropriateness, and feasibility were developed a priori in a deductive manner (Elo & Kyngäs, 2008; Graneheim et al., 2017) based on theoretical conceptualizations (Proctor et al., 2011). This approach was used with the purpose of examining the qualitative material for indicators of intervention fit, in line with the study objectives. The unit of analysis (Graneheim & Lundman, 2004) was each separate survey response. After removing responses where social workers indicated not having met any youth to work with MCMW during that week, the total number of units of analysis was 18 in Study I. The open-ended question in the youth questionnaire generated six responses.
Matilda Karlsson (MK; first author) read the qualitative survey responses several times and identified meaning units belonging to either of the four overarching themes. Where relevant, MK subsequently grouped meaning units within the overarching themes into subcategories by searching for similarities and patterns (Graneheim et al., 2017). In this step, subcategories were created by grouping meaning units based on manifest, rather than latent, content. Subcategories can thus be considered as descriptive of commonalities in the responses within the themes (Graneheim et al., 2017). This step was not performed for the youth data as there were too few responses for grouping into subcategories. During the analysis process, the entire research team met to discuss the analysis. Occasional disagreements were discussed in the whole group until consensus was reached. No software was used in the analysis of the qualitative material.
Results Study I
Fidelity
Program Participation. Youth outcomes can be found in Table 2. Seven youth (78%) recalled participating in MCMW during the past 8 months. Of these, five were either still participating in or had completed the program at T2. Less than half of the youth (43%) had attended 16 sessions or more, which would be the minimum anticipated number of program sessions at program completion (if each of the 16 chapters were to be completed in one weekly or biweekly session each). The median number of sessions attended was 10. Two youth reported starting the program but discontinuing before completion. The open-ended responses as to why participation was discontinued included the perception of not being helped by the program and being occupied with work.
Table of Results Youth-Level Outcomes Study I. (N = 7/9). a
Note. T1 = baseline; T2 = follow-up; CSQ-8 = Client Satisfaction Questionnaire; SOLI = Sustainability of Living Inventory; MLSE = My Life Self-Efficacy Scale; RS-14 = Resilience Scale; AIR = American Institutes for Research.
Knowledge of participation and youth level outcomes; n = 9, the remainder of questions were answered only by those who recalled participating (n = 7).
Adaptations. Social workers described making adaptations to the program in five ways. First, social workers improvised activities. For example: “We talked about [a strategy not included in the manual] and how one can start to recognize negative thoughts.” (SW1). Second, social workers allowed youth participants to have support people in the room. For example: “[The youth] had their partner with them, [the partner] participated with their reflections.” (SW2). Third, social workers reported providing practical support. For example: “I gave the youth support in writing a resumé and application letter, and we talked about jobs they have applied to” (SW1). Fourth, social workers reported completing home assignments during sessions. For example: “We completed the take-home assignment during the session because the session would otherwise be too short, and we had an interpreter booked for one hour” (SW1). Finally, social workers described using sessions in a way not connected to the youth's goal. For example: “[The youth] didn’t really have any goals in this area but I found it important to talk about and check off (…)” (SW1).
Acceptability Youth
Overall, youth who recalled participating in MCMW during the past 8 months reported high rates of client satisfaction (M = 27.3; Table 2). Youth expressed a general appreciation of the program through the open-ended responses. For example, youth reported experiencing MCMW as: “Fun” (Y1), “So appreciated” (Y3), and “I think it is good” (Y4).
Acceptability Social Workers
Social workers reported finding all MCMW modules to be acceptable apart from “building a network” toward which they were negative, and “prioritizing” toward which they were neutral (Figure 3).
Service Provider Assessment of Program Acceptability, Appropriateness and feasibility, Heat Map, Study I. N = 2.
Qualitative responses from social workers resulted in three categories under the theme of appropriateness. The first included general comments where social workers expressed appreciation or disapproval but did not specify any reasons. For example: “I am very positive toward the basic idea of this module” (SW1), “The planning tool in [this session] was great!” (SW1), and “I am quite critical towards this session”(SW1). Social workers also expressed opinions concerning the composition, that is, the structure or use of some program activities: “I would not recommend this [activity] as a take-home assignment at all. [It] can uncover a lot of emotions and isn’t something a youth should be left alone with” (SW1), and “The take-home assignment is the same thing we talked about in the session and mostly feels like repetition” (SW1) are examples from this category. Finally, social workers were concerned about triggering content in a specific module: “I can imagine these tips as being a bit triggering for youth and perhaps even condescending” (SW1).
Appropriateness Youth
No youth reported perceiving the program as having led to increased difficulties in managing problems. All but one of the youth who recalled participating in MCMW perceived the program as at least somewhat helpful (Table 2).
At the group level, no statistically significant differences in any of the youth-level outcomes used to assess appropriateness were found between T1 and T2 (Table 2). There was no effect for social support (g = −0.04; confidence interval [CI] = [−0.63, 0.55]), small and insignificant effect sizes for positive routines (g = −0.37; CI = [−0.97, 0.26]), resilience (g = −0.46; CI = [−1.07, 0.19]), and self-determination (g = −0.22; CI = [−0.81, 0.39]), and a moderate but insignificant effect size for self-efficacy (g = −0.63; CI = [−1.28, 0.05]).
Individual analysis revealed that across outcomes, with the exception of social support and self-determination, more youth exhibited reliable change in the positive direction (n = 2–4) compared to the number of youth exhibiting reliable change in the negative direction (n = 1). Only one youth exhibited reliable change in the negative direction across more than two measures; this youth did not recall participating in MCMW. When removing the two youth who did not recall participating in MCMW from the analysis, deterioration was found only for the measures of social support (n = 1) and self-determination (n = 1). Reliable change in the positive direction was found for a varying number of youth across all measures (n = 1–3). Examples of qualitative comments from youth concerning perceived appropriateness included: “It is a very interesting program for us” (Y2) and “Unbelievably good, I feel like I have been able to start up many things and learned a whole lot too” (Y3).
Appropriateness Social Workers
Regarding the fit of the program components to the youth, social workers were generally positive. Building a network was the only component that generated negative responses across all items. Social workers were neutral toward the appropriateness of the component about prioritization (Figure 3).
We identified three categories within the theme of appropriateness among the qualitative responses. First, social workers were concerned with issues of the program being abstract vs. practical. They highlighted the importance of the program being useful for youth as opposed to being too abstract and theoretical. For example, “For this youth it has been difficult with the abstract thinking that is required by parts of this module and previous modules” (SW1) and “This module works very well and gives practical support for the youth” (SW1). A second concern was that of language. The appropriateness of the material for those youth whose first language was not Swedish was debated: “It is not adapted at all to youth who don’t speak Swedish, and a lot of things can be interpreted in several different ways” (SW1). Finally, social workers expressed concerns about the relevance of some program activities in relation to the youth's needs: “[This exercise] did not seem relevant for this youth whatsoever since [they] don’t have any difficulties in those areas (…)” (SW1) and “Many questions in [these exercises] seem totally irrelevant unfortunately, or I don’t understand how [the exercises] should be used” (SW1).
Feasibility Youth
We identified only one of the youth comments at T2 as referring to feasibility, as it concerned the structure rather than the content of the program: “It was disorganized, I did not like it” (Y6).
Feasibility Social Workers
Social workers were positive when it came to the possibility of successfully delivering the program activities in most of the MCMW components, the component concerning network building and prioritization being exceptions. The item concerning user friendliness (Q4) generated the most negative responses across components (Figure 3), mainly due to an experience of general disorganization and unclarity of the manual rather than program content (as discovered when examining the qualitative responses).
Responses to the open-ended questions under the theme of feasibility were coded into two categories. The first concerned the material structure. Social workers generally expressed a perception of the material as disorganized: Working with the manual in general is difficult. It is messy and appendices, modules and work sheets are in so many different places which makes it hard to get a good overview and understanding of how to work with it in a simple and clear way. (SW1)
Study II
Study Design, Study II
The study design was the same for Study II as for Study I.
Participants and Procedure, Study II
All youth who were participating in YP and the social workers that were providing direct services to youth participants were eligible for participation in Study II.
Recruitment Procedure, Study II
Youth. Youth participants were recruited to YP through the organization's standard procedure between June 2021 and September 2022. Program staff informed youth about the research study and provided written information about the research their rights as participants. If interested, youth were referred to the research team who contacted youth to provide further verbal information and obtain informed consent. All youth participants were offered a gift card voucher worth 99 Swedish crowns (approx. 10 €) each time they completed a questionnaire.
Social Workers. Social workers tasked with providing YP to youth were contacted via email by a researcher who provided written information about the study in conjunction with an invitation to participate. Social workers were included in the study upon giving their informed consent to participate.
Participants Study II
Youth. Twenty youth were referred to the study and 19 were recruited. Twelve (63%) youth responded to both the T1 and T2 survey (Figure 4). The sample included slightly more female (58%) participants than male (42%). The mean age was 20 years. Most participants were born outside of Sweden (89%) and had a first language other than Swedish (83%). The majority (95%) had at least one parent that was alive at baseline, and 32% had a parent that was living in Sweden at the time. A little more than half (63%) were attending school at baseline. Participant characteristics are summarized in Table 3.
Flowchart Detailing Youth Participant Flow Through Study II.
Youth Background Characteristics at Baseline, Study II (N = 19).
Social Workers. Twelve YP social workers were recruited, and nine (75%) completed at least one questionnaire. All participating social workers were employed as mentors and worked with providing YP directly to youth. Background information was not collected on the social workers.
YP Program Description
YP is a mentorship program that provides practical and emotional support as well as skill building within four areas: (1) mental and physical health, (2) society and housing, (3) employment and education, and (4) network and community. The program was originally developed to support unaccompanied minors in preparing for leaving OHC, but the target group has since been broadened to include all care leavers. Youth self-refer to YP but may be recommended to do so by the social services or other adults in their network. The YP program theory is based on a holistic approach in supporting adolescents and young adults, aged 16–25 years, in their transition from OHC to independent living through mentorship and group activities. A detailed description of the YP program theory can be found in Figure 5.
Youth Programme Program Theory.
Program coordinators recruit, train, and assign mentors to youth and conduct regular follow-ups. The mentors receive a comprehensive online material covering the theoretical framework and methods of YP prior to on-site training. All social workers have appropriate prior academic training and work experience, such as teachers or social workers.
An individual needs assessment is conducted upon entering the program in which youth and program coordinator set up a personal development plan based on locally developed assessment tools and the youth's self-identified goals. Each youth is assigned a mentor (referred to as social worker from hereon) for approximately 6–12 months. The social worker and the youth meet once a week to work on the personal development plan, which can be revised at any time upon request from the youth. The meetings take place in the program facilities or in public places.
The social worker's role is to support personal development through emotional support, and confidence and skill building, rather than solving problems for the youth. The program also offers group activities such as help with homework or job interview training. Optional material and resources that may be used to guide the planning of the sessions is available through a mobile application. After the mentorship program is carried through, youth may keep participating in group activities for as long as they wish.
Data Collection Youth, Study II
The data collection method was the same for Study II as for Study I. One participant completed the questionnaire with a researcher and an interpreter, and one completed the questionnaire in English with a researcher.
Youth Measures, Study II
We used the same questionnaire with the same measures in Study II as in Study I. The measures were chosen based on constructs that the intervention aims to improve as defined by the program theory. For complete descriptions of the questionnaire and measures, refer to the corresponding section for Study I. Internal consistency for scale level outcomes in Study II are reported below.
SOLI. Internal consistency on this scale was adequate in Study II (α=.92).
MSPSS. Internal consistency on this scale was adequate in Study II (α=.86).
MLSES. Internal consistency on this scale was adequate in Study II (α=.84).
RS. Internal consistency on this scale was adequate in Study II (α=.82).
AIR self-determination scale. Internal consistency on this scale was adequate in Study II (α=.82).
Data Collection Social Workers, Study II
Social workers received a weekly email with a link to a questionnaire about their perception of their ongoing work with the program.
Social Worker Measures, Study II
Social workers received a weekly email between May 2021 and July 2022 with a link to a questionnaire about their perception of their ongoing work with the program. We used the same weekly questionnaire in Study II as in Study I.
Quantitative Data Analysis, Study II
We used the same quantitative data analysis methods in Study II as in Study I.
Qualitative Data Analysis, Study II
We used the same qualitative data analysis methods in Study II as in Study I. After removing responses where social workers indicated not having met any youth to work with YP during that week, the total number of units of analysis in Study II was 44. The open-ended question in the youth questionnaire generated nine responses.
Results Study II
Fidelity
Program Participation. A summary of youth outcomes can be found in Table 4. A majority of youth recalled participating in YP (n = 11; 92%). All youth who recalled participating were still either participating in or had already completed the program at T2. Eight youth (73%) estimated having attended less than 16 program sessions (Table 4), which corresponds to the minimum anticipated number of sessions after 8 months if the sessions occur biweekly as described. The median number of sessions attended was five.
Table of Results for Youth-Level Outcomes, Study II. N = 13–11. a
Note. T1 = baseline; T2 = follow-up; CSQ-8 = Client Satisfaction Questionnaire; SOLI = Sustainability of Living Inventory; MLSE = My Life Self-Efficacy Scale; RS-14 = Resilience Scale; AIR = American Institutes for Research.
Positive routines and social support (n = 13); self-efficacy, resilience, self-determination, and knowledge of participation (n = 12); the remainder of questions were answered only by those who recalled participating (n = 11).
Adaptations. Social workers reported making adaptations to the program that we coded into three categories. First, they reported prioritizing urgent practical needs of the youth instead of following the original plan for a session. Examples of this were: “This was not a planned activity, but something that the youth came and requested support with, so in other words everything was adapted to these needs” (SW1), and “At the last session I helped the youth with a practical matter. This was not planned but took over two hours” (SW2).
Second, social workers detailed occasions when they had made alterations to the program by changing standard procedure. For example: I adapted due to it not being appropriate working with the originally planned session in a café. Instead, we have looked at websites mediating housing and talked about how to apply for an apartment. The main goal was put on hold and instead we worked with another, still important theme. (SW3) In general the youth [who are concerned by a certain immigration law] don’t have a personal development plan apart from what is required for [being eligible for residence permit]. They have too little time and they need to put all focus into getting all the pieces [required] in place and thereby not risking deportation. (SW1)
Finally, social workers expressed adjusting the program to increase individual fit through individual tailoring. This was seen as a natural and important part of the program: “[The adaptations] mainly concerned the need for everything to always be adapted to a level that suits the youth who sits in front of you” (SW4) and “Not adapted so that it deviates from the personal development plan but adapted to the youth. The same activity can look very different depending on which youth has set this goal” (SW1).
Acceptability Youth
Youth participating in YP expressed high client satisfaction (mean = 28.9; Table 4). The qualitative responses also revealed a general appreciation for the program. Some examples were: “It was great” (Y1), “This program has meant very much to me” (Y2), and “I think it is good to have a contact person to meet and that it is possible to come to drop-in once a week” (Y3).
Acceptability Social Workers
Social workers delivering YP assessed acceptability positively across components apart from the component “network and community” toward, which views were neutral (Figure 6).
Service Provider Assessment of Program Acceptability, Appropriateness, and Feasibility, Heat Map, Study II. N = 9.
We identified four trends in social workers’ comments about acceptability of the program. First, they described program activities as being fun to work with, “It was a lot of fun” (SW4) or “Worked with study support, word comprehension. It was a challenge for me not being a pedagogue, but quite fun still” (SW5). Moreover, they reported feeling that their work with the program was important: “It feels like an important process” (SW1) and “It is an important intervention since having a job and income is important to our youth” (SW6). Third, program activities were described as interesting to them: “It is interesting to work with the goals that concern completing school” (SW1). Finally, social workers mentioned several times experiencing the program as rewarding. Examples were: “The best part about job interview training is that both the youth and the mentor can see the progress very clearly” (SW1) and, Incredibly rewarding to work with these concrete development processes, and to see how the youth's confidence at large grows in front of your eyes when you point out the progress or when showing older assignments to show how the youth wrote back then compared to now. (SW1)
Appropriateness Youth
No youth reported that participating in YP had led to increased difficulties in managing their problems. On the contrary, 91% perceived the program as being either a little or very helpful.
At the group level, no statistically significant differences in any of the youth-level outcomes used to assess appropriateness were found between T1 and T2 (see Table 4). We found no effect for social support (g = −0.01; CI = [−0.50, 0.52]), self-efficacy (g = −0.01; CI = [−0.54, 0.51]), or self-determination (g = −0.17; CI = [−0.69, 0.37]). There was a small but insignificant effect size for positive routines (g = −0.23; CI = [−0.74, 0.29]) and resilience (g = −0.3; CI = [−0.84, 0.24]).
At the individual level, an equal number of youth exhibited reliable change in the positive and negative direction for social support, self-efficacy, and self-determination (n = 2–4). No youth exhibited reliable change in any direction regarding positive routines. Resilience scores indicating a reliable change in the positive direction were found for a higher number of youth (n = 4) than in the negative direction (n = 2). When including only the youth that indicated remembering participating in YP (n = 11), self-determination and resilience stood out as measures where more youth improved (3–4) than deteriorated (0–1).
Two youth exhibited reliable change in the negative direction across more than two outcome measures. One of these youth indicated not remembering participating in YP. The other one had the lowest total client satisfaction score out of all the youth participating (18 out of 32).
Qualitative responses indicated that youth found the program to be appropriate in relation to their needs and expectations: “It has helped me as I wanted” (Y4), “It was good, I got almost all that I needed and all kinds of help and support” (Y5), “I think that it has really been a security and (…) it will really be a great help for me because the rest of society is failing me right now” (Y3), and “It was a good support and people were nice and kind by helping me to reach my goals” (Y6).
Appropriateness Social Workers
Social workers were generally positive toward the appropriateness of all YP components. One exception was network and community, toward which they were neutral (Figure 6).
Social workers addressed four aspects of appropriateness. First, they considered the program's ability to meet the needs of the youth. Opinions regarding this could vary, for example: “What I can perceive as problematic is that the youth needs so much more. Because of this, we rarely have enough time to work efficiently with what we had planned” (SW2) or “[This activity] is [in the program] because of the large need, if a youth gets a job, there is usually a large development in other areas of our program too” (SW4). Social workers also expressed perceiving some activities as frustrating to youth due to development being slow. For example: “It is tedious for the youth to spend loads of time on their goals when there is not much happening” (SW4), and “Even if this is the youth's own goal these things can be frustrating for them, so it is important to make all progress known to the youth, even the small ones” (SW1). Third, social workers described an increased preparedness for independence in youth throughout the program: “They find the words, let go a little bit of the nervousness and they become prepared for the different questions they could get” (SW1). Finally, comments concerned the suitability of the program to specific youth: “I feel that it is a suitable activity but simultaneously that there is a need for other activities for the process to move a bit faster” (SW4) and “With this specific youth it is going very well because the youth welcomes the structure and is very engaged in their own development process” (SW1).
Feasibility Youth
Youth expressed an appreciation of the flexibility of the program. This was perceived as important for the program to work in their specific situation: “So far, I think it works well and I like that it is on one's own terms and that there is no pressure to meet but it is based on oneself” (Y7) and “It [was] really good and I love that they have been able to adapt so much to my situation” (Y3).
Feasibility Social Workers
Social workers perceived all program components as feasible, although they were neutral toward the component network and community. In addition, they experienced the component of health as difficult to work with (Figure 6). Reported reasons for this were societal barriers faced by the youth and mentor (reported in open-ended questions).
Six categories of qualitative responses were relevant to the feasibility of YP. First, social workers reported difficulties in carrying out program activities due to external conditions. Examples given were certain (immigration) laws that affect the future of the youth or not having an appropriate venue to meet: The goal of working with their resumé had to be set aside since we have not been able to meet in the office, and instead we have met at a café. Since the youth needs an interpreter, this type of focused work is difficult. (SW5) It is difficult to work when the youth face the risk of not getting a residence permit. The youth needs a job, but only really a stable job (…) [authors’ note: due to the requirements for getting a residence permit]. (SW7)
Fourth, social workers reported some activities requiring a lot of preparation, both from themselves and the youth. For example: “Since this is an activity where the results can only be seen after some time it takes a whole lot of planning and follow-up. It requires really being focused and working both pedagogically, structured, and encouraging” (SW1). Fifth, technical issues related to the mobile application were perceived as obstructing the work, “The youth had not gotten their account in [the application] so we could not work with the activities (…)” (SW8). Finally, social workers stressed the need to not rush things, and certain activities requiring a lot of time. For example: “It is important to let it take its time, and to remind yourself that this youth should have these tools with them, even when the program is over” (SW1).
Discussion and Application for Practice
Given the current lack of transition support services in Sweden, the goal of both studies reported on in this paper was to identify areas of importance for further development to increase intervention fit of two ILS under development: MCMW and YP. We asked three research questions: (1) To what extent did MCMW and YP follow the basic plan for service delivery? (2) To what extent did social workers adapt program content to individual circumstances? (3) To what extent were MCMW and YP perceived as acceptable, appropriate, and feasible from the perspective of youth and social workers?
Neither MCMW nor YP entirely followed the basic plan for service delivery as many youth had not participated in the expected number of sessions at T2. However, several youth were still in the program at T2 in both studies, indicating that they may complete the program but at a slower pace than expected. This could mean that the program content takes a longer time than expected to deliver. As none of the programs are evaluated yet, it is unclear at this stage what, if any, consequences this has for program outcomes. A relationship between the amount of a prevention program received (i.e., “intervention dose”; Rowbotham et al., 2019) and program outcomes have been found in studies of preventive programs (JaKa et al., 2019; Kogan et al., 2016; Wiest et al., 2023). The impact of duration and pace on ultimate program outcomes for the programs examined in these two studies remains an empirical question that could be assessed in a future large-scale study. No youth expressed dissatisfaction with program duration or frequency of meetings.
Social workers in both studies reported making adaptations to the original program plan. The need for a certain level of program flexibility to be able to adapt to individual youth and situations was specifically highlighted by social workers in study II. Flexibility and adaptability (i.e., degree to which a program can be tailored to local contexts or needs) tend to be intervention characteristics that are beneficial to uptake (Damschroder et al., 2022b). However, it is important that adaptations are being made only within the realm of an “adaptable periphery” (Damschroder et al., 2009) and not to an extent where core components, required for the program to work as intended in the program theory, are being compromised (c.f. Castro & Yasui, 2017).
Attending to urgent practical needs was one of the most common adaptations that social workers made across both studies. For MCMW, this is striking, as this program was designed to be complimentary and not a replacement for the support that youth already receive from CWS. As most youth who participated in MCMW were currently living in OHC, this indicates that urgent practical needs are not sufficiently being addressed by other CWS functions such as for example the young person's case worker, foster parents, or a specifically assigned contact person connected to their residence. Given that MCMW was introduced as a pilot program within the CWS, it is notable that youths now seek assistance from this program for matters beyond its original scope. This may be a sign that urgent practical needs of these youth are not being met by the services they are currently receiving.
Lack of practical support from CWS in the transition from OHC has been highlighted in previous research (Bečević & Höjer, 2024; Bengtsson, 2022; Höjer & Sjöblom, 2014a). A prior study showed that only 19 out of 65 interviewed youth who recently transitioned from OHC in Sweden felt that they could ask someone from their formal network (including but not limited to their social worker) for help with practical day-to-day matters (Höjer & Sjöblom, 2014a). This is particularly important considering that youth in OHC value and seek practical support in addition to emotional support from social service professionals granted that enough trust is built (Cudjoe et al., 2022; Hiles et al., 2013). The youth who participated in YP were self-referred, meaning that the same support system may not be expected for these youth. Since a lack of support could likely be the reason for youth self-referring to the program, this could explain a high need for practical support in this group.
Youth and social workers found both programs acceptable. Acceptability has been found to be particularly important for supporting initial intervention uptake (Proctor et al., 2023). Moreover, prior research indicates that higher rates of program acceptability among service users can enhance positive intervention outcomes (Mira et al., 2019). Youth and social workers perceiving the programs to a high degree acceptable is thus viewed as promising for both programs.
Youth and social workers in both studies generally found the programs to be appropriate. The individual analyses of pre–post differences showed mixed results. However, we did not find clear indications of iatrogenic effects from the youth self-assessment-questions or from the individual analysis. Just as with desirable intervention outcomes, iatrogenic effects are a result of interactions between characteristics connected to the individual service recipient, the intervention itself, and the environment in which the intervention is provided (c.f. Moos, 2012). Poor intervention fit that result in iatrogenic effects could, in other words, be viewed as a sign of program inappropriateness. To detect such signs of inappropriateness early on is therefore essential for developing appropriate interventions that do not cause harm (Rhule, 2005).
Social workers in Study I perceived the MCMW manual as not feasible without revision, due to being disorganized and as such, not user friendly. In Study II, social workers reported structural barriers related to the legal and policy level (i.e., belonging to the outer setting; Damschroder et al., 2022b) as a main threat to feasibility. The discrepancy between the goals of the activities in the program and the limiting societal preconditions was described by social workers as a source of frustration among themselves and the youth they worked with. Legislation and policies constituting structural barriers to the provision of support to individuals have been highlighted in other contexts and in working with other populations (e.g., Richert et al., 2023). Bassett (2020), for example, described program staff in student support programs for low-income, first-generation university students feeling powerlessness at only being able to “teach the rules of the game” but not to change them. This resembles the experience of frustration described by YP social workers in working with youth without resident permits.
Working with the topic of the young person's social network seems to be the most difficult for social workers based on their assessments of acceptability, appropriateness, and feasibility. This finding was similar in both substudies and corresponds with findings from a review on interventions’ impact for youth transitioning from OHC in Australia, where social support was an outcome that the reviewed programs were less successful in promoting (O’Donnell et al., 2020). A weak social network is also an area in which OHC experienced youth have been shown to be particularly vulnerable (Blakeslee, 2015; Höjer & Sjöblom, 2010, 2014a; Österberg et al., 2016; Rutman & Hubberstey, 2016).
Social networks (both formal and informal) that youth can rely on for emotional and practical support are important (Marion et al., 2017; Rutman & Hubberstey, 2016) and constitute an essential pillar in the transition process specifically (Best & Blakeslee, 2020; Reid, 2007) as well as developmental processes in general (Harris & Orth, 2020; Masten & Barnes, 2018). A healthy social network has been found to be a general protective factor (Rickwood et al., 2005; Serie et al., 2021). This means that finding acceptable, appropriate, and feasible ways to work with social network components needs to be prioritized (Okland & Oterholm, 2022). It is worth noting that youth in both Study I and II expressed explicit appreciation for the support they receved through the programs.
This investigation has two key strengths. First, we shed light on the opportunities and barriers for program uptake through multiple data sources and multiple perspectives. This has provided a more profound and comprehensive picture of the different aspects of acceptability, appropriateness, and feasibility from youth and social worker perspectives. Second, the purpose of the study was achieved by recruiting a small sample. Even though the low sample size may be considered a limitation from a methodological perspective (particularly for study I where the number of participating social workers was very low; n = 2), we view a low sample size as a strength from the ethical perspective as both programs are newly developed and the research questions did not require a large sample size (Orsmond & Cohn, 2015). By recruiting a small number of participants, we were purposively conservative to avoid exposing more youth than necessary to programs which may not have been experienced as acceptable, appropriate, or feasible. Over and above this, we discuss below some key points that are important to consider when interpreting the results of the studies.
Due to the low sample size, we performed a Wilcoxon signed-rank test for all youth-level outcome measures in addition to the paired-samples t test to check for robustness of results. We found no meaningful differences between the results of the two tests that would alter the interpretation or conclusions drawn in any of the studies. Due to the low sample size, we were unable to conduct subgroup analyses.
The Leeds Reliable Change Calculator provides methods for two different analyses out of which only one was used in this study, namely the reliable change analysis, which is based upon the LRCI (Moreley & Dowzer, 2014). The other function of the calculator is to provide the calculations of whether the change is clinically significant (i.e., a clinical significant change [CSC] function). Different criteria can be used to determine CSC, such as mean values on the measures used from previous studies of clinical and/or comparison norms which are then used in the analysis as cutoff scores for determining CSC (Moreley & Dowzer, 2014). As the type of data needed for this type of analysis were not available for relevant clinical or comparison norms for the measures used in this study this function was not used.
The RCI was developed in a medical context, and terminology such as “clinical change” or “deterioration” may not be as appropriate or relevant descriptions considering the target groups or outcomes for programs like the ones presented here. For instance, the concept “clinical change” could perhaps better be referred to as “meaningful change.” Previous authors have, however, highlighted the potential usefulness of “clinically meaningful change” when assessing social work outcomes (Baviskar & Bergström, 2023). It should be noted that the purpose of using the RCI in these studies was not to assess ultimate program outcomes but to assess early indications of intervention fit.
Follow-up for youth participants occurred 8 months after referral to the study. For MCMW, this corresponded to the anticipated end of the program. It was originally communicated that YP was expected to range from 6 to 12 months, but as most participants remained in the program longer than six months this was revised to approximately 12 months during the study period. As the length of YP was more flexible, the 8-month follow-up may not mark the end of the program. As most participants in both programs indicated that they were still participating at T2, and that they had participated in fewer sessions than expected at that time, conclusions of the programs taking longer than expected to deliver remain.
We were not granted ethical permission to collect information about youth or social workers that did not provide consent to participate. Thus, the number of youth participating in, and social workers delivering, the programs but unwilling to participate in the research is unknown to us, as are their characteristics. This is a limitation.
The scales used to assess social worker implementation outcomes (AIM, IAM, and FIM) were skewed with most participants indicating high scores, a known issue with these measures (e.g., Hamm et al., 2023). Although this can be viewed as a limitation, we believe that it increases the added value of the qualitative responses. Social workers tended to provide more elaborate qualitative responses when they perceived there was an issue compared to when they were satisfied. This could make up for some of the negative skew from the quantitative measures.
The methods we have used for scoring and interpreting the AIM, IAM, and FIM measures may be considered unconventional. The low sample size and our ambition to inform further program development by assessing social workers’ attitudes towards individual program components required some methodological creativeness. We found the methods that we have used to fulfill the study objectives. As we have not identified any measures designed for the purpose of program development, the alternative would have been to develop our own questions. We consider the use of well-known measures specifically designed to capture concepts in line with the theoretical framework as a strength over inventing new measures, even if not used entirely as originally intended.
The results rely on self-reported data collected through rating scales. Social desirability bias (Piedmont, 2023) and recall bias (e.g., Khare & Vedel, 2019) may have influenced the participants’ responses. Particularly, we cannot guarantee the accuracy of youth self-reports of program attendance. This shortcoming could be avoided in future studies by including additional measures for tracking attendance (c.f. Brusco & Watts, 2015).
Generally, we interpret the above results as promising and encouraging of continued development of both programs and future investigation into program effectiveness. Five points should be considered by researchers and practitioners in further program development, investigation, and delivery of these programs.
First, it was clear from the results that the material for MCMW needs revision to increase user friendliness. Second, there was a considerable drop-out rate in both studies. We were unable to determine if drop-out from the study coincided with drop-out from program participation. Potentially high drop-out rates therefore require consideration from both practice and research perspectives with this target group (Bell et al., 2018). One possibility for future research would be to include more research sites to increase sample size, particularly when it comes to social workers delivering MVMW. Third, both programs seemed to take longer to deliver than planned. As the empirical material did not offer any reasons for these delays, this is something that could be investigated further. Fourth, establishing relevant standard cutoff values for outcome measures, for example, by conducting cross-sectional studies in the general population using the same instruments would be useful to allow for CSC assessment of program outcomes in future studies. Fifth, continued investigation into ways of increasing the acceptability, appropriateness, and feasibility of social support/social network components is important both from a researcher and practice perspective, considering the importance of strong social support for this population (Evans et al., 2022; Wickramaratne et al., 2022).
The findings presented in this paper highlight the importance of conducting small-scale studies when developing new programs to empirically assess intervention fit at an early stage. Both programs under study were currently under development as a response to an identified lack of programs providing support to youth transitioning from OHC in Sweden. The most prominent concerns to fidelity were difficulties in staying within the intended program time frame (MCMW), keeping continuity in meeting with the youth (MCMW and YP), and that sessions were used to solve practical issues outside program scope (MCMW and YP). MCMW and YP were generally perceived as acceptable, appropriate, and feasible by youth and social workers. The findings show that the MCMW material needs revision. No major concerns were identified with regard to potential iatrogenic effects in either of the programs at this stage. In conclusion, the findings from this study encourage further development and, in the future, further investigation into implementation, delivery, and effectiveness of both programs.
Footnotes
Ethics Approval Statement
Ethical approval for this research was granted by The Swedish Ethical Review Authority on 2021-02-11 (Approval 2021-00149).
Consent for Publication
Not applicable.
Consent to Participate
Participants gave their informed consent to participate at the beginning of the first digital questionnaire.
Authors’ Contributions
MK: data curation, formal analysis, investigation, methodology, visualization, writing–original draft, and writing–reviewing and editing. TS: conceptualization, funding acquisition, methodology, supervision, and writing–review and editing. MB: conceptualization, funding acquisition, investigation, and writing–review and editing. DK: writing–review and editing. TO: conceptualization, methodology, funding acquisition, investigation, project administration, supervision, and writing–reviewing and editing.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Forskningsrådet om Hälsa, Arbetsliv och Välfärd (Grant 2020-01287 and 2021-01352).
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: TO, MB and TS are authors of My Choice—My Way! We declare no known conflicts of interest, financial or otherwise.
Data Availability
The data used in this study is not publicly available as ethical consent for sharing data was not applied for or approved.