Adapted,Adopted,and Novel Interventions: A Whole-Population Meta-Analytic Replication of Intervention Effects

Abstract

Background

A challenge to implementation is management of the adaptation-fidelity dilemma or the balance between adopting an intervention with fidelity while assuring fit when transferred between contexts. A prior meta-analysis found that adapted interventions produce larger effects than novel and adopted interventions. This study attempts to replicate and expand previous findings.

Methods

Meta-analysis was used to compare effects across a whole-population of Swedish outcome studies. Main and subcategories are explored.

Results

The 523 studies included adapted (22%), adopted (33%), and novel (45%) interventions. The largest effect was found for adapted followed by novel and adopted interventions. Interventions in the mental health setting showed the highest effects, followed by somatic healthcare and social services.

Conclusions

These results replicate and expand earlier findings. Results were stable across settings with the exception of social services. Consistent with a growing body of evidence results suggest that context is important when transferring interventions across settings.

Keywords

Fidelity cultural adaptation prevention efficacy effectiveness evidence-based

The science of social intervention (Sundell & Olsson, 2017) is concerned with the systematic development and testing of intentional change strategies to promote health, prevent ill health, or maintain healthy developmental trajectories. The ultimate goal of providing social interventions is to impact societal health and wellbeing (Eichas et al., 2019; O’Connell et al., 2009) by targeting diverse populations across prevention level (i.e., universal, selective, indicated) and setting (e.g., social services, mental health). In order to bring the science of social intervention to scale for widespread public health gain, however, interventions need first be developed, tested for their efficacy and effectiveness, and finally assessed for their readiness for broad dissemination and implementation, where a stakeholder (e.g., policy-maker, practitioner, researcher) interested in using an evidence-based intervention (EBI) makes the decision to put it to use in practice (i.e., scale-up; Aarons et al., 2017). Although there are several, and somewhat varying, guidelines available and in use for evaluating claims of intervention effectiveness (e.g., California Department of Social Services, 2006; Gottfredson et al., 2015), there is widespread support for the understanding that in order for an intervention to be deemed evidence-based, it must first be assessed for its effectiveness according to an accepted standard. Bringing the science of social intervention to scale through the dissemination and implementation of EBIs is, however, challenging (Dymnicki et al., 2017).

Underlying the idea that EBIs should be disseminated to achieve widespread public health impact is an expectation that the positive effects found in efficacy and effectiveness trials will be maintained when interventions are moved from their original setting to new settings. A consistent challenge to the transfer and implementation of interventions is the management of the adaptation and fidelity dilemma (von Thiele Schwarz et al., 2019). This refers to the balance between adopting and delivering an intervention with fidelity while also assuring the intervention fits the local context which may necessitate a certain level of adaptation to the intervention under consideration. The current literature acknowledges that adaptations can be needed when using an EBI while it also highlights the importance of implementing interventions with fidelity (Baumann et al., 2017; Rabin et al., 2018).

Fidelity refers to the degree to which a given intervention is implemented as intended by developers and in relation to previous intervention trials when interventions are transferred to new settings (Carroll et al., 2007; Hasson, 2010). A common argument is that positive effects can only be maintained if interventions are implemented with fidelity in their new setting (Carroll et al., 2007; Gottfredson et al., 2015). However, documented challenges exist to the maintenance of fidelity when transferring interventions across settings (Bonell et al., 2021) and there is a growing literature supporting the idea that planned adaptation is necessary when transferring interventions across settings to maintain effectiveness (Ferrer-Wreder et al., 2012; Moore et al., 2021). For a given stakeholder interested in using a given EBI the dilemma of adaptation versus fidelity needs to be resolved: adapt the EBI to the local context by making systematic changes in specific ways (e.g., add content, remove content, change delivery method) or adopt the EBI as described previously with fidelity.

When the focus of implementation of an EBI is to maintain fidelity (i.e., adoption), stakeholders are expected to deliver the EBI without diverging from instructions on how the intervention is to be delivered. This includes delivering the intervention in the same way (e.g., 30-min sessions), using the same format delivery and channel (e.g., group or individual setting), and with the same content as implemented in a prior setting (von Thiele Schwarz et al., 2019). How fidelity looks may then differ depending on if the EBI is manual based, where the delivery guidance is explicit and generally highly specified, or if it is a general practice or a broad standardized approach (e.g., Responsible Beverage Service training), where there may be general guidance or material available but no manual.

When an EBI is to be adapted to a new setting, the adaptations can vary widely across dose, length, content, format, setting, and target population (Escoffery et al., 2018; Moore et al., 2013). Target population adaptations can be adaptations to an EBI being implemented to a population for which the EBI was not originally designed (Lee et al., 2008). For example, Ahmad et al. (2007) took a manualized eye movement desensitization and reprocessing treatment for adults and adapted the material to suit children. Format adaptations are adaptations made to the delivery of an EBI. For example, Calbring et al. (2007) took a published self-help book and implemented the material in an online program. Pragmatic adaptations are adaptations to the session or material length. For example, Livheim et al. (2015) adapted a manualized ACT protocol to be given in 6 weeks rather than 8 weeks. Cultural adaptations are those where the EBI is adapted to fit the cultural norms of the target setting or population (Barrera et al., 2017; Resnicow et al., 2000). For example, Kling et al. (2010) developed a parent management training program in Sweden based on components in programs from the United States. However, they culturally adapted the program by removing time-outs, as this was not an accepted disciplinary approach in Sweden. Planned adaptation, however, should not be conflated with deviations from fidelity. When interventions undergo an adaptation process, they still need implementation with fidelity in order to achieve desired results.

Many social interventions in use today are locally developed (Bergström et al., 2022; Sundell et al., 2015). Thus, stakeholders have a third option available to them when approaching the provision of a social intervention: develop a novel intervention. Ultimately, the decision regarding which strategy to choose will rest on opportunities and barriers present within a given practice setting. An increased understanding of how these approaches impact client outcomes may help increase our understanding of how to transfer interventions across settings to achieve the widespread goal of impacting societal health and wellbeing. However, the available evidence supporting the approach to scale-up and spread—adapt an existing EBI to fit the local context, adopt an existing EBI with fidelity or develop a novel intervention—is limited.

To address the knowledge gap concerning the impact of these approaches to intervention delivery, we have previously conducted a meta-analysis of social interventions delivered in Sweden and Germany (Sundell et al., 2015). The findings across samples suggested that adapted, adopted and novel interventions were all effective. However, results from the Swedish sample showed that adapted interventions were the most effective but made up only a small proportion of the total number of included studies (approx. 10%). In addition, compared to adapted interventions and novel interventions, the group of interventions adopted without any adaptation had the lowest average effect size. This result was stable when controlled for variations in study design and sample size. However, the Swedish sample in the prior study included only 139 studies and the heterogeneity of interventions (e.g., intervention type, prevention level) limited the ability to draw conclusions about the subcategories of approaches (e.g., cultural adaptation, pragmatic adaptation). In addition, social interventions are used in different settings, (e.g., social work, mental health, somatic health). The discourse on evidence and what constitutes an evidence-based practice differ between settings as well as the relationship between research and practice, potentially impacting how choice and delivery of interventions are approached (Liedgren & Kullberg, 2022). Although interventions from different settings and prevention level were included in the prior meta-analysis, the limited sample size did not allow further investigation of these aspects.

The aim of the current study, therefore, is to build on the study described above by expanding and focusing on the Swedish findings by including a larger population of studies conducted in Sweden. We (a) further explore the effectiveness of social interventions by replicating an earlier meta-analysis comparing interventions imported from other contexts with adaptation, interventions adopted from other contexts without adaptation, and novel interventions that have been evaluated in controlled research and (b) compare the effectiveness of these different types of interventions in various types of settings. The following questions are asked:

What are the frequencies and characteristics of adapted, adopted, and novel interventions found in the population of controlled intervention research undertaken in Sweden and published in peer-reviewed journals between 1990 and 2019?

To what extent are approaches to adaptation, adoption, and the development of novel interventions related to intervention outcomes?

To what extent are adapted, adopted, and novel interventions implemented in different settings related to intervention outcomes?

Method

Design and Setting

We used meta-analytic techniques on the population of controlled research conducted in Sweden and published during the investigation period.

Eligibility Criteria

Inclusion Criteria

The publication reported on a study which evaluated a behavioral, psychological, or social intervention.

The publication reported on a study which was undertaken in Sweden and the principal investigator was employed by a Swedish university or organization.

The publication was from a scientific journal and was subjected to peer review prior to publication.

The study was published between 1990 and 2019.

The study design reported in the publication was a randomized or non-randomized controlled design.

The study included a behavioral, psychological, or social outcome measure.

Exclusion Criteria

The study reported did not include an outcome measure at the client, patient, or user level (e.g., only included measures of professional behavior change).

The study reported on an intervention designed to impact somatic health without including at least one behavioral, psychological, or social component.

The study reported on an intervention designed to impact pedagogical or didactical outcomes only (e.g., methods to teach children math skills).

Information Sources

This study is a retrospective analysis and based on information provided in the published literature.

Search Strategy

We conducted a whole-population study and used systematic methods to find all publications meeting our inclusion criteria. A previous study (Sundell & Åhsberg, 2016) demonstrated that “Sweden” or “Scandinavia” is rarely (approximately 10% of cases) used in the title or among key words in published articles on Swedish intervention research. Despite this we have attempted to follow as closely as possible PRISMA reporting standards (Supplementary information 1) and have supplemented these with additional efforts aimed at identifying the entire population of studies meeting our inclusion criteria. The starting point for the current study was the 139 published articles reporting findings from controlled research and included in Sundell and Åhsberg (2016). From this initial pool of studies, the search was carried out in five steps. First, all 191 researchers that were previously identified as having published at least one controlled study during the period 1990–2014 (Sundell & Åhsberg, 2016) were contacted directly. Second, bibliographic searches were conducted to identify research produced by these researchers during the period 2015–2019. Third, a broad search of the six largest Swedish research funders: the Swedish Research Council for Health, Working Life and Welfare (Forte), the Swedish Research Council for Sustainable Development (Formas), the European Research Council (ERC), the Swedish Research Council (VR), the Swedish Innovation Agency (Vinnova), and the Swedish Crime Victim Authority (BRÅ), database was searched for grants awarded to conduct controlled studies during the period 2015–2019 (search terms: evaluation, randomized). Fourth, we conducted a broad search of planned controlled studies registered at clinicaltrials.com (search terms: Swed*, random*, effect*, evaluat*, RCT) as well as studies registered with ISRCTN at www.isrctn.com (search terms: Swed*, mental and behavioral disorder). Fifth, the researchers and studies identified in the searches above were then included in searches in unified index EBSCO Discovery (Stockholm University library; search terms: Swed*, random*, effect*, evaluat*, RCT). Details regarding the search strategy are published previously (Sundell & Åhsberg, 2016).

The studies identified were then pooled and duplicates removed. If individual study results were reported in several publications, the first publication was used as the source for data extraction and coding.

Data Items and Data Extraction

Study Characteristics

Extracted study characteristics include date of publication, study setting (e.g., social services), design (e.g., randomized control), type of control condition (e.g., wait-list control), study population (e.g., adults), follow-up period, prevention level (e.g., indicated), study sample size (excluding dropouts), and effect size estimate. If study characteristic (e.g., setting, prevention level) was not explicitly stated in the publication, a determination was made based on the information provided. If we were unable to determine a characteristic by the information provided the data was coded as missing. Data was extracted by two authors with very good (83%) to perfect (100%) agreement on coding of individual items in a random sample of 40 studies (Cohen's kappa 0.63–1.0; Sundell & Olsson, 2021).

Included Intervention

If the study included more than two arms (i.e., experimental, control), the group that was highlighted as the primary focus of the study was selected. If there was no clear information about the priority of the groups, then the first described group was selected and included in this study.

Intervention Type

Main intervention type (i.e., adapted, adopted, novel) was further defined in terms of nine mutually exclusive subcategories (Table 1). Determination of intervention type was based on the information provided in the study. If authors referred to their own previous studies in their intervention descriptions, these studies were referred to for additional information. No additional attempts were made to obtain information outside of that published in the sources described above. The five authors independently coded intervention type, by using a coding manual (available from authors upon request). The coding manual was developed and tested iteratively by the authors. Any disagreements in coding and interpretation of the coding manual were discussed prior to final coding. Interrater reliability on coding of type of intervention was assessed as very good on a random sample of 40 studies (κ = .80 main; κ = .88 subcategories).

Table 1.

Main Categories, Subcategories, Definitions and Examples of Coding Guidance Used in Coding Included Studies.

Main category	Subcategory	Definition	Examples
Novel	Innovation	The intervention is completely developed based on broad theoretical concepts or models, therapies and/or literature findings.	Overmeer et al. (2011) developed a course based on the biopsychosocial model to give to physical therapists who would treat patients with musculoskeletal pain.
	Reinvention	The intervention is developed based on empirically established international or national intervention programs, the newly developed program takes inspiration from the referenced programs. The intervention program may also be a re-development of a referenced program due to numerous different modifications (i.e., that capture more than x of the adaptation subcategories).	The Healthy School Start by Nyberg et al. (2015) had several different components; brochure, motivational interviewing for parents, and classroom activities. The brochure was created for this study based on a literature review about children's dietary and physical activity habits. Motivational interviewing for the parents was implemented by MI counselors. The classroom component was based on prior effective school interventions. A manual was developed for teachers.
	Combined	The intervention includes both materials that are newly developed and components from a referenced intervention that are combined into a single treatment. A combined intervention may also include referenced interventions that are given together as a single treatment/program/service, that have previously not been tested together.	Bergman Nordgren et al. (2013) combined different treatment protocols from previous internet-based CBTs for different anxiety disorders, this was a novel approach for the ICBT they used.
Adoption	Explicit	The intervention is explicitly adopted as an empirically supported intervention. The intervention is directly translated without any modifications.	The multidimensional treatment foster care (MTFC) is manual based, and Hansson and Olsson (2012) testing the effectiveness in an RCT, explicitly follow the manual.
	Broadband	The intervention is deemed as a standardized treatment, that may not have an explicit reference but is given as standard (e.g., case management, Responsible Beverage Service training).	Jensen et al. (1994) assessed the effectiveness of rehabilitation programs used in usual care for neck and back pain patients. No manual was mentioned, clinicians followed their treatment as usual.
Adaptation	Cultural adaptation	The intervention is adapted to suit the cultural norms of the Swedish context.	Skärstrand et al. (2013) took the Strengthening Families Program and adapted the material and format to explicitly fit into Swedish culture and conditions (e.g., some material was removed, and some added for cultural relevance).
	Pragmatic adaptation	The intervention is adapted for practical reasons, such as adding or removing materials, increasing, or decreasing number of sessions, or session length.	Öst et al. (2004) extended their previously tested 12 session exposure and cognitive therapy treatment to 16 sessions, with the ambition to improve treatment outcomes.
	Format adaptation	The interventions’ delivery format is adapted (e.g., group therapy given as individual therapy) or a former intervention is adapted to be given online.	Attention bias modification intervention (Boettcher et al., 2013) was adapted to be given online instead of in person at a clinic. Prior studies have solely implemented the intervention in in-person clinic settings.
	Target population adaptation	The intervention is adapted to suit a specific target population's needs, such as making material easier to read for children, or modifying language to suit a student population.	A smoking cessation intervention (Müssener et al., 2020) previously developed for adult populations, was tailored for adolescents by adapting materials that were (un)suitable for adolescents. For example, removing material taregting smoking during pregnancy and birth.

Outcome Measures

The primary outcome measure in each study was used and when no such information was provided, the first measure listed in the method section of the publication was used in the analysis.

Methodolgocial Quality of Included Studies

We assess differences in risk of bias (Higgins et al., 2023) by comparing reported methodological quality across our three categories of intervention (adapted, adopted, novel). Data from each included study was extracted in relation to adherence to reporting standards as described in CONSORT (Moher et al., 2010), TREND (Des Jarlais et al., 2004), and Prevention Science (Flay et al., 2005). Items are reported individually across groups and in a combined index consisting of 16 items (for more information about assessment of methodological quality see Olsson & Sundell, 2023).

Statistical Procedures

Effect sizes were calculated according to (Lipsey & Wilson, 2000). When outcomes in a publication were reported to be statistically significant (p ≤ .05) but no additional information on effect size was offered (n = 4), the effect size was set to the average effect for the remaining sample (d = .48). When studies stated non-significant effects but without reporting actual effect size (n = 25), the effect size was set to 0. Since extreme effect sizes may have a disproportionate influence on conclusions drawn from statistical analyses, we checked for outliers by using Cook's distance (Cook, 1977; Walfish, 2006). Three effect sizes were identified as outliers. To reduce the impact of these outliers, their values were substituted by one that equaled the highest effect that fell within the normal range (d = 2.3). This had a negligible impact on study results when compared to analyses using the original data. Comprehensive meta-analysis software was used to synthesize data. As the sample exhibited a large proportion of random variance both in the total population (Q(522) = 2489.15, p < .0001), and on the reduced sample (Q(295) = 1589.91, p < .0001), analyses were interpreted from the random effects models. Publication bias and study heterogeneity in the population of studies included here has been investigated and reported in a prior study (Olsson & Sundell, 2023).

Sensitivity Analysis

As design and sample size have been shown to have significant effects on outcomes, the previous meta-analysis conducted analyses on the entire sample and on a reduced sample in which studies with a non-randomized experimental design and studies with sample sizes of less than 50 were removed (Table 3). We replicate these analyses to test the robustness of our results.

Results

General Description of the Included Studies

We found 523 unique studies fulfilling our inclusion criteria that included enough information for data extraction. In the 523 studies, 426 unique interventions were studied (Table 2). Table 2 provides an overview of the number of interventions evaluated within specific fields of practice as well as the intervention most often studied within each respective field.

Table 2.

Number of Interventions in Included Studies (k = 523) by Field and Most Frequent Intervention Studied within the Field.

Field	Studies (k)	Interventions (n)	The most frequently studied intervention
Adult psychiatry	204	163	Internet-based CBT (k = 57)
Somatic health and medical care	113	101	Internet-based CBT (k = 16)
Social services – individual care	49	47	Responsible beverage service (k = 3)
Social services – children and family	44	42	Comet (k = 5)
Child and adolescent psychiatry	37	33	Internet-based CBT (k = 9)
Social services – elderly care	33	27	Case manager (k = 4)
Occupational rehabilitation	24	23	Yoga and Yomi (k = 3)
Social services - disability	8	9	Social skills group training (k = 2)
Correctional services	7	9	Yoga (k = 2)
Dentistry	4	6	None were more frequent
other (physical training, anti-bullying programs, memory training)	5	7	None were more frequent
Total number	528	426	Internet-based CBT (k = 86)

The included interventions aimed to target 52 unique problem areas within 14 ICF categories. The most common areas included mental functions (n = 289; e.g., treatment of substance abuse), activities and participation in major areas of life (n= 63, e.g., treatment of chronic pain), activities and participation related to general tasks and demands (n= 50; e.g., exercise programs for the elderly); interventions designed to impact body functions (n = 21 e.g., internet-based treatment for stress-related incontinence and n = 23 e.g., excercise program in order to regain entrance to the labor market); and, activities and participation related to interpersonal interactions and relationships (n = 38; e.g., programs to prevent criminal behavior among youth).

In all, the studies included in the review included 296 different primary outcome measures. Of these, 72 were used in two or more studies. Examples include the Alcohol Use Disorders Identification Test (AUDIT; 14 studies), the Insomnia Severity Index (ISI; 12 studies), and the Activities of Daily Living (ADL) Staircase (8 studies).

Frequencies and Characteristics of Adapted, Adopted, and Novel Interventions

Of the 523 studies, 21% (k = 111) described adapted interventions, 33% (k = 170) adopted interventions, and 46% (k = 239) novel interventions (Table 3). Table 3 provides a summary of study characteristics. The most common setting for the studies was within mental health 46% (k = 240). The majority (87%, k = 454) of the studies were RCTs. Most control group participants (63%) received another intervention (e.g., a competing intervention), and the remainder (37%) received no intervention or were on a waiting list. Most studies (55%) had a follow-up period of 12 months or more after baseline (M = 11.9 months, SD = 14.6) and included on average 164 study subjects. Significant differences between adapted, adopted, and novel intervention studies were found across a range of study characteristics (Table 3).

Table 3.

Characteristics of Included Studies (k = 523) and Tests of Differences in Study Characteristics between Adapted, Adopted, and Novel Intervention Groups.

	Total (k = 523)	Adapted (k = 112)	Adopted (k = 171)	Novel (k = 240)	Significance tests	p
Setting
Mental health (%)	46	60	46	40	χ²(2) = 12.02	.0024
Social services (%)	26	17	35	25	χ²(2) = 10.70	.0047
Somatic healthcare (%)	27	23	20	35	χ²(2) = 12.71	.0017
Indicated prevention (%)	73	80	71	72	χ²(2) = 2.73	.02442
Active control group (%)	37	36	30	43	χ²(2) = 7.30	.0226
RCT (%)	87	96	79	89	χ²(2) = 19.48	.0001
Number of months between pre- and follow-up measures (mean)	11.9	11.2	14.2	10.6	F(2,519) = 3.25	.0395
Number of subjects (mean)	163.7	138.5	166.8	173.3	F(2,490) = 0.82	.4426

Effect Sizes for Adapted, Adopted and Novel Interventions

Adapted interventions were found to have the highest standard mean effect size (d = .509), followed by novel interventions (d = .407), and adopted interventions (d = .372; Table 3). There were significant differences between adapted and adopted interventions, Q(1) = 9.35, p < .002, and between novel and adopted interventions, Q(1) = 5.97, p < .05. No significant difference was found between adapted and novel interventions, Q(1) = 1.08, p > .05.

The result for the reduced sample was similar to that of the full sample (Table 4). The effect size was significantly higher in studies of adapted interventions compared to adopted interventions, Q(1) = 4,74, p < .05. There were no significant differences between adopted and novel interventions, Q(1) = 1,974, p > .05, or between adapted and novel interventions, Q(1) = 1.53, p > .05.

Table 4.

Results of the Standardized Mean Difference Effects of Main Intervention Types Including Subcategories.

		Total study population (k = 523)			Excluding non-randomized studies and small study populations (k = 296)
Types of interventions	d	95% confidence intervals	k	d	95% confidence intervals	K
Adapted	.509	.436–.582	112	.471	.377–.566	58
Cultural	.222	.068–.376	11	.191	.023–.358	7
Pragmatic	.595	.438–.752	33	.551	.328–.775	16
Format	.552	.423–.681	39	.589	.424–.753	21
Target population	.508	.342–.674	29	.372	.179–.564	14
Adopted	.372	.322–.421	171	.341	.271–.410	84
Explicit	.376	.297–.436	84	.356	.261–.452	41
Broadband	.367	.304–.449	87	.325	.223–.427	43
Novel	.407	.363–.451	240	.403	.351–.455	154
Innovation	.305	.249–.360	114	.282	.216–.349	62
Reinvention	.523	.447–.599	102	.508	.425–.591	77
Combined	.492	.302–.682	24	.471	.239–.702	15
Total	.414	.384–.443	523	.397	.359–.435	296

d = Weighted effect size according to random effects model; k = number of studies.

Effect Sizes for Adapted, Adopted and Novel Intervention by Subcategory

Adapted Interventions

For adapted interventions (Q(3) = 14.18, p < .003), pragmatic adaptations had the highest standard mean effect (d = .595), followed by format adaptations (d = .552), target population adaptations (d = .508), and finally cultural adaptations (d = .222) (Table 3). In the analysis of the reduced sample (Q(3) = 12.83, p < .005), the pattern of results was not maintained. Here, format adaptations were found to have a higher standard mean effect (d = .589) than pragmatic adaptations (d = .551).

Adopted Interventions

There were no significant differences between the two subcategories of adopted interventions in the full sample, Q(1) = 0.35, p > .05, or in the reduced sample, Q(1) = 1.96, p > .05 (Table 3).

Novel Interventions

Out of the three subcategories of novel interventions (Q(2) = 21.96, p < .0001), reinterventions had the highest standard mean effect (d = .523), followed by the combined (d = .492), and finally innovations (d = .305). This pattern of results was maintained in the reduced sample (Q(2) = 17.95, p < .0001; Table 4).

Effect Sizes by Setting

Interventions provided in the mental health setting showed the highest standard mean difference, Q(2) = 95,12, p < .0001 (d = .577), followed by somatic healthcare (d = .361) and last social services (d = .238) (See Table 5).

Table 5.

Results of the Standardized Mean Difference Effects of Interventions by Setting.

	Total study sample (k = 523)									Excluding non-randomized studies and small study populations (k = 296)
	Mental health			Social services			Somatic healthcare			Mental Health			Social services			Somatic healthcare
Type of intervention	d	95% confidence intervals	k	d	95% confidence intervals	k	d	95% confidence intervals	k	d	95% confidence intervals	k	d	95% confidence intervals	k	d	95% confidence intervals	k
Adapted	.662	.542–.792	67	.197	.101–.293	19	.397	.289–.506	26	.660	.488–.831	33	.188	.071–.304	12	.330	.203–.456	13
Adopted	.500	.407–.592	79	.269	.204–.333	58	.325	.206–.444	34	.429	.307–.551	39	.261	.177–.344	34	.302	.027–.577	11
Novel	.587	.501–.672	96	.227	.164–.289	61	.365	.293–.437	83	.601	.496–.705	61	.219	.151–.286	42	.352	.261–.443	51
Total	.577	.522–.633	242	.238	.199–.278	138	.361	.306–.418	143	.564	.491–.636	133	.228	.182–.275	88	.343	.266–.420	75

d = Weighted effect size according to random effects model; k = number of studies.

Methodological Quality of Included Studies

Methodological quality across groups of adapted, adopted, and novel interventions can be found in Table 6. Adopted (M = 8.6) interventions had significantly lower methodological quality compared to both adapted (M = 9.5) and novel (M = 9.8) interventions (p < 0.001). Adapted and novel interventions more often used blinding, reported intervention fidelity, used randomization (RCT), conducted intent to treat analyses and adjusted for baseline differences (adapted 41%, 69%, 96%,60%, 69%; novel 45%, 55%, 89%, 58%, 55%) compared to adopted interventions (26%, 47%, 79%, 46%, 47%, 0.001 ≥ p ≤ 0.024).

Table 6.

Methodological Quality of the Included Studies by Intervention Group (k = 523).

	Intervention group			Tests of differences across groups
	Adopted	Adapted	Novel	X²(2) p
Total number of publications	171	112	240
Protocol and conflict of interest
Published research protocol (%)	24	34	36	6.87 0.032
Disclosure of conflict of interest (%)	49	54	50	0.91 n.s
Research sample
Inclusion and exclusion criteria (%)	92	96	94	1.13 n.s
Demographic and clinical variables	73	80	80	4.14 n.s
Estimation of statistical power	36	40	44	3.00 n.s
Data collection
Primary outcome measure (%)	33	21	46	11.1 0.004
Reported psychometrics (%)	57	68	63	3.25 n.s
Blinded study (%)	26	41	45	16.38 0.004
Unintended outcomes (%)	10	16	14	2.59 n.s
Fidelity
Reported intervention fidelity (%)	47	69	55	13.21 0.001
Study design
Randomized control trial (%)	79	96	89	19.48 0.0001
Attempts were made to minimize selection bias between study arms^a	44	52	52	2.77 n.s
Analysis
Reported missingness (%)	76	90	79	9.13 0.01
Intent-to-treat (%)	46	60	58	7.45 0.024
Adjusted for baseline differences (%)	47	69	55	13.21 0.1
Reporting of results
Descriptive statistics (%)	77	85	81	3.21 n.s
Effect sizes (%)	44	61	51	7.70 0.021
Average quality index	8.6	9.5	9.8	F = 11.27 0.0001

All exact p-values < 0.05 are reported in order to enable Bonferroni adjustments.

Discussion and Applications to Practice

The purpose of this study was to explore the effectiveness of social interventions by replicating and expanding on an earlier meta-analysis comparing novel, adapted, and adopted interventions. Our results largely replicate our earlier findings. As in the earlier study, we find that novel interventions make up the largest proportion of interventions studied, followed by adopted and adapted interventions. Novel interventions are undoubtedly developed to meet a perceived need in society and this option for intervention delivery is likely preferred when access, awareness, or knowledge of EBIs for specific populations, targeting specific outcomes is scarce. Indeed, although many systematic reviews have identified interventions with strong empirical support for their effectiveness, systematic reviews also consistently find few to no clearly effective interventions across several populations and/or outcomes (e.g., Olsson et al., 2021; Petersen et al., 2022).

We aimed to investigate the extent to which approaches to adaptation, adoption, and the development of novel interventions were related to intervention outcome. We find that adapted interventions produced the largest standard mean effect size followed by novel interventions and adopted interventions. This is in line with a body of research reporting successful adaptations to EBIs in more homogeneous samples (Gladstone et al., 2021; Rajabiun et al., 2021; Stahmer et al., 2019). These findings were stable when we controlled for study design and sample size. In addition, the absolute standard mean effect sizes across intervention type found in the current study are higher than those reported in the prior meta-analysis. Despite our finding of differences in standard mean effect sizes across adapted, adopted, and novel categories, the standard mean effect sizes in all categories fell within the medium range and were significantly different from zero which means on average adapted, adopted, and novel interventions all produce effects in controlled research. It should be noted that uncertainty around the point estimate was large for all categories of interventions. In addition, adopted interventions as a group had the lowest methological quality and as such the highest risk of bias across groups of intervention. It is, however, unclear as to how this may have effected results.

Although statistically significant, the measured difference among the three types of interventions is rather low (d's ranging from .37-.51). Whether this is a meaningful difference is an important question as the mean effect differs by just more than a d of .13 between adapted and adopted interventions. Another way to describe this difference is by converting d to number needed to treat, with the result of 26.08. Or put differently, of one hundred treated individuals four more are successful in adapted interventions compared to adopted interventions. Ethically, one could argue that this difference is not negligible, especially when extrapolated to an entire population, but a more definite conclusion would require an economic evaluation of the included interventions, which was not possible in this study.

Additionally, we aimed to expand the prior meta-analysis by exploring the extent to which the setting within which adapted, adopted, and novel interventions were implemented was related to intervention outcome. We found the same pattern of results in the standard mean effects in mental health and somatic healthcare settings. Although these results were fairly stable when controlling for study design and sample size within mental health, they were not stable within somatic healthcare. This, however, may be due to the small number of studies within the adapted and adopted categories in these analyses. Studies conducted within the social services showed a different pattern of results. In the social service setting, adopted interventions evidenced a higher standard mean effect, followed by novel interventions. Adapted interventions had the smallest mean effect within a social service setting. These results were stable when we controlled for study design and sample size. This result may be due to the relatively low number of adapted interventions within the social services setting found in this study. It may, however, be related to specific characteristics of the interventions, delivery, or research environment within the social services context (Olsson & Sundell, 2016; Sundell & Olsson, 2021). The extent to which there are any specific characteristics of social service interventions or the social services setting that might impact intervention effectiveness needs to be further investigated.

As in the prior meta-analysis, we investigated subgroups of adapted, adopted, and novel interventions. We have also refined and expanded the range of subgroups in this study. Within adopted interventions, cultural adaptations (i.e., adaptations made to the intervention to suit cultural norms) were found to have the fewest number of studies within and across subgroups as well as the smallest standard mean effect size within and across subgroups. This is interesting as this area of adaptation is arguably the most prominent within the literature on intervention adaptation (Barrera et al., 2017; Gonzales, 2017; Mejia et al., 2017). It should be noted, however, that much of the extant literature on cultural adaptation addresses cultural adaptations to EBIs for use with minority populations within the same setting and/or context within which the EBIs were developed, as opposed to cross-cultural adaptations (i.e., when EBIs are transported to new cultural contexts such as countries) which is what we report in the current study. This type of transfer introduces not only changes in the cultural norms of target populations but can also mean changes in broad societal norms and how services are delivered. Similarly, within the novel subgroup, combined interventions (see Table 1) were relatively few and innovative interventions exhibited the lowest standard mean effect size within the novel subgroup. As these subgroups represent a refinement and expansion of coding definitions, we are unable to compare these results to the prior meta-analysis. However, across subgroups the results of the current study find standard mean effect sizes to be higher than that of the prior meta-analysis as well as exhibiting more variability in standard mean effect sizes across subgroups.

Although this study has several strengths there are also four main limitations that should be highlighted. First, although intensive work was done to develop and agree upon the coding manual, the boundaries between the subcategories were not always clear-cut. This introduces not only a methodological challenge but also reflects the complexity of intervention development, evaluation, and spread. The ability to conduct valid and reliable meta-analyses is constrained by the information contained in the report of original research (for a short discussion see e.g., Olsson et al., 2022); however, the intervention development and/or adaptation process may not always be clearly described in publications describing research on interventions. It is a common finding, for example, that despite methodological guidance regarding the inclusion of clear descriptions of interventions assessed in evaluative research (Schulz et al., 2010), investigations and reviews on the effectiveness of interventions consistently point to shortcomings in the extent to which interventions are developed and described (Glasziou et al., 2008; Maden et al., 2017; Olsson et al., 2023). In this study, it is therefore not unlikely that we have categorized interventions as novel when they are in fact adapted interventions. This is due to at least two reasons. First, it is difficult to determine when an intervention is in fact novel as opposed to inspired by prior research on existing interventions. In addition, in reporting intervention studies, authors may limit their descriptions of the intervention development process in order to focus on the reported research findings. Interviews with intervention developers have uncovered for example that the transparent reporting of adaptations in scientific journals involved a conflict between transparency and practical concerns such as word count (von Thiele Schwarz et al., 2018). This in turn may mean that the category “novel interventions” in this study is inflated. Despite this, the interrater reliability between raters was high.

Second, we did not assess the extent to which within study fidelity was tracked or maintained. Although our analysis includes adapted interventions, it should be stressed that these represent intentional a priori efforts to adapt EBIs prior to implementation (as opposed to therapist or program drift). Thus, the findings do not provide information about the impact of adaptations at other time points, for example, when done when the intervention is used in practice. We are, however, unable to assess the extent to which the adapted, adopted, and novel interventions included were implemented with fidelity within each study.

A third challenge is that there are several possible confounders related to method and design present in this study. For instance, the review included different subsets of psychosocial interventions, where some aspects of the intervention itself might be related to the degree and types of adaptation. In the present analysis, we controlled for variation in methodological quality in the included studies by re-analyzing data when articles reporting non-randomized trial designs or low sample sizes were excluded. This did not impact main results and indicates robust findings. However, the impact of other possible confounders remains to be investigated in future research.

Finally, it should be noted that our analysis includes research conducted in Sweden only. The extent to which these findings are generalizable to other contexts and settings is unknown. In addition, we were unable to investigate the reasons why stakeholders choose one approach over another. One could choose to develop a novel intervention when solid evidence on existing interventions is missing or the development of novel interventions may be made when stakeholders lack the ability to search for research on existing interventions and their effectiveness. Cost may also impact choice of approach to intervention development, adaptation or adoption. We were however unable to investigate this in the current study.

The research to practice pathway is based on the idea that interventions are developed and tested and used in the way they were designed. The results reported here are in line with a growing body of evidence suggesting that there is a need to take the fit between the EBI and the context into account when implementing interventions, as shown in the superiority of adapted EBIs and interventions specifically developed for a certain context. This complicates the accumulation of knowledge across studies, with implications for how primary studies as well as reviews of intervention studies are conducted. Primary studies that illuminate what works for whom and how, that is, measurement of the moderators and mediators of intervention effects that are adapted and measured across studies will be vital for ensuring the possibility for an accumulation of knowledge across studies. This type of measurement is also important for increasing our understanding of the core components of interventions. Review methods that can take the variation between cases into account are called for. In conclusion, we find that standardized mean effects for interventions across categories are significantly different than zero, indicating that adapted, adopted, and novel programs all provide benefit to service users. Standardized mean effects for adapted interventions were, however, larger than that for novel and adopted programs and smallest for adopted programs.

Footnotes

Author Contributions

Authors have contributed to this manuscript in the following ways: Tina M Olsson Author’s role: Investigation, data curation, writing original draft, review, and editing, funding acquisition.

Ulrica von Thiele Schwarz: Writing original draft, review, editing and funding acquisition.

Henna Hasson: Writing original draft, review, editing and funding acquisition.

Emily G Vira: Formal analysis, writing original draft, review, and editing.

Knut Sundell: Conceptualization, methodology, formal analysis, investigation, data curation, writing original draft, review, and editing, funding acquisition. All authors read and approved the final manuscript.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Funding for this research has been provided by two grants from the Swedish Research Council for Health, Working Life and Welfare (Forte; Dnr: 2019-01737 and 2018-01315) as well as a grant from the Swedish Research Council (Dnr: 2016-01261).

The datasets used and/or analyzed in the current study are available from the corresponding author on reasonable request.

ORCID iDs

Tina M. Olsson

Knut Sundell

References

Aarons

G. A.

Sklar

Mustanski

Benbow

Brown

C. H.

(2017). “Scaling-out” evidence-based interventions to new populations or new health care delivery systems. Implementation Science, 12(1), 111. https://doi.org/10.1186/s13012-017-0640-6

Ahmad

Larsson

Sundelin-Wahlsten

(2007). EMDR Treatment for children with PTSD: Results of a randomized controlled trial. Nordic Journal of Psychiatry, 61(5), 349–354. https://doi.org/10.1080/08039480701643464

Barrera

Berkel

Castro

F. G.

(2017). Directions for the advancement of culturally adapted preventive interventions: Local adaptations, engagement, and sustainability. Prevention Science, 18(6), 640–648. https://doi.org/10.1007/s11121-016-0705-9

Baumann

A. A.

Cabassa

L. J.

Stirman

S. W.

(2017). Adaptation in dissemination and implementation science. In Brownson

R. C.

Colditz

G. A.

Proctor

E. K.

(Eds.), Dissemination and implementation research in health: Translating science to practice (Vol. 2, pp. 286–300). Oxford University Press.

Bergman Nordgren

Carlbring

Linna

(2013). Role of the working alliance on treatment outcome in tailored internet-based cognitive behavioural therapy for anxiety disorders: Randomized controlled pilot trial. JMIR Res Protoc, 2(1), 1–8. https://doi.org/10.2196/resprot.2292

Bergström

Sundell

Olsson

T. M.

Leander

Åström

(2022). Interventions in child welfare - a Swedish inventory. Child & Family Social Work, Early View, 28(1), 1–8. https://doi.org/10.1111/cfs.12946

Boettcher

Leek

Matson

Holmes

E. A.

Browning

Macleod

Andersson

Carlbring

(2013). Internet-based attention bias modification for social anxiety: a randomised controlled comparison of training towards negative and training towards positive cues. PloS One, 8(9), e71760.

Bonell

Prost

Melendez-Torres

G. J.

Davey

Hargreaves

J. R.

(2021). Will it work here? A realist approach to local decisions about implementing interventions evaluated as effective elsewhere. Journal of Epidemiology and Community Health, 75(1), 46–50. https://doi.org/10.1136/jech-2020-214287

Calbring

P. E. R.

Gunnarsdóttir

Hedensjö

Andersson

Ekselius

Furmark

(2007). Treatment of social phobia: Randomised trial of internet-delivered cognitive-behavioral therapy with telephone support. British Journal of Psychiatry, 190(2), 123–128. https://doi.org/10.1192/bjp.bp.105.020107

10.

California Department of Social Services. (2006). CEBC - California Evidence-Based Clearinghouse for Child Welfare . Retrieved from https://www.cebc4cw.org/

11.

Carroll

Patterson

Wood

Booth

Rick

Balain

(2007). A conceptual framework for implementation fidelity. Implementation Science, 2(40), 1–9. https://doi.org/10.1186/1748-5908-2-40

12.

Cook

R. D.

(1977). Detection of influential observation in linear regression. Technometrics, 19(1), 15–18. https://doi.org/10.2307/1268249

13.

Des Jarlais

D. C.

Lyles

Crepaz

(2004). TREND Group. Improving the reporting quality of nonrandomized evaluations fo behavioral and public health interventions: The TREND statement. American Journal of Public Health, 93(3), 361–366. https://doi.org/10.2105/AJPH.94.3.361

14.

Dymnicki

A. B.

Wandersman

Osher

D. M.

Pakstis

(2017). Bringing interventions to scale: Implications and challenges for the field of community psychology. In Bond

M. A.

Serrano-Garcia

Keys

C. B.

Shinn

(Eds.), APA Handbook of community psychology: Methods for community research and action for diverse groups and issues (pp. 297–310). American Psychological Association. https://doi.org/10.1037/14954-017

15.

Eichas

Ferrer-Wreder

Olsson

T. M.

(2019). Contributions of positive youth development to intervention science. Child & Youth Care Forum, 48(Special Issue: Positive Youth Development Across Cultures), 279–287. https://doi.org/10.1007/s10566-018-09486-1

16.

Escoffery

Lebow-Skelley

Haardoerfer

Boing

Udelson

Wood

Hartman

Fernandez

M. E.

Mullen

P. D.

(2018). A systematic review of adaptations of evidence-based public health interventions globally. Implementation Science, 13(1), 125. https://doi.org/10.1186/s13012-018-0815-9

17.

Ferrer-Wreder

Sundell

Mansoory

(2012). Tinkering with perfection: Theory development in the intervention cultural adaptation field. Child and Youth Care Forum, 41(2), 149–171. https://doi.org/10.1007/s10566-011-9162-6

18.

Flay

B. R.

Biglan

Boruch

R. F.

Gonzalez Castro

Gottfredson

Kellam

S. G.

Moscicki

E. K.

Schinke

Valentine

J. C.

(2005). Standards of evidence: Criteria for efficacy, effectiveness, and dissemination. Prevention Science, 6(3), 151–175. https://doi.org/10.1007/s11121-005-5553-y

19.

Gladstone

T. R. G.

Rintell

L. S.

Buchholz

K. R.

Myers

T. L.

(2021). Adaptation of an evidence-based online depression prevention intervention for college students: Intervention development and pilot study results. Social Sciences, 10(10), 398. https://doi.org/10.3390/socsci10100398

20.

Glasziou

Meats

Shepperd

(2008). What is missing from descriptions of treatment in trials and reviews? British Medical Journal, 336(7659), 1472–1474. https://doi.org/10.1136/bmj.39590.732037.47

21.

Gonzales

N. A.

(2017). Expanding the cultural adaptation framework for population-level impact. Prevention Science, 18(6), 689–693. https://doi.org/10.1007/s11121-017-0808-y

22.

Gottfredson

D. C.

Cook

T. D.

Gradner

F. E. M.

Gorman-Smith

Howe

G. W.

Sandler

I. N.

Zafft

K. M.

(2015). Standards of evidence for efficacy, effectiveness, and scale-up research in preventions science: Next generation. Prevention Science, 16(7), 893–926. https://doi.org/10.1007/s11121-015-0555-x

23.

Hansson

Olsson

(2012). Effects of multidimensional treatment foster care (MTFC): Results from a RCT study in sweden. children and youth services Review, 34(9), 1929–1936. https://doi.org/10.1016/J.CHILDYOUTH.2012.06.00

24.

Hasson

(2010). Systematic evaluation of implementation fidelity of complex interventions in health and social care. Implementation Science, 5(67), 1–9. https://doi.org/10.1186/1748-5908-5-67

25.

Higgins

J. P. T.

Savovic

Page

M. J.

Elbers

R. G.

Sterne

J. A. C.

(2023). Chapter 8: Assessing risk of bias in a randomized trial. In Higgins

J. P. T.

Thomas

Chandler

Cumpston

Page

M. J.

Welch

V. A.

(Eds.), Cochrane Handbok for Systematic Reviews of Intervenitons version 6.4. Cochrane Training. Available from www.training.cochrane.org/handbook.

26.

Jensen

I. B.

Nygren

Å.

Lundin

(1994). Cognitive-behavioral treatment for workers with chronic spinal pain: a matched and controlled cohort study. Occupational and environmental medicine, 51(3), 145–151. https://doi.org/10.1136/oem.51.3.145

27.

Kling

Forster

Sundell

Melin

(2010). A randomized controlled effectiveness trial of Parent Management Training with varying degrees of therapist support. Behavior Therapy, 41(4), 530–542. https://doi.org/10.1016/j.beth.2010.02.004

28.

Lee

S. J.

Altaschul

Mowbray

C. T.

(2008). Using planned adaptation to implement evidence-based programs with new populations. American Journal of Community Psychology, 41(3-4), 290–303. https://doi.org/10.1007/s10464-008-9160-5

29.

Liedgren

Kullberg

(2022). Easy ride or born to be wild’? The traveling of evidence-based social work to Sweden. European Journal of Social Work, 25(2), 224–237. https://doi.org/10.1080/13691457.2021.1918064

30.

Lipsey

M. W.

Wilson

(2000). Practical Meta-Analysis. Applied Social Research Methods. Sage.

31.

Livheim

Hayes

Ghaderi

Magnusdottir

Högfeldt

Rowse

Turner

Hayes

S. C.

Tengström

(2015). The effectiveness of Acceptance and Commitment Therapy for adolescent mental health: Swedish and Australian pilot outcomes. Journal of Child and Family Studies, 24(4), 1016–1030. https://doi.org/10.1007/s10826-014-9912-9

32.

Maden

Cunliffe

McMahonb

Booth

Carey

G. M.

Paisley

Dickson

Gabbay

(2017). Use of programme theory to understand the differential effects of interventions across socio-economic groups in systematic reviews-a systematic methodology review. Systematic Reviews, 6(266), 1–23. https://doi.org/10.1186/s13643-017-0638-9

33.

Mejia

Leijten

Lachman

J. M.

Parra-Cardona

(2017). Different strokes for different folks? Contrasting approaches to cultural adaptation of parenting interventions. Prevention Science, 18(6), 630–639. https://doi.org/10.1007/s11121-016-0671-2

34.

Moher

Hopewell

Schulz

K. F.

Montori

Gotzsche

P. C.

Devereaux

P. J.

Elbourne

Egger

Altman

D. G.

(2010). CONSORT 2010 Explanation and elaboration: Updated guidelines for reporting parallel group randomised trials. British Medical Journal, 340(c869), 1–28. https://doi.org/10.1136/bmj.c869

35.

Moore

Campbell

Copeland

Craig

Movsisyan

Hoddinott

Littlecott

O’Cathain

Pfadenhauer

Rehfuess

Segrott

Hawe

Kee

Couturiaux

Hallingberg

Evans

(2021). Adapting interventions to new contexts-the ADAPT guidance. British Medical Journal, 374(1679), 1–10. https://doi.org/10.1136/bmj.n1679

36.

Moore

Bumbarger

Cooper

(2013). Examining adaptations of evidence-based programs in natural contexts. Journal of Primary Prevention, 34(3), 147–161. https://doi.org/10.1007/s10935-013-0303-6

37.

Müssener

Linderoth

Bendtsen

(2020). mHealth smoking cessation intervention among high school students: 3-month primary outcome findings from a randomized controlled trial. PLoS ONE, 15(3), 1–11. https://doi.org/10.1371/journal.pone.0229411

38.

Nyberg

Sundblom

Norman

Å.

Bohman

Hagberg

Elinder

L. S.

(2015). Effectiveness of a universal parental support programme to promote healthy dietary habits and physical activity and to prevent overweight and obesity in 6-year-old children: The Healthy School Start Study, a cluster-randomised controlled trial. PLoS One, 10(2), 1–19. https://doi.org/10.1371/journal.pone.0116876

39.

O'Connell

M. E.

Boat

Warner

K. E.

2009). Preventing mental, emotional, and behavioral disorders among young people: progress and possibilities. The National Academies Press.

40.

Olsson

T. M.

Kapetanovic

Hollertz

Starke

Skoog

(2023). Advancing social intervention research through program theory reconstruction. Research on Social Work Practice. Special Issue: Program Theory in Social Interventions Developed in Practice Settings, 33(6), 642–655. https://doi.org/10.1177/10497315221149976

41.

Olsson

T. M.

Långström

Skoog

Andrée Löfholm

Leander

Brolund

Ringborg

Nykänen

Syversson

Sundell

(2021). Systematic review and meta-analysis of noninstitutional psychosocial interventions to prevent juvenile criminal recidivism. Journal of Consulting and Clinical Psychology, 89(6), 514–527. https://doi.org/10.1037/ccp0000652

42.

Olsson

T. M.

Sundell

(2016). Research that guides practice: Outcome research in Swedish PhD theses across seven disciplines 1997-2012. Prevention Science, 17(4), 525–532. https://doi.org/10.1007/s11121-016-0640-9

43.

Olsson

T. M.

Sundell

(2023). Publication bias, time-lag bias, and place-of-publication bias in social intervention research: An exploratory study of 527 Swedish articles published between 1990-2019. PloS ONE, 18(2), e0281110. https://doi.org/10.1371/journal.pone.0281110

44.

Olsson

T. M.

Sundell

Leander

(2022). The devil is in the details: Unpackaging the systematic in systematic reviews: Short reply to reconsidering the effectiveness of mentoring for prevention of juvenile criminal recidivism. Journal of Consulting and Clinical Psychology, 90(8), 652–654. https://doi.org/10.1037/ccp0000748

45.

Öst

L. G.

Thulin

Ramnero

(2004). Cognitive behavior therapy vs exposure in vivo in the treatment of panic disorder with agoraphobia (corrected from agrophobia). Behaviour Research and Therapy, 42(10), 1105–1127. https://doi.org/10.1016/j.brat.2003.07.004

46.

Overmeer

Boersma

Denison

Linton

S. J.

(2011). Does teaching physical therapists to deliver a biopsychosocial treatment program result in better patient outcomes? A randomized controlled trial. Physical Therapy, 91(5), 804–819. https://doi.org/10.2522/ptj.20100079

47.

Petersen

Davis

R. C.

Weisburd

Taylor

(2022). Effects of second responder programs on repeat incidents of family abuse: An updated systematic review and meta-analysis. Campbell Systematic Reviews, 18(1), e1217. https://doi.org/10.1002/cl2.1217

48.

Rabin

B. A.

McCreight

Battaglia

Ayele

Burke

R. E.

Hess

P. L.

Frank

J. W.

Glasgow

R. E.

(2018). Systematic, multimethod assessment of adaptations across four diverse health systems interventions. Frontiers in Public Health, 6(102), 1–11. https://doi.org/10.3389/fpubh.2018.00102

49.

Rajabiun

Myers

Goldberg

Fortu

Sullivan

Berthaud

Wahome

Downes

Cabral

H. J.

(2021). Outcomes from the adaptation of an evidence-based peer linkage and re-engagement intervention for women of color with HIV in three urban clinics. AIDS and Behavior, 26(2), 415–424. https://doi.org/10.1007/s10461-021-03395-6

50.

Resnicow

Soler

Braithwaite

R. L.

Ahluwalia

J. S.

Butler

(2000). Cultural sensitivity in substance use prevention. Journal of Community Psychology, 28(3), 271–290. https://doi.org/10.1002/(SICI)1520-6629(200005)28:3<271::AID-JCOP4>3.0.CO;2-I

51.

Schulz

K. F.

Altman

D. G.

Moher

, & the CONSORT Group. (2010). CONSORT 2010 Statement: Updated guidelines for reporting parallel group randomised trials. BMC Medicine, 8(18), 1–9. https://doi.org/10.1186/1741-7015-8-18

52.

Skärstrand

Sundell

(2013). Evaluation of a swedish version of strengthening families program. The European Journal of Public Health, 24(4), 578–584. https://doi.org/10.1093/eurpub/ckt146

53.

Stahmer

A. C.

Rieth

S. R.

Dickson

K. S.

Feder

Burgeson

Searcy

Brookman-Frazee

(2019). Project ImPACT for toddlers: Pilot outcomes of a community adaptation of an intervention for autism risk. Autism, 24(3), 617–632. https://doi.org/10.1177/1362361319878080

54.

Sundell

Åhsberg

(2016). Trends in methodological quality in Swedish controlled trials of psychological and social interventions. Research on Social Work Practice, 28(5), 568–576. https://doi.org/10.1177/1049731516633297

55.

Sundell

Beelman

Hasson

von Theile Schwarz

(2015). Novel programs, international adoptions, or contextual adaptations? Meta-analytical results from German and Swedish intervention research. Journal of Clinical Child & Adolescent Psychology, 45(6), 784–796. https://doi.org/10.1080/15374416.2015.1020540

56.

Sundell

Olsson

T. M.

(2017). Social Intervention Research. Oxford Bibliographies. https://doi.org/10.1093/OBO/9780195389678-0254

57.

Sundell

Olsson

T.M.

(2021). Svenska effektuvärderingar av beteendemässiga, psykologiska och sociala interventioner 1990-2019. Swedish Research Council for Health, Working Life and Welfare. Retrieved from https://forte.se/app/uploads/2021/04/fort-0035-rapport-effektutvarderingar_ta.pdf

58.

von Thiele Schwarz

Aarons

G. A.

Hasson

(2019). The value equation: Three complementary propositions for reconciling fidelity and adaptation in evidence-based practice implementatin. BMC Health Services Research, 19(868), 1–10. https://doi.org/10.1186/s12913-019-4668-y

59.

von Thiele Schwarz

Förberg

Sundell

Hasson

(2018). Colliding ideals - an interview study of how intervention researchers address adherence and adaptations in replication studies. BMC Medical Research Methodology, 18(868), 1–10. https://doi.org/10.1186/s12874-018-0496-8

60.

Walfish

(2006). A review of statistical outlier methods. Pharmaceutical Technology, 30(11), 82. https://api.semanticscholar.org/CorpusID:114377662