Assessing the Implementation of Interventions Addressing Socioeconomic Inequalities in Cancer Screening in High-Income Countries

Abstract

Background

The context of an intervention may influence its effectiveness and success in meeting the needs of the targeted population. Implementation science frameworks have been developed, but previous literature in this field has been mixed. This paper aimed to assess the implementation success of interventions, identified from a systematic review, that reduced inequalities in cancer screening between people in low and high socioeconomic groups.

Design and methods

The implementation framework by Proctor et al. was utilised to assess the potential success of 6 studies reporting on 7 interventions in the “real-world” environment. A standardised rating system to identify the overall implementation success of each intervention was established.

Results

Four interventions (57%) demonstrated high potential to be implemented successfully. Interventions included enhanced reminder letters and GP-endorsed screening invitations, containing evidence on the acceptability, from participants and stakeholders, appropriateness and direct cost of the intervention.

Conclusion

While some interventions reduced socioeconomic inequalities in cancer screening participation, there have been missed opportunities to integrate the experiences of the targeted population into design and evaluation components. This has limited the potential for transferability of outcomes to other settings.

Significance for public health

Interventions may be effective within a research context and the sample population but may not equally benefit all when implemented. Implementation science frameworks are useful in understanding how and why an intervention may succeed or fail in the “real-world” environment. This project assessed the implementation success of interventions, identified from a systematic review that reduced inequalities in cancer screening between people in low and high socioeconomic groups. Over half of the interventions that produced a positive impact in the research context had high potential for implementation success. This assessment demonstrated that no one size fits and for the socioeconomically disadvantaged, input from these participants along the implementation spectrum is a crucial element for ensuring transferability to other settings. To make sure finite research funding is used effectively, engagement with disadvantaged populations is of public health importance because programs or services need to be tailored to their specific needs to reduce inequalities.

Introduction

The effectiveness of an intervention and its success in meeting the needs of the targeted population may be influenced by the context in which it has been implemented. Although controlled studies aim to replicate the true environment, where the intervention will be implemented and test it with a representative cohort, outcomes of the intended effect can only be proven within the study environment. Pfadenhauer et al. ¹ defines implementation (evidence or theory-based) as a process of fitting, assimilating or putting an intervention to use – within an organisation or setting. Effectiveness, implementation and context are all inextricably linked,^2,3 but previous literature has lacked information on implementation and contextual factors; a major barrier to appraising the transferability and applicability of interventions into the “realworld” environment. ²

Previous research has insufficiently understood the influence of implementation and context, contributing to the already existing gap between research and practice. Interventions consist of multiple components interacting either in isolation or combination with one another. ¹ This can make it difficult to conceptualise and assess the implementation of an intervention beyond the controlled environment. ¹ Consideration of whether an intervention has an impact on an intended outcome also requires appropriate assessment of the contextual factors, such as the geographical, socioeconomic, and organisational settings in which the intervention is to be delivered. ¹ Implementation of an intervention may be difficult to maintain, particularly those which fail to be embedded into process and practices. Often, this is because they are unable to gain the support of health service managers or users, or have inadequate resources for implementation, which results in services reverting to more familiar approaches.

Several implementation science frameworks have been developed to evaluate the success of an intervention and its implementation. ⁴ Evaluating and understanding the processes involved in implementing interventions into the “real-world” environment is important and necessary to improve, replicate, scale-up and embed evidence-based interventions into systems. Proctor et al. ⁵ provides a concept for implementation outcomes defining it as the “effects of deliberate and purposive actions to implement new treatments, practices and services.” This framework has eight outcomes, including acceptability, adoption, appropriateness, (implementation) costs, feasibility, fidelity, penetration and sustainability. Various disciplines have utilised this framework bringing interventions to practice and informing future development, showing that intervention delivery is complex and requires considerable time and resources. Monitoring implementation and accounting for changes is essential in ensuring the intervention makes sustainable and positive changes.

The aim of this research was to assess the implementation success of interventions from a previous systematic review, which evaluated the impact of interventions addressing socioeconomic inequalities in cancer-related outcomes between low and high socioeconomic groups within high-income countries. ⁶ Interventions considered as efficacious in reducing socioeconomic inequalities in cancer from the systematic review were assessed, and further evaluated in this paper for implementation success.

Design and Methods

Included interventions

The outcomes from a prior systematic review conducted by the research team informed the studies and interventions included in this assessment. ⁶ The systematic review identified 7 interventions from 6 studies that reduced inequalities in cancer-related outcomes between low and high socioeconomic groups within high income countries, and all were included in this assessment. Interventions improved participation in cancer screening in low socioeconomic groups, and either had similar or no effect on high socioeconomic groups, reducing the inequality. These included:

• enhanced reminder letters;

• GP-endorsed screening invitations;

• text-message reminders;

• organised implementation screening programs; and

• pre-formulated implementation intentions specifying the when, where and how a behaviour can be modified.

The methods, including study and intervention characteristics have been described in detail, in a separate publication reporting on the outcomes of the systematic review. ⁶

Implementation assessment framework

An environmental scan was conducted to identify an implementation science framework to assess the potential success of interventions in the “real-world” environment. Proctor et al.'s taxonomy of implementation outcomes was selected in preference of others because it is a conceptual framework that addresses several outcomes in a transparent and comprehensive manner. This framework considers acceptability, adoption, appropriateness, feasibility, fidelity, cost (incremental or implementation), penetration and sustainability outcomes. According to Proctor et al. the first six outcomes are relevant to earlier stages of implementation, penetration relevant to mid-implementation and sustainability to longerterm implementation. ⁵ The characteristics of the included interventions determined the level of analysis necessary to evaluate implementation success.

Analysis of implementation outcomes

Determining stage of interventions

Interventions in early phases of development were defined by Proctor et al., ⁵ as promising interventions being tested in research settings by research-based providers for efficacy. All interventions included in the assessment met this definition as all were conducted in a research setting without previous use in practice.

Included outcomes for assessment of early stage interventions

Acceptability is the perception among implementation stakeholders that a given treatment, service, practice or innovation is agreeable, palatable or satisfactory, and should be measured from the perspectives of stakeholders and consumers. This outcome was selected in assessing the interventions because receiving input from the perspectives of low socioeconomic groups and stakeholders is essential in promoting engagement. Appropriateness is the perceived fit, relevance or compatibility of the innovation or evidence- based practice for a given practice setting provider, or consumers; and/or perceived fit of the innovation to address an issue or problem. This was selected as an outcome as inequalities in cancer outcomes indicate that specific considerations are required to understand the complexities involved in engaging with disadvantaged populations, such as the socioeconomically disadvantaged. Cost of implementing the intervention (direct and indirect costs) was selected as it helped determine impact of the implementation effort, and whether the intervention would be a valuable investment in improving health outcomes. Implementation cost varies depending on the setting in which the intervention is delivered and its complexity, which were characteristics considered throughout assessment. The remaining implementation outcomes recommended for the assessment of early stage interventions (adoption, feasibility and fidelity) were not included in the assessment as they were considered to overlap and interrelate with outcomes chosen for assessment and did not provide further information to inform assessment.

Extraction of information for assessment of outcomes

Implementation outcomes described above were either inferred or measured in terms of study design elements, expressed attitudes and opinions, intentions, or reported or observed behaviours. Acceptability was measured based on the content and complexity of the intervention, and involved feedback provided from participants and/or stakeholders. Appropriateness was measured similarly to acceptability, but also considered whether the intervention was fit-for-purpose in addressing socioeconomic inequalities in cancer-related outcomes. The costs of implementing the intervention were identified, and if possible, comparisons of costs with current implementation strategies were provided.

This information was extracted for each study and by implementation outcome – acceptability, appropriateness and cost. Refer to Table 1 for a summary of the information extracted on implementation outcomes by each study.

Measurement of implementation outcomes

While Proctor et al. ⁵ suggested several approaches to measuring outcomes of implementation, such as qualitative and quantitative surveys, ways in which to measure other implementation outcomes were not proposed. A standardised rating system was used to identify the overall implementation success of an intervention because of the differing nature of the populations, interventions and contexts. A detailed explanation of the implementation rating system by implementation outcome can be seen below.

Using implementation outcomes to assess implementation success

To determine the potential for an intervention to be implemented successfully into the “real-world” environment, interventions were rated by implementation outcome. In this assessment, implementation success was modelled to reflect:

$$\begin{array}{l}\text{Implementation success =}\\ \text{Intervention efficacy + implementation outcomes (acceptability, appropriateness and cost)}\end{array}$$

The overall potential implementation success score, ranged from 0 to 6, which was calculated based on the sum of each individual implementation outcome. An overall score of 0 indicated no potential implementation success, 1 or 2 low potential implementation success, 3 or 4 moderate potential implementation success and 5 or 6 indicated high potential implementation success.

Results

This assessment focused only on the results of the interventions that reduced inequalities in cancer screening between people in low and high socioeconomic groups, and their implementation outcomes.^7-12 Four interventions were assessed to have the potential to be implemented successfully beyond the study environment and included enhanced reminder letters and GP-endorsed screening interventions.

Acceptability

Three studies (50%) assessed the acceptability of four interventions in addressing socioeconomic inequalities in cancer screening through focus and advisory groups.^10-12

The studies that implemented focus groups and promoted collaboration with participants in low socioeconomic groups and stakeholders in the development or delivery method of the intervention displayed high acceptability (n=2, 67%).^10,11 One study (33%) that engaged with either participants or stakeholders to determine the transferability and applicability of the intervention in the “real-world” environment showed moderate acceptability. ¹²

The remaining half of the studies (n=3, 50%) did not provide evidence that indicated an interventions acceptability from a participant or stakeholder perspective.^7-9

Appropriateness

Nearly all studies (n=5, 83%) included details of activities used to assess the appropriateness of interventions in addressing socioeconomic inequalities in cancer screening.^8-12 Of these studies, all showed high appropriateness, specifically targeting interventions towards socioeconomic disadvantaged populations compared to developing an intervention that targeted the general population.

The remaining study (17%) did not investigate sub-group populations, including socioeconomic disadvantaged populations, resulting in low appropriateness in addressing inequalities in cancer screening. ⁷

Implementation costs

Direct costs. Four interventions in three studies (50%) were assessed with low implementation costs.^10-12 Of these interventions, all required one-off costs to be implemented into existing program infrastructure, ranging from AUD $142,113.09 to $143,343.52.^10-12 Interventions included enhanced reminder letters and GP- endorsed screening invitations. Fifty percent of the studies (n=3) did not report on the cost of implementing an intervention into practice.^7-9

Indirect costs. Additional resources needed to deliver interventions were reported for one-third of the studies (n=2).^7,8 The studies outlined that written material was needed to deliver pre-formulated implementation intentions and text-message reminders.^7,8 Four studies (n=67%) failed to report on the additional resources needed for the development and delivery of interventions.^9-12

Table 1

Summary of implementation outcomes by each study.

No studies provided information on the investment in additional training, or the skills and knowledge staff needed to deliver the interventions (n=6, 100%).^7-12 In addition, no study (n=7, 100%) reported on the additional time required to deliver an intervention. ^7-12

Discussion

To understand the gap between research and practice, interventions that reduced inequalities in cancer screening between people in low and high socioeconomic groups were assessed for their potential success in the “real-world” environment. While these interventions improved participation in screening for people in low socioeconomic groups, reducing socioeconomic inequalities, few demonstrated the capacity to be translated from the controlled environment, where they were developed and tested, to practice, where their impact is needed. Interventions with high potential implementation success included enhanced reminder letters and GP-endorsed screening invitations.^10-12 The remaining interventions showed either no implementation success, ⁹ or low or moderate implementation success.^7,8

Acceptability is far more complex than just participation rates and intervention uptake, demonstrating that participant satisfaction and stakeholder involvement is critical to measuring success. ¹³ Assessment of interventions showed that acceptability was mixed among the population samples, and those that were assessed with similar potential for implementation success had alike characteristics. Interventions with high potential for implementation success either used focus groups during the developmental phase to better understand the reasons for low socioeconomic groups not participating in cancer screening programs or collaborated with participants and established stakeholder groups, such as Primary Care Advisory Groups to facilitate implementation into existing infrastructure.

The enhanced reminder letters were designed with input from low socioeconomic groups, through engaging with this population to explore reasons for non-uptake of colorectal cancer screening. ^10,12 High levels of participant involvement helped researchers target low awareness and address the inaccurate risk perceptions of colorectal cancer screening, creating high intervention acceptability. ^10,12 This may have motivated low socioeconomic groups to engage with cancer screening services, but few interventions involved participants to be part of the study design and implementation process. The complexity and time required for stakeholders to deliver an intervention are important additional outcomes to consider. An intervention that relies on automated and customised text-messages, phone calls and reminders would be resource intensive for healthcare professionals to deliver. It may also lead to low socioeconomic groups responding negatively and therefore, reporting relatively low acceptability. ¹⁴ Future interventions need to collaborate with the targeted population and various stakeholders, and recognise the complexities involved in developing an intervention to ensure scalability. ¹⁵

While not commonly an outcome evaluated in previous literature, intervention appropriateness is important but there is no general, one size fits all. An intervention is deemed appropriate depending on individual circumstances and whether the identified need of the intervention is achieving positive outcomes in the best possible way. Five studies were assessed as having high appropriateness, ^8-12 however, the methods used to encourage intervention uptake varied. Community-based participation was often used to recruit members across all socioeconomic groups, a well-valued approach for tailoring programs or services to meet the specific needs of a targeted population. ¹⁶ Moreover, altering the intervention so that the language can be understood by people with varying levels of health literacy is important, and only three interventions considered this. Literacy level is a component of language and it is likely that when this view is adopted, interventions will be perceived as useful and compatible by disadvantaged populations.^16,17

Direct costs for implementing interventions were most frequently reported, but evidence was limited. An acceptable costeffectiveness ratio is difficult to assess objectively because the complexity of an intervention and the setting in which it is delivered varies. It demonstrates that cost should not be considered in isolation of other components. Four interventions were considered to have relatively low implementation costs, including enhanced reminder letters and GP-endorsed screening invitations.^10-12 These interventions were delivered in supportive environments, where the National Health Service Bowel Cancer Screening Program had been running for years, and only a one-off upfront cost was required to modify the system due to already established pathways and high engagement. Although Reeves et al. ¹⁸ highlights that resource-intensive interventions subject to high initial investments, can be equally as effective if scaling-up and funding by health systems is justified. In this assessment no study explored how implementation costs would be borne by health systems at large, which is a necessary question to address to ensure intervention sustainability.

The interventions included in this assessment successfully reduced inequalities in cancer screening between low and high socioeconomic groups. An essential component in understanding an interventions ability to transition from the controlled to “realworld” environment depends on how it is delivered. Many interventions in the assessment were not considered as products ready “right out of the box” for implementation, potentially explaining the long transition from research to practice. While an intervention may be efficacious, transitioning it into the “real-world” environment is a stepped, nuanced process, and one size may not fit all. ¹⁵

Study limitations

Due to the heterogeneity of the interventions in terms of context, population recruitment and methodology, it was difficult to make comparisons between interventions. A specific framework ⁵ was utilised to assess the potential implementation success of interventions. While many of the interventions were considered to have limited implementation potential in the “real-world” environment, it does not necessarily mean that the interventions were not successfully implemented into practice. In addition, Proctor et al. ⁵ did not propose or validate a standardised measurement tool, and the tool (i.e., rating system) used in this assessment were not formally tested. The ratings were assigned by the research team and therefore, may have introduced bias, influencing the findings and promoting the successfulness of some interventions in reducing socioeconomic inequalities in cancer screening.

Conclusion

Few interventions have utilised an area-based measure to account for differences in cancer screening participation by socioeconomic status. Interventions which considered implementation outcomes had higher potential for success, indicating the importance for evaluating such methodologies in achieving ongoing improvements in diagnosis and screening outcomes. This study identified that interventions which use an area-based measure to improve socioeconomic inequalities, often miss opportunities to integrate the experiences of the targeted populations into the design and evaluation of interventions, limiting their potential transferability of outcomes to other settings. In designing future programs or services, the context in which interventions are delivered must be suitable for the targeted population and meet the specific needs of that population.