Sage Journals: Discover world-class research

Abstract

The Canadian youth services system is fragmented with less than one third of youth accessing the mental health services they need. Experts have called for systems transformation that will increase the integration of youth services and take advantage of complementary services, such as peer supports. Further, researchers have suggested that there is a need to identify the unique contribution and underlying mechanisms that support client recovery within youth peer support interventions. This paper describes the steps taken to implement a hybrid realist and participatory evaluation examining peer support services for youth (14–26 years old) with mental health, physical health and/or substance use challenges. We describe the procedures followed to engage peers in the design of the study and how this was integrated with a realist approach. We also provide a detailed description of the related adaptations to the methods applied within the second stage of the study. Lessons learned through the integration of the two methods are provided as well as potential implications for the findings and related research.

Keywords

community based research methods in qualitative inquiry mixed methods par—participatory action research qualitative evaluation

Introduction

Youth are at increased risk of developing mental health and substance use issues with approximately 75% of lifetime adult mental disorders commencing before the age of 18 (Kessler et al., 2005). In Canada, the youth services system is fragmented (Biglan, Flay, Embry, & Sandler, 2012; Kirby, Keon, & Dinsdale, 2006; Waddell et al., 2005) and researchers have identified that less than one third of youth access the mental health services they need (Merikangas et al., 2011). Barriers to help-seeking among youth are well documented and include such factors as perceived stigma, difficulties recognizing symptoms, and a preference for self-reliance (Gulliver et al., 2010). Further, stigma has been described as the principal barrier to the delivery of care (Sartorius, 2007). Experts have called for systems reform to support increased integration of services, access to timely supports and that take advantage of complementary services, such as peer supports (Malla et al., 2018; Settipani et al., 2019).

Recently, there has been a surge of interest in peer support programs (O’Connell et al., 2018) including youth serving organizations (Ontario Centre of Excellence for Child and Youth Mental Health, 2018). This is likely the result of identified benefits within the adult literature (Cabassa et al., 2017; Chinman et al., 2014; Lloyd-Evans et al., 2014) as well as recent interest in peer support as a method to integrate youth voice within services (Daley & Egag, 2019; Salt et al., 2017). Some examples of uptake in peer support within Canada are represented in the scaling of integrated youth services (Halsall et al., 2018, 2019) such as ACCESS OpenMinds (https://accessopenminds.ca/) Foundry in British Columbia (https://foundrybc.ca/), and the Youth Wellness Hubs Ontario (https://youthhubs.ca/en/). In addition, there are models that utilize peer support to enhance housing stabilization for youth such as the Housing Outreach Project—Collaboration (HOP-C; HOP-C Working Group, 2020) and Housing First for Youth (Gaetz, 2017). Although there is an expanding body of research examining youth peer support, there continues to be a need to identify the unique contribution and underlying mechanisms that support wellness within youth peer support interventions (Ali et al., 2015; Gopalan et al., 2017). Youth peer support models align with adult peer approaches, however research demonstrates that peers have a more critical influence during the developmental period of youth and young adulthood (Patton et al., 2016), and as such, they may be particularly well-suited to provide support for this age group. Although, youth peer support services have proliferated across youth service settings and have been designed to address specific developmental issues of adolescence and emerging adulthood, there is very little rigorous research examining their impact and why they might be effective (Gopalan et al., 2017).

Within the context of peer support programming, “peers” have been identified as sharing a key characteristic with the client and benefits are hypothesized to result as a function of having peer status (Simoni et al., 2011). Furthermore, peer training is typically limited in comparison with professional service providers in their field and their work is purposeful in relation with the specific function of a standardized role (Simoni et al., 2011). There are a range of different youth peer support services models. These include programs that integrate youth peer support workers within overall clinical teams that are led by non-peers, partial peer-delivered services (wherein peers have more independent roles) and independent youth peer support worker programs that are entirely led by peers (Gopalan et al., 2017). Youth peer support worker roles have included skill-building for mental health coping, emotional support, service navigation and education, action planning, engagement, coordination support and evaluation (Gopalan et al., 2017).

There is a fair amount of evidence regarding adult peer support for mental health issues and several recent reviews have been conducted (Cabassa et al., 2017; Chinman et al., 2014; Lloyd-Evans et al., 2014), however, the empirical base for youth peer support interventions is not as robust (Gopalan et al., 2017; Kirsch et al., 2014; Ontario Centre of Excellence for Child and Youth Mental Health, 2018). There are several concepts within the social cognitive theory (Bandura, 2005, 2011) that may be relevant to better understanding how peer supports might be effective, such as self-efficacy and observational learning through social modeling. Self-efficacy relates to an individual’s perception of their personal ability to succeed (Bandura, 2005) and is particular to specific situations and varies across contexts and pursuits (Bandura, 2011). Learning through social modeling involves the development of new behavior through the observation of similar others’ behavior and the related consequences (Bandura, 2011).

For example, peers may provide additional support to clients by sharing effective coping strategies (Kirsch et al., 2014). Barton and Henderson (2016) argue that peer support workers may help clients by demonstrating successful functioning in the community. Furthermore, clients may learn through the observation of their peer supporter’s behavior but also through the observation of consequences that result from these actions (Barton & Henderson, 2016). The concept of self-efficacy is also relevant to client recovery in peer support (Alvarez-Jimenez et al., 2016; Beeson et al., 2017; Hosek et al., 2011; Lloyd-Evans et al., 2014; Simoni et al., 2011). Within peer support, clients may develop stronger coping self-efficacy through the application of peer’s recommendations and through experiences of success in their recovery journey.

Finally, peer support services may help to overcome stigma (Alvarez-Jimenez et al., 2016; Barton & Henderson, 2016; Blixen et al., 2015). For example, group-level interventions to reduce self-stigma in people with mental illness have been found to be effective (Thornicroft et al., 2016). In addition, interpersonal contact with individuals suffering from mental health issues is associated with reduced stigmatizing attitudes (Couture & Penn, 2003) and identification with stigmatized mental health groups may help to maintain positive self-esteem when confronted with stigmatizing conditions (Corrigan & Watson, 2002). Therefore, connection with peer support staff who share lived experience with mental health, substance use, or other social challenges may help to increase opportunities for social learning from peer experience. Further, clients may experience reduced self-stigma and enhanced perceptions of belonging to a positive reference group through their identification with a positive peer role model.

It is important to identify the unique contribution that is provided by peer support interventions (Cabassa et al., 2017). In addition, research should determine “what aspects of peers are most important?” (Simoni et al., 2011, p. 357) as well as the specific conditions that facilitate effective peers support services (Chinman et al., 2014). These research directions have been reiterated in the youth peer support literature. Gopalan and colleagues (2017) conducted a scoping review of US studies examining youth peer support services to identify the range of models and outcomes. They recommend that future research in youth peer support focus on specific mechanisms of influence, and in particular, whether alignment between peer traits and characteristics influences treatment outcomes. For example, it is recommended that evaluations examine recruitment and training of peers to clarify their roles within organizations as well as further defining core competencies through standardized measurement (Gopalan et al., 2017). Similarly, in their systematic review of online youth peer support groups for mental health, Ali and colleagues (2015) identified that studies demonstrated promising outcomes in the reduction of anxiety and substance use but that there is a need for future research to identify the unique contribution of youth peer support. In addition, Barton and Henderson (2016) suggest that research examining youth mental health peer support programs should examine the dynamics of the relationship between peer support worker and client more closely. This is particularly relevant as peer support is being identified as a key feature in emerging integrated youth service models of care that are being implemented and scaled-up internationally (Halsall et al., 2019; Hetrick et al., 2017; Settipani et al., 2019).

Realist evaluation is an approach that can be structured to address the empirical gaps in the field of youth peer support as it is designed to answer the question: “what works, for whom, how, why and in what circumstances?” (Pawson, 2013; Rycroft-Malone et al., 2016, p. 3). Within realist evaluation, mixed methods are recommended (Manzano, 2016; Pawson, 2013; Salter & Kothari, 2014) whereby qualitative methods are used to explore program processes and theory while quantitative data collection is used to measure outcome patterns (Pawson, 2013). Realist evaluation also recognizes the complexity of programs as well as the systems in which they are embedded and is designed to capture the dynamics of this complexity (Pawson, 2013).

One key practice within realist evaluation is the use of Context-Mechanism-Outcome-Configurations (CMOC) designed to test assumptions about how and why a program might be effective. “A CMOC is a hypothesis that the program works (O) because of the action of some underlying mechanisms (M), which only comes into operation in particular contexts (C)” (Pawson, 2013, pp. 21–22). The term “context” relates to characteristics of the conditions wherein a program operates that are implicated in the functioning of the program and can include features of the client, the environment, surrounding culture and any other system that may be relevant. Outcome patterns describe the variation in program impacts, including differences related to client demographics and contextual features that may affect outcomes. Mechanisms “are causal forces or processes which operate at a different level of the system than the outcome that they generate” (Westhorp, 2019, p. 53). Mechanisms are often described as the reasoning or interpretations that participants generate in response to program offerings and these are the essential components that lead to program impacts. Although realist evaluation is well-equipped to examine how and why youth peer support might be effective, it is theory-driven and does not privilege the voice of lived experience.

While realist evaluations are theory-driven and designed to identify the underlying mechanisms that create program impacts, these methods lack the direction and intuition that can be afforded through the inclusion of lived experience within participatory approaches. Participatory evaluation with youth is an approach wherein young people are actively engaged in the research process and involved in identifying the research focus as well as methods for data collection and utilization (Checkoway & Richards-Schuster, 2003). This approach promotes the inclusion of program beneficiaries and key stakeholders and can enhance the tailoring of design and implementation to the program context. In addition, it can support the integration of subjective reasoning from the reference point of those with lived experience in order to test theoretical assumptions. Participatory approaches enhance ethical considerations and maximize community benefits (LaFrance & Nichols, 2010; Smith, 2012). In addition, youth participatory evaluation can support positive youth outcomes, increased organizational capacity, and enhanced quality, reliability and validity of data (Zeldin et al., 2012). These two methods are complimentary as they integrate knowledge from theory and practice. In addition, each approach follows an iterative and adaptive cycle to tailor methods as insight is gained through each inquiry process.

Although youth participatory evaluation has been recognized as a valuable approach, challenges often exist related to a lack of adult acceptance of youth ideas (Checkoway & Richards-Schuster, 2003) and this can lead projects to be susceptible to tokenism (see Hart, 1992). As such, there continues to be a need for practical examples in the literature that describe how to implement successful strategies. In addition, there is an identified need for guidance for researchers implementing realist evaluation in terms of how to deal with challenges describing complex processes, managing a flexible research approach and developing candidate theoretical explanations that include consideration of context, mechanism and outcomes (Marchal et al., 2018).

Context

The Transitional Aged Youth (TAY) program at LOFT (Leap of Faith Together) Community Services implements peer support services alongside case management, mental health and housing support services. In 2018/2019, the TAY program worked with approximately 800 young people between the ages of 14–26 to provide an array of support including walk-ins, housing, curriculum-based support groups, social drop-ins and on-campus supports for post-secondary students. Generally, the youth served by the TAY program reflect the diversity of the population in downtown Toronto with representation from many countries from across the world (Arora, 2019). The majority of clients are experiencing both mental health and substance use issues and a smaller proportion are coping with a chronic physical illness or a dual diagnosis (developmental disability combined with a mental health issue). In addition, there is a proportion of youth served by the TAY program who are in need of housing stabilization. The overall objective of the program is to build a sense of autonomy in clients and increase their life skills capacity to help them achieve their personal goals and maintain personal wellness, regardless of the level of complexity of their challenges.

This study was designed in collaboration with LOFT to examine the peer support services within the TAY program. In the following section, we describe the realist and participatory procedures that were used to initiate the research. Then we describe the revised research questions and methods that were developed based on initial findings.

Purpose

This paper responds to the gaps identified above while describing the implementation of a hybrid realist and participatory evaluation examining peer support services for youth with mental health, physical health and substance use challenges. The initial research study applied the social cognitive theory as a theoretical frame and was focused on the following research questions: 1) In terms of LOFT peer support for youth, what works, for whom, why and in what circumstance? and 2) What research questions are of interest to LOFT staff? Procedures related to engaging peer support staff in the design and implementation of the study are described along with methods used to integrate these practices within a realist approach. Lessons learned through the integration of the two methods are provided as well as implications for the findings and related research.

Methods

Realist approach

Following the realist approach (Manzano, 2016; Pawson & Tilley, 2004), an initial exploration of the formal program theory was conducted through a review of relevant literature, program documents and interviews with key stakeholders, including peers. Although there was not an existing logic model for the youth peer support services within the TAY program, Daley had contributed to a best practices document (Vanderheul et al., 2018) that elaborates in detail on peer principles of practice, values, core competencies, recommendations for hiring and recruitment, and key operational supports such as peer supervision and policy development. Halsall used the information from this document to create a draft logic model. This was shared with Daley, revised and then used to contribute to initial CMOCs (see Table 1).

Table 1.

Prospective Research Strategy. This Table Describes the Relationship between the Revised Evaluation Questions and initial Context-Mechanism-Outcome Configurations (CMOC) with the Preliminary Codes that Arose from the First Round of Analysis and Related Methods.

New Research Questions Based Current Issues Identified by LOFT TAY Staff	Initial Codes	Interview Guide Questions	Surveys
1. How does peer support practice differ from traditional therapeutic practice?	- Best practices - Unique peer skills	- Is the client relationship with peer supporters different than with case managers? If so, how? - What factors influence the setting of boundaries between peers and clients? - How can self-disclosure be used to support peer practice? Risks? Benefits? Are there ways that self-disclosure can be used that can be beneficial to peers? Please describe.
2. What are the on-going barriers that exist for peer supporters in the workplace?	- Current issues in Peer Support - Risks in Peer Support - Pressure to exemplify perfection in health and recovery - Story-telling/self-disclosure - Stigma - Boundaries	- How are peers perceived in the workplace? - Have you experienced stigma in your role? Please describe. - Has this influenced your professional development? Please describe. - How can organizations help to protect peers from burnout?
3. What is the nature and value-add of peer story-telling?	- Unique peer skills - Story-telling/self-disclosure	- How have you used your story in your peer practice? - How have clients reacted? - Were there any resulting consequences? Please describe.
Client-related CMOCs
1. C (Peer supporters share similar experiences and recovery journeys with clients) + M (Peers demonstrate positive identity and wellness while moving forward in recovery & clients develop a more positive evaluation of shared social reference group) → O (Clients experience enhanced positive identity, decreased self-stigma and enhanced wellbeing)	- Peer wellbeing - Stigma - Similarity between peer and client - Self-disclosure - Peer positive identity	- Have your previous experiences been helpful to you in supporting clients? If so, how? How do clients use this information? Has this resulted in any impact on clients…? Probes ▪ Self-perceptions? ▪ Stigma? ▪ Application of new strategies? ▪ Self-efficacy? ▪ Self-determination? ▪ Social skills? ▪ Social connections?	Client online survey Clients: Therapeutic Alliance Scale for Children–Youth version
2. C (Peer supporters organize social events) + M (Opportunities to participate in social activities & clients build their sense of social connections) → O (Reduced social isolation)	- Social connection - Peer successes	See interview question for CMOC #1 What are the ways, if any, that peer support can enhance client social connections?	Client online survey
3. C (Peers bring lived experience and practical knowledge with respect to successful coping / overcoming challenges) + M (Peers offer guidance based on their lived experience in addition to other mental health supports clients are receiving & clients recognize the value of peer advice and apply strategies) → O (Clients experience success in applying strategies and increased self-efficacy / self-determination)	- Unique peer skills - Similarity between peer and client - Self-disclosure - Hope - Role-modeling - Self-efficacy/self-determination	See interview question for CMOC #1	Client online survey Clients: Youth Efficacy/Empowerment Scale—Mental Health (self subscale)
Peer-related CMOCs
1. C (TAY is adaptable to peer voice) + M (opportunities for peers to contribute to organizational development & peers perceive their voice as relevant and valuable) → O (Peer supporters continue to develop self-efficacy and self-determination)	- Peer success - Positive organizational atmosphere - Team building - Inclusion - Peer visibility - Peer to peer support - Peer positive identity	- Have you had opportunities to provide input into program/service development? Please describe. How has this been used? What has this meant for you? Have you experienced any barriers to providing input?	Peers: Youth Efficacy/Empowerment Scale—Mental Health (self and systems subscales)
2. C (Organization provides appropriate supervision) + M (continued support for peers in self-care and sustainable work strategies & peers recognize and implement appropriate boundaries) → O (Peers develop stronger self-care and enhanced resilience)	- Peer supervision—flexibility/support - transitions	- In your experience within supervision, what has been the key personal insights you have developed? How, if at all, have they helped you?	Peers: Youth Efficacy/Empowerment Scale—Mental Health (self subscale)
2. C (Organizations create supportive peer supporter training and development) + M (opportunities for peers to experience increased growth and career development & peers learn professional skills and practical career knowledge) → O (Peers develop enhanced self-sufficiency and ability to transfer professional skills to support career development)	- Peer skill development - Peer case management responsibilities - Best practices	- How, if at all, has your experience as a peer influenced your career development?	Peers: Youth Efficacy/Empowerment Scale—Mental Health (self subscale)

In addition, we conducted semi-structured focus groups and interviews designed to identify preliminary context, mechanism and outcome patterns as well as general successes and challenges in order to refine initial CMOCs. Examples of interview guide questions included: Please describe how peer support services work? From your perspective, can you explain why peer support services are helpful to promote client recovery? What kind of clients benefit most from peer support programming? To achieve saturation within qualitative research, it is recommended that data be collected until no new findings are captured within further interviews (Morse & Maddox, 2014). Recognizing that the key internal program stakeholders represent a limited number of possible participants, we conducted initial interviews with peer staff, non-peer staff (including program administrators) and students that work in the TAY program at LOFT (other participants N = 15; peers N = 8). A second round of interviews (N = 7) were conducted with peers to refine and confirm initial theoretical explanations and to triangulate outcome data. For a summary of staff roles and involvement in the research, please see Table 2.

Table 2.

LOFT Staff Involvement in the Organization and the Research.

Role at LOFT (N)	Peer Co-Researcher	Peer Staff	Non-Peer Staff	Management
Role Description	- Previous client of LOFT - Administrative and program design responsibilities - Project management	- Previous client of LOFT - 18–30 years old - Several staff also involved with group facilitation and case management	- Care navigators, team leads, group facilitators, other administrative staff	- Program oversight - Peer supervision
Role in the Research	Peer Co-Researcher	Peer Staff	Non-Peer Staff	Management
Co-researcher (involved in engagement and coordination of staff, co-design of methods, data collection, co-facilitated second workshop, analysis and interpretation of the data and authorship of publications)	✓
Participated in presentation about the study and evaluation content	✓	✓	✓	✓
Contributed to discussion on emergent issues in peer support practice and exploration of research questions of interest	✓	✓	✓	✓
Participated in online workshop exploring new research questions, initial CMOCs and proposed tools	✓	✓	✓	✓
Invited to provide feedback on revised data collection strategy	✓	✓		✓
Participation in first round of interviews and focus groups	✓	✓	✓	✓
Participation in second round of interviews and survey	✓	✓
Supported development of the research proposal				✓
Provided administrative support for the study implementation	✓			✓

The initial interviews and focus group were recorded, transcribed and uploaded to QSR NVivo. Meyer and Ward (2014) recommend combining deductive and inductive approaches within research design to take previous knowledge into account, while remaining open to developing new insight. We followed a mixed inductive–deductive analytic approach wherein we created an initial codebook that was based on the research questions and current behavior change theory in peer support. In addition to identifying these initial codes, we applied an exploratory thematic analysis (Braun & Clarke, 2006), which involved familiarization with the data, generation of codes, identification of themes, revision of themes, and definition and specification of themes.

The first round of coding was completed by three coders reviewing separate transcripts. A second round of coding of all the transcripts was completed by one coder (Halsall), where relevant codes were revised and grouped into context, mechanism and outcome categories. There were also multiple themes that were related with current issues in peer support that did not align with CMOCs but related with research questions identified by staff. A third round of coding was completed by Daley and agreement was reached among the three coders with respect to the initial themes and sub-themes identified. The interview themes were compared with the themes that arose from the all-staff discussion as well as program documents to inform a second round of data collection with a focus on two goals: 1) to further refine and test initial program theories (realist objective) and 2) to capture information to respond to the research questions identified by TAY staff (participatory objective).

Participatory Approach

The participatory approach was launched with an all-staff workshop that included both peer staff, non-peer staff and students from the LOFT-TAY program. The workshop focused on the study purpose and background, general evaluation principles and an exploratory discussion to collect feedback on initial design and current issues of interest within peer support. The study and evaluation content were delivered in a short PechaKucha-style presentation. In our previous work, this method was identified by youth advocates as a novel way to communicate information about evaluation. Pecha Kucha (Japanese for chit-chat) is a presentation style whereby 20 slides (mainly images) are shown for 20 seconds creating a concise and fast-paced message designed to increase audience engagement (see www.pechakucha.com).

Notes were captured during the discussion and major themes were combined with information from program documents and interview codes to create revised research questions. These new research questions along with initial CMOCs were presented to staff in a second online workshop, along with proposed tools and draft interview guide and survey questions. The workshop was facilitated online as public health restrictions related to the COVID-19 pandemic did not permit for an in-person activity. Staff were invited to review the revised evaluation strategy and provide feedback. Research questions, CMOCs and methods were presented to LOFT staff before submitting the revisions as an ethics amendment to the Royal Ottawa Health Care Group Research Ethics Board.

In addition to involving the broader staff in identifying the major research directions and methods, Daley participated in regular meetings and contributed key insight to each stage of design and development. Along with other research staff, she received a training on the study background research, related theory and qualitative research methods. She was involved with assembling program documents, facilitating engagement with staff, tool selection and design, transcription of interviews, data analysis and interpretation, and revising manuscripts.

The initial realist and participatory research methods as well as the second round of data collection are outlined in Figure 1. The second round of procedures were designed based on the findings from the first stage of data collection. As described in Figure 1, these procedures have been underway since the early fall 2020. As a result of the current COVID-19 pandemic public health restrictions, all data is being collected online or over the phone. The next section describes the procedures for the second round of data collection.

Figure 1.

This figure displays each component of the study and presents the timeline for implementation.

Second Round of Data Collection

Peer interviews

Based on the initial themes identified in the first round of analyses and on staff discussions, new research questions and corresponding methods were created (see Table 1). The second round of interviews have been conducted with peers to examine CMOC hypotheses more in-depth and to explore current issues in peer support that were identified by staff at the initial workshop. Similar to the initial data collection, we used semi-structured interviews and a deductive-inductive analysis approach. Revised interview guide questions are presented in Table 1 and in the supplementary file entitled “Revised Peer Interview guide_Final.”

Client online survey

As a result of pandemic restriction measures, peer support services are being delivered online and an online survey was designed to be administered to clients (N = 80) and peers (N = 7). We are in the process of recruiting clients through online peer support therapy groups. The client survey questions are focused on demographics, process issues related with peer support services, perceptions about assigned peer supporter and peer support practice and positive mental health outcome indicators. The single-item measure of self-rated mental health and life satisfaction which have been used as measures of positive mental health (see Orpana et al., 2017) have been included. These are being used as outcome variables, along with responses to the survey item identifying new social connections developed through LOFT.

In addition, the survey includes the Youth Efficacy/Empowerment in Mental Health Scale–Self subscale¹ (YES-MH; Walker & Powers, 2008) and a modified version of the Therapeutic Alliance Scale for Children: Youth version² (TASC; Shirk & Saiz, 1992; Shirk, Karver, & Brown, 2011). The TASC was modified so that respondents reference their peer rather than a therapist connection. The TASC has been included as a measure of how well the client relates with their peer and will be used to represent the context variable within the first client CMOC. The YES-MH is designed to examine youth perceptions of their ability to manage their own mental health (Walker & Powers, 2008). This measure will be used to examine efficacy or empowerment as a possible mechanism that supports client wellbeing (see third CMOC in Table 1).

Peer online survey

An online survey was distributed to peers. The peer survey included the YES-MH–self subscale and in addition, the system subscale. The system subscale is designed to examine respondent perception of their ability to support care providers to improve services and to help other youth navigate services (Walker & Powers, 2008). This measure is being used to test the peer CMOCs (see Table 1). Self-efficacy and empowerment are represented as an outcome within the first peer CMOC and may also be related to the outcomes in the second CMOC (stronger self-care and enhanced resilience) and third CMOC (enhanced self-sufficiency and ability to transfer professional skills).

Each survey respondent will receive an online gift card valued at 20$. The data from the two surveys will be used to triangulate the interview findings and to test proposed relationships within initial CMOCs. Logistic regression analysis will be performed to determine the relationships among proposed context, mechanism and outcome variables. Both online surveys are being used to test initial CMOCs (see Table 1). The survey findings will also be useful to triangulate the qualitative findings being derived from the interviews. Finally, the data will be used to examine relationships among proposed context, mechanism and outcome variables.

Knowledge mobilization

A third workshop is planned for the winter or early spring of 2021 to present the final data to staff and to facilitate a discussion to capture their input to support decision-making in utilization of the findings. The LOFT-TAY staff are also involved in the provision of peer support training for the Youth Wellness Hubs Ontario (YWHO; www.youthhubs.ca) initiative. YWHO is a network of integrated youth service hubs that is being implemented across the province of Ontario. Findings from this study will be used to inform the training and development of peer support services within the YWHO network.

Discussion and Lessons Learned

This study describes the process for engaging peers in a participatory evaluation approach within youth peer support services. It also provides an in-depth description of how to implement a realist approach that applies an exploratory mixed methods strategy. In addition, it describes how realist evaluation can be combined with participatory methods to create an integrated strategy. We experienced challenges and opportunities that are useful to share with other researchers interested in applying related methods or working within similar contexts.

We argue that this project would not have been as successful if the proposed method had included only a realist aspect, as it would have been more difficult for the lead researcher to develop partnerships with the LOFT staff. Nevertheless, it was also difficult to integrate a fixed direction within a participatory approach. In this case, the realist approach was not a method that was selected by peers. As such, it was of key importance to clearly communicate the rationale and potential utility of applying the realist approach in order to generate buy-in. Realist evaluation methods are concerned with using theory to show how programs work and we found that it was challenging to find agreement on the theoretical explanations needed for design and implementation of the approach. In some respects, realist methods stand in contrast to participatory methods. Allowing theory to inform research design requires a top-down approach, while using lived experience to guide research strategy follows a bottom-up direction. Yet, realist evaluation has significant practical value to understand the inner-workings of interventions and these benefits helped to justify the means. In addition, the realist approach requires strong involvement from key stakeholders and an iterative approach whereby early findings are used to design methods going forward. This process was easier to combine with participatory methods whereby staff perceptions were used to revise the evaluation strategy, while also refining initial CMOCs.

A major ethical issue in participatory research is related with the tension between maintaining participant anonymity and creating opportunities to be included as authors. Other researchers have discussed the issue of maintaining anonymity and giving voice to participants in research with Indigenous communities and projects using visual methods (Evans, 2004; Wiles, Coffey, Robinson, & Heath, 2012). Evans (2004) notes that anonymizing individuals within community-based research can diminish authority and that potential community benefits should be considered,

In qualitative branches of sociology, history, anthropology, or Indigenous studies, and especially in the context of much community centred research, anonymity can obscure community authority and voice, .... In fact, misplaced confidentiality can “disappear” people and communities…Against the loss of research results is the issue of benefits, the benefits to communities that come of effective participatory research, truly community centred research projects, and research results that community members can interpret, own, and in which they see themselves reflected and named. (Evans, 2004, pp. 73–74)

Similarly, Wiles and colleagues (2012) acknowledge that imposing anonymity denies participants the right to choose to be identified. “Being identified was often about more than just being seen in a visual image; it was also about having a message or viewpoint heard and having views made visible.” Evans also recognizes that younger participants may not be able to fully understand the range of impacts that could result from the use of their data and that their views may change with the passage of time.

When co-researchers are contributing as participants and as authors, it may not be possible to maintain their confidentiality with respect to the data they provide. The potential risks and benefits should be discussed with the co-author so that they can make an informed decision with respect to whether they would like to include their name. In initiatives where young people are engaged as partners to enhance services, youth advocates are often called upon to share their personal stories of experiencing mental health issues with the objective to raise awareness or generate funding (Daley & Egag, 2019).Yet, youth advocates may experience discrimination as a result of disclosing this information. In addition, sharing personal stories can be disempowering when it is not accompanied by the inclusion of their personal views in decision-making and this has become a form of tokenism within youth engagement in mental health. It is important to provide youth advocates a forum to share their own views and to understand how their story is being used (Daley & Egag, 2019). Engaging these youth advocates in research expands their capacity to contribute to key issues by combining their lived experience with knowledge of research and theory. It also creates a potential avenue for career advancement in research. The field of youth mental health research would benefit significantly from having researchers with lived experience and these youth have the potential to become key leaders in the future.

There is an added advantage to the involvement of peers in co-design as they bring the lens of lived experience of being a program beneficiary who has received services in the past. Therefore, they are able to draw on the knowledge they have developed from both receiving peer services and delivering them and can offer insights and practical learning from both of those experiences. As a result, the contributions to the study made by the peers involved a heightened recognition and sensitivity for the challenges that clients were experiencing as well as an in-depth understanding of how service provision can be adapted and improved. Through their involvement, this research study was conceived and implemented using the perspectives of research, practice and lived experience.

Involving peers in shaping the research direction, methodological decisions, analysis and interpretation significantly influenced the nature of the findings. For example, the initial research proposal used the social cognitive theory to frame the research questions. Through collaborative discussion and reflection on the themes, we were able to expand our considerations to new social theories. In addition, through discussion we were able to find alignment between language derived from contextual experience and theoretical concepts. This process required a fair amount of time and dialogue in order to identify coherent explanations. We hope that this process will bring stronger meaning to both theory and practice and, hopefully, will allow the findings to be better understood and applied.

To elaborate, a new theory of importance that was identified through co-design with peers was the social identity theory (Tajfel, 1974 ]). Social identity is defined as “that part of an individual’s self-concept which derives from his knowledge of his membership of a social group (or groups) together with the emotional significance attached to that membership” (Tajfel, 1974, p. 69). The first CMOC is informed by social identity theory and identifies a potential influence on stigma. Themes that are related to the first CMOC are: “Peer wellbeing,” “Stigma,” “Similarity between peer and client,” “Self-disclosure” and “Peer positive identity.”

The second CMOC highlights the importance of social contexts and the potential for significant influence on social connections. The themes that substantiated this CMOC included “Social connection” and also “Peer success.” Peer success was an important theme as many social activities and events that were coordinated for clients were peer-led projects and were perceived as significant successes. Finally, the third CMOC is closely aligned with the social cognitive theory as it describes observational learning through peer modeling and the development of self-efficacy. Implicated themes include: “Unique peer skills,” “Similarity between peer and client,” “Self-disclosure,” “Hope,” “Role-modeling” and “Self-efficacy/self-determination”).

Strengths and Limitations

There are several limitations and strengths that should be noted with respect to the methods applied. Initially, a secondary analysis of client assessment measures had been planned to capture outcome measures related to client recovery. As we began to compile the assessments, however, it was determined that there were not sufficient numbers of the identified measures completed with clients receiving peer support services. Therefore, we altered our strategy to collect new outcome indicators, but were limited to one time-point due to the short term of the grant. In addition, many of our procedures had to be facilitated online as a result of COVID-19 pandemic restriction measures. It was challenging to deliver workshop content in this medium and broader staff engagement was stronger in the earlier workshops and decision-making that were facilitated in person.

Internal evaluations can be perceived as less objective (Conley-Tyler, 2005) and although this approach involves external researchers, internal peers have been involved as key partners and in identifying research priorities. LOFT takes a significant organizational learning approach and peers have been involved in shaping the program over the years. In many ways, peers take ownership of the program and may introduce bias in terms of how they perceive the services. Yet, the major focus of this research is not on performance measurement, but rather takes an exploratory approach to identifying relationships among program variables. Therefore, we argue that the intimate knowledge and understanding of context and experience provided through the participation of peers and other program staff considerably outweighs the drawbacks associated with potential bias.

One of the methodological strengths of this study is the comprehensive participatory approach that was applied. The participatory design included multiple levels of participation, involving all staff in major decisions and one peer researcher who collaborated on a regular basis in the co-design and implementation of methods and played a significant role in analysis and interpretation. It is hoped that these features will strengthen the validity and relevance of the findings and amplify the utility.

In addition, we felt that the realist methods enhanced the design through the inclusion of theory with an emphasis on identifying explanatory relationships. These considerations may support the broadening of the application of the findings to related contexts in order to enhance services. Finally, we hope that the integration of the two methods allow the combination of the strengths of significant depth of practical insight and lived experience along with broader considerations of scientific practice to enhance the contribution of the study.

Conclusion

This paper provides an in-depth description of the design and implementation of a hybrid realist-participatory evaluation that was conducted to examine youth peer support services. Challenges, successes and lessons learned are shared with respect to combining the two methods. This paper will be useful for practitioners, advocates, and researchers who are working in organizations or contexts where youth engagement is a key element of practice and who are interested in understanding how the program or initiative functions. It is also useful for researchers who are interested in learning new strategies to combine theory with practice in their work.

Supplemental Material

Supplemental Material, sj-pdf-1-ijq-10.1177_1609406921995680 - Exploring Peer Support Services for Youth Experiencing Multiple Health and Social Challenges in Canada: A Hybrid Realist-Participatory Evaluation Model

Supplemental Material, sj-pdf-1-ijq-10.1177_1609406921995680 for Exploring Peer Support Services for Youth Experiencing Multiple Health and Social Challenges in Canada: A Hybrid Realist-Participatory Evaluation Model by Tanya Halsall, Mardi Daley, Lisa Hawke and Joanna Henderson in International Journal of Qualitative Methods

Footnotes

Acknowledgment

We would like to acknowledge the efforts of LOFT Community Services and in particular, the TAY staff.

Author Contributions

TH was responsible for the conception and design of the work, drafted and revised the manuscript, and approved the final submitted version. MD made substantial contributions to the design of the work, substantively revised the manuscript and approved the final submitted version. LH and JH substantively revised the manuscript and approved the final submitted version. All authors have read and approved the manuscript.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical Approval Statement

Ethics approval was received from the Royal Ottawa Health Care Group Research Ethics Board for the methods related to the initial research questions and informed consent was obtained before all surveys, interviews and focus groups (REB# 2019007). In addition, a data sharing agreement was created between LOFT Community Services and the Royal’s Institute of Mental Health Research in order to standardize the management of data and ensure confidentiality is maintained.

Full Copies of Interview Schedules/Focus Group Schedules

Full copies of the interview guides are provided.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project is funded by the Social Sciences and Humanities Research Council of Canada (SSHRC; award # 892-2018-3059). The funder is providing the salary for the second author as well as funding support for participant payments, related travel, meeting costs, supplies and publication fees. The funder did not play any role in the design of the study and collection, analysis, and interpretation of data and in writing the manuscript.

ORCID iD

Tanya Halsall

Notes

Supplemental Material

Supplemental material for this article is available online.

References

Ali

Farrer

Gulliver

Griffiths

K. M.

(2015). Online peer-to-peer support for young people with mental health problems: A systematic review. JMIR Mental Health, 2(2), e19.

Alvarez-Jimenez

Gleeson

J. F.

Rice

Gonzalez-Blanch

Bendall

(2016). Online peer-to-peer support in youth mental health: Seizing the opportunity. Epidemiology and Psychiatric Sciences, 25(2), 123.

Arora

. (2019). Toronto: A data story on ethnocultural diversity and inclusion in Canada. Catalogue no. 11-631-X. https://www150.statcan.gc.ca/n1/pub/11-631-x/11-631-x2019002-eng.htm

Bandura

(2005). The evolution of social cognitive theory. In Smith

K. G.

Hitt

M. A.

(Eds.), Great minds in management, (pp. 9–35). Oxford University Press.

Bandura

(2011). The social and policy impact of social cognitive theory. In Mark

M. M.

Donaldson

S. I.

Campbell

(Eds.), Social psychology and evaluation (pp. 31–71). The Guilford Press.

Barton

Henderson

(2016). Peer support and youth recovery: A brief review of the theoretical underpinnings and evidence. Canadian Journal of Family and Youth/Le Journal Canadien de Famille et de La Jeunesse, 8(1), 1–17.

Beeson

E. T.

Whitney

J. M.

Peterson

H. M.

(2017). The development of a collegiate recovery program: Applying social cognitive theory within a social ecological framework. American Journal of Health Education, 48(4), 226–239.

Biglan

Flay

B. R.

Embry

D. D.

Sandler

I. N.

(2012). The critical role of nurturing environments for promoting human well-being. American Psychologist, 67(4), 257.

Blixen

Perzynski

Kanuch

Dawson

Kaiser

Lawless

M. E.

Seeholzer

Sajatovic

(2015). Training peer educators to promote self-management skills in people with serious mental illness (SMI) and diabetes (DM) in a primary health care setting. Primary Health Care Research & Development, 16(2), 127–137.

10.

Braun

Clarke

(2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101.

11.

Cabassa

L. J.

Camacho

Vélez-Grau

C. M.

Stefancic

(2017). Peer-based health interventions for people with serious mental illness: A systematic literature review. Journal of Psychiatric Research, 84, 80–89.

12.

Checkoway

Richards-Schuster

(2003). Youth participation in community evaluation research. American Journal of Evaluation, 24(1), 21–33.

13.

Chinman

George

Dougherty

R. H.

Daniels

A. S.

Ghose

S. S.

Swift

Delphin-Rittmon

M. E.

(2014). Peer support services for individuals with serious mental illnesses: Assessing the evidence. Psychiatric Services, 65(4), 429–441.

14.

Conley-Tyler

(2005). A fundamental choice: Internal or external evaluation? Evaluation Journal of Australasia, 4(1–2), 3–11.

15.

Corrigan

P. W.

Watson

A. C.

(2002). The paradox of self-stigma and mental illness. Clinical Psychology: Science and Practice, 9(1), 35–53.

16.

Couture

Penn

(2003). Interpersonal contact and the stigma of mental illness: A review of the literature. Journal of Mental Health, 12(3), 291–305.

17.

Daley

Egag

(2019). Creating, managing and supporting spaces for young adult experts and peers. LOFT Community Services.

18.

Evans

(2004). Ethics, anonymity, and authorship in community centred research or anonymity and the island cache. Pimatisiwin, 2, 1.

19.

Gaetz

. (2017). This is housing first for youth: A program model guide. Canadian Observatory on Homelessness Press. https://www.homelesshub.ca/sites/default/files/attachments/COH-AWH-HF4Y.pdf

20.

Gopalan

Lee

S. J.

Harris

Acri

M. C.

Munson

M. R.

(2017). Utilization of peers in services for youth with emotional and behavioral challenges: A scoping review. Journal of Adolescence, 55, 88–115.

21.

Gulliver

Griffiths

K. M.

Christensen

(2010). Perceived barriers and facilitators to mental health help-seeking in young people: A systematic review. BMC Psychiatry, 10(1), 113.

22.

Halsall

Manion

Henderson

(2018). Examining integrated youth services using the bioecological model: Alignments and opportunities. International Journal of Integrated Care, 18(4), 1–12.

23.

Halsall

Manion

Iyer

S. N.

Mathias

Purcell

Henderson

(2019). Trends in mental health system transformation: Integrating youth services within the Canadian context. Healthcare Management Forum, 32(2), 51–55.

24.

Hart

R. A.

(1992). Children’s participation: From tokenism to citizenship. UNICEF International Child Development Centre.

25.

Hetrick

S. E.

Bailey

A. P.

Smith

K. E.

Malla

Mathias

Singh

S. P.

O’Reilly

Verma

S. K.

Benoit

Fleming

T. M.

(2017). Integrated (one-stop shop) youth health care: Best available evidence and future directions. The Medical Journal of Australia, 207(10), 5–18.

26.

HOP-C Working Group. (2020). Stabilizing pathways out of youth homelessness: A practice guide. Canadian Observatory on Homelessness Press. https://www.homelesshub.ca/stabilizingyouthpathways

27.

Hosek

S. G.

Lemos

Harper

G. W.

Telander

(2011). Evaluating the acceptability and feasibility of project accept: An intervention for youth newly diagnosed with HIV. AIDS Education and Prevention, 23(2), 128–144.

28.

Kessler

R. C.

Berglund

Demler

Jin

Merikangas

K. R.

Walters

E. E.

(2005). Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National comorbidity survey replication. Archives of General Psychiatry, 62(6), 593–602.

29.

Kirby

M. J.

Keon

W. J.

Dinsdale

H. B.

(2006). Out of the shadows at last: Transforming mental health, mental illness and addiction services in Canada (Final report of the standing senate committee on social affairs, science and technology). https://repository.library.georgetown.edu/handle/10822/802965

30.

Kirsch

D. J.

Pinder-Amaker

S. L.

Morse

Ellison

M. L.

Doerfler

L. A.

Riba

M. B.

(2014). Population-based initiatives in college mental health: Students helping students to overcome obstacles. Current Psychiatry Reports, 16(12), 525.

31.

LaFrance

Nichols

(2010). Reframing evaluation: Defining an indigenous evaluation framework. The Canadian Journal of Program Evaluation, 23(2), 13–31.

32.

Lloyd-Evans

Mayo-Wilson

Harrison

Istead

Brown

Pilling

Johnson

Kendall

(2014). A systematic review and meta-analysis of randomised controlled trials of peer support for people with severe mental illness. BMC Psychiatry, 14(1), 39.

33.

Malla

Shah

Iyer

Boksa

Joober

Andersson

Lal

Fuhrer

(2018). Youth mental health should be a top priority for health care in Canada. The Canadian Journal of Psychiatry, 63(4), 216–222.

34.

Manzano

(2016). The craft of interviewing in realist evaluation. Evaluation, 22(3), 342–360.

35.

Marchal

Kegels

Van Belle

(2018). Theory and realist methods. In Emmel

Greenhalgh

Manzano

Monaghan

Dalkin

(Eds.), Doing realist research (pp. 41–58). Sage. https://doi.org/10.4135/9781526451729

36.

Merikangas

K. R.

Burstein

Swendsen

Avenevoli

Case

Georgiades

Heaton

Swanson

Olfson

(2011). Service utilization for lifetime mental disorders in US adolescents: Results of the national comorbidity survey–adolescent supplement (NCS-A). Journal of the American Academy of Child & Adolescent Psychiatry, 50(1), 32–45.

37.

Meyer

Ward

. (2014). ‘How to’ use social theory within and throughout qualitative research in healthcare contexts. Sociology Compass, 8(5), 525–539.

38.

Morse

J. M.

Maddox

L. J.

(2014). Analytic integration in qualitatively driven (QUAL) mixed and multiple methods designs. In Flick

(Ed.), The SAGE handbook of qualitative data analysis (pp. 524–539). Sage.

39.

O’Connell

M. J.

Sledge

W. H.

Staeheli

Sells

Costa

Wieland

Davidson

(2018). Outcomes of a peer mentor intervention for persons with recurrent psychiatric hospitalization. Psychiatric Services, 69(7), 760–767.

40.

Ontario Centre of Excellence for Child and Youth Mental Health. (2018). Evidence in-sight: Youth peer support training literature review and environmental scan. Ontario Centre of Excellence for Child and Youth Mental Health.

41.

Orpana

Vachon

Dykxhoorn

Jayaraman

(2017). Mesurer la santé mentale positive au Canada: Validation des concepts du Continuum de santé mentale—Questionnaire abrégé [Measuring positive mental health in Canada: Construct validation of the mental health continuum-short form]. Health Promotion and Chronic Disease Prevention in Canada: Research, Policy and Practice, 37(4), 123–130. https://doi.org/10.24095/hpcdp.37.4.03

42.

Patton

G. C.

Sawyer

S. M.

Santelli

J. S.

Ross

D. A.

Afifi

Allen

N. B.

Arora

Azzopardi

Baldwin

Bonell

Kakuma

Kennedy

Mahon

McGovern

Mokdad

A. H.

Patel

Petroni

Reavley

Taiwo

Waldfogel

… Viner

R. M.

(2016). Our future: A lancet commission on adolescent health and wellbeing. The Lancet, 387(10036), 2423–2478.

43.

Pawson

Tilley

(2004). Realist evaluation. British Cabinet Office.

44.

Pawson

Ray

. (2013). The science of evaluation: A realist manifesto. Sage.

45.

Rycroft-Malone

Burton

C. R.

Wilkinson

Harvey

McCormack

Baker

Dopson

Graham

I. D.

Staniszewska

Thompson

(2016). Collective action for implementation: A realist evaluation of organisational collaboration in healthcare. Implementation Science, 11(1), 17.

46.

Salt

Parker

Ramage

Scott

(2017). Community-based mental health services hubs for youth environmental scan. PolicyWise for Children & Families.

47.

Salter

K. L.

Kothari

(2014). Using realist evaluation to open the black box of knowledge translation: A state-of-the-art review. Implementation Science, 9(1), 115.

48.

Sartorius

(2007). Stigma and mental health. The Lancet, 370(9590), 810–811.

49.

Settipani

Hawke

L. D.

Cleverley

Chaim

Cheung

Mehra

Rice

Szatmari

Henderson

(2019). Key attributes of integrated community-based youth service hubs for mental health: A scoping review. International Journal of Mental Health Systems. 13(52), 1–26. https://doi.org/10.1186/s13033-019-0306-7

50.

Shirk

S. R.

Karver

M. S.

Brown

(2011). The alliance in child and adolescent psychotherapy. Psychotherapy, 48(1), 17.

51.

Shirk

S. R.

Saiz

C. C.

(1992). Clinical, empirical, and developmental perspectives on the therapeutic relationship in child psychotherapy. Development and Psychopathology, 4(4), 713–728.

52.

Simoni

J. M.

Franks

J. C.

Lehavot

Yard

S. S.

(2011). Peer interventions to promote health: Conceptual considerations. American Journal of Orthopsychiatry, 81(3), 351.

53.

Smith

L. T.

(2012). Decolonizing methodologies: Research and indigenous peoples (2nd ed.). Zed Books.

54.

Tajfel

(1974). Social Identity and intergroup behaviour. Social Science. Information/Information sur les Sciences Sociales, 13(2), 65–93.

55.

Thornicroft

Mehta

Clement

Evans-Lacko

Doherty

Rose

Koschorke

Shidhaye

O’Reilly

Henderson

(2016). Evidence for effective interventions to reduce mental-health-related stigma and discrimination. The Lancet, 387(10023), 1123–1132.

56.

Vanderheul

Daley

Bourke

Stalker

(2018). LOFT community services transitional age youth department: Best practices for supporting youth peer support workers. LOFT Community Services.

57.

Waddell

McEwan

Shepherd

C. A.

Offord

D. R.

Hua

J. M.

(2005). A public health strategy to improve the mental health of Canadian children. The Canadian Journal of Psychiatry, 50(4), 226–233.

58.

Walker

J. S.

Powers

L. E.

(2008). Administration and scoring of the youth efficacy/empowerment scale—Mental health and the youth participation in planning scale. Research and Training Center on Family Support and Children s Mental Health, Portland State University.

59.

Walker

J. S.

Thorne

E. K.

Powers

L. E.

Gaonkar

(2010). Development of a scale to measure the empowerment of youth consumers of mental health services. Journal of Emotional and Behavioral Disorders, 18(1), 51–59.

60.

Westhorp

(2019). Understanding mechanisms in realist evaluation and research. In Emmel

Greenhalgh

Manzano

Monaghan

Dalkin

(Eds.), Doing realist research (pp. 41–58). Sage. https://doi.org/ 10.4135/9781526451729

61.

Wiles

Coffey

Robinson

Heath

(2012). Anonymisation and visual images: issues of respect, ‘voice’ and protection. International Journal of Social Research Methodology, 15(1), 41–53.

62.

Zeldin

Bestul

Powers

(2012). Youth-adult partnerships in evaluation (Y-AP/E): A resource guide for translating research into practice. ACT for Youth Center of Excellence, Cornell University.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.10 MB