Abstract
Although male-focused sexual and relationship violence (SRV) prevention programs are widely promulgated, limited guidance concerning how programs should be implemented for rigorous evaluation exists. To help develop evidence for such guidance, this paper reports on 20 interviews with leaders at youth-serving community-based organizations (CBOs) and educational institutions, which are sites for male-focused SRV prevention programs. This study examined: (1) how programs can be designed to engage male participants; and (2) how youth-serving CBOs and educational institutions can partner with researchers for evaluations. Findings underscore the importance of attending to the unique needs of program participants, their parents/guardians, and host organizations.
Keywords
Introduction
The high prevalence and detrimental consequences of sexual and relationship violence (SRV) underscore a critical need for effective violence prevention programs (Breiding et al., 2014; Centers for Disease Control and Prevention [CDC], 2019; DeGue et al., 2014). Relationship violence (RV), which includes psychological, sexual, or physical abuse; stalking; or the threat of such abuse within intimate partnerships (Breiding et al., 2015); and sexual violence (SV), nonconsensual sexual activity within or outside of intimate partnerships (Breiding et al., 2014), often first surface during adolescence (Ball et al., 2012; Foshee et al., 2014; Smith et al., 2018). Intervening early to prevent first-time SRV perpetration is the key to prevent both immediate and long-term negative physical, mental, and social outcomes associated with both victimization and perpetration (Ackard et al., 2007; Banyard & Cross, 2008; Roberts et al., 2003).
SV and RV are gendered issues (see Graham et al., 2020 for a more thorough discussion of this topic). Research on SRV suggests gendered disparities in SRV prevalence and impacts (Adhia et al., 2019; Breiding et al., 2014; Dank et al., 2014; García-Moreno et al., 2013; Mennicke & Kulkarni, 2016; Smith et al., 2018). Boys and men are more likely than girls and women to perpetrate SV and severe types of RV that cause physical injury and death. Girls and women are more likely to be victims of these violence types (e.g., Archer, 2000; García-Moreno et al., 2013). Moreover, limited available research suggests that transgender and non-binary individuals are among those at highest risk for experiencing SRV (Dank et al., 2014; Hoxmeier, 2016; Stotzer, 2009). Studies also indicate that, particularly among male-identified people, some gender-related beliefs, norms, and attitudes increase the likelihood of perpetrating SRV (e.g., endorsement of hostile masculinity or hostility toward women; Casey et al., 2017; Dardis et al., 2015; Tharp et al., 2013; Thompson et al., 2013, 2015). As such, it is understandable that the SRV field pays considerable attention to engaging boys and men in SRV prevention programs, a trend that has led to the development of male-focused prevention programs and strategies (Casey et al., 2018; Graham et al., 2021).
Limited guidance exists for how community-based and community-delivered, male-focused SRV prevention programs might be designed and implemented in diverse settings to meet the needs of their target audiences, while also allowing for rigorous evaluation of effectiveness (Embry et al., 2022; Graham et al., 2021). For example, mixed evidence exists concerning whether programs would be most effective when delivered in person (e.g., Coaching Boys Into Men [CBIM]; Miller, Jones, Culyba, et al., 2020; Miller, Jones, Ripper, et al., 2020) or online (e.g., RealConsent, Salazar et al., 2014). Additionally, it is unclear whether programs for boys and men would be best received in single-gender or mixed-gender groups (Black et al., 2012; Noonan & Charles, 2009) and who the most effective program facilitators might be (e.g., Jaime et al., 2016). Such guidance is important because prevention programs are typically delivered to boys and young men in the context of their participation in other organizations and programs, including at educational institutions (i.e., colleges, schools, and universities), athletic programs, and community-based organizations (CBOs). Consequently, violence prevention researchers must often partner with the institutions and organizations that are the sites of such programs to conduct studies of prevention effectiveness. Likewise, researchers seeking to conduct experimental and large-scale studies of SRV prevention programs should be prepared to conduct multi-site studies to help ensure programs are effective in various organizational contexts and with diverse participants. Notably, few multi-site randomized studies of SRV prevention programs have been completed, and only a small portion of these examined the effectiveness of programs delivered in settings other than schools or universities (DeGue et al., 2014; Graham et al., 2021; Reyes et al., 2020).
The research described in this paper helps address these gaps in research guidance by examining data from semi-structured, in-depth key informant interviews with 20 SRV prevention program facilitators and potential partners (i.e., those in leadership/staff positions at youth-serving CBOs, community colleges, private or public educational institutions) involved with the implementation and/or evaluation of male-focused SRV prevention programs. The research team completed data collection for this study as part of the Guys Relate study, a larger research project focused on readying Wise Guys: The Next Level (WGNL), a community-based, male-focused SRV prevention program, for a rigorous, multi-site evaluation.
Wise Guys: The Next Level Program Description
The Children's Home Society of North Carolina (CHSNC) developed WGNL. This program was an evolution of an evidence-informed intervention named Wise Guys that was developed in the 1990s by the same organization to address healthy relationships and masculinity as well as teen pregnancy prevention with young adolescent boys (see Gottsegen & Philliber, 2001; Gruchow & Brown, 2011; Herrman et al., 2016, 2017 for details on the earlier program and its evaluation). Given CHSNC's decades of experience in violence prevention, supported by a Cooperative Agreement from the CDC, the NC Rape Prevention Education (RPE) Program awarded funding to expand WGNL implementation. However, since WGNL had not been formally evaluated, CHSNC in partnership with a team of University of North Carolina at Chapel Hill (UNC-CH) researchers sought and were awarded additional CDC funding to evaluate this practitioner-developed program with the Guys Relate study. In its most recent, practitioner-developed form, WGNL was delivered in community settings (e.g., community colleges, boys and girls clubs) to groups of 5–20 male participants ages 14–25. Typically, participants engaged in 12, 60-minutes, interactive sessions facilitated by a trained prevention educator using a manualized curriculum. Topics included conflict resolution, healthy relationships, effective communication, SV, and RV, among others (see Embry et al., 2022 for additional details on WGNL).
Guys Relate Study Description
The Guys Relate study research team collaborated closely with CHSNC and other community partners and undertook a five-step process to convert WGNL into an intervention that could be evaluated with a multi-site randomized design in different community-based settings, including public and private schools and universities, community colleges, and youth-serving CBOs (Embry et al., 2022). As part of the overall study, the research team collected qualitative data to better understand how WGNL and other male-focused SRV prevention programs might be best designed and implemented with boys/young men in diverse educational and CBO settings and how such programs could be rigorously evaluated by practitioner-researcher teams in rigorous, multi-site studies. Such guidance is necessary for increasing the reach, effectiveness, and efficiency of these program offerings.
Current Study
In this study, we focus on data gathered in the Guys Relate study from 20 key informant interviews. The research team conducted these interviews to address the following two research questions: (1) how can practitioners and researchers design, recruit for, and implement SRV prevention programs in a way that successfully engages boys/young men?; and (2) how can youth-serving CBOs and educational institutions successfully partner with researchers to implement and rigorously evaluate SRV prevention programs for boys/young men?
Methods
As part of the overall study, including efforts to engage with and recruit potential study sites, the research team, made up of two doctorate-level researchers, one master's-level researcher, and a bachelor's-level research assistant, used a semi-structured interview guide to invite insights from key informants. Interviews were conducted to develop an understanding of how WGNL and other male-focused SRV prevention programs might be optimally implemented in diverse educational and CBO settings. The team sought to collect key informants’ perceptions of the program curriculum's content, format, acceptability, usefulness, and barriers and facilitators to implementation. The team also investigated how practitioner-researcher collaborations might be established and enhanced to facilitate research on SRV prevention programs. The UNC-CH Institutional Review Board evaluated all study protocols and deemed them exempt from further review (IRB #17-3101).
Recruitment and Sample
The research team invited 28 potential key informants to participate in interviews. To be invited, individuals had to be serving/have served in leadership or central staff positions at youth-serving CBOs, community colleges, private or public educational institutions, and/or Title IX programs, and/or a position that was responsible for preventing or responding to SRV at their organization. In addition, potential key informants were invited to participate if their institution or organization had been identified as a potential site where a rigorous SRV prevention research study could be conducted. The 28 organizations on the list were identified by nominations made by: (1) the study's primary community-partner, CHSNC; (2) the Guys Relate study Research Advisory Board that was made up of local community, state-level, and national experts in SV prevention; and (3) research team members. Key informants interviewed represent nine counties across NC; three are rural and six urban (Health Resources and Services Administration, 2018).
Two members of the research team contacted potential key informants by telephone to invite them to participate in a telephone interview and within 24 hours of that call, sent each informant a follow-up email with information about the study. The study invitation notified potential key informants that they could be entered into a drawing for a $150 gift card following participation. These team members repeated these outreach efforts each week for about three weeks. At the end of recruitment, 20 informants agreed to participate, three informants declined to participate, and five had not responded to the team's invitations.
Data Collection
The same two team members conducted individual phone interviews with 20 key informants. Team members used a semi-structured interview guide developed by the research team that had been piloted prior to the interviews with people (n = 6) who were working or had worked in violence prevention practice and/or research. The interviews lasted 21 to 63 minutes each (M = 40; Mdn = 40). The interviewers used a standard guide across interviews, though the research team made minor adaptations to certain questions to ensure their relevance for participants’ organizational setting and population served (e.g., whether the setting was an institution of higher education or youth-serving CBO). Before data collection started, the interviewers informed all participants that their participation was voluntary and that they could skip any questions or end their participation at any time.
All interviews were audio-recorded and subsequently transcribed verbatim by a professional transcription service. At the end of the interview, the interviewer offered key informants the opportunity to be entered into a drawing for a $150 gift card.
Analysis
Two research team members led data analysis. Prior to starting data analysis, the full research team created a codebook of a priori codes (i.e., codes specified prior to starting data analysis; Elliott, 2018) developed from the interview guides. Next, the two analytic leads read all of the interview transcripts to ensure familiarity with the data. Then, these team members selected two transcripts to code to ensure that they were coding deductive themes consistently. When coding differences emerged, the team members met with a third member of the team to discuss the differences until consensus was reached. The two analytic leads met weekly to identify, refine, and apply both a priori and emergent codes (i.e., codes drawn from the data themselves; Elliott, 2018) across analyses. The full research team reviewed and discussed these initial analyses monthly to identify and incorporate emergent codes into the final codebook. The coders re-visited transcripts throughout the coding process to ensure that they applied all codes to all transcripts. Last, one team member reviewed all fully coded transcripts to identify overarching themes that arose across multiple transcripts. As described by Hennink et al. (2017), the research team found that both code (“heard it all”) and meaning (“understand it all”) saturation were reached during the coding process, with no additional codes and themes arising from the interviews by the end of coding (p. 605). The team completed coding and analysis with Atlas.ti 8 software.
Results
Themes derived from these data elucidated key informants’ recommendations regarding design and implementation of SRV prevention programs for boys/young men, as well as methods for rigorous SRV prevention program evaluation conducted by teams of researchers and community-based practitioners. Specific findings and themes relating to each of the research questions are described next.
Research Question 1 Results: Program Design and Implementation
Many informants offered their perspectives on recruitment and engagement of boys/young men in SRV prevention programs, as well as ways to implement such programs to help ensure their active participation. Prominent themes that arose in these areas included information on how to educate boys/young men (e.g., format), program facilitator and session dynamics, implementation facilitators and barriers, and implementation logistics.
How to Educate Boys and Young Men?
Interviewers asked informants questions regarding how to format (i.e., structure program sessions, including how they are delivered) a male-focused SRV prevention program. Informants spoke to several issues, including group formats, online versus (vs.) in-person learning, single-gender vs. mixed-gender program offerings, and tailored vs. one-size-fits-all curricula.
Group Format
When asked their thoughts on an ideal SRV prevention program format, many informants noted that in their experience, facilitating program sessions in an in-person group setting in which the same participants start and end the program together (i.e., a cohort) was successful. Specifically, many shared that they felt a group format was preferable because it facilitated sharing as well as peer accountability, noting that the group cohort format of WGNL was a strength of the program. As stated by one informant, a group format “…works in…favor of teaching…because once one [person] becomes receptive,… typically it’s a domino effect and everyone, you know, buys into the program” (CBO employee). Many emphasized the importance of fostering a sense of cohort and group identity. Reasons cited for fostering group identity varied but included that boys/men often do not have spaces to talk about sex/masculinity; participants might be more likely to share in a comfortable group setting; and that cohorts can support each other through an intensive process of addressing harmful norms about masculinity. In the words of one informant: …I think that the group dynamic of it works…and that’s because you get various perspectives, and you know, a lot of times, teenaged boys, they sort of feed off of one another…once you get one to share, it becomes easier for the others to share, and you, you know, sort of get into their headspace and figure out where they're at. And, that also works in the favor of teaching them because once one becomes receptive, it typically sort of—it's a domino effect and everyone, you know, buys into the program (CBO employee).
Some informants also highlighted potential challenges of using a group format, expressing that it can be more difficult to facilitate participants’ engagement in sensitive conversations, particularly when the participants do not know each other. For example, one informant shared that there can be “…very uncomfortable conversation[s] because some people have had experiences that they have never talked about…, and it's gonna regurgitate some of those memories, so…one of the problems [of groups] would be to get people to talk” (CBO employee). Some informants shared other challenges that can emerge in group settings like defiant participants, problems getting young participants to maturely discuss masculinity and SRV-related topics, developmental differences among participants, and resistance to being in a shared learning environment that feels like school (i.e., “classroom-like”) during times they would not be in school or a classroom.
Online vs. In-Person Learning
Views were somewhat mixed concerning which program format(s) might be best received by boys/young men. Some informants expressed caution with online learning directed towards youth participants, noting that participants do not take online learning as seriously as in-person learning; aspects of the program can be lost without in-person program components (e.g., “chemistry” with the program facilitator); and potentially harmful SRV online behavior is prevalent. For example, one informant stated: It would be possible to create an [application] app or do something online, but I’m not sure that it would have the same effect as the—you know, the person-to-person interaction… I think a lot of things get lost in translation when there’s not the person-to-person interaction as far as, you know…semantics…tone of things (CBO employee).
On the other hand, some informants shared support for an online SRV program or app, highlighting that participants can interact via online communities, also noting that many youth already engage with each other regularly and are comfortable accessing information online. Further, many expressed interest in exploring the utility of an app, not to supplant, but rather to complement in-person curricula like WGNL, and some informants highlighted that there could be an opportunity to leverage social media as a component of such programs. One informant offered the following thoughts on combining online and in-person options: “I just think it's [the SRV program is] shorter and then just to have some online options…you know, having access to kind of a one-on-one coach after it's over would be great, too” (community college employee).
Single-Gender vs. Mixed-Gender Forums
When asked about implementing SRV prevention programs in single-gender and mixed-gender groups, many informants felt that mixed-gender programs would not be successful because part of program success depends on a forum (e.g., gathering, meeting, platform, “space”) that is “just male.” In a “just male” forum, many informants specifically highlighted the importance of creating trust and rapport between the facilitator and participants to foster conditions in which difficult SRV discussions could be held. One informant stated “…so they [boys/young men] will openly engage…but they…definitely are not gonna do it in a mixed-gender group” (community college employee), while another stated that they “… think by being an all-male program, that creates an atmosphere that’s not as threatening for some of the material that they do discuss” (state health department employee).
One-Size-Fits-All vs. Tailored Curricula
Multiple informants shared their perspectives on whether male-focused SRV prevention programs should be designed and implemented in the same way (i.e., “one-size-fits all”) for all boys/young men or tailored (i.e., adapted) for particular sub-groups of boys/young men. Many informants emphasized the importance of resisting a “one-size-fits-all” model, suggesting SRV prevention programs should be flexible enough to be tailored to specific sub-groups of boys/young men. For example, informants described having success tailoring SRV prevention programs to meet the needs of fraternities, athletes, young men of color, and gay men. Here, tailoring refers to adapting a program in terms of content, format, or other aspects of program design and implementation to meet the specific and potentially unique needs of particular sub-groups of boys/young men.
Program Facilitator and Session Dynamics
Informants were also asked questions that elicited information on program facilitator and session dynamics. Their responses provided insights regarding the importance of facilitator rapport and relatability as well as interactive learning.
Facilitator Rapport and Relatability
Some informants flagged the importance of a program facilitator connecting early with the male participants, one stating “the presenter, the instructor has to really knock a homerun the first session to get them coming back” (community college employee). Some informants also noted that the way participants perceive facilitators matters. Specifically, participants want to be able to see themselves in their facilitators, for example having facilitators who look like them in terms of race, sex, and age, and facilitators they view as “men.” One informant stated “…I think students are not as interested in [and] engaged with folks that they don’t feel like they can identify with” (university employee).
Similarly, many informants highlighted that program participants want a facilitator who is relatable, and that part of being a relatable facilitator means “…hav[ing] their own personal story to share… somebody like my age [i.e., older than the participants] who's going to come in and tell about how they made a mistake when they were 16 years old” (CBO employee). Some informants also elaborated on useful strategies that facilitators can use to build trust and rapport with participants. These strategies included engaging in person with potential SRV prevention program participants before they participate in the program, having a program launch party with food and music, and having an open discussion with potential participants about what the program covers before the sessions start.
Interactive Learning
Many informants also felt that facilitators should create a learning environment that makes participants feel like they are part of the session, rather than passive attendees, and emphasized that the program session format should be interactive, rather than predominantly didactic. One informant highlighted the importance of “makin’ sure that…he [the facilitator] is including the class, the classroom participants as opposed to just havin’ a dissertation speech. He’s askin’ questions and they [the participants]…feel like they're part of the class as opposed to…watchin’ a movie” (CBO employee). Another informant shared “…what I see when I speak to [college] students generally about things, the best ways to do things are through scenarios, through, um, experiences they may already have, unpacking situations, as opposed to just delivering information” (university employee).
Implementation Facilitators and Barriers
Informants were also asked to share their thoughts on potential facilitators (i.e., that promote participation) and barriers (i.e., that discourage participation) to boys’/young men's participation in SRV prevention programs. These are summarized next.
Facilitators of Program Participation
Overall, informants individually reported several components that they think encourage boys/young men to participate in programs like WGNL, which included offering the program in a conference format (e.g., a half-day event with speakers and activities instead of multiple weekly sessions), requiring a less intense commitment for participation, and ensuring there is a strong social component. Some informants also recommended using recruitment strategies that involve men, particularly those with influence in their community, reaching out to other men or friends reaching out to friends. Many informants also flagged the importance of offering incentives, such as gift cards, food, headphones, or t-shirts, to encourage program attendance.
Some informants also spoke to the importance of obtaining parent/guardian buy-in for program participation of those under age 18, noting that parent/guardian buy-in increases participation in and support of WGNL, which might be true for other SRV prevention programs, too. Some also discussed their strategies for parent/guardian engagement, which included frequently contacting parents/guardians by telephone to discuss the program, as well as securing verbal parent/guardian permission for adolescents’ participation. Additionally, some informants emphasized that those offering prevention programs must be mindful of the beliefs held by parents/guardians and the broader community in which the program is being implemented. Such knowledge could help groups offering programs anticipate parents’/guardians’ and community members’ concerns and tailor program content to the specific community, which may, in turn, increase program support and participation. Although, such tailoring presents problems for program evaluation across program sites because the program will no longer be directly comparable from site to site. As stated by one informant: I think…the challenge is…schools are understandably concerned if the content is too conservative they will offend some students and parents. And if the content is…too progressive…it will offend and scare some students and parents. I think in each of these…[situations], the curriculum would have to be tailored quite significantly for each [audience]…to meet the expectations of each group,…which makes evaluation really, really hard (university employee).
For example, one informant (state health department employee) noted that inclusion of program content on gender identity was not acceptable to some parents/guardians in their community nor was the use of the term “sex,” which in turn, had implications for potential program participation.
When discussing the process of incorporating outside organizations’ educational services, for example a CBO such as CHSNC coming into a school to offer a prevention program, many informants discussed the relevance of creating memorandums of understanding (MOUs) to guide the work. One informant noted: …if we're partnering with another organization, there's definitely a need to have a memorandum of understanding between that organization and the college;…however, with those MOU's, they're not, you know, a 15-page document. It's maybe a one-page document that explains what the college will do for the organization, what the organization will provide for the college (community college employee).
Relatedly, many informants reported that their organization did not have a standardized or formal process for incorporating educational programs from outside organizations. To facilitate incorporation and collaboration between organizations, several informants emphasized the importance of collaboration and partnerships, as well as ensuring the programs being offered mutually benefit each organization involved. Moreover, multiple informants noted that for such partnerships to be considered, outside organizations need to be endorsed or vouched for by individuals or organizations that the host organization trusts.
Barriers to SRV Program Participation
Informants also shared their insights on why boys/young men might not participate in SRV prevention programs, highlighting potential reluctance about the program material, concerns about how programs are framed (or described) to potential participants, parent/guardian and administrator hesitancy about participation, and scheduling challenges. Some informants noted that potential participants may be resistant to being taught about relationships, sex, and/or masculinity, and that these topics alone are not sufficient to generate motivation to participate. One informant shared: “…I think it’s hard to recruit students in college to want to be a part of something that’s like healthy relationships….They don’t think about it that way, and they don’t want…to be taught anything. It feels like health class to them” (university employee). Relatedly, multiple informants noted that SRV prevention programs need to be better explained and marketed to target audiences, taking into account what actually motivates boys/young men to participate in these programs. Some informants shared ideas about connecting SRV content to other issues affecting boys/young men to increase interest in and motivation to participate in SRV prevention, such as men's mental health, job skills, and leadership skills.
Relating back to parent/guardian program buy-in, some informants shared that they have encountered administrators and/or parents/guardians who were concerned about over-burdening busy young people by adding SRV prevention program participation to their schedules, underscoring the need for buy-in from more than the program's potential participants. Some informants also noted that challenges often arise when trying to choose an accessible time to offer programming. Suggestions for avoiding WGNL-related scheduling challenges and address hesitancy around over-burdening young people that might apply to other programs included offering the program on a weekend or incorporating the program into an existing class, rather than asking young people to use their free time or skip other activities.
Regarding barriers to organizational partnerships to offer SRV prevention programs, many informants emphasized that there are frequently organizational limitations to supporting such partnerships and implementing program offerings like WGNL. Informants noted that these limitations often included limited budgets and staff capacity. Informants consistently emphasized that the already stretched capacity of CBO staff and staffing shortages greatly constrains organizations’ ability to support and implement these programs. Several informants also noted that they have experienced administrative blocks and “red tape” that make it difficult to introduce an outside program into a new CBO or educational institution. Additionally, some informants stated that when they have sought to bring programming into new organizations, they at times encountered hesitancy and skepticism from that organization's leadership about having “outsiders” teach the youth they serve. Several informants went on to share that they think this skepticism reflects the need for internal champions to generate organizational buy-in for SRV prevention programs like WGNL.
Research Question 2 Results: Organization-Researcher Prevention Research Collaboration
Interviewers also asked informants about ways to support rigorous evaluations of SRV prevention programs conducted by collaborative teams of practitioners and researchers. Informants spoke primarily to issues concerning incentives for implementing organizations for research participation, the IRB process, and organization-researcher communication.
Incentives for Organizations’ Participation
Several key informants emphasized the importance of providing incentives to organizations to support program implementation and their participation in research to help ensure they have the capacity to collaborate. One informant shared that Those agencies [CBOs/educational institutions] have programming needs often because they have slim budgets. They’re underfunded and need support. And so, I think it’s really hard to implement and evaluate a program in…that context of scarcity…that these organizations are experiencing (university employee).
Additionally, some informants highlighted that giving organizations more money without also helping fund dedicated staff time will not increase their ability to host a program like WGNL. In one informant's words …even at our little school, things get busy and our principal, you know,… “No problem” …but…it [the work]…wasn’t assigned to anybody. So it just kind of fell by the wayside…So…I think that’s, that's the biggest challenge is just…the lack of resources and time…for…school counselors or whoever your contact person is gonna be within to think about some of the stuff that…we're talking about (community college employee).
The IRB Process
Overall, many informants felt challenged by the IRB process for engaging in research. Relatedly, multiple informants shared that in their experience partnering with researchers, they lacked a sufficient timeline and clear expectations around engaging with the IRB process. One informant shared With IRB,…I think that’s something that for…me was a big lesson learned…,and I think a lot of it had to do with the topic [SRV] that we’re talking about as well, because we’re asking the types of questions that we’re asking students on those things (university employee).
These informants wanted more information before starting a research project and longer lead times for their participation in the IRB process. For instance, one informant noted I guess, for me…a timeline would have, um, been good to see what the expectations were of what…I needed to be doing…I think a conversation [upfront] with…whatever organization you’re trying to…get to help you facilitate that study, a conversation about human subjects, and IRB and what that looks like [would be good] (university employee).
Organization-Researcher Communication
Many informants flagged issues they had witnessed or experienced with organization-researcher communication when attempting to partner on SRV prevention studies. Several informants shared that CBOs need explanations of technical research terms and processes, but they also do not want to receive unnecessary information or feel talked down to by the research team. In the words of one informant I do think that, especially when you’re bringing partners to the table who are not versed in, or, not in the academy and are not versed in that language, you know…sometimes people can get really turned off…by…kind of seemingly nonchalant throw down of acronyms that everybody is just supposed to know. And so…, you know, I think…there has to be—we have to be able to figure out a way, how to like, engage people in our processes that are not their processes without sounding patronizing (university employee).
Additionally, some informants shared discomfort with randomized study designs as some participating sites could be in a control condition that receives no intervention, expressing also that upfront and honest communication about study design is important to the success of organization-researcher partnerships. Several informants noted that clear communication from researchers regarding the rationale for a control condition, as well as how the control condition would be assigned to their organization was critically important for ensuring successful engagement with their organization, as well as critical for ensuring the study's success. Likewise, several informants also asked for researchers to offer an established alternative programming for the control condition.
Discussion
By inviting the perspectives of 20 leaders from youth-serving CBOs or educational institutions, this research explored how practitioners and researchers can collaboratively design, recruit for, and implement SRV prevention programs in ways that successfully engage young male-identified participants. The research also examined how youth-serving CBOs and educational institutions can successfully partner with researchers to plan rigorous evaluations of male-focused SRV prevention programs.
Notably, little guidance is available concerning optimal implementation and rigorous evaluation of community-based and community-delivered, male-focused SRV prevention programs designed for boys/young men. Additionally, limited research exists on these programs’ effectiveness as well as what aspects of these programs (e.g., program components, delivery methods, formats) might ensure effectiveness in reducing SRV among their target audiences (Embry et al., 2022; Graham et al., 2021). It is possible that existing programs of this kind are not meeting the needs of boys/young men and/or the organizations or institutions in which these programs are commonly delivered (i.e., CBOs, schools, colleges, or universities). Accordingly, these informants’ unique, largely untapped perspectives are critical to guiding practitioner-researcher partnerships for both implementing and evaluating SRV prevention programs in real-world settings.
The study also brought to the fore many recommendations for ways to format and implement SRV prevention programs. Regarding program design and implementation, informants had somewhat mixed opinions about offering these programs in-person vs. online or through an app, with some support for both (or a combination of) these two implementation modes. Notably, current study informants may not be able to judge what's “best” in terms of format if they do not have experience delivering programs in multiple formats. Extant research has found both in-person (e.g., CBIM) and online programs (e.g., RealConsent) somewhat effective in reducing some types of SRV victimization and perpetration (see Graham et al., 2021). Nonetheless, additional research is warranted in this area, particularly concerning potential hybrid (i.e., a mix of in-person and online programming) models for program implementation and boys’/young men's perspectives on various delivery modes, particularly given the increasing demand for virtual programming in the wake of the COVID-19 pandemic.
Regarding the implementation of single-gender vs. mixed-gender programs, many informants thought it important to offer single-gender (male only) programming. Arguments in favor of each format exist in broader scholarship in this area (Noonan & Charles, 2009). However, the evidence regarding which format(s) and for whom each format is most effective is limited and mixed. For example, findings from one study with African American middle school students indicated that girls benefited the most from participation in a mixed-gender SRV prevention program, while boys benefited most from a single-gender format of the same program (Black et al., 2012). On the other hand, a recent meta-analysis of the effectiveness of SV prevention bystander programs on college campuses found no significant difference in the effects of single-gender and mixed-gender program delivery formats (Jouriles et al., 2018). These findings suggest that the effectiveness of single-gender and mixed-gender programming may differ depending on, for example, the participants’ age as well as race/ethnicity, as well as program foci. Accordingly, future research should explore potential consequences of offering programs in each format. For example, single-gender programs may ostracize potential participants who do not identify with the male-female gender binary and/or as heterosexual and could serve to reinforce harmful gender norms, such as hypermasculinity, that are known risk factors for SRV (Graham et al., 2021; Noonan & Charles, 2009; Tharp et al., 2013).
Many informants also called attention to the promise of tailoring SRV programs to particular sub-groups of boys/young men (e.g., fraternities, athletes), having observed success with this practice in their own work. This finding is in line with extant research on one of the most rigorously evaluated SRV programs for boys—CBIM—which has some demonstrated effectiveness in reducing use of SRV behaviors among male athletes (see Casey, Graham, & Greer, 2022 and Graham et al., 2021 for reviews of relevant studies; Miller, Jones, Ripper, et al., 2020). Moving away from or supplementing a universal, one-size-fits-all approach to SRV prevention with tailored programming may be promising, particularly for addressing SRV among sub-groups of boys/young men at heightened risk of engaging in SRV (see Malamuth et al., 2018 for a discussion of the potential for SV programs to increase violent behavior among men at high-risk for perpetration). On the other hand, tailored programs could be stigmatizing by calling attention to sub-group identities (rather than addressing the needs of all boys/young men in a single program), or such programs could produce non-significant program effects compared to other non-SRV-focused programming (see Miller, Jones, Culyba, et al., 2020 which compared a tailored male-focused SRV prevention program to job readiness training).
Interestingly, in their discussions of facilitators and barriers to SRV prevention program participation among boys/young men, many informants highlighted the importance of paying close attention to how the program is marketed to potential program participants, their parents/guardians, and organizations who might offer the program. Based on interviewees’ responses, getting buy-in from all of these parties likely includes tailored marketing of the program description, the session schedule, descriptions of how participants will benefit from the program, and more. For that reason, future efforts should involve boys/young men, their parents/guardians, and program host organizations in determining how to market programs to ensure high participation in SRV prevention programs.
Informants also shared their own experiences with facilitators and barriers to practitioner-researcher partnerships focused on rigorously evaluating male-focused SRV prevention programs. Informants primarily discussed the importance of providing sufficient compensation for organizations participating in research, such as by ensuring funding is allocated to cover organization staff effort to participate in the research project, as well as concerns about both the IRB process and organization-researcher communication. These same issues have been documented elsewhere in scholarship pertaining to practitioner-researcher collaborations in violence and public health intervention more broadly (e.g., Pinto et al., 2019; Wojcik et al., 2021), though less so in published literature on SRV prevention with boys/young men (Embry et al., 2022). However, it seems the whole field of SRV prevention may benefit from attending to compensation, both monetary and otherwise, for partner organizations; providing clarity on IRB processes in language that is meaningful and understandable to practitioners; and communicating clear, feasible project timelines to partner organizations, taking into account their input and expertise in planning processes.
Practice Implications
This study has many potential implications for SRV prevention practice (see Table 1 for a summary of these implications). Many informants’ shared that based on their SRV prevention experience, practitioners planning to implement such programs should consider: (1) providing male-focused SRV prevention programs to groups of participants that start and end the program together (i.e., cohorts); (2) having facilitators that program participants are likely to relate to (e.g., in terms of race/ethnicity, life experience, other aspects of identity); and (3) offering program sessions that are interactive as opposed to traditionally didactic to help ensure engagement of program participants. In addition, findings indicate that the needs and preferences of potential program participants and their parents/guardians (for those under age 18) should be fore-fronted in program development and delivery. For potential participants, program implementers should take into account the technology they use, spaces they occupy, program session schedules, and program marketing. Likewise, program implementers may also need to consider how to best reach parents/guardians as well as how to market and describe the program to parents/guardians in meaningful ways, including its potential benefits to program participants and others (e.g., their peers, current or future intimate partners, or broader society). Last, many informants spoke to the funding and staffing needs of program host organizations. A clear recommendation that emerged is that funding should be made available to potential host organizations (e.g., perhaps in the form of local, state, federal, or foundation grants) that can cover staff time and effort to implement male-focused SRV prevention programs, whether or not formal evaluation is being conducted.
Implementing and Evaluating Sexual and Relationship Violence (SRV) Prevention Programs for Boys and Men: Key Practice and Research Implications.
SRV, sexual and relationship violence; IRB, Institutional Review Board.
Research Implications
As for research implications, study findings highlight lessons regarding both the development of successful practitioner-researcher partnerships and areas for future research. Regarding successful partnerships, these results and those of others (e.g., Pinto et al., 2019; Wojcik et al., 2021) underscore that researchers should develop strong, reciprocal relationships and lines of communication with the organizations with whom they hope to collaborate well before designing a research project. Having this lead-time provides opportunities to both understand and meet the needs of partner organizations in terms of staffing, funding, and other aspects of supporting their work on the research project and vice versa. Additionally, this study pointed to the importance of avoiding research jargon, building in ample time to discuss the IRB process and timeline with partner organizations, as well as discussing and choosing potential study designs with partner organizations (rather than for them).
Regarding areas for future research, the implications gleaned from this study support expanding the SRV prevention field's use of participatory research models that meaningfully engage partner organizations as well as boys/young men and their parents/guardians in research projects from start to finish (for examples see Banyard et al., 2020; Irby et al., 2018; Oldfield et al., 2018). Such research strategies will enhance studies that seek to rigorously evaluate male-focused SRV prevention programs as well as studies that gather the much-needed input of diverse boys/young men concerning optimal program formats (e.g., in-person vs. online, single-gender vs. mixed-gender) and marketing/recruitment for such programs.
Strengths and Limitations
This study has both strengths and limitations. Although these findings are from a relatively small group of participants all located in NC, this research collected in-depth qualitative data capturing the perspectives of a unique population. As most male-focused SRV prevention programs are delivered in CBO and educational settings, understanding the experiences, perspectives, and recommendations of leaders in these organizations who are responsible for SRV prevention activities can help guide both implementation and evaluation of such programs. Without their buy-in and support, these programs will not be delivered or evaluated. Additionally, although the research team may not have asked all questions that could have shed light on the overarching research questions, many strategies were taken to ensure the rigor of the qualitative methods used (e.g., piloting interview guides, iterative coding, double-coding interviews, use of open-ended questions). Despite the limitations, this study serves as a novel, starting place for ongoing conversations and research to help elucidate how to best implement and rigorously evaluate community-based, male-focused SRV prevention programs.
Conclusion
Findings from these 20 in-depth key informant interviews constitute a unique and important source of perspectives on the implementation and evaluation of male-focused SRV prevention programs by practitioner-researcher teams. In efforts to implement and evaluate such programs, more attention should be paid to the unique needs of potential program participants, their parents/guardians, and program host organizations. Both program implementation and evaluation require substantial commitments of organization staff time and effort for program management and delivery, and funding should be made available to cover these costs. Increasing use of participatory research models may enhance the SRV prevention field's ability to conduct meaningful research that both rigorously evaluates male-focused SRV prevention programs as well as gathers the much-needed input of diverse boys/young men concerning optimal formats and marketing approaches for male-focused SRV prevention programs.
Footnotes
Acknowledgments
For their contributions to the project that informed this work, the authors thank and acknowledge the staff members at the North Carolina Children's Home Society, especially Rodney Absher and Rick Brown, the University of North Carolina at Chapel Hill colleagues, Venita Embry, Steve Marshall, Alexis Moore, Justin Quimbo, Jada Walker, and Belinda Rose-Young, as well as current and former staff members at the Division of Violence Prevention at the National Center for Injury Prevention and Control at the Centers for Disease Control and Prevention, Drs. Rosalyn Lee, Megan Kearns, Sue Neurath, and Beverly Fortson.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Center for Injury Prevention and Control of the Centers for Disease Control and Prevention (Grant No. 5U01CE002842-02). This project was also supported by the University of North Carolina Injury Prevention Research Center, which is partly supported by a grant (R49/CE000196) from the National Center for Injury Prevention and Control, Centers for Disease Control and Prevention. Points of view in this document are those of the authors and do not necessarily reflect the official position or policies of the Centers for Disease Control and Prevention.
