Abstract
Based on interviews and case studies in nonprofit organizations, this article examines the perspectives of evaluation advocates undertaking evaluation in culturally diverse contexts where interpersonal barriers may exist. Participants provided insights into how to overcome issues by developing meaningful interpersonal relationships, finding shared goals, and providing encouragement. This article presents practical strategies that anyone promoting evaluation may find helpful to develop positive interpersonal relationships and equitable and inclusive working environments.
Keywords
Introduction
Evaluation advocates are not evaluators, or people with evaluation skills and knowledge. Rather, they are individuals who motivate others and provide energy, interest, and enthusiasm by connecting evaluation with colleagues’ personal aspirations and the organizational goals to make judgments about effectiveness (Rogers et al., 2022; Rogers & Gullickson, 2018; Silliman et al., 2016). The purpose of this article is to present findings that answer the research question: How do evaluation advocates in nonprofit organizations effectively engage colleagues in evaluation? This introduction first outlines the research context and the theoretical frame. The remainder of the article details the methods and then presents the findings in three key areas. It discusses the results in relation to the extant literature, the theoretical elements of cooperative teamwork, and evaluator competencies. The article concludes by outlining the study's strengths, limitations, and implications for the field of evaluation.
Nonprofit human social service organizations strive to address inequalities and human service issues by fulfilling unmet community and social needs. A challenging dimension of the nonprofit workforce is diversity among team members, which includes both paid and volunteer staff. Nonprofits rely upon support and engagement with the client group for human resources related to service delivery and governance support; volunteerism is a distinguishing feature of the sector (Carman et al., 2008; Powell et al., 2017). There can be multiple intersections of diversity, including age, gender, cultural and linguistic background, and levels of education (Chouinard, 2014; Hyde, 2004). Particularly in culturally diverse contexts, communication issues can arise when two or more diverse groups who have been socialized in different ways and have had distinct personal experiences interact (Sheu & Sedlacek, 2004). Therefore, implementing evaluation can be challenging if it involves using technical languages such as values, criteria, merit, and other hard-to-define terms.
Another challenging issue related to workforce dynamics in the nonprofit sector is that there is potential resistance to change (Gill, 2010). Gill (2010) suggests that some people in nonprofit organizations, even when they are shown to have a limited impact, would rather keep on doing what they have always done rather than change. This can result in a refusal to question basic assumptions about how the organization is operating and reluctance to face hard choices about where to place limited resources (Gill, 2010).
Related to the resistance to change in nonprofit organizations is resistance to evaluation. Some employees may have had negative experiences in the past that make them reluctant to participate, or they are not able to allocate sufficient time to evaluation among other competing priorities (Chaudhary et al., 2020; Hyde, 2012). Some employees of nonprofit organizations have negative emotions associated with evaluation (Geva-May & Thorngate, 2003). They may feel threatened about the thought of exposing programs that have not worked. They may feel anxious about exposing their personal weaknesses. Employees may become defensive, fail to cooperate, actively try to prevent evaluation from being undertaken, or avoid participating in any evaluative activities (Perrin, 2014; Taut & Brauns, 2003). Extending this disconnect to the language used around evaluation, some employees may not understand the relevance, or they may find evaluation to be confusing if difficult terminology and abstract constructs are used (Mason & Hunt, 2018; Picciotto, 2017).
In summary, trying to solve social problems, operating in competitive funding environments, and producing information for multiple audiences, the nonprofit sector can be a challenging setting in which to make evaluation relevant, meaningful, and useful (Bach-Mortensen & Montgomery, 2018; Campbell & Lambright, 2017). Although nonprofit organizations can use evaluation to improve services, demonstrate effectiveness, and meet the information demands for accountability purposes (Gill, 2010; Moxham, 2014), some people may resist becoming involved with evaluation. The lack of dedicated resources, disconnected priorities, previous negative experiences, feelings of distrust and anxiety, and difficulties with evaluation concepts and terminology can all be barriers (Chaudhary et al., 2020; Donaldson et al., 2002; Mason & Hunt, 2018; Rogers et al., 2021).
Therefore, locating this research in the culturally diverse and resource-limited nonprofit sector was of value for elucidating organizational interpersonal dynamics. The empirical research described in this article focused on understanding how evaluation advocates worked with coworkers over the long term and explored interpersonal dynamics in depth. However, as well as elucidating practical examples from evaluation advocates, the research also drew upon theory to assist with understanding what was happening in this social situation.
After initially drawing upon Patton's (1997, p. 44) “personal factor,” interactive evaluation practice was found to be relevant and useful for this research as this approach to evaluation focused on interpersonal interactions (King & Stevahn, 2013). Interactive evaluation practice is based on social psychological theory of social interdependence theory. This research used the specific component of the social interdependence theory, the elements of cooperative teamwork, to make sense of the interactions occurring between colleagues (Johnson & Johnson, 2003).
Social interdependence theory can help determine the degree of cooperation, identify how employees in an organization promote mutually beneficial goals, and maintain productive relationships while implementing evaluative initiatives, facilitate interactions, and manage conflict (King & Stevahn, 2013). Social interdependence occurs when team members have mutual ambitions and when the consequences for an individual are dependent upon each other's actions (Johnson & Johnson, 2002). The way interdependence around goals is structured influences expectations and interactions, which ultimately determine outcomes; competitive goals lead people to oppose each other, but relationships founded on cooperative goals can support mutual success (Johnson, 2003).
For the application of this theory in practice, Johnson and Johnson (2003) undertook research on the concepts of interdependence, interaction, and outcomes to empirically validate the elements of effective cooperative teamwork. The theory posits that to increase the likelihood of successful working relationships, five elements of cooperative teamwork must be purposefully structured. The five elements are positive interdependence, individual accountability, social skills, group processing, and promotive interaction (Johnson & Johnson, 2015). The methods section describes how these five elements were considered during data analysis, and the discussion section presents the findings in relation to the elements of cooperative teamwork.
Method
This section articulates how, drawing upon the literature and the elements of cooperative teamwork, this study took a qualitative interpretivist perspective to collaboratively generate knowledge about evaluation advocates. Qualitative methods were chosen to increase understanding of this phenomenon by generating detailed information about the strategies the participants use. Qualitative methods enabled the researchers to emphasize and preserve the meanings and perspectives of participants and incorporate multiple constructed realities via nuances, feelings, underlying reactions, nonverbal responses, and verbal and written feedback (Denzin & Lincoln, 2005). The researchers jointly created and interpreted the findings with participants, with the researchers drawing on the elements of cooperative teamwork to understand whether the findings resonated with a broader evidence base (Gay & Weaver, 2011).
Design
The researchers used semistructured interviews (n = 17), obtaining perspectives from the recruited participants, and multiple case studies (n = 4), triangulating the findings with additional data from within the organizational contexts (Patton, 2002; Stake, 2006). The purpose of combining the data from the semistructured interviews with the case studies was twofold. First, the researchers sought to illustrate how these individuals operated in context and interacted with their coworkers in a real-world setting, including obtaining the perspectives of the evaluators with whom they were working. Second, the goal was to produce findings that could be considered in relation to the elements of cooperative teamwork to determine whether the findings were similar to occurrences in other social circumstances.
The interview schedule for the participants included the following questions:
How do you encourage others to engage with evaluation? Example? How do you handle people who've had bad evaluation experiences in the past? In relation to evaluation, what do you consider to be important when working in culturally diverse teams? How does (participant) encourage others to be involved with evaluation? Please give an example of a time when this worked well. Any reflections on members of (participants)'s team and their involvement with evaluation?
The case study interview schedule for the participants’ colleagues and evaluators who had worked with the participants included the following questions:
Sample
The research used purposive sampling (Punch, 2014) within Australian nonprofit organizations, and the key criterion was that an evaluator or colleague nominated potential participants because they considered them champions of evaluation. This meant that the evaluators considered the potential participant to be an advocate for evaluation, a promoter of evaluative thinking and methods, someone who cares about the findings and encourages use, promotes the benefits, and generates support among their peers (Rogers & Gullickson, 2018). The evaluators or colleagues had worked with the participants as internal or external evaluators or with the participant on an evaluation team. Of the 17 participants (10 females and 7 males), all had been employed in the nonprofit sector for over 10 years. Participants had strong personal cultural and linguistic connections with the organizations’ target audience or had worked with the target audience for more than five years. In total, six participants self-identified as having a culturally and linguistically diverse background; four participants identified as being Aboriginal and/or Torres Strait Islander Peoples, and two grew up outside of Australia, one in the United Kingdom and one in a western African nation.
Participants in this research worked in organizations that supported the elderly, homeless, and minority groups to engage with the wider community; developed community-driven projects; and provided services for newly arrived refugees. They also supported Aboriginal and Torres Strait Islander Peoples and marginalized groups to achieve equity and overcome barriers to accessing services, provided counseling and assistance to families in crisis, and supported women and children escaping domestic violence. Table 1 displays details about participants with pseudonyms and selected demographics.
Participant Details.
Note. *Senior—executive; Middle—some direct reports; Project—no direct reports.
Data Collection and Analysis
The first author conducted all the interviews (2017–2018) at a location of the participants’ choosing; they averaged 70 minutes in duration. All interviews were digitally recorded, transcribed, and then sent to each participant for review. The participants interacted with the interviewer in a way that indicated that they felt comfortable enough to engage honestly and openly with the topic. Participants responded to the questions in a consistent way with minimal differences among participants from different organizational levels, cultural and linguistic backgrounds, or genders.
Data collection for the four case studies (2017–2018) involved additional semistructured interviews, a document review, and a site visit. Between five and seven interviews were conducted for each case and included at least one evaluator among the interviewed consultants, colleagues, and board members who worked with the participant. Documents included evaluation reports, minutes from meetings, organizational charts, annual budgets, communication logs, work plans, internal reviews, and reports from public and private sources. A site visit occurred at all four of the organizations with the primary purpose of conducting the interviews required.
This research used Stake's (1995) suggested approach of collecting and analyzing data simultaneously. Initial coding to nodes in NVivo (2012) of the first 11 interviews and one case study followed the topics from the interview protocol and theoretical framework; codes were identified prior to the analysis (a priori) based on the interview questions (n = 3) and elements of cooperative teamwork (n = 5) (positive interdependence, individual accountability, social skills, group processing, and promotive interaction). The first author then coded each transcript to the additional five codes that emerged inductively from the data to capture the unanticipated concepts, insights, and emergent findings (Stake, 2006). The research involved a continuous process of interpretation of data, reflection, and reality testing with participants. Ethics approval was sought and approved by University of Melbourne Human Research Ethics Committee (HREC: 1647875), and a letter of endorsement was received from the organizations involved in the case studies.
Findings
This section presents findings related to the ways in which the participants promoted evaluation among their colleagues. It elucidates how they overcame issues by developing meaningful interpersonal relationships and understanding their colleagues as individuals by highlighting key quotations from the research. The section begins by focusing on the strategies participants used to foster positive personal relationships: understanding colleagues as individuals, tailoring communication, and incorporating conflict resolution strategies. It then illustrates how they collaboratively developed mutually beneficial shared goals (positive interdependence) and, in the final segment, shares their strategies for providing appropriate levels of encouragement (promotive interaction).
Foster Positive Personal Relationships
In this study, mutually beneficial working relationships were foundational to all interactions. Participants cultivated meaningful relationships and used those personal connections to make the evaluation relevant. They created and tended to relationships to clarify the purpose of the interaction around evaluation for that individual. As Michael stated, “It's about relationships. So, for me, whenever we’re trying to work with our staff, we have to work with them as an individual, as people, and then we move on to do the work, the complexities of our lives, so that relationship factor is really key.”
Participants highly valued their positive working relationships with their coworkers. Making interpersonal connections with coworkers to find appropriate ways of incorporating evaluation was fundamental. As Duncan articulated, “While the [evaluation] activities took much time implementing, you would also spend time with staff… it was important to sit and listen and before you get into work business. It was great getting to know staff personally rather than just on a work level.” Interactions on a one-on-one level were where participants, their colleagues, and evaluators reported that they excelled. From that basis, participants were then able to engage constructively with their colleagues, encourage them to listen to what they had to say, engage them to assist, influence their work practices, bring their resources, harness their expertise, influence other people's behavior, and persuade them of the potential for evaluation to assist the organization to achieve its goals.
Participants reported that it took time to form strong interpersonal relationships to understand their colleagues’ individual aspirations, the underlying purpose of the interaction, and their key motivational drivers, including enablers and barriers to participation. They recommended having as many conversations as required over time for colleagues to understand each other. This way of working favors long-term working relationships that are based on trust and reciprocal engagements. If the process is rushed, time-bound, or not based on a positive interpersonal dynamic, then significant barriers may prevent the interaction from being mutually beneficial. Alice reported that she took time over multiple opportunities to understand how her own worldview was different from others: Those words around respect and humility and reflective practice are really important. I think working across cultures is a great opportunity which people are stupid not to pick up to reflect on their own cultural values. The journey is often within yourself, ‘cause it's only then that you realize the world you come from has this frame. The philosophical underpinning this set of values, and while they’re yours, they may not be others, and they are not one better than the other. . . . I always remain open to the one-on-one conversations with people, but what I’m actually starting to do is actually find people who are in the organization who are able to have those conversations as well… “Go and talk to this person,” and he had different perspectives about why we’ve come to where we are.
Although participants focused on supporting, encouraging, and promoting evaluation to colleagues, they reportedly listened, communicated effectively, and developed rapport by being respectful, humble, empathetic, and inclusive. Participants also used their networks to find the means to source the expertise required to resolve issues and demonstrated their commitment by modeling the desired behavior.
Most recommendations from participants for dealing with conflict or with people who had negative experiences with evaluation echoed answers from questions about being inclusive and understanding that people had different worldviews. They recognized the challenges of working with people coming from culturally diverse backgrounds who may approach things differently and have different value systems. For example, a number of Michael's colleagues clearly articulated how he effectively prevented conflict and dealt with conflict resolution by providing emotional support, encouragement, and offering safe opportunities to engage with evaluation.
Although cultivating relationships took time and required a high degree of personal investment, these strategies were reportedly worth the considerable effort. Participants engaged with peers on an individual level to find out what their aspirations were and then, by identifying their individual needs, preferences, and learning styles, showed them how evaluation could assist. No one size fits all. Participants considered who was involved and how best to bring those talents, perspectives, and worldviews together. Participants promoted equity and inclusion; they made coworkers feel comfortable to participate by being welcoming, respectful, supportive, and valuing their involvement. Participants recognized that to ensure the highest quality evaluation initiative and work of the organization overall, they had to facilitate the inclusion of as many people with different worldviews as possible.
Although asking about the ways that participants worked to overcome communication barriers, the interview schedule included a specific question about their strategies for communicating appropriately in culturally and linguistically diverse contexts. There were two reactions: some were reluctant to answer, and others reacted negatively to the question. For example, there were comments such as, “I don’t think it's any different… We just have cultural awareness and people awareness across everything we do” (Charles), “We’re all human beings” (Mary), and “There was nothing out of the ordinary that we had to be mindful of. No, there was nothing in relation to that” (Sally). Many participants resisted the notion that they were tailoring their approaches in relation to the concept of being culturally appropriate, such as Michael's comment: “I try not to look at the diversity. I just try and look at people. There's almost this political correctness that drives a thinking and a language.”
Some participants who were willing to discuss their strategies framed their responses along with other forms of diversity. For example, Fred gave this response: “I think definitely because it is so culturally diverse and because you’ve got people from such a range of backgrounds, you have to do a multi-pronged approach, an approach which can change and cater for everyone's needs.”
Most participants were unwilling to delineate individuals on cultural and linguistic grounds but instead considered multiple kinds of diversity. This approach underlined the strategies they deployed. Participants acknowledged that those involved were not only people of different cultural and linguistic backgrounds, but different ages, genders, levels of engagement with technology, educational backgrounds, location (urban or remote settings), levels of tolerance of administration, and different learning and engagement styles. Participants recognized they had as many people with different worldviews in their evaluation team as they had people in their team, but they did not single people out based on any one of these characteristics. They wanted many different people to contribute to enhance the work they were doing. Participants worked to make sure everyone had the opportunity to make a valuable contribution.
Despite in some cases resisting answering the question about communication strategies in culturally and linguistically diverse contexts, participants provided numerous examples of tailoring their approaches to meet the needs of their coworkers. Instead of adapting their behavior for a specific cultural group, participants developed communication strategies that offered many different options. The aim was to reach as many people in their organization as possible. Participants attempted to tailor each one of the messages they were trying to communicate, or ways of seeking information, from the entire organization down to the individual level.
There were examples of participants working in inclusive and multimodal ways. To forge an inclusive pathway to evaluation, participants had to break down the barriers among the evaluation team, coworkers, management, and funders. They acted as brokers who acknowledged that everyone was interested in different things. Participants used their knowledge about the different communication needs and styles to adapt the information and tools and make sure they could include multiple audiences. For example, participants had to translate the terminology and processes related to evaluation in different ways to varied audiences to make them acceptable, meaningful, and eventually useful. Alice called herself a “code-breaker” and an “interpreter” and gave this example of how inappropriate evaluation terminology can be: Even to say something like, “Look, I think we’ll probably need to do a program logic on this.” What does that mean to most people? Nothing. Nothing. So, I always avoid all that stuff. I read it myself, and I try and learn and then sort of be an interpreter and minimize the switch-off factor that goes with listening endlessly to a language and a thinking process that doesn’t fit with the reality of an Aboriginal community organization and working with women facing family violence.
Find Shared Goals
To address the challenges preventing the uptake of evaluation, the predominant strategy participants used was to find shared goals between evaluation and their colleagues’ aspirations. Participants reported that they established working relationships based on trust and made connection among their coworkers, evaluation, and organizational goals. Participants found shared goals by connecting their colleagues with evaluation to achieve the purpose of the organization, by finding a way to connect evaluation to the motivations of their colleagues, or by a combination of both these strategies. An essential component was to keep the focus on the desired outcomes of the organization. They attempted to ensure that evaluation activities were not just an end, but a personally connected means to achieving the core purpose.
Participants shared examples of explaining the importance of evaluation to their colleagues by linking directly to the desired social outcomes the organization was working toward. Participants helped peers to see how their contribution to evaluation could assist with achieving their aspirations, how they could help others, and how they could progress toward achieving the organizational goals. They maintained focus on the long-term, big picture and directly linked it to the organization's desired social outcomes. Participants shared analogies, metaphors, diagrams, and lists that promoted a long-term, overarching perspective to illustrate the benefits of cooperation and how valued everyone's contribution was for mutual success. They used events happening across the organization as catalysts to promote evaluation as a tool that could assist.
Their strategies went beyond connecting individuals to organizational goals. Personalization and adaptability were also important as participants identified what aspect of the goals appealed to their individual coworkers, showed individuals how they would personally benefit from evaluation, and helped them achieve their unique aspirations. As Charles said, They want to feel like they’re doing something meaningful, too. So it's not too hard. I think it's connecting their jobs, their work and their clients through to being able to speak succinctly to the quality, and for others to be able to speak to the quality of the work we do as well.
Some participants said understanding the motivational triggers for individuals was important because they may be the only factors powerful enough to draw someone closer to participating in an evaluation activity. Jenny's determination was evident in this quotation: I’m dogged, no question about that, absolutely dogged. And I don’t apologize for that at all because I can see the value of it. And I work on people [laughs] so that they eventually see the value of it. And they may not see it in the same aspect as I do, but I will find something that will connect them to the importance of this area of work… You need to sell the benefit and try to demonstrate how it ultimately makes their job easier not harder.
Participants also removed barriers that prevented colleagues from participating and found solutions to address the underlying reasons why a peer might resist evaluation. They considered if fear, dislike of change, mistrust, risk of personal criticism, or time pressure were issues, and then worked in constructive ways to address these barriers. Participants kept the focus on the desired outcomes of the organization and reinforced, through participatory processes and group problem-solving, that everyone needed to contribute whether they were to achieve their ambitions. They attempted to ensure that the evaluation activities were not an end, but a means to achieving the overarching organizational purpose.
Predominantly, participants worked in organic ways, using the tools they had access to, and created connections that would be meaningful for their individual colleagues. Examples from participants included developing visual representations, using the organizational plans that were already in place, and sharing stories that developed a common vision for what the advocate wanted to see in the future. Therefore, participants were effectively promoting group work around evaluation by tailoring their messages to connect with the shared goals. They created the conditions for positive interdependence to occur and established the basis of mutually beneficial cooperative working relationships that included evaluation. Participants were thereby well-positioned to use the evaluation for the purposes of improving, informing decisions, communicating to multiple audiences, and influencing peers because they demonstrated how it could help achieve their aspirations and how it connected to organizational goals.
Provide Encouragement
Participants promoted evaluation by providing their peers with reassurance, enthusiasm, and support. They used their skills to make evaluation appealing, promoted their coworkers’ success, and provided help, assistance, resources, materials, praise, feedback, and suggestions. Alice stated, “I think I bring a very positive, hopeful, fun—it's worth doing because you feel the benefit. For me, it's a benefit proposition—I work with our team to see the benefits and the value proposition.” Participants in the research provided many examples of how they enthusiastically encouraged and supported their colleagues to engage with evaluation. Participants also made suggestions of what they would not do; they would not link evaluation with performance management, not use confusing terminology, and minimize any top-down approaches.
Participants adapted the ways they provided encouragement depending on their degree of influence. In the case studies where three out of four participants were in high positions on the organizational hierarchy, they used formal strategies to persuade their peers to incorporate evaluation into their routine operations. For example, in Case Study A, Fred established peer support mechanisms and working groups. He prioritized the consideration of the divergent preferences for working, engaging, learning, and contributing when he communicated about evaluation. A colleague felt that Fred had been creative in thinking about different techniques regarding how to engage the group as much as possible. She noted that a skill of Fred's was to “facilitate an experience. I did feel like there was a lot of autonomy to approach this stuff in a creative way.”
However, project-level participants, who had less power to initiate formal strategies, had to rely upon informal opportunities to make connections, insert enthusiasm, and interact in meaningful ways. The findings illustrate that participants predominantly used informal strategies to persuade their reluctant peers to incorporate evaluation such as providing random acts of encouragement. Case Study B provides an example of some of these informal strategies. Sally played a prominent role in assisting the evaluators to work with her colleagues to adapt the tools to the context in which they were working. She was a source of support and constant encouragement for her coworkers. A colleague said, Sally monitored and provided knowledge on how to use the tools with the participants. Sally would conduct practice sessions with us to ensure we were delivering the tools correctly. There was constant contact and constant communication… very approachable… I felt extremely supported.
Discussion
This research found that advocates effectively engage colleagues in internal evaluation in three key ways. First, they understood their coworkers as individuals by taking the time to foster meaningful positive personal relationships; they tailored their communication to be appropriate and took a proactive approach to deal with conflict. Second, they found shared goals and worked toward those goals in mutually beneficial ways using analogies, metaphors, and simple explanations. Third, they provided appropriate levels of encouragement at the right time, right place, and in the right way. The discussion draws upon the findings in combination with relevant sources of literature and theory; it begins by discussing the findings in relation to the elements of cooperative teamwork and culminates in a list of interpersonal strategies mapped to evaluator competencies.
Findings Mapped to Elements of Cooperation
This segment discusses the strategies participants used to engage their colleagues by highlighting alignment with relevant literature and the elements of cooperative teamwork. Specifically, participants illustrated the following elements of cooperative teamwork through detailed, creative, and innovative exemplars: social skills, positive interdependence, and promotive interaction.
Social Skills
The theoretical element of social skills encompasses all actions undertaken to effectively interact with other people (Johnson & Johnson, 2015). This element, according to Johnson (2003), is about a skill set: “Promoting the success of other group members requires participants to have (or to be taught) the interpersonal and small group skills needed for high-quality cooperation as well as to be motivated to use them” (p. 939). Communication skills, conflict resolution skills, and cultural competence are required to get to know, trust, accept, and support coworkers, converse unambiguously, resolve conflicts, and value and respect diversity (Johnson & Johnson, 2003; Stevahn, 2013).
Above all other strategies, participants used their social skills to be inclusive and engage a wide variety of diverse colleagues in their nonprofit organizations around evaluation. To ensure the highest quality evaluation initiative and work of the organization overall, participants recognized the value of facilitating the inclusion of as many people with different worldviews as possible. Participants provided numerous examples of communication tensions and disconnect in their workplaces. However, while they understood the potential for there to be communication barriers in a nonprofit context, they were more concerned with how to include their coworkers’ views and encourage, support, and empower others to participate and contribute in meaningful and appropriate ways.
In the literature, their way of working is also referred to as individualized consideration or understanding their coworkers as individuals (Bass, 1999, p. 16). It appears in relation to developmental and transformational leadership and in the descriptions of people who champion initiatives within organizations (Howell et al., 2005; Taylor et al., 2011). According to Bass (1999), individualized consideration is when the person pays attention to the differences among peers and discovers what motivates each individual. Examples from the literature have shown these individuals to be flexible and adapt their approach to be useful for different audiences. Howell (2005) provides astute examples of individualized consideration, referring to individuals who can hear and read people to understand their needs and then process information in ways that will be useful for them. Similarly, the different examples analyzed in this research demonstrated participants’ ability to tailor the promotion of evaluation to the unique circumstances that arose as participants worked alongside their colleagues and identified their individual needs, preferences, and learning styles.
Specific to communication skills, participants illustrated individualized consideration in practice when they reported that they were aware of the multiple layers of diversity that existed in their organizations. They recognized not only multiple layers of cultural and linguistic differences among their peers but also differences in education levels, aptitude for technology, age, gender, ability, tolerance for administration tasks, and preferences for learning and engagement. Participants tailored their communication to their specific audience. They harnessed their knowledge brokering skills and worked with others using a team approach to provide evaluative information to the right people, in the right way, at the right time to generate further momentum. They were wary of labeling any one difference as more important than another. The participant responses aligned with this statement from Kirkhart (2010): “Categorical labels attached to personal characteristics such as race, ethnicity, religion, gender, age, sexual orientation, language, disability, and social class often give the false impression that naming accurately represents human diversity” (p. 401). To deal with this degree of diversity and effectively promote evaluation, they attempted to understand their coworkers as individuals and communicate in ways that created supportive, inclusive, and equitable environments to enable the participation of as many people as possible.
In regard to managing conflict, apart from Case Study A where Fred instigated a formal change management plan that included a communication strategy, there was minimal discussion about preplanning communication strategies in advance or putting in place any conflict resolution strategies. Although the literature suggests that planning for managing conflict is imperative (Preskill & Boyle, 2008a; Stevahn, 2013), taking a more subtle approach may be warranted in cross-cultural settings. In this research, participants, in alignment with individualized consideration, were using their intuition, ability to adapt to the context, and capacity to tailor the information on an “as needed” basis. Participants were encouraging their colleagues to discuss their differences with and issues regarding evaluation. They sought to clarify the problems and barriers and collaboratively find and implement solutions. Participants were encouraging everyone to participate and have their say, recognizing that potential solutions could emerge from anyone at any point in the process. Existing research supports harnessing cooperation and incorporating competition in a constructive way in competitive, confrontational, and cross-cultural organizational contexts may be a beneficial approach (Deutsch, 2011).
In regard to cultural competence, participants understood the context, they privileged multiple perspectives of their coworkers, and they highly valued personal relationships. In accordance with individualized consideration, they displayed an approach that fostered involvement, promoted equity, tailored engagement, and created a supportive environment for participation, no matter who was involved. Participants valued diversity, particularly recognizing the nuanced diversity within the group, as a resource that could strengthen the process of their evaluation initiatives as recommended in the literature (Symonette, 2004; Yarbrough et al., 2011). They would use different strategies in different circumstances to be inclusive, respectful, and ensure that the voices of those less powerful were being listened to and taking part in the decision-making. Strategies incorporated strengths-based approaches, the promotion of social justice, a holistic way of thinking, and aimed to “infuse the cultures of individuals, program implementers, groups, and communities into the inquiry process” (Thomas & Parsons, 2016, p. 11). Participants recognized that including multiple perspectives was essential to understand the evaluation context and agreed with this sentiment: “Gathering as many perspectives as possible will provide the most accurate view of the program being evaluated” (Casillas & Trochim, 2015, p. 33).
The strategies examined in this research are similar to the extensive references from the evaluation field that illustrate working in ways that respect cultural diversity and are appropriate to the context (American Evaluation Association [AEA], 2011; Casillas & Trochim, 2015; Chouinard, 2014; Kirkhart, 2015; Symonette, 2004; Thomas & Parsons, 2016; Yarbrough et al., 2011). The way participants applied individualized consideration was also in alignment with the American Evaluation Association statement on cultural competence as it includes acknowledging the challenges of cultural identity as an essential practice: “Attempts to categorize people often collapse identity into cultural groupings that may not accurately represent the true diversity that exists… The culturally competent evaluator recognizes and is responsive to differences between and within cultures and subcultures” (AEA, 2011, p. 7).
However, as participants were focused on their relationship with their colleagues as individuals, many were reluctant to call their way of working cross-cultural, culturally responsive, or culturally appropriate evaluation. Although the examples from participants also resonated with the literature that focuses on cross-cultural practices, they either reacted negatively to the use of this terminology or were concerned that this description was too narrow. They did not think it adequately encapsulated diversity in regard to age, gender, levels of education, preferences for learning and engagement, and levels of aptitude with technology and administration tasks. Participants did not want to use singular cultural dimensions to define their peers but rather preferred the option of acknowledging the multiple intersections of cultural diversity (Kirkhart, 2010, 2015).
Positive Interdependence
Building upon the foundation of meaningful interpersonal relationships with their individual colleagues, participants found shared goals by connecting the individual aspirations of peers and the organizational values with evaluation in combination. Participants found positive reasons why individuals should participate, identified the reasons why they were not able to contribute to the evaluation, and then worked constructively to address the barriers so coworkers could support evaluation to facilitate goal achievement. These strategies can be theoretically linked to the element of cooperative teamwork called positive interdependence, which involves creating shared identity, determining interconnected roles, clarifying the rewards, and identifying and allocating available pooled resources (Johnson et al., 2011).
Participants’ strategies also resonate with the examples of how evaluators can develop positive interdependence from the literature. Donaldson et al. (2002) suggest that discussing the risks and benefits of cooperation around an evaluation initiative is a strategy for managing evaluation anxiety. They recommend outlining the major benefits to individuals from evaluation and at the same time articulating how to manage the risks so the downside is negligible, “but you have to talk this through or they don’t realize it” (p. 268). Other suggestions from the literature involve developing plans for evaluations in collaborative ways that help to ensure participants are working toward a common goal. Preskill and Boyle (2008b) suggested: “Plan for and design the ECB [evaluation capacity building] effort… design their capacity building activities in ways that engage participants, are appropriate for their level of need, cover key concepts early on, are based on goals, and are practical” (p. 164). Stevahn (2013) suggested that a shared team goal will help to clarify the work focus, and allocating tasks will make it easy to understand who will be responsible. It should be meaningful, authentic, clear, tangible, and able to be divided into defined tasks. King (2007) also suggests developing “a purposeful, written capacity-building plan that explicitly frames the evaluation process to teach people evaluation skills and positive attitudes” (p. 52).
Promotive Interaction
The findings on providing encouragement align theoretically with promotive interaction, which is about meaningful interpersonal ways of providing support and reinforcement that enable the common goal to be achieved (Stevahn, 2013). Encouraging, facilitating, helping, and exchanging are terms used to explain how promotive interaction involves individuals working together toward the group goal (Johnson & Johnson, 2015). In all the strategies provided, a common thread was talking to different people at different times about how seemingly unrelated topics connected to evaluation. Making interpersonal connections among coworkers to find appropriate ways of incorporating evaluation was fundamental. The literature includes similar examples of how to provide encouragement. For example, a mix of formal and informal strategies that range from reorienting workspaces to establishing peer support structures and communities of practice can be beneficial (King & Stevahn, 2013; Preskill & Boyle, 2008a; Taylor-Powell & Boyd, 2008). Taylor-Powell and Boyd (2008) suggest not underestimating the power and potential for informal strategies being pervasive and influential: Use every opportunity, every teachable moment, and every serendipitous occasion to build evaluation capacity. This can include inserting comments in e-mails, engaging in hallway discussions, sharing articles and resources, and asking questions — approaches that are not necessarily planned or formal. (p. 67)
Findings Mapped to Evaluator Competencies
This segment collates the findings and strategies for how to work responsively and inclusively around evaluation in organizations from this research and maps them against the interpersonal domain of two sets of evaluator competencies. Evaluation societies are currently mapping what is required to do evaluation competently. They have identified interpersonal skills such as cultural competence, communication, facilitation, and conflict resolution as essential competencies (AES, 2013; AEA, 2018; ANZEA, 2011; CES, 2010). Garcia and Stevahn's (2019) research provides insights into how evaluator competencies are conceptualized across developed evaluator competency frameworks. They conducted a cross-comparison of the frameworks and confirmed that the interpersonal domain (social skills for constructive relations) was a foundational common component (p. 4). This was further confirmed by the AEA Evaluator Competencies Task Force (Tucker et al., 2020).
Derived from this research on evaluation advocates, the overarching results included three key findings: fostering positive personal relationships (understand colleagues as individuals, tailor communication, and incorporate conflict resolution strategies); finding shared goals; and providing appropriate encouragement. The evaluation advocates’ practical examples on the interpersonal skills, ways of understanding the context, and tailoring information to the needs of individuals may be useful for anyone attempting to make evaluation appealing and engaging for nonevaluators. Although many of the strategies for building relationships take considerable amounts of time and energy, such as exploring the underlying value systems of colleagues, they may over time be worthwhile. Participants reported that time allocated to understanding the needs, strengths, values, and knowledge systems of their peers was worth the effort to find an alignment with the evaluation.
Table 2 documents these strategies mapped to the elements of cooperative teamwork (Johnson & Johnson, 2015), the interpersonal domain of the American Evaluation Association Evaluator Competencies (AEA, 2018), and the Australasian Evaluation Society competency framework (AES, 2013) to illustrate the competencies in practice. Presenting the findings in this table highlights that the participants’ strategies were most strongly aligned to the social skills element of cooperative teamwork and the competencies that relate to listening, trust, communication, and facilitation. The table illustrates participants’ key strategies in relation to the eight AEA (2018) 12 AES (2013) interpersonal competencies.
Practical Strategies Mapped to the Interpersonal Domain.
Strengths, Limitations, and Implications
This segment will discuss the limitations of this research, methodological strengths, and the implications of these findings for the field of evaluation, including evaluation educators, internal evaluators, evaluation advocates, external evaluators, and theorists and researchers of evaluation. The unique value of this research was its focus on evaluation advocates working in organizations in the nonprofit sector. Undertaking research in the nonprofit sector was ideal for elucidating interpersonal dynamics and understanding why and how interactions were happening in a real-world context.
Given that nonprofit organizations around the world are dealing with challenging social problems with limited resources, the study's main limitations relate to the lack of resources available for organizations to participate in the research. This meant that methods that involved extensive observation and opportunities to view participants in action, such as ethnography and phenomenology, were not available. Additionally, it would have been ideal to undertake a case study with another individual who was influencing their organization from a low hierarchical position, but this was also not possible. For that reason, it was difficult to distinguish between the impact of project-level people who advocate for evaluation from people in senior-level positions who recognize the value of evaluation as an organizational management tool.
Despite these limitations, the researchers obtained sufficient high-quality data because they could leverage their professional experience working in nonprofit organizations, networks, years of rapport building, and establishing trust. They also ensured that the process and interactions were mutually beneficial. The interview data were self-report; however, participants were given opportunities to review transcripts, and most provided verification and/or clarification for the accuracy of responses. Also, although the participant sample size and number of case studies were relatively small and dealt with nonprofit organizations only, triangulating participant interviews with case study data strengthened confidence in the results. The interviews and case studies in combination were appropriate for the context and resulted in sufficient data to answer the research question. The research design also included reciprocal arrangements negotiated with the case study participants that aimed to contribute to organizational development. This enhanced the amount and quality of the data in this research study.
Overall, the value of this research was that it investigated people who had genuine relationships with colleagues that resulted in effective engagement around evaluation. Researching the interpersonal skills of people working in these challenging settings provided useful insights and strategies that illustrate how individuals can overcome their organizational issues. Thereby, this research identified strategies for engaging with peers and promoting evaluation that was creative and innovative and of greater range and depth than what might have existed in a context where there were minimal communication issues or less resistance to evaluation.
For evaluation educators, the strategies found here may provide a useful way of presenting the interpersonal skills required to do the evaluation. Educators and trainers could use the strategies to help evaluators assess their interpersonal skills and garner a more accurate picture. Research demonstrates that evaluators may assume (erroneously) that they can relate to people, be culturally sensitive, and attend to multicultural issues (Dewey et al., 2008). This need is being addressed by an increasing focus on including this skill as part of training and professional development for evaluators (King & Stevahn, 2013; Rogers et al., 2022). However, this competency domain receives less attention in formal educational courses than other domains (Dewey et al., 2008; Gullickson et al., 2019), and the literature is lacking on how to apply these competencies in practice (Garcia & Stevahn, 2019). Therefore, finding creative approaches to using the strategies to elicit constructive feedback may overcome people's overly positive self-assessment of competence in this area (Dewey et al., 2008).
For internal evaluators and evaluation advocates working in organizations, these strategies could form the basis of self-assessment activities where feedback is sought formally or informally from colleagues to determine the extent to which the strategies are being implemented appropriately in practice. The practical examples may provide inspiration for alternative ways of interacting or as topics of discussion with colleagues, other evaluators, and in communities of practice.
External evaluators may not have opportunities to develop relationships within organizations, understand the nuances of differences within specific cultures, and interact with individuals on the same level over the long term. They may need, therefore, to engage the support of evaluation advocates who can enact these strategies. This research reinforces the importance of recruiting and retaining the evaluation team evaluation advocates who have meaningful relationships and can understand their colleagues as individuals. A field guide for evaluation advocates may be a useful tool for identifying these essential individuals (Rogers et al., 2022).
For theorists and researchers of evaluation, further investigation is required to help elucidate the intersections between these strategies and culturally responsive evaluation. Although in this study participants’ strategies illustrated that they work in ways that respect cultural diversity, are appropriate to context, and are in alignment with the culturally responsive evaluation literature, some participants were reluctant to use terminology such as cross-cultural, culturally responsive, or culturally appropriate. They did not want to separate cultural competence from other types of social skills. Researchers and practitioners have been critical of approaches that do not pay enough attention to culture, racial differences specifically, and underlying power dynamics (Chouinard, 2014; Hyde, 2004; Thomas & Parsons, 2016). For these scholars, acknowledging everyone as diverse may not go far enough to address the underlying power imbalances and may well continue to perpetuate inequalities among staff.
Recent literature is calling for cross-cultural evaluation approaches and participatory ways of working to go beyond just looking at the team dynamics and the situation in front of people and instead to think about the underlying system (Chouinard, 2014). Froncek and Rohmann (2019) have also articulated a need to address the underlying system in critiques of social interdependence theory. They suggest that the theory lacks guidance for minimizing the negative outcomes of encouraging participation when there are power imbalances in the workplace. Froncek and Rohmann (2019) draw upon the procedural justice literature that proposes that individuals develop their socioemotional interests through their relationship with the group. Hence, theorists and researchers could determine if individuals would be willing and able to increase emphasis on social and emotional needs of coworkers at the social inequality level.
Conclusion
It is difficult to convince colleagues to do something they do not want to do; in the nonprofit context, evaluation is often one of those things. A range of interpersonal challenges can prevent individuals from engaging in evaluation and can ultimately inhibit organizations from using evaluation for improvement, accountability, and communication purposes. This research elucidated how nonevaluators were effectively able to persuade their reluctant peers to incorporate evaluation into their routine operations.
Participants achieved this feat by understanding colleagues as individuals, finding shared goals, and providing appropriate encouragement. No matter their position on the organizational hierarchy or their cultural or linguistic background, participants valued developing strong positive relationships and forming meaningful connections with individuals in their workplace. They valued these relationships above skills related to communication, conflict management, and cultural competence. Their inclusive social skills were the most important foundation that underpinned their attempts to find shared goals and provide encouragement. Finding shared goals meant identifying what their peers were aspiring toward and tangibly demonstrating how evaluation could help them achieve those desired outcomes. Providing encouragement meant supporting and acknowledging progress in meaningful ways that maintained momentum.
This research found that when there was a base of meaningful relationships and colleagues understanding each other as individuals, then establishing shared goals and providing encouragement were the basic tenets for effectively encouraging others to do something they did not want to do. The research aligns with social interdependence theory and provides an example of what can be achieved through cooperative working relationships. Specifically, the participants’ practical interpersonal strategies, documented in this article, may be relevant for others attempting to assess and develop their interpersonal skills, develop positive interpersonal relationships, and develop equitable and inclusive working environments in relation to evaluation.
Footnotes
Acknowledgments
This research presents the results of the first author's doctoral thesis research. The second, third, and fourth authors served as supervisors. All would like to thank Professor Janet Clinton, committee chair, for her insightful expert counsel.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
