Abstract
An increasing number of foundations are embracing racial equity/equity as a core value, and it is influencing how they see themselves and operate. However, evaluation has for the most part remained untouched. Knowing how race/racism has influenced both, philanthropy and evaluation, deepens our understanding of how philanthropic evaluation practice may unintentionally reinforce racism. Equitable evaluation shifts the current evaluation paradigm to one that centers equity/racial equity, so that it is more aligned with the values and intentions of current day philanthropic endeavors.
The current moment finds philanthropy moving to embrace issues of equity more fully in its operations, investments, and initiatives. This means touching upon issues that are inherently complex and underpinned by social norms, biases, and privilege, all of which are internalized at the individual, organizational, and systemic level—perpetuating advantages for some and disadvantages for others. Foundations are realizing that to address equity they must address race and racism within their institutions and in their external actions and decisions. Efforts focusing on diversity and inclusion are intended as steps in that journey, as a means to an end but not the end in and of itself. Given this increasing interest in issues of equity in foundation work, evaluation practitioners who work in this sector now have the opportunity to rethink their relationship to evaluation with a particular focus on how racial equity shapes practice (see Appendix for definitions of key terms used in this article).
Race, racism, and racialization are integral components to the founding and development of the United States, and without addressing racial equity, any broader achievement of equity will be elusive. In this article, I invite my fellow evaluation practitioners who work in philanthropy to reflect on the field’s history, harness progress toward and interest in equity, and grow together. We need to reflect on our history because it sheds light on the implicit and explicit beliefs, values, and intentions that frame our current approaches and frameworks. We must harness progress toward and interest in equity by being comprehensive in our assessment of the ways in which inclusion, diversity, and equity manifest in internal policies, processes, and practice. This includes interrogating the ways in which values, assumptions, and beliefs (particularly those related to race) influence strategy, grantmaking, other investments, and evaluation. Finally, we can grow together by working in partnership to assure that the methods and tools used to understand conditions and assess the effectiveness of foundation solutions model the principles of racial equity that are being implemented through grantmaking and other foundation practices. One such approach currently being advanced is “equitable evaluation” (EE), which shifts the current evaluation paradigm to one that explicitly names equity as the purpose and considers the ways in which equity shapes evaluation practice. This necessitates the exploration of historical and present-day systematic and structural barriers that surface the role of racism (along with other “isms”) as contributing factors to the issues many foundations seek to address. In focusing on these three opportunities, we can begin to unpack and make transparent the foundations of evaluation practice in philanthropy and assure that they are consistent with present-day context, efforts, values, and intentions.
I write from the perspective of an evaluation practitioner who works with foundations across the United States. I seek to change the conversation about how we conceptualize and utilize evaluation in service of something more than data and information and to move us toward a fuller expression of values and a more robust commitment to a world in which race is not a key determinant of conditions, experience, opportunities, and outcomes. It is important to note that my perspective is one that integrates both my professional and personal identities. Professionally, I am an evaluation practitioner who has worked with foundations across the country for the past 25 years, often working closely with their nonprofit partners to strengthen their evaluation skills. Personally, I am a middle-aged, African American, cisgender, heterosexual female who grew up outside of Philadelphia, PA, went to an all-girls preparatory school, and am one of two children of still-married, college-educated professional parents, who are happily retired on to the seventh hole off a golf course in Arizona. During the course of my career as an evaluator, my role has evolved from being the diversity hire (the one person of color on a primarily White evaluation team) to being part of a small pool of evaluation professionals of color invited to submit proposals. I share these details because they shape how I see the world and my role as an evaluator. I am objective but not neutral: I believe evaluation has the potential to be an instrument that contributes to the world in which I want to live, one in which my race is not an indicator of, well, anything.
Reflecting on Our History
We are at a seminal moment in our nation’s evolution. Will we move forward and embrace the complexity, diversity, and some would say, promise of what we are now (or could be)? Or will we turn back time to the days when women, people of color, and anyone not part of the dominant group (White, Protestant, heterosexual, and adult males) are seen and treated as less able and less important and as subject to the decisions and actions of those in power? The jury is out, and the world is watching.
The current state of our country’s affairs speaks to how uncomfortable we are talking about equity, race, and racism and the historic, institutionalized, legalized, and racialized ways privilege is perpetuated in the United States. A strong undertow that has brought us to today is racism. In the United States, when we talk about racism, we must remember that race is a social construct (meaning that it has no biological basis) created by a group of people (White Europeans) to rationalize and legalize the sale and purchase of other people (Africans and Others; Baker, 1998; Smedley & Smedley, 2005). As notions of race were constructed in the United States, a platform for institutionalizing privilege and racism was born. That platform has been systematically integrated into the rules that govern this nation through policies related to housing, education, and criminal justice system, to name a few (Glaser, Spencer, & Charbonneau, 2014). This contributed to individual and institutionalized racism, which reinforces implicit and explicit differential treatment, expectations, and opportunities based on race (Glaser et al., 2014). Over the centuries, there have been attempts to disrupt racism through presidential decrees such as the 1862 Emancipation Proclamation (Dunn, 2015), federal cases such as the 1954 Supreme Court Brown v. Board of Education decision (Kluger, 2004), and federal legislation such as the 1964 Civil Rights Act (Risen, 2014). In the 21st century, attention has focused on law enforcement and the criminal justice system, where people of color, primarily African Americans, have higher rates of arrest, incarceration, and sentencing severity than their White counterparts for the same crimes (Chin, 2016; A. V. Hall, Hall, & Perry, 2016; Kutateladze, Andiloro, Johnson, & Spohn, 2014; Lawrence, 2015; Tolliver, Hadden, Snowden, & Brown-Manning, 2016). These efforts seek to change the policies and practices that reflect a racist orientation. They have not, however, changed the dominant narrative that attributes the ability for rational thought and cognitive aptitude to one group (White people) to be superior to that of others (in this case, people of color, primarily African Americans). This unconscious internalization of what may be referred to as the “white racial frame” (Feagin, 2013) has colored the ways our minds work. Feagin (2013) defines the White racial frame as “an overarching white worldview that encompasses a broad and persisting set of racial stereotypes, prejudices, ideologies, images, interpretations and narratives, emotions, and reactions to language accents, as well as racialized inclinations to discriminate” (p. 3, Feagin’s emphasis). The White racial frame, says Feagin, is “the dominant racial frame that has long legitimated, rationalized, and shaped racial oppression and inequality in this country. [It] is a centuries-old worldview that has constantly involved a racial construction of societal reality by white Americans” (2013, p. x).
Framing is important to the conversation about evaluation because it speaks to a tendency to accept what is consistent with the framing as normal, perhaps even right. Because framing is so unconscious and easy, the mind does not seek alternative hypotheses or challenge assumptions. Kahneman (2011) added to the research on framing by positing that it occurs on two levels—Systems 1 and 2. System 1 is our default. House (2017, p. 179) describes it as “mostly automatic, autonomous, and subconscious.” It is what comes easiest to us and what is reinforced through experience, practice, and images that normalize our framing as natural and correct. It provides us with comfort, security, and to some degree, stability. It is thinking (or framing) that goes unchallenged. System 2 thinking is higher order thinking that forces us to slow down, concentrate, and check validity (Kahneman, 2011). System 2 makes us stop, reassess, explore alternatives, and seek new information—but only if System 1 sends an alarm that something is not right. This is challenging because System 1 seeks coherence and ease. In short, if our thinking feels right and consistent with previous experiences and there are sufficient data to support that thinking (regardless of validity), we conclude that it is right—no need to think any further. However, System 2 does come into play when System 1 gets stuck. System 2 can entertain doubt and explore multiple scenarios and make intentional choices. But System 2 is lazy (House, 2017) and won’t work really hard to overcome System 1 bias. For me, this translates to the conclusion that if both Systems 1 and 2 are grounded in White racial framing, they merely reinforce each other.
So what does all of this have to do with philanthropy and evaluation? A great deal, in fact, because racial equity is a norm that does not exist within the White racial frame (historically or currently). The pursuit of equity necessitates those who have amassed tremendous wealth and engaged in philanthropic endeavors to acknowledge and reflect upon the ways in which privilege (and thus racism) have been key contributors to that wealth. For evaluation as a field, the notion of equity challenges what practitioners accept as valid, rigorous, and objective. Equity asks us to consider multiple truths (some perhaps more important than others); to weigh the complexity of our current society, the multiple communities that exist within that society, and the multiple identities we each carry; and to discover new and multiple definitions of validity.
Philanthropy and Evaluation: The Evolution of a Partnership
Before I move to a consideration of equity’s current role in philanthropy, I want to begin by looking at how the practice of evaluation came to be part of philanthropy. It’s important to remember that evaluation was not seeded in the fertile soil of philanthropy; rather, it was transplanted from the federal government and academic research institutions (Hogan, 2010). Evaluation’s initial purpose was to inform the allocation of public dollars and the effectiveness of those investments (Preskill & Russ-Eft, 2015; Shadish, Cook, & Leviton, 1991). It was not intended to interrogate and inform the development of solutions that addressed structural and social policies and practices that perpetuate inequality or promote equity. The evaluation DNA that philanthropy adopted included definitions that reflected current thinking about what was considered truth (ontology), knowing (epistemology), and value (axiology) which led to a specific and somewhat narrow portfolio of approaches (methodology; Mertens, 2012). So where we are now is not surprising in my opinion.
If we are to move the practice of evaluation forward, we must consider and reflect on several elements or decision points in the evolution of evaluation in philanthropy and evaluation as a field (see Table 1). By naming these key junctures and recognizing the values, context, and intentions that were at play (whether implicit or explicit), we can decide which to hold tightly and which to release or hold loosely. This can create space for different, inclusive, and diverse perspectives, thereby deepening our knowledge and ability to move toward an equitable society and helping us rethink the role evaluation can play.
Our History: Four Things to Remember.
Evaluation is a child of government and behavioral sciences
As a field, evaluation was a potential tool to answer questions about government expenditures and the social good they were intended to manifest (Preskill & Russ-Eft, 2015; Shadish et al., 1991). It was a way to answer questions related to accountability and how best to use the public dollar. This was a direct response to deeper and larger investments in social programs designed to help those who needed it most, primarily focusing on education, health care, and other social programs aimed at the individual. For the most part, evaluation became program evaluation (Shadish & Luellen, 2005). The methods of scientific research—including controlling and isolating for contributing and confounding factors, controlled environments, and questions of dose—became part of the underpinning of evaluation, regardless of foci and regardless of context and population. Randomized controlled trials became the standard of evidence to determine effectiveness (Hogan, 2010).
Evaluation in foundations was conceived in a narrow context by a few individuals
In thinking about present-day evaluation practices, we must consider the when, why, and who that brought evaluation to philanthropy in the beginning. Foundations embraced evaluation as a way to increase federal and community interest in how they were spending their assets, quell questions about the resulting benefits of their expenditures, and assuage mixed feelings about the tax benefits afforded to them (P. D. Hall, 2003). Evaluation was one way foundations could prevent further inquiry (imposition and interference) into foundation operations and decisions while also improving and standardizing (to the degree possible) decision-making and expenditures related to foundation mission (P. D. Hall, 2003). The champions of bringing evaluation to foundations were high-wealth, White male industrialists, and scientists who were leading some of the early U.S. foundations. This is the when and the why (P. D. Hall, 2003).
The early forefathers of foundations were primarily White, middle-aged, heterosexual male executives, individuals such as Ford, Rockefeller, and Sage. They ran large national companies engaged in industry and manufacturing. This is the who (P. D. Hall, 2003). It is not surprising to me that these men thought of evaluation as a tool for documenting outputs and costs as well as assessing efficiency and compliance. The burden of proof regarding effectiveness and contribution to foundation mission was on grantees as opposed to foundation leaders who were making decisions about foundation strategy and its execution.
These early conceptions of evaluation in philanthropy remain so deeply rooted that foundation staff and their evaluation partners have forgotten that they were planted by someone—and that the someone matters. Because this history is unknown or the underlying assumptions and purpose not challenged, evaluation’s potential role and appropriate approaches are limited. Remembering this narrow context frees us to let go of that which does not serve us while holding to the values embedded in the original intention to support learning, growth, and impact.
Evaluators needed a new market
In the 1980s, shifts in the federal sector left the growing field of professional evaluators without a place to practice their craft (P. D. Hall, 2004; Madaus, Stufflebeam, & Scriven, 1983). During that time, seasoned professional evaluators saw foundations as a potential audience. They brought to their new workplaces mind-sets, methods, and tools that had served them well when conducting evaluation research or longitudinal studies for the federal government or in research institutions. Their focus was to understand the return on investment of the public dollar and the effectiveness of large-scale programs. Context and culture were often things that were controlled for as opposed to elements to consider as parts of program design or evaluation approach (P. D. Hall, 2004; Madaus et al., 1983).
What strikes me as interesting here is that evaluation arrived in foundations with established principles and practices even though the context was different. That is not to say there was not variety and experimentation in the evaluation methods—there was (and is). However, what remains core is a shared opinion of what qualifies as knowledge and as evidence. There is a strong bias toward data that are quantifiable, perceived as objective, and meet the agreed-upon standards of scientific rigor and validity.
Harnessing Interest in and Progress Toward Equity
Currently, there is a burgeoning interest in equity in philanthropy, reflecting an expansion from a narrower focus on diversity and inclusion in this sector (Nielsen & Huang, 2009). Early advocates such as PolicyLink and the Philanthropic Initiative for Racial Equity, along with active voices that include the National Committee for Responsive Philanthropy, D5, and funders like the Annie E. Casey Foundation and the W. K. Kellogg Foundation, have challenged philanthropy to move beyond merely assuring equal opportunity and diversity. Recent players such as the Ford Foundation, Meyer Memorial Trust, and the Weingart Foundation have added their voices to the discussion.
There has also been increased attention on equity considerations in specific grantmaking strategies. Often faulted as looking for a magic fix, foundations frequently fall victim to seemingly new models that they believe will make their work less complex (Schambra, 2008). Unfortunately, these magic fixes have often influenced how we think about evaluation, thus reinforcing elements of the White racial frame and the scientific model expressed through the fixation on quantitative proof and replication. As a result, well-intended grantmaking and collaboration strategies have sometimes inadvertently perpetuated particular ways of being, thinking, and doing when race and racism were not explicitly named and addressed.
In recent years, intermediary philanthropic associations have also begun to focus more attention on equity. It is important to note that the Association of Black Foundation Executives (ABFE), founded in 1971, has been a long-standing advocate and leader pushing the field of philanthropy to embrace principles of diversity and inclusion in service of racial equity. CHANGE Philanthropy (formerly known as Joint Affinity Groups) was founded in 1993 to unify identity-focused philanthropic affinity groups into an empowered coalition. The seven core partners work to integrate diversity, inclusion, and social justice into philanthropic practice, transforming the sector’s culture to be one that embraces equity. Newer to the conversation are the United Philanthropy Forum (formerly the Regional Association of Grantmakers), founded in 1995, and Grantmakers for Effective Organizations (GEO), founded in 1997. GEO recently hired a senior advisor to lead its racial equity work, and the organization has dedicated a portion of its website to equity, which speaks to its long-standing support of organizations such as PolicyLink, CHANGE Philanthropy, ABFE, and others (GEO, 2017). The United Philanthropy Forum website (2017) identifies advancing racial equity, diversity, and inclusion as a strategic focus. It is too early to tell the degree to which these efforts, which now include mainstream and identity-focused organizations, can transform current foundation practices.
Finally, although there is much happening in philanthropy in regard to equity, with race increasingly being identified as a core element of the conversation, the situation is nevertheless challenging, given that the wealth of many foundations has grown due to the ways that U.S. laws and rules privilege the privileged. The assets of many foundations are a consequence of successful capitalist ventures (in industry and technology), inherited family wealth, health-care conversion legislation, and technology holdings. The majority of these financial assets are held by White people (P. D. Hall, 2006; Zunz, 2011). All of that said, it would be remiss not to note that there are and have been efforts to increase the racial diversity of staffed foundations as a way of informing the work of foundations to explicitly hold race as a core frame and equity or social justice as a desired end. These efforts over the past 10–15 years have been successful in keeping attention on the role of race in philanthropy (Bearman, Ramos, & Pond, 2010; D5, 2016).
Philanthropy and Evaluation: How the Partnership Might Be Transformed
Despite the increasing number of foundations that are embracing equitable change, one area that remains somewhat untouched in philanthropy is evaluation practice—and here we come to the final element of our history that we need to remember: Foundations and evaluators have a history of coming together and pushing evaluation practice. Over the years, foundations and evaluation leaders have joined together based on shared values and intentions to rethink evaluation approaches and tools within the foundation context (Patrizi, 2006). This has often been driven by shifts in foundation grantmaking strategies (e.g., strategic philanthropy and collective impact) as well as a deeper desire for evaluation to reflect particular values (e.g., multicultural evaluation, feminist evaluation theory, culturally responsive evaluation, and empowerment evaluation).
Additional efforts include the Evaluation Roundtable. For almost 20 years, foundation evaluation leaders have come together regularly through the Roundtable to explore current evaluation practice (Patrizi Associates, 2010). Benchmarking research leading up to convenings, along with a deep focus on peer sharing and learning, has advanced the Roundtable’s own practices and that of the larger field (evaluation) and sector (philanthropy). This is space where evaluation and program staff can step back and delve into substantive issues related to the ways in which evaluation is best serving the strategic and programming needs and intentions of foundations. Topics are selected based on degree of interest across many foundations and selected based on environmental scans and interviews with foundation staff by the Center for Evaluation Innovation, which hosts the Evaluation Roundtable. In reflecting on conversations with participants of the Evaluation Roundtable, I have noticed that there does not seem to have been much formal attention paid to the role of race in evaluation beyond acknowledgment of the need for a more diverse pool of evaluators and evaluation staff within foundations. Hiring more diverse staff is an important step, but this step alone is not sufficient to override the inherent bias in what we consider evidence, knowledge, and valid data that are often referred to as “objective” and “rigorous.” The fact that race and how it shapes evaluation practice has not been an emerging theme in these conversations is not surprising given the “who” (White males and females) who have historically comprised foundation evaluation staff and the evaluators who work with them. However, as demographics shift within foundations and the work of foundations themselves shifts (moving from interventions focused on individuals to strategies that address structural barriers that perpetuate racism), it is likely and necessary that conversations regarding equity—in particular, racial equity—will evolve.
We know that evaluation practice in foundations continues to evolve (Behrens & Kelly, 2008; Braverman, Constantine, & Slater, 2004; Inouye, Yu, & Adefuin, 2005). We also know from a recent report (Center for Effective Philanthropy & Center for Evaluation Innovation, 2016) that foundation staff (within evaluation and program units) have three wishes for the future in terms of evaluation. They want to be more strategic in the ways they plan for and design evaluations, so that information collected is meaningful and useful. They want to use evaluation data to make decisions and improve practice. And they want to be more transparent about their evaluations and share what they are learning externally. While racial equity is not mentioned explicitly in this report, it may be that we are evolving toward evaluation that more fully recognizes context and cultural validity and that explores questions that deepen understanding of historic and present-day structural and systematic barriers to desired outcomes. Given the increasing focus on equity among foundations, it is not a far leap to see how evaluation practice that specifically models equity and explores barriers to it might be an area of transformation.
Foundations and the evaluation field have also come together to diversify the talent pool of evaluators to include individuals with experiences and mind-sets that are more connected to that of the communities being served. They have also sought to expand the theories and methods utilized by evaluators (Madison, 2007), such as culturally responsive evaluation (Hood, Hopson, & Kirkhart, 2015), feminist theory (Podems, 2010), multicultural evaluation (Inouye et al., 2005), and Indigenous evaluation framework (LaFrance & Nichols, 2008).
Efforts such as the Building Diversity Initiative/Graduate Education Diversity Internship, the Robert Wood Johnson Foundation Evaluation Fellowship Program, and most recently Leaders in EE and Diversity have all been funded and supported by foundations, the W. K. Kellogg Foundation, the Robert Wood Johnson Foundation, and the Annie E. Casey Foundation, respectively. As someone who has worked as a consultant for the past 25 years, it is clear to me that these foundations have had a longtime commitment to addressing issues of disparity, disproportionality, and inequity and clearly see evaluation as an important component of their tool kit to effect systemic and structural change. These efforts are important in that they have created a new pool of evaluators who are more ethnically, racially, and culturally diverse. These new evaluators have been exposed to evaluation theories and methods that recognize complexity and culture as central to understanding the evaluand. However, these approaches do not seek to transform or diversify the places where evaluation occurs or the evaluative culture in which these new evaluators might be entering. In foundations, the evaluation culture has remained for the most part unchanged. Although methods have evolved somewhat, definitions of validity and rigor that were introduced in the 1970s remain the gold standard. If foundations are not ready, willing, and able to entertain new, different, or multiple ways of designing, implementing, and using evaluation, we may be placing the burden of leading and changing the status quo on people of color.
Growing Together
To fully unpack privilege, notions of objectivity, rigor, and validity must be challenged or must evolve. Historically, foundations have explored new ways of thinking about and doing their strategy, grantmaking, and programmatic work (Soskis & Katz, 2016). Similarly, foundations must do the same with how they conceptualize and uses evaluation, so that it keeps in step with foundations’ evolution in their larger work. In not doing so, foundations run the risk of using a tool (evaluation) from a different place and time, reflecting different values and intentions to assess the worth, merit, and value of their new endeavors (Patton et al., 2014; Scriven, 1991). A new way of doing evaluation is particularly critical for those foundations engaged in work in the service of equity. This means moving beyond the demographics of the evaluator and the cultural competence of the methods to reconceptualizing and expanding definitions of truth, knowing, and evidence, so that they represent and reflect diversity, equity, and inclusion (Eliadis, Furubo, & Jacob, 2011).
As someone who has worked with foundations for more than two decades and witnessed the shifts in focus and practice in regard to the role of race and racism in the sector, I call on my evaluation colleagues and foundation partners to ensure that evaluation does not remain untouched and unchallenged in this transformation. Specifically, I ask that we recognize the following: Evaluation is based on a paradigm. It is not objective. It reflects a way of defining, describing, and analyzing the world, a vantage point that for the most part still reflects its historical underpinnings. It is rooted in a particular moment in time that was informed by the values, experiences, and needs of a small group of White males. The formative years of evaluation were strongly influenced by scientific research and by the need to assess the best use of federal funds for social programs. It was transplanted with that DNA into philanthropy. Diversity and lived experience, although valuable, often merely “color” methodological approaches that inherently give greater value and validity to certain types of data and analysis as opposed to others. Evaluation is political. At its simplest, evaluation is the systematic “process of determining the merit, worth and value of things” (Scriven, 1991, p. 1). Who gets to decide, the questions, the process, and the criteria for determining merit, worth, value, or significance—all of these matter.
As evaluation practitioners move forward, we can evolve and expand what we mean by value (axiology) to one that centers human and civil rights, we can recognize that knowledge is a social construct (epistemology), and we can honor multiple realities (ontology), not all of which are of equal importance (Mertens, 2012).
Race, Equity, and Evaluation: We Don’t Know Enough
Grantmaking focused on racial equity is based on several principles of practice, such as conducting structural analyses of inequity, including stakeholders and affected communities in the codesign of strategies, and attending to funders’ own institutional policies and practices related to diversity, inclusion, and equity (Grantcraft, 2007; Philanthropic Initiative for Racial Equity, 2014; Redwood & King, 2014). For example, the Fund for Shared Insight (2017), a national funders’ collaborative that launched in 2014, believes that foundations can be more effective by encouraging and incorporating beneficiary feedback through their nonprofit partners and understanding the connection between feedback and better results. Here, “beneficiary” might be code for the marginalized, low-income, and people of color communities that many foundations focus upon.
These principles of practice emphasize that data about the problem being addressed—as well as about the outcomes of the change initiative—should at minimum be disaggregated, so that differential effects by race, ethnicity, gender, language, or a myriad of other dimensions can be spotted and accounted for. They do not, however, translate into how evaluation as a practice and process should evolve to reflect racial equity and in service of equity.
Given the influence of foundations on evaluation practice, it is time to explore how to systematically integrate what some practitioners call “EE” into philanthropic institutions (Dean-Coffey, Casey, & Caldwell, 2014). Developed initially by a collaborative team of Luminare Group, the Center for Evaluation Innovation, and the Dorothy A. Johnson Center for Philanthropy, EE explicitly accounts for the context of structural barriers to social change and recognizes that culture is bound to this context (Dean-Coffey et al., 2014). As the number of foundations that have an equity focus is growing, it is time to examine the “fit” of existing evaluation approaches to the principles and values of equity-focused grantmaking and understand better what it takes to build the organizational capacity, will, and wherewithal to engage in EE.
United Nations Children’s Fund has a particularly robust definition of equity-focused evaluation (Bamberger & Segone, 2011). That definition helped in the formulation of thinking about equity and evaluation in U.S. practice. Context, culture, and intention matter. As foundations interrogate their internal policies and practices with regard to human resources, investments, and grantmaking, they cannot leave evaluation untouched. It, too, is a system guided by values (often implicit) that need to be made explicit, challenged, and aligned ideally with the stated intentions and principles of the foundations and, at a minimum, with foundations’ grantmaking strategies. This means moving beyond the diversity of the evaluator and including the voices of beneficiaries. It means fundamentally redefining what we mean by evidence, truth, and knowing in service of equity.
A current effort in this vein is the Equitable Evaluation Project (2017), a project with which I have been closely involved. The Equitable Evaluation Project centers on an engagement model that the Center for Evaluation Innovation has used successfully to trigger new thinking on other emerging topics in evaluation and philanthropy (e.g., evaluating advocacy field building, network strategies, and social innovation). Through a set of exploratory interviews, a framing paper, and a literature review and via a 1-day convening, the Equitable Evaluation Project explored what equity programming looks like in practice; identified current practice in the evaluation of equity programming; developed insights, principles, and recommendations for effective EE; and identified areas where further development of the field’s thinking, capacity, and commitment are needed. The findings were shared through the Equitable Evaluation Project website and through publications and presentations. The framing paper offered a set of principles for EE (see Table 2) and identified a set of orthodoxies that are barriers to foundations operationalizing EE (Luminare Group, Center for Evaluation Innovation, & Dorothy A. Johnson Center for Philanthropy, 2017). The Equitable Evaluation Project is now a multiyear initiative with foundation partners who are interested in promoting EE and learning from those taking the first steps to adopt EE in practice.
Equitable Evaluation Principles.
Moving Forward: Grounding Philanthropic Evaluation Practice in Equity
Philanthropy, uniquely positioned in our nation’s social fabric, has the ability and, one could say, the responsibility to interrogate assumptions, experiment with strategies, intentionally learn and share, and push thinking and practice. It has done so successfully in the past, yielding new insights about and strategies for some of our most pressing and challenging social issues (Anheier & Hammack, 2010). These efforts include but are not limited to income inequality, education reform, health-care policy, and lesbian, gay, bisexual, transgender, and queer equality (Anheier & Hammack, 2010; Lewis, 2015).
Now is an opportune time for foundations and evaluators to work together to ensure that evaluation practice remains in step with current contexts, intentions, and values. To that end, as an evaluation practitioner, I propose that funders and evaluators of equity efforts, particularly those focused on racial equity, commit to these emerging principles of EE. This means changing the paradigm in which evaluation is rooted to one that holds equity at its core, challenging current notions of “evidence” and “objectivity,” which work within the construct of White racial framing. In doing so, evaluators working in philanthropy can resituate evaluation practice and create a new container in which existing and often underutilized theories and approaches (e.g., culturally responsive evaluation, feminist theory, multicultural evaluation, and Indigenous evaluation framework) may become more valued and common.
Anticipated Challenges
Change is hard. I find myself constantly challenging my assumptions and making my biases and intentions as clear as possible, so that I (and my client partners) can navigate them together to make decisions consistent with the world we jointly want to see and cocreate. As evaluation practitioners, we are in a place of uncertainty and likely more turmoil. The world is changing. Quickly.
In the past, evaluation has often offered, perhaps falsely so, a degree of certainty, clarity, and simplicity that brings comfort and safety. This holds true for foundations and for many evaluators. As foundations continue to wade into the uncharted territories of authentic community engagement as well as systems and structural change that rewrites the rules that privilege some and deprive and punish others, it is likely that there will be more and different questions than there will be answers.
This is not work for everyone. EE requires a way of being and thinking that is grounded in a moral imperative (equity as an end, not only as a means) and that embraces the complexity of human identities, including those of the evaluator. It will challenge ideas of “validity” and “rigor” that codify bias as well as ways of knowing, truth, and evidence that are defined as objectivity. But for those foundations stating that racial equity is their work, it is imperative that, in their introspection and transformation, they do not leave evaluation practice uninterrogated or unchanged.
As I stated earlier, philanthropy has a tendency to look for a quick fix or to be enamored of that which is new or emerging. If equity is in fact the moral imperative that more foundations are gravitating toward, then the practice of EE cannot remain something that is talked about only at conferences or through social media. This moment in time calls for foundations to look inside at the ways in which their practices and ingrained White racial framing prevent evaluation from being as powerful a tool, as it can be in service of their missions.
We have to do more than recognize bias and racism in our evaluation practices. Instead, we must go further to develop new and different ways of designing, implementing, and using evaluation. Building on what House (2017) writes, this means challenging our default System 1 thinking, pushing our System 2 thinking to create new pathways that honor complexity and systems thinking, and eventually rewiring our System 1 default. In philanthropy, that means looking at the rules and norms that have been put in place, rules and norms that are interpreted as fair and rigorous but that unintentionally reinforce bias and racism when it comes to evaluation. It means moving beyond the demographics of the evaluator and moving toward demographic and philosophical characteristics that ensure a worldview and analytical frame that centers racial equity as core.
The Promise and the Potential
Seen through this new lens, evaluation can become not only a means of assessing the “merit” or “worth” of an intervention of the collective strategy of a foundation and the ways in which that intervention contributes to equity, but evaluation can also become a tool in the service of equity. I offer the following recommendations for U.S. foundations, as they move this work forward: Acknowledge that evaluation reflects a paradigm that cloaks privilege and racism as objectivity and neutrality. Educate foundation staff about the origins of evaluation within philanthropy, noting its strengths and limitations, so that they can become better producers, consumers, and users of evaluation and in doing so push the field of evaluation to continue to evolve. Explore the ways in which foundation practices (orthodoxies) are barriers to the adoption of EE principles, and identify and share approaches that interrupt those habits. Move beyond methodological approaches and evaluator demographics to address culture and context, and in so doing, unpack our definitions of evidence, knowledge, and truth, so that we may create new ones grounded in this time, place, and set of intentions. Continue to diversify and expand the talent pool of evaluators, and ensure that their training (both formal and informal) introduces and nurtures a myriad of new and different ways to conceptualize evidence, knowledge, and truth in service of greater validity and rigor. Continue to foster and provide resources for evaluation practice in philanthropy, so that evaluation remains aligned with the values shaping other foundation practices.
Several of these ideas are bubbling up in a variety of settings, but what is needed now is an overarching frame that reminds us of the “why” and the “to what end.” In this case, it is evaluation that yields more meaningful and relevant insights and information as well as greater racial equity—not “either/or” but both.
Conclusion
The work is uncharted, but the process is not. Transparency, experimentation, and deliberate inquiry and learning are practices and mind-sets we as evaluators hold as core to our discipline. It may be that the stakes are higher now given the political and socioeconomic landscape and given what it seems the future may hold, but there are lessons learned and models that can be used to move this work forward. Methods such as culturally responsive evaluation (Hood, 2004; Hood et al., 2015) and multicultural evaluation (Inouye et al., 2005) point the way to new ways of doing our work, as do organizations such as the Center for Culturally Responsive Evaluation and Assessment and initiatives such as the American Evaluation Association’s Graduate Education Diversity Internship program (Collins & Hopson, 2014; Symonette, Mertens, & Hopson, 2014).
Evaluation was birthed in a place and time by a select group of individuals for a narrow range of efforts and intentions. It was transplanted into philanthropy with a preexisting set of “truths,” which for the most part have remained unchallenged. Now that more foundations are moving to embrace the values of equity/racial equity and seek to manifest them in all their operations, we as their evaluation partners must be in step with them and do the same. As professionals and practitioners, we can no longer sit on the sidelines wearing the cape of objectivity and neutrality, a cape that shields beliefs and assumptions about knowledge, rigor, and evidence and which elevate a Western White worldview. Imagine what might be possible if we collectively conceptualized, implemented, and utilized evaluation in a manner that is consistent with and promotes racial equity as a step toward equity more broadly. Everyday narratives that continue to marginalize, minimize, and disrespect people of color and those with less privilege could be replaced with ones that do not demonize and place blame on the individual. They could instead lift up the historical, contextual, and powerful dynamics that create and sustain oppression and shed light on the strategies and solutions which can shift the “rules of the game,” so that equity is achievable. Evaluation could have a powerful role in ensuring that individuals, communities, and peoples not only share equitably in the knowledge, wealth, and resource of society but also contribute to the creation of those elements.
Footnotes
Appendix
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.