Abstract
Evaluating and Valuing in Social Research by Schwandt and Gates (2021) describes the global evolution in the understanding of social issues and offers innovative approaches to valuing and evaluating that fit where we are now. The presentation of this evolution begins with the past pattern of framing social issues as problems amenable to project-based solutions. When projects operationally defined social issues, evaluating and valuing efforts needed only to assess the project outcomes, generalizing findings as indicative of progress on the social issue at its core. The authors note that increasingly this project-based view of social issues and its underlying assumptions are giving way to an appreciation that the wicked complexity of contemporary social context, and the issues therein, do not easily translate into project-based solutions. Social issues evolve too quickly to easily translate into a single or even multiple project designs.
Schwandt and Gates identify the challenges of conducting social research in the context of complex contemporary social issues and the need for greater transparency, relevance, and efficacy in valuing and evaluating. The authors accomplish this by examining past valuing and evaluation frameworks, current or conventional framing, and suggested alternative approaches to framing. This treatment of valuing and evaluation includes a review of the underlying assumptions that define alternative frameworks. In building their argument, the authors offer a persuasive case for a morally committed social science and present strategies for valuing and evaluating that are influenced by situated practical reasoning, systems thinking and complexity science, boundary critique, co-production and deliberation, activity model of intervention, and self-determination and systems change Ethical, moral, and procedural questions emerge from the authors’ discussion of the past, current, and new approaches to valuing and evaluating.”
Overview
This is an ambitious book. The authors have produced a richly sourced volume. Every reading and re-reading of the text reveals more of the book's complex, multilayered argument. The book begins with a foundational chapter on research ethics that, among other points, makes the argument that researchers cannot rely solely on theoretical and methodological expertise when addressing how to determine what actions to take in examining social issues. The next chapter dismisses once popular claim of a value-free science, instead portraying the production of social science knowledge as value-laden and, therefore, requiring transparency. The first two chapters combine to take the reader through ways of thinking about facts and values, the myth of value neutrality and a call for morally informed research.
Chapter 3 describes current normative evaluation practice as comprising “the conventional frame.” “Frame” is defined as “a way of representing evaluation knowledge as well as a guide to thought and action with respect to positioning evaluating in policy and program planning and decision making” (p. 67). Conventional framing includes the traditional methodological, technical, and instrumental activities used to determine the value of planned interventions, policies, and projects. The chapter discusses the conventional frame's definitions of social problems, problem solving processes, definition and role of stake holders, and perspective on the nature and use of evidence, and situates them in relationship to the demands of contemporary social science practice.
The next two chapters recount the augmentation of the conventional frame and the subsequent development of alternative frames better suited to tackle the wickedly complex issues of the day (Rittel & Webber, 1973). The expansion of the conventional frame provides a response to the technical, social, and political environments and, in turn, alters these contexts. For example, according to Schwandt and Gates, evaluators have introduced practices intended to develop more engaged and vibrant partnerships with various impacted stakeholder communities than are common in the conventional frame. These and the other augmentations discussed in Evaluating and Valuing in Social Research moves evaluation beyond the conventional project-oriented stance to one that addresses broader issues and concerns.
However, expanding the conventional frame is insufficient to respond to the current context wherein facts are uncertain and contested, complexity is the norm, and values are in dispute and unstable. The alternative frames cited by Schwandt and Gates in chapter 5 argue that valuing and evaluating must evolve to adequately conceptualize and contribute a more robust understanding of social issues. In this transition, a more collaborative style of social inquiry alters the role of stakeholder groups. Examples of alternative frames for evaluation and research include acknowledgement of the white racial frame and Indigenous and Culturally Responsive Evaluation frameworks.
Chapter six provides a case illustrative of alternative framing, of the type discussed in chapter five. This case study is followed by an appendix which introduces additional methods, such as process tracing and contribution analysis, that reflect the alternative frames for valuing and evaluating. The concluding chapter explores the implications of adding the word “professional” to the meaning of words like valuing and evaluating. Initially the chapter establishes the ethically responsible posture towards valuing and evaluating that includes active questioning and critical reflection supported by individual evaluators and researchers in collaboration with stakeholders. Subsequently, Schwandt and Gates identify five key commitments, stating that commitments “signal responsibility and both a willingness and resolve to do something” (p. 163). The five commitments include a commitment to “(1) supporting learning and acting through evaluating, (2) focusing on the primacy of practice, (3) embracing an epistemology and politics of participation, (4) accepting professional expertise and responsibility as a civic matter, and (5) critically questioning the normative ideas of improvement” (p. 163). Each of these commitments is discussed at length, are “focusing on the primacy of practice” and “embracing an epistemology and politics of participation” (p. 163). The book closes with the implications of these commitments for researcher education and research on evaluation.
Structurally, the book contains helpful end of chapter aids such as an annotated bibliography of the most important resources tapped for the chapter. Additionally, a “Bridge to Practice” section provides discussion questions and possible workshop, or course discussion prompts that reinforce the points made in the chapter. Throughout the text, figures and examples are used to facilitate the reader's understanding of the constructs and to ground the theoretical discussions in practice. A glossary at the end of the book is another helpful tool. Among the potential uses of the book, I would view it as a companion text in an evaluation course or a stand along text for professional development efforts. While the book is structured to facilitate instruction, senior scholars and practitioners will find the content a valuable resource.
Determining Versus Developing Value
Evaluating and Valuing in Social Research takes a stance towards evaluating and valuing that matches up with current broader and more fluid assumptions about social issues. The authors describe movement from determining value through gathering evidence in the conventional framing of evaluation and valuing, to developing value through making sense of value perspectives in the alternate framing. The book further articulates that, in the former, the project is synonymous with the definition of the social issue of interest and, in the latter, the project represents only one set of efforts to address what is conceived as a broader and more complex set of social issues. Determining value reflects the approach of gathering evidence to provide the response to questions focused upon project or intervention value. Developing value connotes a more engaged process wherein the social researcher collaborates with stakeholders to co-construct the findings to be reported. The authors also acknowledge that “it is not uncommon that evaluating aligned with the notion of developing value will make use of some of the ideas and practices associated with determining value of planned interventions” (p. 141).
The book makes no effort to be procedurally explicit. Instead, it provides guiding questions and suggested exercises after each chapter, which bring the content into focus. The book successfully pulls together the thinking of social scientists who are leading the transformation of valuing and evaluating into a more civically engaged and morally driven enterprise. By reviewing what has been and what is, the book leads to a vision of what ethical reasoning in evaluating and valuing in social science practice could be.
A Morally Committed Social Science
Evaluating and Valuing in Social Research advances a much-needed conversation in the social science practices of program evaluation, policy analysis, and applied research. A rich and diverse literature is sampled along the way, which provides the reader with provocative questions and illustrations. Through the text the reader is asked to lean into the important questions about what constitutes professional practice for contemporary social challenges and the values that underly them. Schwandt and Gates advocate “for developing researchers and evaluators committed to civic participation and social justice coupled with critique of unjust arrangements and practices” (p. 162). This advocacy they argue, “should not however be mistaken for preparing these professionals as activists … we seek to decenter researchers’ and evaluators’ claims to authority, instead emphasizing that what they bring to investigations are capacities for collaboration, for skeptical scrutiny and for fostering the incorporation of scientific (and Evaluative) thinking into democratic deliberation” (p. 162). It is incumbent upon the evaluator and researcher to acknowledge the key elements of their practice. That there can be no professional code of conduct that recommends the “right” choices for how to engage the questions of the day, as the authors state, begs the question of what options are available or under development to inform practice.
The fragmented and divisive nature of contemporary social issues demands new relationships between valuing, evaluating, learning, and doing. Professional conduct requires taking on ethical responsibility for practices and, as advocated by Schwandt and Gates, exhibiting “democratic professionalism” by fostering “a way of working on co-owned, shared problems” (p. 186).
Valuing and Evaluating in Social Research offers five commitments, a plethora of considerations, and relevant strategies which combine to provide the practical wisdom necessary for taking next steps with due consideration.
