Abstract
This commentary takes up the question raised by Socius editor Adam Gamoran regarding the role of social scientists in reducing inequalities. Gamoran asks, “Can we make changes to existing practices or do we need radical reconstruction?” In considering Gamoran’s question and two springboard articles published in the special collection, the author considers three questions: First, does the field truly understand enough about the causes of inequalities to be sure that the solutions generated are appropriate? Second, what does it mean to assess the feasibility of solutions within a system that operates on the assumption that inequalities are part and parcel of its economic and social systems? Finally, how do social scientists think about the power of their research to make transformational change, when current strategies operate inside existing systems of power and inequality? After musing on these questions, the author offer a few suggestions for remaking our research to foster systems transformation.
For the master’s tools will never dismantle the master’s house. They may allow us temporarily to beat him at his own game, but they will never enable us to bring about genuine change.
“What,” Adam Gamoran asks in the editor’s essay that animates the theme of Socius’s special collection, “has been sociology’s role in responding to inequality?” His answer? “By one reckoning, not much.” To be fair, Gamoran goes on to say that sociology—and the social sciences more generally—have contributed enormously to our understanding that inequalities exist, to what level, where, and for whom. Sociology and other social sciences have also made clear that the impact of inequality—racial/ethnic, socioeconomic, gendered, ability-based, religious, linguistic, or intersections of these and many other forms of inequality—has powerful and indeed tragic effects for people worldwide. In the face of increasing inequality (Gamoran 2022), many social scientists have begun to wonder whether research on inequality can contribute to meaningful change. The original articles (by DiPrete and Fox-Williams 2022 and Nalani, Yoshikawa, and Carter 2022) and the commentaries that follow pose the question of what social scientists can do differently for research on inequalities to make a dent in the problem.
Several scholars of sociology have ruminated on, and offered antidotes to, this problem throughout history, as DiPrete and Fox-Williams (2022) make clear in a thorough and thoughtful analysis of the dominant literature in sociology. Drawing from that literature, they argue that sociological research needs to do more than describe and analyze the nature of inequality to shift the way decision makers think about the root causes of inequality. Instead, they argue, social scientists should construct research that assesses the feasibility of strategies designed to redress the root causes as documented in existing descriptive and analytic studies.
Nalani et al. (2022) take a different route by examining six pathways in which some, often small, change has occurred, and they posit that because inequalities derive from numerous systems-level inequities, research to reduce inequalities must be multidimensional. Some of the pathways focus on the conduct of research, but others are more of what one might refer to as public scholarship, in which a scholar uses research to advance change (e.g., legal cases, activist projects).
Both arguments are compelling and thought provoking. Both provide useful ways forward for social scientists interested in crafting research with the goal of reducing or redressing inequalities. Both raise for me three fundamental questions. First, does the field truly understand enough about the causes of inequalities to be sure that the solutions generated are appropriate? Perhaps the struggle to reduce inequalities is, in fact, the result of partial or flawed understandings. Second, assuming that there is enough—or accurate—understanding of the sources of inequality, what does it mean to assess the feasibility of solutions within a system that operates on the assumption that inequalities are part and parcel of its economic and social systems? Third, if radical reconstruction—systems transformation—is the answer, then is the right kind of research being done? That is, how do social scientists think about the power of either feasibility studies or “strategic levers” (Nalani et al. 2022:000) to make transformational change, when both sets of strategies operate inside current systems of power and inequality? This final question is thorny and invokes the Lorde quotation I used to open this commentary: How do we rebuild the master’s house if we are only using the master’s (research) tools? Lorde was referring to transforming a world that routinely subjugates women, but the overarching sentiment applies. Can we make change inside a system that relies on inequalities? In what follows, I begin with the question of understanding and conclude by suggesting approaches to systems transformation research.
Do We Really Understand?
In making the call for more feasibility research, DiPrete and Fox-Williams offer a range of compelling examples that would appear to advance efforts to reduce inequalities. At the same time, however, the call for more feasibility research as opposed to research in the service of better understanding inequities overlooks some important considerations in social science research, some of which are brought to light in the second article, by Nalani et al. A primary question that stands out for me when considering the value of understanding-focused (or frame-shifting, in DiPrete and Fox-Williams’s lexicon) research is, Do we really understand? Are the calls for frame shifting based on all the evidence social scientists can ascertain about particular social phenomena? Before social scientists move away from descriptive and analytical research designed to shift frames of mind about why inequalities persist, we need to be certain that we have sufficiently understood the sources of inequalities. As DiPrete and Fox-Williams outline, many sociological theories inform frame shifting, but whose theories are they?
Specifically, were the people who experienced the inequalities consulted about the questions that need to be studied? Equally important, who answered the questions, and how were they asked and answered? What frame is getting shifted and by whom? In most of the examples provided by DiPrete and Fox-Williams, the frame shifting was being done by sociologists and economists, groups that, although changing, remain predominantly white and upper middle class. The methods used in the studies cited were, by and large, statistical. To what extent were the populations being studied involved in the design of the research? To what extent were their voices centered (Doucet 2019)? What other kinds of methods could have fleshed out or expanded the understandings ascertained in those studies?
I can think of any number of examples to illustrate this problem, but an example from education reform may be useful. Allensworth et al. (2009) documented a flawed attempt both to shift an education inequality frame and to reduce education inequalities. In 1997 Chicago Public Schools drew from research that documented a relationship between college-going and enrollment in high school college preparatory courses. Previous analyses indicated that the more courses students took, the more likely they would be to attend college, countering the belief (shifting the frame) at the time that the best path to equity was not to force students who were uninterested in college to pursue college-preparatory courses and instead to provide youth with choices in their educational trajectories.
Drawing from findings suggesting that a more constrained and “rigorous” curriculum would increase student learning, the district mandated that all ninth grade students in Chicago Public Schools, regardless of background or preparation, would enroll in college-preparatory algebra and language arts courses. As documented in the analysis, the district policy did not end inequalities, especially in mathematics instruction; if anything, the inequalities were exacerbated, with failure rates increasing and grades dropping for the most vulnerable students. Longer term outcomes indicated that students of interest did not continue to enroll in mathematics courses, the policy had no influence on graduation rates, and final high school grade point averages dropped slightly for all but the lowest achieving students. The authors concluded at the end of the study that “extant research was limited in its application to a universal mandate” (Allensworth et al. 2009:382) and that the quality of the instructional context did not shift with the policy. Research shifted the frame, and a policy was enacted, but both were done without the perspectives of teachers, students, and other school personnel who might have raised concerns about the mechanisms of the policy. The policy resulting from the frame-shifting research was flawed, as was the frame shift itself, because the research on which a particular understanding had been based was incomplete, especially with regard to the voices of key players.
This type of policy misstep is well documented throughout the history of the social sciences in any number of attempts to reduce a range of inequalities, from race- and ethnicity-based inequalities to socioeconomic to ability-based inequalities, largely because, I would argue, researchers are predominantly members of dominant groups and have failed to acknowledge the experiences and perspectives of people being studied. Indeed, researchers often resist involving the people being studied, on the assumption that objectivity results from distance, when in reality it obscures clear vision. As a group, social science researchers do not always have the experiences from which to draw to ask all the necessary questions to understand phenomena from multiple perspectives or frames. Much frame-shifting research, despite earnest attempts to explain inequalities in ways that reframe deficit narratives of historically marginalized groups, nevertheless remains rooted in a white, upper-middle-class view of the world, precisely because social science researchers are predominantly white and upper middle class. Another way to think of this is to call upon the wisdom of bell hooks (2000), who reminded us that those at the center cannot see their own privilege and likely cannot see the margins adequately. Even with advances in social science and even with attempts to diversify the ranks of social scientists, social science researchers remain at the center, and descriptive or analytic research does not reach all the way to the margins.
Is Feasibility an Answer?
The call for feasibility studies is similar to the call for design-based research in education studies, wherein researchers and teachers work together to design and test curriculum or pedagogy for feasibility and usability in actual classrooms—without controls—in multiple iterations until the affordances are fully developed and the constraints managed. The power of such studies, whether labeled feasibility or design studies, is that the research could document why, when, and how a strategy does not work. The subtle difference between feasibility and design-based research is that feasibility studies tend to look for whether a strategy or tool will work, whereas design-based studies try to iterate on the strategy or tool to account for the vagaries of the context or system in which the tool is situated. Either way, when done well, this kind of research relies heavily on the voices, experiences, and expertise of those who engage in the practice that researchers are attempting to improve. Indeed, the practitioners are part of the research team, rather than just being subjects of a feasibility or design study.
Design-based or feasibility studies can also eschew the concept of “fidelity” to the intervention in favor of understanding how, when, and why practitioners feel the need to deviate from an intervention’s procedures because of setting constraints, the needs of those being served, or the sense that the intervention is not living up to its goals. Such research assumes that the expertise of the practitioner can inform the field’s understanding of why a solution does or does not work, rather than trying to control the practitioner’s moves. In education research, for example, this kind of work pushes against so-called teacher-proof curricular designs, toward designing with the challenges teachers face in mind, together with the needs of their students. Powerful feasibility or design research, thus, should assume a kind of partnership with those enacting a given intervention and sets up the stance that both researcher and practitioner can learn from the result.
When considering feasibility studies as an alternative to descriptive or analytic studies designed to shift frames, however, I found myself wondering if feasibility is enough, despite believing that feasibility studies are a preferred alternative to impact studies alone, which generally reveal whether, and not why, an impact occurs. As I read DiPrete and Fox-Williams’s argument about feasibility research, my first reaction was that this is an idea that can truly push the field forward. But how far forward? If feasibility is determined in terms of the affordability of a solution, then can we really expect to reduce inequalities? The realist in me appreciates the question of economic feasibility, but I see it as substantively different from the question of whether practitioners could feasibly enact a strategy if given adequate financial resources.
Another challenge to consider in advocating for feasibility studies is that the feasibility of an approach is rarely a matter of only one individual’s practice or a single setting or context. The feasibility of an approach is mired in multiple and intersecting systems. Teachers attempting to enact well-evidenced curricula, for example, can see their best efforts undone if other challenging elements of the system—such as lack of adequate time for instruction, chronic absenteeism, lack of nutritional food available in the community, or weak health and human services—remain stable. Even more insidious, but still a function of the system, rather than of the treatment being tested, is the fact that practitioners with differing levels of preparation will respond differently to a given intervention. If the system does not change to ensure adequate preparation, then an intervention could be rendered infeasible when the real problem is that another treatment—appropriate preparation—is not simultaneously enacted. In other words, feasibility studies need to take into account not only multiple factors, but also multiple systems of change, including time.
The need to examine multiple and interlocking systems is well documented in several of the Moving to Opportunity (MTO) studies (Gennetian et al. 2012; Ludwig et al. 2013), in which families found themselves better off economically as a result of moving to better housing in safer neighborhoods but saw mixed impact, and in some cases negative impact, on a range of other factors. Specifically, MTO families saw little to negative impact on their children’s long-term school achievement, especially in reading and mathematics, which are predictors of later school success (Leventhal, Fauth, and Brooks-Gunn 2005; Ludwig et al. 2013; Sanbonmatsu et al. 2006). Qualitative studies, however, suggest why school achievement effects were mixed at best, especially for older children, demonstrating that the schools their children attended in new neighborhoods were not necessarily of better quality for the children of the MTO families (Leventhal et al. 2005). In other words, moving to lower poverty neighborhoods reaps benefits in certain social systems (e.g., access to nutritious food networks, safety systems, and physical and mental health resources, as well as lower crime), but it does not solve all systems-level challenges and may even introduce greater racial discrimination, profiling, and tracking of youth both in and out of school, especially for boys (Clampet-Lundquist et al. 2011; Leventhal et al. 2005). That said, more recent studies (Chetty, Hendren, and Katz 2016) have shown positive effects on young children’s later college-going (with college defined as any postsecondary activity), dependent on the age children were at their move to new housing (as well as on the types of vouchers the families received). MTO children who moved before the age of 13 showed positive effects on college-going, whereas MTO children who moved after the age of 13 were significantly less likely than their control group peers to attend any postsecondary experience. The positive effects are not necessarily visible in earlier and traditional studies of school achievement, again suggesting that reducing inequalities requires studying across multiple systems. The negative effects for older youth—combined with qualitative analyses of the experiences of Black males, in particular—suggest as well that intersectional racial and gendered inequalities across multiple systems (schools, law enforcement, and neighborhoods) may not be countered by reducing economic inequalities in one system alone. Feasibility studies thus need to contend with complex questions of effectiveness of a reform for whom, when, under what conditions, and why to understand the true feasibility of a given approach.
These points take me to the central concern of my commentary: I am not convinced that the answer lies in choosing feasibility studies over descriptive or analytic studies. Should the social sciences, as Gamoran asks in his editorial essay, and as Nalani et al. suggest, aim for radical reconstruction? What if, in trying to reduce inequalities, social scientists are targeting the wrong goal? What would happen if social scientists asked how to transform systems of oppression rather than how to reduce inequalities? This is a semantic difference, but words matter. The language we use signals and frames what people, including researchers, assume about the world. To focus on reduction rather than transformation assumes that some level of inequality must exist. To focus on inequalities, rather than the systems that produce them, assumes that research designs should work on diminishing symptoms rather than causes. Even to focus on systems without naming them as systems of inequality, particularly race-based inequality, avoids facing the demons that stand in the way of reducing inequalities.
If system transformation, or radical reconstruction, is the goal, then social scientists have a great deal of work to do, because such a goal requires the field to change in significant—radical—ways. Are social scientists up to the task? Some might argue that the goal is to transform systems is monumental, perhaps even utopian to the point of naiveté. But if that is the assumption, then the field is locked in a system based on inequality, and social science research will never achieve the goal of even reducing inequalities. If the social sciences cannot lead the way on how research can transform systems, then what field can?
Remaking Our Tools: Toward Systems Transformation
A radical research goal requires the field to consider radical questions of research transformation. First, is research implicated at all? Can social scientists transform through research, especially research that has been taught, practiced, and conducted within the system one aims to transform (see Lorde, quoted previously, on this point)? I believe that research can make a systems-level difference but that social scientists must take a step back and reconsider the methods used, the voices privileged, and the practices for teaching (or learning from) a new generation of scholars. Nalani et al. point the way for systems transformation by considering multiple pathways and by offering several suggestions for changing our practices, including centering the questions that emanate from lived experience, especially the lived experience of people who have been historically marginalized. Nalani et al. also underscore the importance of naming and embracing our own biases rather than claiming to set them aside through methods and discourses deemed objective (cf. Doucet 2019)
Second, how do research questions change? Here, I want to underscore a focus on the questions, not just the methods, to which I will turn next. Transformation is different from reduction. Systems are different from the individuals affected by inequalities. Researchers need to frame questions to focus on the systems themselves rather than the inequalities that systems produce. Researchers should write questions to consider intersectionality, across a range of dimensions, more fully. For example, separating distributional and relational inequalities for analytic purposes is worthwhile (Nalani et al. 2022), but the separation in practice does not serve the goal of system transformation. Instead, social scientists should look at the systems-level relationship between these two types of inequalities, zeroing in on how they feed each other and documenting the myriad ways that inequalities in one sector maintain and even reproduce inequalities in other societal sectors. Social scientists need to study how discourse practices, in particular, allow researchers, even those who want to transform systems, to participate in practices that reproduce inequalities.
Third, how do research methods change? If we refocus on transforming systems, then what would our stated outcomes be? How would our measurements change? In particular, if we really want to think about making change, then we need to change multiple parts of the systems, as well as multiple interlocking systems. Are our current research methods up to the task? We can certainly document change using qualitative methods, but those methods demand a small(ish) scale to allow deep dives into studying mechanisms. Social scientists can engage in more systems-level design-based research, but design-based work is time and labor intensive. Documenting affordance and challenges to the feasibility and usability of a solution is time consuming, and that is only part of design work. One then must tweak and iterate. Typically, multiple iterations are necessary before one can pronounce a solution feasible and usable for a given context. And the solution is limited to that context until it can be translated and examined in a new context.
The question of transforming at scale is the most perplexing. Design research typically happens without an impact analysis that gives confidence that the solution produced the desired change. What if the change is due to cohort effects? The skill of the leader, teacher, or facilitator? For many, designs that allow causal inference are the coin of the realm when thinking about impact and scale, but causal inference methods have their own challenges in the quest to transform systems. Causal inference methods require the researcher to home in on certain aspects of a design to test its efficacy, thus controlling everything else as noise. The challenge with this approach is that inequalities are often a function of the noise being controlled. The idea of controlling for race, for example, is antithetical to how race works in society (Doucet 2019). Racial identifications shape both how people move through various systems and how they are positioned within those systems. To search for strategies that “work” regardless of race overlooks the way the system sets up racial identities (or other ways people identify and are identified) as unequal. Solutions whose impacts are assessed by controlling for the very quality that produces inequalities are unlikely to reduce inequalities and are even more unlikely to transform systems of inequality.
As both DiPrete and Fox-Williams and Nalani et al. illustrate in their analyses, the work is highly complex, whether we think of the goal as reducing inequalities or transforming systems. If we build system-transforming interventions, then how do we separate dimensions for analysis? Social scientists need to work together to design methods that examine the effect of the intersection of dimensions that shape solutions, which is not captured by parsing or controlling for them. One suggestion is to better integrate multiple methods (not just “mix” methods), but that requires new—more collaborative, more practice based, and more integrated—training of future researchers (see Nalani et al.). If the reality is that change, even radical change, in one dimension of a system will be swamped by lack of change in another, but our only methods for robustly assessing effects is to parse and control different dimensions, then how do social scientists study the transformation of systems?
DiPrete and Fox-Williams might respond to this question with the idea of studying the feasibility of a given solution. I agree that in current social systems, feasibility studies—or design studies, from my vantage point—are the best hope. However, as valuable as feasibility studies might be within our social systems as currently defined, how do feasibility studies make change if social scientists must use the frames and work inside the systems that allow inequalities to occur in the first place to evaluate the feasibility of our designs? What is feasibility in systems transformation? This is a critical point, because a practice or policy that is deemed infeasible might be the very practice that would transform the system, if the system features that prevented the practice from succeeding were what was considered infeasible, rather than the practice itself. To examine the feasibility of a practice within a given system is to accept the system as good or appropriate or valid. A teacher who struggles to enact a curriculum intended to provide children the space to learn at their own pace likely finds the curriculum infeasible because the system is designed to provide a low-cost education to children at scale, which requires that children and teacher move through a curriculum en masse and at a set pace, regardless of whether learning is happening. An inquiry-rich, socially just curriculum that takes too much time will be infeasible in such a system, regardless of what research says about how people learn best. Similarly, a curricular solution that demands costly equipment is infeasible in a system that distributes resources inequitably.
If, however, the goal is to transform systems rather than to reduce inequalities, then feasibility takes on a different meaning. Feasibility of a solution from a systems transformation approach means that researchers would assess a solution in terms of its ability to disrupt, not to work within, a given set of structures. Feasibility would take aim at the systems practices and policies that enable or constrain research-based interventions. Feasibility’s focus would be on the structures of systems, rather than on individual actors, texts, or activities.
In sum, if the goal is to reduce inequalities, then social scientists should aim for transforming systems that shape opportunity. To transform systems, social science needs to transform research questions, designs, and methods of measurement and analysis. Social scientists need to transform how we draw conclusions and make claims. Researchers need to transform translation of claims into action, with a focus on systems-level structures, practices, and discourses. To do so will require new ways of working, which at base should include the strategies suggested by Nalani et al. (i.e., act on the basis of person experiences and interrogate uses of evidence; integrate research synthesis, communication, and research-policy-practice partnerships; and shift training of future researchers). To that list, I would add the following:
Diversify the ranks of social scientists, by recruiting to the field and by engaging community members in participatory research. In other words, ask the people who have had experiences with inequalities to participate as coresearchers at every step of the research design. This approach can achieve two goals: it provides insight to researchers currently in the field, and it potentially recruits a more diverse pool of future researchers to the field.
Seek to understand researcher positionality in all research designs and methods, with an eye toward questioning one’s assumptions about the constructs under study so that designs do not overlook key variables and to ensure that instruments are designed, and analyses conducted, with attention to multiple perspectives on the constructs.
Deliberately seek diverse perspectives on the questions asked, the methods used, the claims made, and the solutions that result. Form collaborative teams that draw from methods beyond the social sciences. To do this work, one needs to resist talking past colleagues from other disciplines and prizing the discourse internalized in graduate education. Learn to speak across disciplinary discourse communities and in so doing, learn to question assumptions embedded in prized disciplinary practices.
Use integrated methods. Here I suggest that moving beyond the language of “mixed” methods and taking seriously the aim to integrate findings so that researchers can delineate mechanisms even as impact is being evaluated. Eschew studies that claim to mix methods by throwing in a few interviews to an impact analysis. Instead, seek ways to use qualitative findings to inform assessments of impact and vice versa. Design studies to be iterative and longitudinal, with cycles of qualitative and quantitative analyses, always targeted on reforming systems structures, practices, and discourses.
Consider how to attend to, rather than control for, key variables. Instead of trying to assess whether strategies work regardless of race, for example, design studies to assess whether strategies work because of race. Center identities associated with unequal and inequitable treatment, rather than attempting to control identities to find the transformational strategies that work for “everyone,” which is, in effect, code for the people already at the center.
Examine the feasibility of interventions across multiple systems; to do this, researchers need to spend time understanding how inequalities are embedded in, and fostered, across multiple systems, which implicates long-term, embedded studies design to understand, but for the purpose of transforming multiple systems-level structures, practices, and discourses.
Craft feasibility studies with transformation of systems, not people and their practices, in mind. That is, aim to break down the structural barriers that make desired practices infeasible within current social systems. Think of feasibility for transformation, rather than feasibility within the system as currently structured. Consider the value of systems-focused, design-based research methods that go beyond a base determination of whether an intervention is feasible and instead focus on how to change the system to embrace and enable the design.
None of the actions I suggest here is easy. In fact, these are incredibly challenging actions that will require intentional and innovative reinvention of the way we do social science work. Each suggestion implicates funders and funding structures, which should require collaborative teams, long-term designs, and community involvement as part of competitive study designs.
To conclude, I return to Gamoran’s question: What is the work of social scientists who want to make the world a more equitable and just place? Should we seek to understand, to shift frames, to test the feasibility of interventions, or to radically reconstruct the social world as we know it? I argue that social scientists need to do all of these things, and to do that, we need to change our discourse to focus on transforming the systems of power and oppression that allow inequalities to exist. If we change our aim and adjust our research practices to focus our aim, then we might hit the target we seek.
