Policymaking Pragmatics: What’s a Qualitative Researcher—Especially a Critical Qualitative Researcher—to Do?

Abstract

Policymakers tend to be suspicious of qualitative research. Distrust of qualitative work is especially acute when a researcher openly embraces critical perspectives and is oriented toward critiquing privilege and providing the sort of knowledge that helps set the stage for challenging the status quo and creating more just and equitable social contexts. The first part of this article uses an adaptation of autoethnographic methodology and the personal essay genre to suggest how all types of qualitative researchers, including those whose research is informed by critical perspectives, can influence state policy. In the second part of the article, the focus shifts to the local level. This part of the article describes a process designed to promote dialogue across differences and generate at least a modicum of consensus about policies that should be adopted and implemented within an organization.

The emergence of critical race theory as a wedge issue in Virginia’s 2021 gubernatorial election, as well as in other recent state and local elections across the United States, highlights the resistance researchers who employ critical perspectives are likely to confront when they enter policymaking arenas. The problems are especially acute for critical qualitative researchers, given recent trends that situate qualitative data at the bottom of the scientific credibility scale (Fielding, 2020). This article describes actions critical and other qualitative researchers can take at both the state and local levels to circumvent policymakers’ biases.

How to Win Friends and Influence State Policymakers (Even If You Are a Critical Qualitative Researcher)

Insights From Early Work

Several years ago, I published an article in a special issue of Qualitative Inquiry that challenged policymakers’ and funders’ push to simplify qualitative research findings (Donmoyer, 2012b). In the call for papers, the special issue’s guest editors had written that simplification is “driven by policy and funding agencies” (Koro-Ljungberg & Mazzei, 2010, p. 1) and declared that policymakers’ and funders’ demand for simplified narratives “can lead to over-generalizations of findings, decontextualization of the research project, and claims that may actually disempower those the research purports to benefit” (Koro-Ljungberg & Mazzei, 2010, p. 1).

The paper I submitted detailed experiences I had had that supported the guest editors’ claims about policymakers’ and foundation funders’ pressure to simplify complex qualitative results. However, I also wrote the following:

My socialization as a qualitative researcher . . . makes me more than a little squeamish about the a priori judging implicit in this special issue. . . . [In short,] I have been influenced not only by the research methods of cultural anthropology but also by that field’s traditional assumptions. For example, rather than judging cultural practices, historically cultural anthropologists were socialized to assume that the practices they observe—even seemingly problematic practices such as leaving elderly grandparents alone on an Arctic ice flow to die of exposure—are functional within the cultural contexts in which they occur. . .

It seems important, therefore, that a special issue . . . that, by design, is intended to be critical of policymakers’ preference for simplification (and, presumably, also certain qualitative researchers’ willingness to accommodate this preference) include at least one article that attempts to bracket judgments about . . . policy and funding communities in the interest of understanding the very different cultures inhabited by policymakers, on the one hand, and most qualitative researchers, on the other. (Donmoyer, 2012b, p. 799)

Reviewers for the special issue seemed to accept this argument, and the special issue’s editors opted to publish the paper.

The bulk of that paper demonstrated, largely through describing interactions I had had with multiple policymakers in different states over many years, that simplification was highly functional (and, in fact, normally quite necessary) in policymaking arenas. I even wrote of a time when I briefly took off my researcher hat to move into a policymaking position. I noted that playing the policymaker role temporarily transformed even me—someone addicted to noticing and documenting complexity—into someone who realized he needed to “keep things simple” to act.

The final section of the 2012 paper provided a brief autoethnographic account of how someone “who was attracted to qualitative methods because they promised to accommodate the complexity of social life, has attempted to influence policymaker and the policymaking process without completely losing his methodological soul” (p. 798). What I wrote in this final section of the earlier paper provides a foundation for what I am writing here.

Basically, in the final section of the earlier paper, I indicated that “I realized early on that if politics is ‘the art of the possible,’ this characterization applies even more so to the process of influencing what is inevitably a politicized policymaking process” (Donmoyer, 2012b, p. 803). I also noted that because of inevitable differences in power when researchers operate in policymakers’ arenas, any qualitative researcher “who wants to influence policymakers must work within policymakers’ frames of reference and use policymakers’ language” (Donmoyer, 2012b, p. 803). I suggested, in short, that qualitative researchers needed to be translators “between the culture of qualitative [researchers] and policymakers’ culture” (Donmoyer, 2012b, p. 804).

Unfortunately, there was not enough space in the 2012 paper to unpack what being a translator entailed. I could not, for example, discuss the specialized problems critical qualitative researchers faced. This article provides an opportunity to build on the insights provided by the earlier paper and delve more deeply into what being a translator means for qualitative researchers, including those influenced by critical perspectives.

Explicating the Translator Role

Embrace Causation

One issue I did discuss in the 2012 paper, albeit quite briefly, was the need for all qualitative researchers to get over their reticence to characterize social phenomena in cause-and-effect terms. Early in the qualitative research movement outside of traditional qualitative fields like Anthropology, Lincoln and Guba (1985) argued that the social world is too complex to differentiate between causes and effects and, instead, was characterized by (and should be portrayed as a process of) “mutual simultaneous shaping.” Over the years, Lincoln and Guba’s thinking about causality has been challenged in a variety of ways (e.g., see Donmoyer, 2012a; Huberman & Miles, 1985; Schwandt, 1997; Thomas, 2021); however, much of the qualitative research community has continued to embrace it. Maxwell (2012) cites the evidence: “Despite some notable exceptions,” Maxwell wrote,

this rejection of causation has been a widespread and pervasive assumption within the qualitative community. . . The recently published two-volume Sage Encyclopedia of Qualitative Research (Given, 2008) had no entries for cause or causation, and the only non-dismissive mention of causation in the third edition of the Handbook of Qualitative Research (Denzin & Lincoln, 2005a) was in the chapter on qualitative evaluation and social policy (House, 2005), where the issue of causation is almost impossible to avoid. (There are no non-dismissive mentions of causation in the fourth edition of the Handbook; Denzin & Lincoln, 2011). The author of one of the finalist dissertations for the qualitative dissertation award given by the Qualitative Research Special Interest Group of the American Educational Research Association in 2011, Brianna Kennedy, confided to me that she felt that “even uttering the word ‘causal’ in qualitative methodology circles was akin to invoking evil spirits.” (pp. 655–656)

Even critical qualitative researchers often at least appear to have accepted Lincoln and Guba’s mutual simultaneous shaping view of social phenomena, even though critical theorists’ claims about hegemonic influences are, at base, causal claims.

Policymakers, by contrast, uncritically accept and engage in cause/effect thinking. They assume the policies they adopt will cause specific things to happen, and they expect researchers to provide evidence that either validates or challenges their assumptions. Consequently, I argued in the 2012 paper that qualitative researchers who hope to communicate with policymakers and influence their actions need to think of the notion of causation as, at the very least, a highly functional fiction, that is, as an admittedly oversimplified (and always probabilistic) short-hand description of what will occur if certain policies are adopted.

I did not use the quite limited space allotted to the final section of the 2012 paper to provide additional support for this recommendation. Here, let me provide at least a modicum of support by quoting from Peter Cohen (1968) paper, “The Very Idea of a Social Science.” Cohen’s paper critiqued an influential book, The Idea of a Social Science, by Peter Winch (2007, c. 2003, 1990, 1985). Winch’s book laid out a vision of the social world and social science research that is strikingly similar to the interpretivist perspective of many qualitative researchers. Like Lincoln and Guba (1985), for instance, Winch rejected cause-and-effect thinking in social science. In responding to Winch’s vision of causation-free social science, Cohen wrote,

One would have to agree that the use of the term “causation” does not have as precise a reference in the social world as it does in the physical world. [However,] if one is to use such criteria, one wonders what is to be offered in place of causation. . . . In fact, one begins to wonder how social policy would be possible without some idea of causation. (p. 416, emphasis added)

Develop Relationships

My attempts to influence state legislative actors over many years also have made me aware of the need to develop relationships with legislators and legislative staff, ideally prior to engaging in official or unofficial lobbying efforts to influence specific legislation. ¹ My experiences in two different states with two somewhat different legislatures suggest this is not difficult to do, although the strategies I employed to develop relationships were somewhat different in each state context.

In one state, I was able to develop relationships directly with legislators. ² As one legislator in this state told me in an interview about how ordinary individuals can influence state policies, “Members [of this state legislature] are very accessible; they want to be” (Fox & Donmoyer, 1990).

One way I began to develop relationships with legislators in this state was to invite legislators to speak at statewide conferences my university colleagues and I had organized on different policy issues. Some legislators even accepted invitations to come along to dinner after a day-long conference, which provided opportunities to develop a more social type of relationship. Social types of relationship building sometimes led to a legislator sending me drafts of bills the legislator planned to introduce, along with a request for comments.

The situation was quite different in another state. Undoubtedly, the difference had a great deal to do with the fact that, unlike the context I just described, the legislature in the other state was not just a 20-min bus ride from my place of employment. In fact, the state capital was in a different city than the city in which I worked. But even if this were not the case, I suspect I would have had to work differently in the second context. In this state, the legislature employed an experienced, exceedingly professional, and quite influential staff, and staff members’ influence had grown once term limits were adopted and legislators’ tenure in their positions was shortened. I quickly concluded, in this context, it was more effective to work with a staff member on issues, at least initially.

No matter the context, however, it was important to follow the guideline articulated in the 2012 paper that proposed working “within policymakers’ frames of reference and . . . [using] policymakers’ language.” Among other things, that meant eschewing abstract theoretical language and, instead, talking, in concrete terms, about what specific things legislators could do and the likely impact the legislation being discussed would have.

Timing and Positional Power

Two other nuts-and-bolts issues are illustrated by what happened when a state legislator, who I had met when he spoke at a conference my colleagues and I sponsored, sent me a bill he planned to introduce to review. The bill addressed what the legislator believed was the failure of schools to adequately teach reading to primary-grade students. Basically, the bill would have required that teachers take additional reading methods courses. I responded, “Do you really think more of the same will improve the situation?” I added, “If you want to address the reading problem, why not consider a one-on-one tutorial program some of my colleagues at the university have imported from New Zealand and are currently experimenting with in one school system in our state?”

The legislator sent a staff member to the university to study the program, was impressed by the fact that the program had outcome measures (which, interestingly, were not standardized test scores), and proceeded to include funding in the state’s new biennial budget to implement the program statewide. The legislator could do this for at least two reasons.

First, all of this happened as the state legislature was preparing to adopt the state’s biennial budget. Had our interactions happened even a month later, it is unlikely the legislator would have been able to do much unless he had a very good memory and attempted to add the funding proposal to the next biennial budget when it was considered 2 years later. (My experience has been that most legislators’ memories are quite short.)

Second, this legislator had positional power because he was a member of the legislature’s budget committee. Consequently, he was positioned to do what most legislators would not have been able to do, at least not so quickly or easily: add funding for the program to the budget bill, even though the state’s Department of Education opposed funding the program because it was not on the Department’s list of funding priorities. ³

What’s a Critical Qualitative Researcher to Do?

Let me turn, now, to the issue I alluded to at the start of this section of the article: dealing with critical perspectives in policy arenas that are not amenable to hearing, much less accommodating, critical perspectives. My specific recommendations on this matter were foreshadowed in the earlier guidelines: work within policymakers’ frames of reference and speak their language.

For example, I have never uttered the words hegemony or hegemonic when state policymakers were within earshot. I even have been exceedingly cautious about using a seemingly innocuous term like equalization with actors from the legislature. I know that, for some legislators, the term equalization conjures up imagery of a zero-sum game in which helping less successful (and less privileged) students automatically means taking resources away from already successful students (whose families may be among a legislator’s strongest supporters). That sort of tradeoff is not necessarily inevitable, of course, but rather than risk raising sometimes unacknowledged concerns in the minds of some policymakers, I long ago concluded it was better to simply talk about helping less successful students learn, which is something I found virtually all policymakers, no matter their ideology or political party affiliation, could support.

Similarly, I generally avoid characterizing anything as discrimination, even when I am convinced that discrimination has occurred or is occurring. Rather than using the label, I have found it more convincing to provide concise narratives, either verbally or in writing, of discriminatory practices in actual situations and their consequences for real people without attaching the discrimination label or engaging in theoretical editorializing of any sort. In other words, I let the qualitative data speak rather than speaking for the data.

The point is, one normally can make a case for a particular course of action in any number of ways, and there is no reason to choose a way that is virtually guaranteed to confuse and/or anger those one is trying to persuade. The good news for all kinds of qualitative researchers, including those guided by critical theory, is that despite most policymakers espoused preference for quantitative work, I have found that most policymakers can be convinced by telling a compelling story, assuming the storyteller eschews theoretical editorializing.

Several years ago, for example, I organized, for the governor’s office and a foundation funder, the development of a series of case studies about what the governor’s statewide reading initiative looked like in selected schools in different parts of the state. As anyone who knows schools would expect, the program looked quite different in different places. These differences, once documented in the case material, initially raised concerns among members of the governor’s staff who, at first, entertained the idea that the governor’s program needed more directives and regulations. As they re-read the cases, however, they began to see that each of the contexts studied differed from the other contexts that were studied and the programmatic differences that were observed were logical responses to contextual variation.

Of course, I could have inserted lots of literature and the theoretical thinking the literature articulates to make that very same point, but many policymakers would have seen this sort of action on my part as intrusive and, consequently, it is unlikely my theoretical editorializing would have been effective. Furthermore, theoretical editorializing was not necessary. If qualitative studies, including qualitative studies that are reflective of a critical sensibility, are well done, the data can make the important points; labels or other forms of theoretical abstraction are not needed.

A Final Point About Influencing Legislators

Before concluding the discussion of using qualitative data rather than theoretical abstraction to convince policymakers to act in certain ways, I should add that, when it was time to convince legislators rather than their staffs, narrative data needed to be translated into brief but telling anecdotes. Most politicians do not have the patience to read detailed case studies. They do respond, however, to oral briefings that include snippets of case narratives. A study I conducted at the start of my career demonstrated that one succinct story of a family with a special needs child was enough to convince the author of a particular piece of legislation to modify the bill he had introduced to allow school districts to carve out exceptions for special needs students (Donmoyer, 1982). No detailed case study was needed. In fact, had a detailed case been provided, it is unlikely the legislator would have read it.

A Local-Level Process to Promote Dialogue Across Differences and Facilitate Consensus About Policy Recommendations From Diverse Stakeholders

Let me now shift the focus to policymaking at the local level, a level that is “closer to home” and, consequently, should be less about influencing those selected to represent us than about having opportunities for direct involvement in policy decision-making. To promote this sort of direct involvement, at the start of my career I developed and explored a process designed to promote interaction and collaboration among different sorts of stakeholders, at least some of whom initially viewed the problem and likely solutions in very different ways. The goal, in short, was to promote what Burbules and Rice (1991), in their discussion of a possible response to the postmodernist thinking, characterized as “dialogue across differences.” In the process I developed, the goal of such dialogue is to generate, even in postmodern times, at least a modicum of consensus among an assembled group of representative stakeholders about what policies and/or practices an organization being focused on should adopt, eliminate, and/or modify.

The process I developed was not all that original. In fact, it was inspired by and extended Schwab’s (1969, 1971, 1973) discussions of deliberation in curriculum making; Schwab, himself, borrowed his notion of deliberation from Aristotle’s discussions of practical reason. Schwab (1969) defined deliberation as follows:

Deliberation is complex and arduous. It treats both end and means and must treat them as mutually determining one another. It must try to identify, with respect to both, what fact may be relevant. It must try to ascertain the relevant facts in the concrete case. It must try to identify the desiderata in the case. It must generate alternative solutions. It must make every effort to trace the branching pathways of consequences which may flow from each alternative and affect desiderata. It must then weigh alternatives and their costs and consequences against one another and choose, not the right alternative, for there is no such thing, but the best one. (pp. 20–21)

I have employed variations of this sort of deliberation process intermittently over the years in a variety of contexts and in a variety of ways. Here, however, my focus will be on my first attempt to implement the process through twice-monthly meetings in an elementary school with an assembled group of diverse stakeholders (four teachers, five parents, the school’s principal, the principal from another district school, and an administrator from the district office) over the course of one school year. I have opted to focus on this early work because my first federal grant provided funding not only to implement the process but also to assemble a team of two doctoral students to work with me to systematically study the implementation process through different forms of participant observation, ⁴ analyzing video recordings of all the group meetings, and pre- and postprocess interviews with all participants. Here, I will use the recently reanalyzed data generated from the study of the process for two purposes: (a) to demonstrate how a critical sensibility can be incorporated into the process and (b) to discuss potential problems that anyone attempting to implement similar sorts of processes should be prepared to confront. Before doing this, however, I need to provide some additional contextual information about the setting in which the deliberation process was initially implemented.

The Context

The school in which the process was first implemented was, in many respects, an ideal site in which to explore the possibilities and pitfalls of engaging in a collective process of critique through a process of deliberation. The school’s teachers almost all had BA or MA degrees from a nearby university, and that university had effectively socialized them to avoid engaging in direct instruction and, instead, teach informally and indirectly. In short, the school’s teachers, to a greater or lesser extent, all believed their students should take charge of their own learning and that a teacher’s role was to provide scaffolding that helps ensure that what students choose to do will be beneficial to them. Even new teachers who were educated elsewhere were socialized by their peers to accept this indirect and informal approach to teaching and learning.

A clear indicator of the teachers’ pedagogical commitments appeared early in the process when the group was still deciding on the focus for its work. One parent suggested the group focus on discipline. Teachers cavalierly dismissed the suggestion by claiming, rather audaciously, that they did not believe in discipline. When parents, at a subsequent meeting, restated the question they proposed studying with more academic-sounding language—“What type of reinforcement and maintenance is established for a child in particular areas?”—the parents’ request was just as quickly dismissed: “‘Reinforcement’ and ‘maintenance’ are too behavioristic for me,” one teacher declared. That comment, effectively, ended the conversation about classroom and school discipline as a focus for our work together before it had barely begun.

The members of the research team were perplexed by why teachers so cavalierly dismissed the idea that discipline mattered in schools, as opposed to taking a more moderate stance such as rejecting certain approaches to discipline and/or promoting other approaches that emphasized developing students’ self-discipline. Telling parents that the teachers in their child’s school do not believe in discipline seemed to be rather risky business. Furthermore, this was not the only time teachers cavalierly rejected what seemed to the research team to be reasonable requests from parents.

What eventually became obvious to research team members was that, for at least some of the school’s teachers, symbolism often trumped substance. For this group of teachers, for example, organizing the curriculum into thematic units through the process they called webbing was not just a way to organize the curriculum. Thematic units and the “ritual” of webbing reinforced teachers’ belief that their school was “uniquely different.” Teachers articulated the uniquely different claim frequently throughout the nearly year-long process in which we were engaged, even though they often had difficulty articulating what made the school different:

Teacher 1: I think it [the school’s program] is different, and . . . [the school] is different, and I think that any parent within the community and any teacher can tell you that it is different and that, because our program is not [pause] it’s not a program that [pause] it’s different and it’s unique. . . . I’m sure of that. I’m just so sure.

Other stakeholders who participated in the deliberation process were not as convinced as the teachers about the school’s uniqueness. During an interview after the study at the center of the study-of-the-study project had been completed, for instance, the central office administrator who had participated in the deliberation process told the research team,

Too many times they [the teachers] wanted to talk about “how we are different,” but they were not able to define what it was that was a difference. And with that in mind, I think it was frustrating to parents and to anyone else, because if you don’t know what it is you’re trying to say you are, then how do you know what it is?

And this is what the participant who was a principal of a different district school said during her postevaluation interview:

I just had a feeling, I don’t know if it’s due to my experience or what, but all the issues that were being raised are issues that could have been raised in any building . . . and I’m not so sure some of those people could see that. A few issues were unique to . . . [the school], but not all of them.

Because the school’s teachers, for the most part, did what they did very well, most parents ultimately were relatively satisfied with the results the school produced, but interviews with parents who had volunteered to participate in the collaborative policy analysis project demonstrated quite clearly that this was not the type of school many of the school’s largely working-class parents would initially have chosen for their children. The district, also, was heading in a different direction at the time of the study: It had endorsed and was promoting an approach to teaching and learning that emphasized direct instruction and the carefully organized, step-by-step mastery of basic skills. All of this made the school an ideal site for watching a deliberation process engaged in by diverse stakeholders unfold over time. One question to be addressed in this article is how much of a critical sensibility can be infused into a process that is occurring in a place where teachers’ ability to cavalierly dismiss parents’ concerns about school discipline suggests the existence of significant power differentials.

Infusing a Critical Sensibility Into a Deliberation Process

As it turned out, as the process unfolded over time, those of us who facilitated the process got much better at forcing everyone to listen to ideas and not cavalierly dismiss an idea before it got a fair hearing. In addition, there were times when teachers held the trump card.

Once the group decided to focus its work on critiquing the school’s approach to reading and language arts, for example, parents had the school’s standardized test scores as exhibits to introduce into the process. Teachers still objected to the use of the scores to measure the school’s effectiveness in the reading and language arts areas. They argued that standardized tests did not really measure what was important in the reading and language arts domains. ⁵ The scores, however, could hardly be ignored.

This parent/teacher standoff was resolved when the facilitators suggested the group not only look at test results but also examine the standardized tests the school used to measure achievement in the areas of reading and language arts. Reading test manual discussions of the psychometric properties of the tests in question turned out not to be appropriate for any of the stakeholders engaged in the deliberation process, but members of the group did closely examine the tests and actual test items. Most group members concluded the test measured some of what they meant by reading and language skills, but even most parents concluded that the scores, in and of themselves, were not an adequate measure with which to judge the school’s teaching of reading and language arts. Some parents, after looking at the tests, even began to wonder whether the tests measured the ability to sit quietly and take a test more than the ability to read and/or write.

Even when teachers initially appeared to have the advantage, their influence could still be somewhat limited. Consider their ability to reference academic theories in support of what was happening in the school. The data, for instance, suggest teachers frequently invoked theory and the research associated with it to support the arguments they articulated and the positions they took on issues. The data also demonstrate that parents, and even the administrators who participated in the process, were not aware of the theories and research the teachers cited and did not have knowledge of rival theoretical perspectives they could use to bolster their positions. Most, however, were not intimidated by the teachers’ citation of theory and research and often challenged the teachers’ position with quite convincing arguments rooted in their experiences and/or commonsense thinking and what could be characterized as folk theories.

On one occasion, for example, a parent challenged the teachers’ assertion that theory and research supported the school’s practice of accepting rather than correcting students’ invented spelling, that is, the spelling of a word that, at best, only approximated a word’s conventional spelling. The parent’s response was balanced and sensible. She indicated she did not disagree with the idea of accepting misspelling when students are in the early grades and just learning to read and write. But by fourth and fifth grade, she declared, teachers should expect students to spell words correctly and correct misspelling if it occurs, so students learn the correct spelling of words before the students move on to middle school. In short, this parent needed no formal theory to make a claim that had considerable face validity.

Another example of parents being able to “hold their own” without being able to reference a formal theory occurred early in the process before the group decided to focus on the teaching of reading and language arts. A parent suggested the group’s work should focus on what the parent called curriculum continuity. What this parent meant by curriculum continuity became clear when the parent noted his son’s teacher for both second and third grade had not taught cursive writing or encouraged the memorization of multiplication “facts,” even though teachers in other second and third classrooms had done so. This put his son at a decided disadvantage when his son entered a fourth- and fifth-grade combination classroom, the parent noted.

Teachers attempted to counter the parent’s curriculum-continuity concern by invoking a theory about individual differences and using that theory to justify teacher autonomy. In this situation, the teachers’ invocation of an academic theory proved completely unconvincing to other members of the group. The story of a child being moved onto fourth grade without being taught the same crucial skills students in other classes had been taught was no match for a pieced-together defense of teacher autonomy supported by a formal theory of individual differences. Folk wisdom and common sense, once again, trumped formal academic theory.

On those rare occasions when teachers were not able to tell compelling stories and/or cite what could be considered folk theories or folk wisdom to challenge teachers’ recitation of theories from the academy, I, in my facilitator role, would check with content-area experts at the nearby university. I reported what I had learned at a subsequent meeting of the group. To be sure, the delay in responding took some of the sting out of the rebuttal, but at least those with less command of the theoretical knowledge from the academy were not at a complete disadvantage. Facilitators also, at times, pointed out during discussions that not all researchers agreed with a theory the teachers had cited.

To summarize, when it came to the citation of theory and research to support claims that were being made during the process, the playing field was not completely level. However, there were ways to ensure that those who lacked academic knowledge were not at an overwhelming disadvantage. In short, as was the case with the review of standardized test data during the process, the evidence that was analyzed (and recently reanalyzed) suggested a process designed to promote dialogue across differences was viable.

Two Cautionary Tales

Not all the evidence about the minimization of power differences in the deliberation process was positive, however. Embedded within the data are at least two cautionary tales for anyone interested in facilitating a similar sort of deliberation process.

Cautionary Tale 1: Style Can Trump Substance

One of the main characters in the first cautionary tale was the parent who had raised some of the most thoughtful points during the process, including the idea that there should be grade-level limits on accepting invented spelling. During her final interview, she told us she went home after most group meetings and cried, largely because she had not gone to college and, consequently, other group participants dismissed her ideas. She did note that I, in my facilitator role, on several occasions called her ideas sophisticated, but she claimed nobody else seemed to agree with that comment.

I did call her ideas sophisticated because they were. Unfortunately, her way of expressing them was not, at least not in the setting in which the study occurred. In short, style at times trumped substance and most participants could not “hear” what this parent was saying much of the time, even though what she had to say was exceedingly valuable.

The opposite happened with another parent in the group, a woman who held a relatively high-level position in the state’s bureaucracy. Like most of the teachers in the school who were married to attorneys or other high-paid professionals, this parent could have been coded as upper middle class. Consequently, she seemed different from many of the other parents, most of whom probably would have been classified as working class. The school staff respected and very much liked the upper-class parent and assumed they all were kindred spirits.

If one listened to what this parent had to say during our meetings, however, it was obvious she was not overly enthusiastic about what was happening in the school. Teachers and even the school’s principal could not seem to hear her critiques, however.

We mentioned what we had heard this parent say at different points in the process to the school principal in our end-of-project interview with him. He was mystified and disagreed with our characterization of this parent’s view of the school. Approximately a year later, however, the school’s principal and I attended the same social gathering, and he sought me out to tell me we were right: This parent definitely was a critic of the school; her actions had made that obvious during the year since our study had concluded. The principal could not figure out why he and his teachers had not viewed her as a critic rather than a supporter during the deliberation process, even though her critical comments were obvious to all members of the research team.

Cautionary Tale 2: The Intrusion of Public Relations Concerns

Above, I alluded to the use of thematic units and a process referred to as webbing in the school. In fact, I characterized it as a kind of ritual with symbolic as well as practical significance. In an early stage of the process, that is, before the group had settled on its reading/language arts focus, Parent 1 questioned whether the school taught the objectives articulated by the district. In response, two teachers volunteered to provide a demonstration of how they used the district’s articulated social studies objectives in planning a thematic unit called “Flying Things” through the webbing ⁶ process. The presentation was impressive and quite convincing. In an interview at the end of the study, however, one of the presenters put what she and her colleague had done in perspective:

• Teacher 2: Very honestly, that was an after-the-fact. . . . I didn’t have those social studies objectives in mind when I did that unit. . . . I had my own goals and rationale for doing it. . . . That’s what I had thought through, but sticking those [district curriculum] social studies objectives in was an afterthought.

• Interviewer: It was a mechanism to look good?

• Teacher 2: What was on the paper? Yes.

• Interviewer: You trucked this out for parents?

• Teacher 2: Yeah. Yes.

• Interviewer: It was a public relations ploy?

• Teacher 2: Right.

Interestingly, another teacher, during her end-of-study interview when there was no longer a need to mount a public relations campaign to defend what was happening in the school, told us the following:

I thought . . . [Parent 1’s] issues were valid. I think he had some legitimate concerns that people were afraid to address. . . . When they [early grade students] get to fifth grade [the grade this teacher taught], I resent having to teach them cursive writing. . . . I resent when they seemingly have no idea about what a noun and verb is. . . . I feel I have two things to do. . . . Not only do I have to foster the creativity [of students], which I feel strongly about doing, but also I have to give a heavy dose of mechanics.

This teacher said absolutely nothing like that publicly during the almost year-long process in which we had collectively engaged. In fact, no teacher ever publicly disagreed with another teacher during the process. As one of the school’s teachers told us ex post facto: “People are so strong that, if you disagree, there isn’t a lot of tolerance.”

Final Comments About the Deliberation Process

Because of the difficulties discussed above, it took an inordinate amount of time to reach agreement even about seemingly uncontroversial matters. For example, we did not finally agree on the decision to focus on the school’s approach to the teaching of reading and language arts until the process was almost half over. On at least one occasion, we ended a meeting believing agreement about the project’s focus had been reached, only to discover the agreement had evaporated by the next meeting of the group.

By the time a consensus finally was reached to focus our deliberation on the school’s approach to teaching reading and language arts, much of the initial enthusiasm for the project had begun to wane, at least a bit. Indeed, by the time flu season hit during the coldest winter months, the teachers involved in the project were no longer talking about making similar sorts of opportunities for dialogue with parents yearly occurrences in their school, as they had done during the first months of the project. To be sure, in postprocess interviews, all participants acknowledged having learned a great deal by participating in the project and most could articulate specific ways they thought differently after participating. It seemed clear, however, that promoting dialogue across differences was difficult and reaching consensus about which courses of action to recommend pursuing was exceedingly difficult.

In most of the other projects I have engaged in since the initial project described here, in fact, I have downplayed the achieving-consensus goal and been content with simply surfacing a range of stakeholders’ thinking and encouraging other stakeholders who think differently to at least hear what other participants said. Currently, for example, the Diocese of San Diego has hired a colleague and me, both, incidentally, non-Catholics, to document what was being said during meetings parishes organized to discuss (a) times participants were disappointed by the Church, (b) times the Church brought participants joy, and (c) participants’ hopes for the future of the church. We are expected to document dialogue across differences, including, for example, between participants who want the church to be more welcoming to LGBT Catholics and those who want the sacraments denied to LGBT Catholics. We are not expected to broker any sort of agreement between those who think differently about actions the Church should take in the future. ⁷

Reflections on Critical Qualitative Inquiry and the Contemporary Policy Context

One of the positive features of qualitative research is its appreciation of the importance of context. Even critical qualitative researchers who tend to focus on macro-level forces that historically were viewed as playing out in mechanistic and predictable ways now embrace the concept of resistance and the related idea that contextual variation will influence how macro forces impact different settings. Consequently, most qualitative researchers are skeptical about the possibility of generalizing study findings or even transferring what was learned in one context to other inevitably different contexts.

I would be remiss, then, if I did not conclude this article by noting that the ideas detailed here should be treated only as “working hypotheses” about what can and will happen in the future. After all, today state legislatures often are pressured to enact prepackaged policies being promoted across different states by powerful interest groups (Anderson & Donchik, 2016; Ferrare & Reynolds, 2016; Malin & Lubienski, 2022). I do not know whether, in this environment, there is still room for individuals to impact legislation at the state level. Also, today local contexts often appear to be far more politicized and the people in them much more ideological than was the case when I experimented with the dialogue-across-differences approach to policy analysis described above. When we conducted the deliberation study, for example, we intentionally selected parent participants who appeared to be both supporters and critics of the school, but we also tried to pick people in both categories we suspected would be able to modify their initial positions based on evidence and argumentation. Judging from news reports of some rather raucous recent school board meetings, those sorts of parents may be difficult to find in some school districts today.

So, the ideas presented in this article should only be used heuristically. They suggest what contemporary actors who wish to influence legislation at the state level or implement decision-making opportunities at the local level could consider doing, but what actors today end up actually doing may require some—possibly even considerable—modification of things done in the past. Of course, I would say the same thing about all qualitative findings . . . and, for that matter, also about the findings generated by quantitative studies, as well.