Abstract
The scholar most associated with "public sociology" responds to steven lubet’s prosecutorial approach and argues that, if looking for falsehoods is the point of empiricist ethnography, looking for falsifications is the point of theory-driven ethnography.
Response to Accuracy in Ethnography: Narratives, Documents, and Circumstances
Ethnographers study others; they are not accustomed to being studied. Steven Lubet, the law professor from Northwestern University, is giving us a taste of our own medicine. In a number of widely read essays, he joined a social movement to pillory University of Wisconsin sociologist Alice Goffman. She is author of On the Run, originally billed as a sensational six-year ethnography that exposed the long arm of the state in a poor African American community in Philadelphia.
Lubet charges her with fabricating evidence and acting unethically. In Interrogating Ethnography, Lubet extends his critique beyond Goffman to ethnography more generally. Delving into more than 50 studies, he searches for doubtful empirical claims, and offers us lessons in what he calls “evidence-based” ethnography. The editors of Contexts have invited me to respond to Lubet’s challenge by reflecting on the meaning and importance of ethnography.
Lubet writes as a lawyer. He argues that, while the legal process inoculates itself against falsehood through the adversarial process, ethnography’s truths are more vulnerable, hiding its evidence behind a veil of secrecy and anonymity and allegedly equating myth and hearsay with reality. Accordingly, Lubet would require ethnographers to rely on multiple sources of evidence, employing documents, fact-checkers, reliable witnesses, and experts. In addition, he calls on ethnographers to follow the example of other scientists and cross-examine each other’s “facts”. This is, indeed, the strategy he follows in contesting some of Goffman and others’ empirical claims. As Lubet puts ethnography on trial, he acts as a stereotypical trial lawyer, ferreting out random errors in monographs to discredit them. Thus, for example, he disputes Kathryn Edin and Luke Shaefer’s claim, in $2 a Day, that students from Mississippi don’t know an elevator when they see one. He goes to great length, consulting all manner of experts, to show this is implausible, irrespective of whether this affects the argument of the book. Because Lubet’s pickings along these lines are quite thin, he goes on to indict ethnographers for covering their tracks through anonymity, thereby making it difficult to fact-check their evidence. At no point, however, is there any serious discussion of the theory or argument of the research Lubet addresses; in this line of reasoning, facts come first, and theory follows. If any of the facts are false, ipso facto, the entire theory is false.
Why has there been so little resistance to Lubet’s indictment? Do we simply disdain the critical intervention of an outsider?
Lubet’s critique warrants a response if for no other reason than such distinguished ethnographers as Gary Fine, Shamus Kahn, Peter Moskos, and Colin Jerolmack have taken Lubet’s charges seriously, while others have joined him in denouncing Goffman. Why has there been so little resistance to Lubet’s indictment? Do we simply disdain the critical intervention of an outsider, just as our own subjects often disdain us? That’s part of the answer, but there is something more profound at stake, namely the resonance of Lubet’s “evidence-based” ethnography program with a conception commonly found in our discipline—that ethnography is a “natural” sociology that gets at the unvarnished “truth”. Insofar as they follow empiricist assumptions, ethnographers are vulnerable to having their research discredited by any empirical slip, no matter how insignificant. Here, I offer an alternative theory-driven approach to ethnography that is no less attentive to evidence but deploys it in a different way, for the growth of knowledge.
My response, which has benefitted from the criticism of scholars Carmen Brick, Andy Chang, Aya Fabros, Shannon Ikebe, Andrew Jaeger, Tyler Leeds, Thomas Peng, as well as three anonymous reviewers and the editors of Contexts, develops through an interrogation of Lubet’s interrogation, pointing to misconceptions underlying his deceptively simple argument. First, all ethnography is “evidence-based”. Evidence is as important for interpretive anthropology as it is for scientific sociology. Just the purpose is different. Second, by introducing a false dichotomy between “evidence-based” and so-called “postmodern” ethnography, Lubet elides a crucial distinction between his own empiricist ethnography, in which theory is grounded in and arises from the supposedly unproblematic “facts” and “theory-driven” ethnography which claims there can be no facts without theory, that is without a lens to select from the infinite manifold that is the world we study. The lens can be the folk theory of common sense or the analytical theory developed by the sociologist qua scientist. Analytical theory, hereafter just “theory”, is not confined to an abstract body of laws or some grand vision of the world. Theory is simply a proxy for enduring bodies of organized sociological knowledge. Theory emerges as a collective endeavor of a community of scholars accumulating knowledge by advancing competing and inter-related research programs.
Thus, theory-driven ethnography takes evidence very seriously, precisely because it focuses on critical observations that allow adjudication between alternative theories or the reconstruction of existing theories. Indeed, for theory-driven ethnography, facts become so crucial that we are often led to examine their production and have to develop auxiliary theories to explain their appearance. Famously, Durkheim’s claim that Protestants commit suicide at a greater rate than Catholics turns out, under closer investigation, to be as much a function of the unwillingness of Catholic officiators to record a death as a suicide as it is of the “egoism” of Protestants. Rather than focusing on critical facts, Lubet’s empiricist ethnography treats all facts as equal, so he hunts through ethnographies searching for “implausible” claims, showing little concern either for the way facts are produced or for the theory they support or challenge. In justifying this slash-and-burn approach, Lubet draws inspiration from empiricist ethnographers for whom, indeed, “truth” springs spontaneously from the “facts”.
The Adversarial Approach at Work
Interrogating Ethnography starts out by embracing a leader of the empiricist school—Mitchell Duneier, author of award-winning ethnographies Slim’s Table and Sidewalk. In an article, “How Not to Lie with Ethnography,” Duneier asks ethnographers to put their research on “trial” by searching for “inconvenient witnesses.” He takes on Clifford Geertz’s fabled account of the Balinese cockfight, asking how it would appear differently had Geertz relied on different participants—namely, the poorer villagers whose social position pushes them to the perimeter of the cockfight, where bets follow a different logic. For Duneier’s critique to be convincing, he would have had to take Geertz’s interpretation of the cockfight far more seriously; found some real, existing “inconvenient witnesses”; dug up some crucial falsifying evidence; and then specified an alternative theory. Instead, his complaint is entirely speculative. Especially telling, Duneier shows no signs of having consulted the voluminous literature on Geertz’s iconic piece. But empiricism gives him the warrant to challenge any study, even when he knows nothing about the topic.
Duneier does better with Eric Klinenberg’s Heat Wave. He pays a short visit to Chicago and, with the help of research assistants, launches a spirited attempt at falsification. Duneier zeros in on crucial empirical “facts” about two communities Klinenberg studied and a third that Klinenberg didn’t study. He concludes that Klinenberg’s “isolation” hypothesis, employed to explain deaths from the heat wave, suffers from the ecological fallacy. The most compelling evidence both for his refutation and his alternative hypothesis draws from retrospective interviews with neighbors and kin of decedents who died 10 years earlier in the heat wave. In response, Klinenberg argued that Duneier’s “replication” uses a questionable definition of “dying alone,” misreads official documents leading to undersampling of death through isolation, and relies on “hearsay.” Klinenberg criticizes Duneier for adopting a widespread but “baseless” public account that attributes the deaths to “substance abuse”—a convenient explanation that blames the victim and exculpates the interviewees from responsibility for the death of friends, neighbors, and relatives.
Here is Lubet’s adversarial approach at work. Curiously, it looks like a cockfight, a fight to the death in which truth wins out. Only, in this case and all too many others, there is no clear victor. The exchange reveals how difficult it is to achieve refutation through replication—as has long been understood in the physical sciences and is now coming to be understood in the social sciences—and calls into question the bedrock assumption of empiricism, the solidity and ultimate incontrovertibility of facts. That is, attempts at refutation show that facts are socially produced. That there is as much conflict over the theories behind the production of evidence as there is over those theories the evidence is supposed to confirm or refute only gives further credence to the priority of theory.
Each contestant insists theirs is the truer rendition of reality, but from the standpoint of science, this looks like a battle between two research programs guided by opposed theories—one looking at individual-level explanations and the other focusing on broader social conditions. As each hacks away with falsifications while defending against the opponent’s falsification, what is billed as a battle for a single truth proves to be an irresolvable contest. Theories refract divergent interests in society, and the possibility that “evidence” is inflected by “interests” eludes both Duneier and Lubet.
Theories refract divergent interests in society; “evidence” is inflected by “interests.”
The Two Dimensions of Social Science
Since Lubet does not spell out the premises of ethnography, we must do so. Social science can be distinguished from the physical sciences by our participation in the world we study. No matter how we try to insulate ourselves from the world beyond, no matter how hard we try to deceive ourselves that we are “objective” observers, the world we study springs back in our face. Indeed, as the university becomes an embattled zone buffeted by external political and economic interests, the mythology of the neutral, outside observer becomes ever-harder to sustain and ethnography—as a methodology that recognizes we are part of the world we study—becomes a prototype for all social science.
Social science, then, is defined by a double hermeneutic. Its scientific dimension is composed of the interaction between theory and data, itself embedded in an academic field. Its reflexive dimension involves the interaction between participant and observer, itself embedded in a field of power. Suppressing the reflexive axis and pretending we are not part of the world we study, leads us back to some form of positivism, while suppressing the scientific axis leads to postmodern interpretations or the transformative projects of participatory action research.
Just as the reflexive axis ranges in its emphasis between the observer and the participant, so the scientific axis is caught between two approaches: on the one hand, an empiricist ethnography in which data speak for themselves and theory emerges tabula rasa from hearing and seeing, and, on the other hand, a “theory-driven” ethnography which works from the premise that to make any sense of the fieldsite requires some sort of prior lens. Without a lens the world is a blur. Thus we don’t enter the field as “neutral” individuals, but with perspectives, defined by common sense as well as accumulated bodies of disciplinary knowledge. The task of theory-driven ethnography, then, shifts from discovering new theory to reconstructing existing theory. Here the crucial criterion is not verisimilitude but the growth of knowledge. The ethnographer is not a heroic individual, the valiant explorer who discovers a new world, but a member of a scientific community engaged in the collective enterprise of advancing knowledge.
The ethnographer is not a heroic individual but a member of a scientific community engaged in the collective enterprise of advancing knowledge.
Tackling Hearsay, Distinguishing Ethnography
Studying the world as we participate in it creates multiple dilemmas, but it has the decided advantage of allowing us to distinguish between hearsay and observed reality. Indeed, its method is to analyze the dialogue between what people say and what they do, between stated norms and actual practices, between justifications and behavior. Intrinsic to participant observation is a suspicion of informants’ accounts as the only version of the truth: they give one side of the truth, they exaggerate. Even when they lie, that tells us a lot about the world they inhabit because, by indicating what people want to hide, they point to deeply held interests.
In this regard, Lubet’s scholarly paragon of rigorous documentation—Matthew Desmond’s Evicted—fares more poorly than Goffman’s On the Run. Desmond’s seamless narrative, dramatizing lives in two desperate communities facing housing insecurity, fails to distinguish between what people say they do and his observations of what they actually do. In excising himself from the field, Desmond leaves readers little idea of how he gathered his vivid and detailed stories, nor the relations he had with his informants. How, for example, might being introduced to the tenants through the landlords evicting them have affected tenants’ self-reports? Lubet is taken in by Desmond’s all-too-brief methodological pronouncements, thereby violating his own strictures against “hearsay”. While it makes her vulnerable to contestation, compared to Desmond, Goffman appears far more honest and careful about not relying on hearsay than Desmond.
Here it is important to compare ethnography with other methods that are more or less helpless before the accusation of hearsay. Survey research, for example, has no immediate check on the data it collects. Any given question can mean different things to different respondents; answers are often shaped by the interview situation. In contemplating their own lives, respondents are likely to confound expected behavior with reality. Survey researchers only know they have been misled when predictions prove to be wrong (as, for example, in elections). The focused interview offers an opportunity for closer interrogation, but discrepancies remain between what people say they do, what they say they should do, and what they actually do. As Lubet points out, Myanmar’s Rohingya refugees spin a particular narrative to the New York Times’ journalists to gain access to relief supplies. And, forsaking his ethnographic sensibility, Duneier questions “witnesses” about the causes of death in the Chicago heat-wave and regards their responses as incontrovertible. At the other extreme, we can look to experimental social psychology, in which people’s behavior is observed in laboratories, but we have no idea how the same people will behave in the much messier laboratory of their own lives.
By entering the time and space of the subject, participant observation provides a continual check between what people do and what they say. It’s not foolproof, of course. Lubet offers us the case of Katherine Verdery, who discovered through Romanian police files that there may have been more going on in her Romanian village than met her ethnographic eye. Still, ethnography gives us the best shot at understanding how and why people produce their distinctive “perspectives.”
Situating the Document
Lubet doesn’t trust the ethnographer, so he proposes we develop a number of external checks on ethnographic data. Above all, he urges us to consult “documents” that offer evidence “frozen in time, unlike fragile human memories that may change with every retelling” (p. 23). But being frozen in time doesn’t make a piece of evidence more reliable. Documents are produced, and they reflect the interests of their producers. The same goes for other external checks: each offers a perspective affected by specific yet unseen interests.
Lubet’s search for a singular truth is ultimately achieved by awarding certain sources authority and legitimacy while denying it to others. The authority reflects power as much as veracity, however: he gives credence to hospital administrators, former public defenders, and police in order to to debunk Goffman’s observation that the police hang around emergency rooms and peruse hospital records looking for suspects. To a serious social scientist, of course, instead of a single data point, these contending views might provide the empirical basis for mapping a field of contestation.
After all, an abiding motivation for ethnographic research is to contest “official” views of the world like those enunciated in reports, in the media, and in common sense. Ethnography created new research programs in the study of work, family, race, class, sexuality by opposing stereotypes and conventional wisdoms. In her ethnography of sex work, Kimberly Kay Hoang’s Dealing in Desire debunks the NGOs view that sex work in the Global South is always a form of coerced trafficking by showing that the sex workers she studies in Ho Chi Minh City enter their occupation voluntarily and actively consent to their different labor processes. By comparing different market niches, Hoang also shows how the sex industry lies at the intersection of multiple circuits of global capital. These conclusions would be impossible without participant observation, which thereby carries its own authority.,
Lubet brings a particular vision of the legal process to his assessment of ethnography: the trial as an adversarial battle for the truth. But an alternative approach is to see the trial as a battle between competing perspectives. Each side appeals to different precedents, assumptions, and interpretations. Generally, ethnographers can map the divergent positions of actors in a broader field of domination; in this view, the law court is a field in which the actors—detectives, witnesses, experts, trial lawyers, defendants, judge, jury—bring their different “outlooks” to bear on the case and those outlooks reflect different positions in the field, which accords different levels of legitimacy to different classes of actors. Recognizing the divergent perspectives, the sociologist may knowingly take the standpoint of any given actor or try to grasp the field as a whole. Either way, Lubet’s attempt to reduce the field to a singular data-point trivializes the complexity of the field as an object of study.
Fieldwork, From Falsehoods to Falsification
If looking for falsehoods is the ground of empiricist ethnography, looking for falsifications is the essence of theory-driven ethnography. That’s what we do with every revisit to the field. Any field-site is infinitely complex, so we need some set of presuppositions, questions, concepts, coding schemes—theory, in the most general meaning of the word—to make sense of it. Theory tells us what to look for and sensitizes us to things out of place. A good theory makes predictions and fosters surprises. Moreover, this should affect the way we write our field notes, allowing for a running experiment of testing and revising hypotheses in a real-world experiment. We do look for confirmations but, for knowledge to grow, it is essential to look for the falsifications that enable us to build theory, brick-by-brick, with every visit to the fieldsite. Without falsifications, theory stagnates.
If looking for falsehoods is the ground of empiricist enthography, looking for falsifications is the essence of theory-driven ethnography.
Theory-driven ethnographers don’t necessarily reveal the iterative process of their day-to-day field work in their published work, but they may divide their extent of field work into “revisits”. Returning to Hoang’s study of Vietnamese sex-workers, we see how she begins with literature that sees the co-production of gender and capital within the categories of “developed/underdeveloped” or North/South, conforming to her observations in the first phase of her fieldwork. In 2009, when she returns for a second stint of field research, Hoang works as a hostess and bartender. She now sees the North-South binary as inadequate to explain the different market niches created by the economic expansion of Vietnam and the rise of Asian capital after 2008: one niche for Western businessmen and another, the highest status, for Vietnamese elites, who use the sex industry to lubricate business deals. And in a third visit, in 2013, Hoang finds that the sex industry has been restructured once again, requiring her to rebuild her theory of the circuits of global capital. Hoang’s successive revisits allow her to falsify and then reconstruct the theory of sex-work in its global context.
Ethics of Ethnography—Research in the Real World
The interaction of theory and data shapes and is shaped by the ethnographer’s real-world immersion, that is the relation between participant and observer. Perhaps, the biggest consternation aroused by On the Run is Goffman’s methodological appendix, in which she describes joining Mike in a murderous pursuit of the killers of his close friend, Chuck. Lubet is shocked that Goffman might have been an accomplice in a crime, what he believes was a felony. I’m shocked that he’s shocked.
Ethnography is the study of the world in the time and space of the participant. If you are studying people who commit crimes, you are likely to get involved in those crimes. To maintain relations with the people you study, you often have to do what they do. If that means committing a crime, you must accept the consequences.
As social scientists, our final goal is not to learn about the case, which is what the participant wants, but to learn from the case in order to expand scientific knowledge.
In my research I, too, have transgressed the law. When I was studying students at the University of Zambia, I was drawn into a 1971 student demonstration. A thousand of us marched on the French Embassy, protesting that country’s economic support for apartheid South Africa. I was arrested along with others. When it came to the trial it never occurred to me to justify my participation in the demonstration as research. My trial became a demonstration of my loyalty to a community suspicious of my motives, and it helped me soften the racial and status divide that posed dilemmas throughout the fieldwork.
But participation also facilitated my study of the forces operating in Zambia’s political field. The protest escalated into a major confrontation, involving the students, university administration, the ruling party, the para-military, the media, the President, and the Ministry of Education. Participation in this tragic unravelling of events allowed me to dissect the contradictory place of the university in society and the clash between competing avenues of upward mobility.
Transgressing the law is but one of the everyday dilemmas that confront the ethnographer embedded in a field of unequal power, forced to take sides in a world which is not one’s own. Ethical dilemmas do not end with the fieldwork. They continue into the written representation of your subjects—to anonymize or not. Lubet complains that “anonymity” inoculates the ethnography against verification. But this elides the immersion of the ethnographer in a power-laden field. So, conventionally “anonymity” is justified on the grounds of protecting subject, even as some argue ethnographers are accountable to their subjects only by being open about the identity of those subjects. Apart from the potential risk to subjects, such fieldwork strategies can make the ethnographer hostage to the subjects’ view of the world. Ethnography would be in the grip of participants’ firmly held common sense—thereby spelling the end of sociology as a science. The point of science, after all, is to reveal what the participant does not see. As social scientists, our final goal is not to learn about the case, which is what the participant wants, but to learn from the case in order to expand scientific knowledge.
Ethnography has its distinctive dilemmas—getting in and getting out, overt vs. covert participation, engaging both the dominant and the dominated, who has the positionality to study whom, studying people whose values are anathema, etc.—but these dilemmas all spring from the relationship between the scientific and reflexive dimensions of ethnography. The agony and ecstasy of ethnography lies in the acuteness of our presence in the world we study. As the walls of academia become thinner, as the research we do has real consequences, as economic and political pressures invade scholarly activities, as we increasingly recognize that we are part of the world we study, these ethnographic dilemmas creep into all social science.
Theory-Driven Social Science
The assault on Alice Goffman brings to mind a similar case of a senior professor hounding a young assistant professor out of academia in the early 1980s. David Abraham received his PhD from the history department at the University of Chicago, writing his dissertation on the collapse of the Weimar Republic. Abraham’s research was well received, earned him a tenure-track history position at Princeton, and a book from Princeton University Press. Gerald Feldman, a distinguished Berkeley historian of Modern Germany, was one of the book’s original reviewers. He grudgingly acknowledged that The Collapse of the Weimar Republic was important and well-researched, even though it was couched in a distasteful Marxist framework. Enter Henry A. Turner, senior history professor at Yale, who was also hard at work on the same topic, but from a different angle. With the help of a collaborator in Germany, Turner found errors in Abraham’s footnotes, and passed them on to Feldman.
Feldman felt betrayed, as though Abraham had deliberately deceived him, and went on a global vendetta, accusing Abraham of fraud. Journals, magazines, and newspapers played host to ferocious exchanges. The fight reached a low when Abraham was accused of lying in the book’s dedication to his parents who had suffered in the Holocaust—this was false, it was said, because they hadn’t died in the concentration camps. Historians were divided. Abraham admitted mistakes, but emphatically denied deliberate fabrication. Some of his errors weakened his argument, some strengthened it. Most made no difference. Still, even though eminent scholars defended Abraham, Feldman prevailed and Abraham was denied tenure at Princeton. Feldman then wrote letters to all and sundry to ensure Abraham didn’t receive any subsequent job offers. Forced out of the history profession, Abraham entered law school and later became a professor of law at the University of Miami.
So far as I know, Lubet has made no attempts to expel Goffman from sociology, although his arguments could be used against her. Unlike history, sociology is not made up of autonomous fiefdoms, impervious to the intervention of colleagues outside the immediate specialty. Still, there are obvious parallels in the prosecutorial mode of operation. In both cases, the critics focus on the evidence rather than the theories being proposed. In focusing on minor, debatable errors, both miss the forest for the trees—in Feldman’s case, because theory was all-important, and in Lubet’s case because it is unimportant. In both cases, an opportunity to expand knowledge was lost in a furious battle over “errors.” It is probably no accident that the victims were both critical young scholars vulnerable to the pressure of tenure and to attacks from senior scholars. Such power differentials are conveniently lost in the shuffle—as if the transcendent goal of empiricism justifies targeting a young scholar.
Ethnography has been too easily hijacked by a common sense view of science, an empiricist view that theory springs tabula rasa from the facts, so that if you get the facts wrong then ipso facto the theory is wrong and your contribution is zero (or even negative). So Lubet becomes like Mike, carrying a metaphorical gun, hunting for the killer-error in every nook and cranny. What is most disturbing is that Lubet has persuaded other ethnographers to join him in his empiricist crusade.
Ethnographers of science, historians of science, and philosophers of science have long since abandoned the empiricist view of science—a discredited perspective that never applied to the physical sciences but was foisted on the insecure social sciences, holding them hostage to arbitrary standards. Starting with Karl Popper’s critique of induction and moving on to Imre Lakatos’s idea of research programs, we have a far better understanding of how science is actually practiced. We start with theory, we end with theory, and evidence drives the mediation between the two.
We start with theory, we end with theory, and evidence drives the mediation between the two.
Within sociology, empiricist ethnography grew up as a reaction to the domination of structural functionalism—the idea that theory should spring from the ground rather than from the ethereal brain of Talcott Parsons. Fine. Grounded theory proved an effective weapon against structural functionalism. But structural functionalism is long-since dead. There is no warrant for ethnographers to continue an outdated philosophy of science, except as a strategy of power to subjugate young insurgents within a contested field.