Abstract
The American Sociological Association recommends that instructors and departments infuse research methods instruction throughout the curriculum (i.e., beyond research methods courses). Sociology textbooks are widely used instructional components and are essential to examine how research instruction extends beyond chapters and courses on research methods. The author uses qualitative content analysis of 27 textbooks for introductory courses (n = 19) and intermediate elective courses (n = 8) to uncover how textbooks illuminate the process of how sociologists know what they know (i.e., do research). Overall, textbooks model asociological and unscientific thinking by using false equivalence practices, reinforcing commonsense explanations, and relying on shortcuts to scientific credibility, among other means. Textbooks describe society using basic descriptive statistical data but provide little explanation of how scholars gather or analyze data or draw conclusions about data. These results suggest that textbooks have gaps in their ability to support research methods instruction throughout the sociology curriculum.
Keywords
The American Sociological Association and scholars of teaching and learning in sociology agree that sociology graduates should have research skills and that methods instruction should occur throughout the sociology curriculum (i.e., the Sociology Literacy Framework [SLF]) (Berheide 2005; Ferguson 2016; Ferguson and Carbonaro 2016; Greenwood 2013; Gunn 2017; Howery and Rodriguez 2006; Kain 1999; Markle 2017; Medley-Rath 2021; Medley-Rath and Morgan 2022; Parker 2011; Pike et al. 2017; Weiss 1987; Wilder 2010; Williams et al. 2016; Williams and Sutton 2011). All undergraduate sociology programs require a research methods course, and most (77.9 percent) require statistics (Sweet, McElrath, and Kain 2014). However, required research methods and statistics courses may leave instructors (wrongly) assuming that students are receiving sufficient methods instruction (see Sweet et al. 2014).
One of the most important things students in introductory sociology should know is “the scientific nature of sociology” (Persell, Pfeiffer, and Syed 2007:305). Moreover, Greenwood (2013:233) argued that introductory sociology courses “may be the first time students learn about using science to study social behavior and how to apply it to their own lives.” Her premise suggests that learning about research methods should occur during that first course. To this end, many introductory sociology textbooks include a chapter devoted to research methods.
Developing sociology students’ research skills empowers students to consume research (Beuving and de Vries 2020; Burgess 1990; Small 2018). Beuving and de Vries (2020) argued that the Western cultural context calls for improved methods instruction because of the supremacy of big data, the public nature of academic discussions (e.g., Twitter, TED Talks), and the public’s declining trust in science (see also Small 2018).
Sociology textbooks are important data for understanding how the research process is shared in instruction because textbooks provide a means of cognitive socialization into the discipline for students and instructors (see Zerubavel 1997). Textbooks, therefore, should do more than tell readers what we know by showing readers how we know what we know. Shedding light on the research process in textbooks for a range of courses serves to inform students (i.e., the public) about how sociologists construct sociological knowledge. The public must understand that sociological findings are not just “opinions” formed by the researcher’s biases but that sociologists take care to minimize bias and use a variety of research tools to produce sociological knowledge. However, sociological knowledge in textbooks reflects a fractured mirror constructed by classical and contemporary sociological research, market interests, disciplinary norms and biases, and logistical constraints (i.e., the length of a semester or student preparedness).
Given the many disciplinary calls for research methods instruction across the curriculum, the importance of research consumption skills for careers and everyday life, and the role textbooks play in communicating disciplinary knowledge (and skills), the instructional narratives about doing research in textbooks matter. Therefore, I use the SLF research-focused competencies and a sociology of knowledge framework in a qualitative content analysis of 27 sociology textbooks (19 for introductory courses and 8 for intermediate elective courses). I uncover how texts make and fail to make the research process known. In other words, how do textbooks demonstrate how sociologists know what they know?
The Sociology of Sociology Textbooks
Sociology of knowledge scholars have long recognized the role of textbooks in creating and maintaining disciplines. Textbooks matter because they “communicate the vocabulary and syntax of a contemporary scientific language” (Kuhn 1970:136), socialize students into specialized experts (Fleck 1981), and maintain a discipline (Lynch and Bogen 1997; see also Dunham, Cannon, and Dietz 2004; Fitzgerald 2012; Perrucci 1980; Tischler 1988). Therefore, sociology textbooks should reflect the discipline (Tischler 1988); however, research finds our texts do not reflect sociology as a discipline. For example, scholars find gaps between the sociology found in introductory sociology textbooks and that found in journals and monographs (Hamilton and Form 2003). Books include dated terms (Best and Schweingruber 2003; Dong 2008), debunked ideas (Nolan 2003), myths (Schweingruber and Wohlstein 2005), and incorrect definitions (Puffer 2009). Textbooks also rely on journalistic accounts rather than sociological research for new topics (Carroll 2017; see also Babchuk and Keith 1995).
Textbooks prioritize topics on the basis of market demand instead of current thinking on a topic, a topic’s use by scholars, or a topic’s usefulness to help students learn more complex material (see Greenwood 2013). How and why topics are left out is due to both “too much” information available (see Abrutyn 2013) and sociology’s long-standing sexism (McDonald 2019), racism (Morris 2015; Wright 2012), ethnocentrism, homophobia, and ableism. Textbook sociology emerges from competing interests and is produced by many “cooks in the kitchen”—the author, editor, copyeditor, graphic designer, and so on—influencing the final product.
The scholarship on textbook sociology (see Appendix A) overwhelmingly focuses on textbooks for a specific course, primarily introductory sociology. I found no studies of social problems textbooks and only a few studies of books for elective courses in sociology. Another gap in the literature is on how textbooks for different courses cover the same topic (e.g., Pearce and Lee 2021). Most textbook studies focus on topic coverage (e.g., race) for a single course.
How Sociologists Know What They Know
Scholars have studied the presentation of sociology as a science in textbooks. Keith and Ender (2004) reported that textbook authors understand sociology as scientific, though they do not always coherently reflect this view in their texts (see also Lynch and Bogen 1997). The literature suggests that textbooks are not adequately demonstrating how sociological knowledge is known.
Understanding how sociological knowledge is known is a key concern of the SLF. Ferguson and Carbonaro (2016) outlined the SLF, and the American Sociological Association endorsed the framework (Pike et al. 2017). The SLF outlines fundamental concepts and competencies that students should have upon degree completion. My study focuses on the competencies that most closely align with illuminating how sociologists know what they know and center research skill development: “apply scientific principles to understand the social world” (i.e., apply), “evaluate the quality of social scientific methods and data” (i.e., evaluate), and “rigorously analyze social scientific data” (i.e., analyze) (Ferguson and Carbonaro 2016:154).
My research addresses several gaps in the literature. First, my sample includes textbooks across the undergraduate sociology curriculum, including texts for several distinct sociology courses. Second, I focus on the overall narrative to uncover how books show students how sociological knowledge is known rather than quantitative measures of topic coverage (e.g., counting topics found in section headers or indices). Third, I draw on a sociology of knowledge theoretical framework with the SLF to guide analysis. This study is the first to evaluate evidence of the SLF in sociology textbooks.
Methods
Sample
I used purposive convenience sampling to select textbooks for introduction to sociology, social problems, race, class, gender, and family courses. I chose books for intermediate-level elective courses on inequality (i.e., race, class, or gender) and family to increase the likelihood of similar topic coverage within all the textbooks in the sample. I selected 27 books for analysis (19 for introductory courses and 8 for intermediate elective courses) (see Table 1) using the inclusion criteria described in Appendix B.
Textbooks Included in the Sample (n = 27).
Only two chapters in Kennedy, Norwood, and Jendian were included. There was not a comparable third chapter to analyze.
Research Questions and Analysis
My overarching research question is, How do textbooks explain how sociologists know what we know? I developed guiding questions to focus the analysis: (1) What themes emerge about the research process? (2) Do textbooks explain how a study was done, or do they only report results? (3) Are quantitative data (e.g., statistics), qualitative data (e.g., interview snippets), or both reported? I looked for descriptions of methodological approaches, research design, sampling procedures, measurement of variables, data collection methods, research questions, and hypotheses. I analyzed how textbooks talked about research.
I closely read each chapter using qualitative content analysis (Schreier 2012) to identify manifest and latent themes. Qualitative content analysis can uncover latent content missed by quantitative studies that count the frequency of a given topic (Shin 2014). I coded chunks of text that talked about research. I prepared a research memo for each chapter, which included quoted material and initial thoughts on what was occurring in the data and, when possible, connecting the data to the research-focused competencies in the SLF. See Appendix C for an example of my coding.
Results
I give agency to textbooks in this article because textbooks are the product of multiple interests. That is, I report what textbooks say rather than what authors write. For specific examples, I refer to a textbook by its authorship.
My objective is to understand the story told in textbooks about how sociologists know what they know. I report results on the emergent themes about the research process (see Table 2). Overall, these textbooks did not adequately explain how sociologists know what they know. Instead, texts modeled unscientific and asociological thinking, failing to model what the SLF advocates, let alone support the development of research competencies among students.
Extent of Themes Found across Textbooks (n = 27).
Textbooks more often placed extensive research details in insets (i.e., outlined boxes) and end-of-chapter activities rather than the main body. For instance, Shepard’s (2018:206) detailed explanation of Eder’s 1995 research on middle schoolers is an inset. Shepard identified what Eder observed (i.e., rituals and daily speech routines of 12- to 14-year-olds), data collection (i.e., observations during lunch and extracurricular activities and informal interviews), and findings. Shepard provided ethical details (i.e., informed consent) and practical details (i.e., audio recording of the interviews). This level of description helps students see how sociologists use scientific principles, but still, Shepard did not explain how the researcher analyzed the data or used theory.
Overall, textbooks lacked detail about how sociologists do research for students to understand how sociologists know what they know. Texts shared study results—whatever might fit into one or two sentences—and mostly left out explanations of the data used, how data were collected or analyzed, information about the sample or research site, or limitations of the data. One exception is that textbooks used examples of conceptualizing variables (e.g., referencing how the U.S. Census Bureau defines family as the basis for the textbook’s definition of family). Books used shortcuts to establish scientific credibility, including reporting sample sizes and study length, author credentials, and using adjectives to describe research. I broaden Ferguson and Carbonaro’s (2016) meaning of the “apply scientific principles” competency to include evaluating the quality of sources of sociological claims. Textbooks made claims without sources, making it difficult for students to learn from where findings come. Furthermore, they used sources from a range of claims makers (e.g., from peer-reviewed research to anecdotes from the author’s classroom), treating them all as equally credible.
False Equivalence Practices
About half of the sampled textbooks used false equivalence practices to present disagreement among scholars or perspectives. False equivalence practices included using citations or unnamed critics or comparing related but different social phenomena. False equivalence practices worked to establish “both sides” of controversial topics.
Books used citations as the basis of “both sides” arguments. Texts used recent peer-reviewed sources for one side and no citations, dated peer-reviewed sources, or non-peer-reviewed sources for the other side (and there were never more than two sides). Instead of showing how sociologists used peer-reviewed research to understand social phenomena, textbooks treated all sources of information as equally valid. For example, Kendall (2017:313) suggested disagreement among scholars: Why does gender inequality increase in agrarian societies? Scholars cannot agree on an answer; some suggest that it results from private ownership of property. . . . However, some scholars argue that male dominance existed before the private ownership of property (Firestone, 1970; Lerner, 1986).
This example illustrated how textbooks fail to use sources (i.e., “some suggest”) and rely on dated references (i.e., “However, some scholars”) to present disagreement. Textbooks also cite reprinted dates without the original publication dates, making debates appear current (e.g., Thompson, Hickey, and Thompson 2019).
Books presented debates between sociologists and others (e.g., social conservatives). For instance, Griffiths et al. (2017:309) wrote, The question of what constitutes a family is a prime area of debate in family sociology, as well as in politics and religion. Social conservatives tend to define the family. . . . Sociologists, on the other hand, tend to define family . . . .
The text provided no citations for this debate. Moreover, the text treated sociological research as equivalent to the opinions of unidentified social conservatives and suggested agreement among sociologists.
Textbooks also compare things that appear similar but are not. For example, Macionis (2019:322–24) includes separate sections on violence against women and men. This organizational choice appears balanced because the text includes men and women. However, his examples for each group are for different forms of violence. Macionis addressed rape and female genital mutilation for women and murder and suicide for men. He tipped his hand when writing by citing only statistics for murder and suicide and in his introduction to the men’s section: “If our way of life encourages violence against women, it may encourage even more violence against men” (p. 323). Conley (2019:354–55) contrasted the reported statistics on hate crimes against Sikhs with the experience of the media’s reports of one White woman who converted to Islam: Even more striking [emphasis added] is what happens to Caucasian Americans who convert to Islam. One woman, despite having fair skin and green eyes . . . wears . . . the hijab . . . [and has] even been told, “Go back to your own country.”
Comparing different social phenomena as if they were two sides of the same coin, consequently, reinforced the notion that groups with privilege were also disadvantaged.
Reinforcement of Commonsense Explanations
Many textbooks posited that sociology is different from common sense and debunk commonsense beliefs about social phenomena. Yet most texts reinforced commonsense explanations by ignoring sociological research findings that refute common sense, repeating common sense without debunking it, or linking topics in a way that supported common sense.
Multiple textbooks, for instance, constructed single parenthood (often as described as single mothers or the result of “out-of-wedlock births” [Desmond and Emirbayer 2016:371]), divorce, and cohabitation as responsible for adverse childhood outcomes, especially poverty (e.g., Barian 2019; Macionis 2019; Olson, DeFrain, and Skogrand 2019; Ritzer and Murphy 2019; Shepard 2018). These claims ignored or minimized research demonstrating that single parenthood is not correlated with poverty in Western Europe to the degree that it is in the United States (see Brady, Finnigan, and Hübgen 2017). Occasionally textbooks pointed out that “Income inequality accounts for a large part of the adverse effects of single parenting on . . . children (Sigle-Rushton and McLanahan 2002)” (Shepard 2018:325; see also Croteau and Hoynes 2020:305–306; Ritzer and Murphy 2019:276). Shepard, however, devoted more space (i.e., three paragraphs) and support (i.e., nine citations from 1977 to 2010) to the harmful effects of single parenthood on children before addressing the role of income inequality in a single paragraph with a single citation.
Books repeated commonsense explanations without any sources: “Women are left alone to raise children as the men leave because they cannot support them or because the mothers are more likely to qualify for welfare if the father is absent” (Ritzer and Murphy 2019:276). Schaefer (2020:235) wrote, “By giving priority to African Americans in admissions, for example, schools may overlook more qualified White candidates.” Neither text included citations to support its claims.
Some textbooks produced a logic that promoted causal claims where correlations (at best) exist, reinforcing commonsense explanations rather than sociological explanations of social phenomena. For instance, texts suggested that topics were linked through placement under the same section header yet provided no evidence that they were related (e.g., teen pregnancy and infant abandonment in Leon-Guerrero 2019).
Other textbooks minimized research reporting correlations because sociologists could not establish causation. For example, Henslin (2017) presented one study that revealed no difference and the contested research by Mark Regnerus that showed negative differences in childhood outcomes for children raised by same-sex couples. He dismissed both studies because neither study could confirm causation. He concluded, “With arguments on either side of this issue rooted not in science but in people’s values, future research, no matter its particular findings, is destined to stir more controversy” (Henslin 2017:480).
Shortcuts to Scientific Credibility: Sample Sizes, Study Length, Credentials, and Adjectives
Most textbooks attempt to establish research credibility by reporting sample sizes, study length, the credentials of the scholars, adjectives to describe a study (e.g., “definitive study”), and by citing research. However, none of these strategies established the scientific credibility of the research or provided students with sufficient information to evaluate the quality of the study for themselves.
Textbooks reported sample sizes and the length of a study to provide quantitative evidence that makes research more credible to an audience with limited research methods literacy. Sample sizes were described for qualitative and quantitative studies but not always precisely. For example, Ferris and Stein (2018) reported the sample size of Edin and Kefalas (“Promises I Can Keep”) precisely: 162. In contrast, Desmond and Emirbayer (2016:375) reported the sample size as “over 160.” Although true, the exact sample size is known. Textbooks also emphasized study length. For instance, Schaefer (2020:18) described Elijah Anderson’s 1990 study of Philadelphia as a 14-year fieldwork study of public behavior. Textbooks used quantitative measures (e.g., study length, sample size) without explaining their relevance to evaluating qualitative research. Therefore, textbooks send the message that sociologists talk to, observe, or survey many people over a long period, and the more people they talk to and the longer they do it, the better.
Another shortcut to scientific credibility textbooks used was reporting a scholar’s discipline (e.g., sociologist, family researcher), institution (e.g., Stanford), or both. Disciplines and institutions were even used in place of scholars’ names and citations. For example, “Some anthropologists and psychologists believe the higher degree of nurturance among women is innate, whereas others believe it is learned” (Stinnett et al. 2016:62).
A different strategy textbooks used to establish scientific credibility was by using adjectives to describe a study. For example, Kendall (2017) described a “definitive study” (p. 306) and a “comprehensive study” (p. 317). Textbooks used adjectives to convince readers that the research was credible instead of providing sufficient details for readers to draw their conclusions.
Occasionally, textbooks used several shortcuts at once. For example, Ferris and Stein (2018:370) used study length, sample size, and adjectives: “Kathryn Edin and Maria Kefalas (2005) spent five years [i.e., study length] doing in-depth research [i.e., adjectives] with 162 low-income single mothers [i.e., sample size] to understand their attitudes about parenthood and marriage.” This example provides several yet less relevant details and excluded more useful information. For instance, it remains unknown how Edin and Kefalas recruited research participants, what their data were, or how they analyzed their data. It is impractical to share all relevant details of a study, but using shortcuts to establish scientific credibility does not reflect the standards used by sociologists and does not give students practice in evaluating the quality of social scientific methods or data.
Three textbooks took a different approach altogether (e.g., Andersen; Coates, Ferber, and Brunsma; and Korgen and Atkinson). They mostly cited only research findings instead of using other shortcuts to scientific credibility. For instance, Collins and Mayer’s (2010) research has found that even as the funding for government programs for the needy has been cut, such programs have also increasingly been expected to subsidize the income of the working poor and provide other safety nets once provided by employers (Coates et al. 2018:123-4).
Textbooks taking this approach still provided context for the research, such as an explanation of the methods used by the researcher or referred to the researcher as a sociologist. However, citing research tells the reader who made a claim, but it does not tell the reader how they reached that conclusion.
Sources of Claims
Textbooks ranged in their ability to support the skill of evaluating the quality of sources of claims. Texts drew on research and claims from other fields (e.g., biology, psychology, anthropology), classroom and popular culture anecdotes, think tanks or government sources (e.g., the Census Bureau, the Pew Research Center), other textbooks, journalist accounts, peer-reviewed research, and unnamed critics. Drawing on a wide range of sources is to be expected; however, texts do not consistently position sources differently, given their variation in credibility. Moreover, textbooks relied on dated sources, lacked citations to support claims, and use sources unclearly.
Textbooks presented academic debates between sociology and other disciplines. For instance, Eitzen, Zinn, and Smith (2018:199) asked, “Is Gender Biological or Social?” and concluded the first paragraph with “let us review the evidence for each position.” The evidence for biology took up twice as much space as the evidence for social. Both sections included two sources each. Additionally, each position was about “gender roles,” not gender, as used in the introductory paragraph. The section concluded with “a sociological perspective,” or the social construction of gender, and the following sections support a sociological perspective. Henslin (2017) did something similar with race in his chapter “Race and Ethnicity.” In this chapter’s first section, “Laying the Sociological Foundation,” Henslin cited an economist, a physical anthropologist, a philosopher, a sociologist, and a journalist to report on the human genome and the number of races and to debunk myths about race. He failed to explain how sociologists construct race in the body of the text (aside from one sentence in reference to the Thomas theorem). Instead, he discussed the social construction of racial classification in an inset. Placing a critical sociological insight about race in an inset instead of the body suggests to readers that this insight is less important.
Textbooks included anecdotes alongside official or government sources and peer-reviewed sources. For instance, Thompson et al. (2019:272) did not distinguish between the credibility of sources. They wrote, “Men and women are different. This has been confirmed in medical journals, self-help and dating guides, novels, movies and television.” In contrast, Korgen (2019) shared statistics on unemployment from the federal government, then shared results from a peer-reviewed study, and concluded the paragraph with a layperson’s experiment from a viral video. Korgen (2019:190) labeled the anecdote as a “layperson’s experiment,” setting it off from government data and peer-reviewed research. Anecdotal evidence also came from in-class activities with the author’s students (e.g., Ferrante 2015; Ray and Sharkey 2019; Schwartz and Scott 2018).
Writing a textbook and updating editions is challenging, so we should expect some reliance on dated sources. Furthermore, some dated references are classics and will have a long life in textbooks (e.g., C. Wright Mills, W.E.B. Du Bois). It is unclear, however, why books used some dated source. For instance, books continue to cite Murdoch’s 1949 and 1957 studies on polygamy, as if no one else has studied the topic in the past 70 years.
Textbooks claimed the relevance of older sources, but without sources to support those claims. For example, Conley (2019) cited a study from 1984 about media stereotypes of Middle Easterners and Arabs (terms he used interchangeably). He continued, “Little has changed since this study came out over 30 years ago, although after 9/11, the emphasis shifted away from stereotypes of Arabs as extremely rich and toward one of Middle Easterners as terrorists” (p. 365). The shift from a positive stereotype (i.e., rich) to a negative stereotype (i.e., terrorist) is a significant shift that Conley dismissed. Not to mention, he included no citation for the post-9/11 stereotypes.
Textbooks inconsistently used sources to support their claims, and citation practices that obscured which source supported a claim. For example, Ritzer and Murphy (2019:273) included no references to support their five claims: “married men and married women live longer, have fewer health problems, have more sex, save more money, and have fewer psychological problems . . . than do unmarried men and women.” Furthermore, the placement of sources made it hard to track which source (if any) supports a claim. Textbooks placed citations at the ends of paragraphs instead of the sentences with the claims (e.g., Andersen 2020; Coates et al. 2018). This practice left readers (wrongly) assuming that the citations support all claims made in the paragraph. Desmond and Emirbayer (2016) and Khan, Sharkey, and Sharp (2019) used endnotes to cite their sources, further obscuring them. Desmond and Emirbayer (2016:19), however, included the name of the scholar and their discipline (e.g., “Evolutionary biologist Joseph Graves, Jr.”) within the body of the text.
How or why a source fits with a claim was sometimes uncertain. For example, Khan et al. (2019:3) wrote about how Mark Granovetter has “studied a wide variety of topics” and how “individual behavior can be influenced by the actions of those around us . . . like . . . a riot.” It is unclear if Granovetter studied riots or if his other research helped sociologists understand riots.
The lack of transparent citation practices in textbooks led to other confusing conclusions. For instance, Lamanna, Riedmann, and Stewart (2018:18) report that research shows that when children and adolescents have deeper religious connections, they tend to have less premarital sex and to be older when they have their first sexual experience and, as adults, more willing to care for their aging parents (Eggebeen and Dew 2009; Gans, Silverstein, and Lowenstein 2009; but see Stark 2009; Wilderman and Percheski 2009).
The textbook did not explain what “deeper religious connections” or sexual experience meant or the ages of the comparison groups. The authors identified two trends correlated with religion, first sexual experience and caring for aging parents, which appear unrelated. By placing citations after multiple claims, textbooks leave readers without a map of each claim’s origin.
Conceptualizing Variables
Textbooks hold potential for explaining how to conceptualize variables and how conceptualization can affect results by describing from where definitions of key terms come. For instance, Croteau and Hoynes (2020:288) stated, “Because precise definitions of asexuality vary, research findings about the number of people who are asexual also vary considerably.” The textbook did not define asexuality and instead offered possible definitions. Cohen (2018:129) reported the contrast between how researchers define lower class (i.e., economic conditions) compared with how research participants identify: Only 7.9 percent of the population identifies as “lower class” in the GSS [General Social Survey], but based on their economic conditions, most sociologists believe that this group is larger. . . . the official poverty rate was 13.5 percent in 2015.
In contrast, Andersen (2020) explains that emotional labor is a term from Arlie Hochschild and defines it according to Hochschild’s meaning. Nevertheless, Andersen (2020:83) also left an opening for the misuse of Hochschild’s term and pointed out that “Technically, because housework is unpaid, it does not fit the classic definition of emotional labor.”
Overall, it was unusual for textbooks to devote much space to conceptualizing variables when presenting research results. More often, texts explained how definitions can vary and be difficult to establish (e.g., family, social class, race). Multiple books reported using the Census Bureau’s definition of family. Some texts point out the limitations of the Census Bureau’s definition and also discuss religious and legal understandings of the term family (e.g., DeFronzo and Gill 2020). Moreover, textbooks did not call this conceptualizing variables even though that process was being modeled for readers.
Basic Descriptive Statistics
Textbooks focused on certain kinds of data over other types of data. Texts provided numerous basic descriptive statistical data, primarily from governments or think tanks. Readers will encounter many basic descriptive statistical data in sociology textbooks: percentages, frequencies, rates, means, and medians. Only three texts (i.e., Desmond and Emirbayer 2016; Kennedy, Norwood, and Jendian 2017; Lauer and Lauer 2019), however, explained these terms in the sampled chapters, one of which had a separate methods chapter.
Rarely did a text share inferential statistics or discuss statistically significant results. Lauer and Lauer (2019:26) were an exception and explained significance before reporting on studies with significant results in their text. In contrast, Stinnett et al. (2016:22) referred to statistical significance. Still, they did not explain why it mattered: “Virtually every study of marital dissolution undertaken since the late 1960s has found both spouses’ age at marriage to be statistically significant with respect to the probability of divorce (South 1995).”
Discussion
Textbooks should help students learn how to think sociologically. Nevertheless, books from across the undergraduate sociological curriculum did not consistently model sociological nor a broad version of scientific thinking. Other scholars have found sociology textbooks lack a scientific framework and promote asociological thinking (Keith and Ender 2004; Lynch and Bogen 1997). The sampled texts did not integrate the most basic research skill from the SLF: applying scientific principles. Instead, readers received false equivalence practices, reinforcement of commonsense explanations, an emphasis on shortcuts to scientific credibility, and sources of claims from unreliable sources. Textbooks were better at describing how to conceptualize variables and providing multiple encounters with basic descriptive statistical data.
For this study, I am using a broad understanding of the conceptualization of variables. Most often, this meant that a textbook explained how the author(s) arrived at a definition for a term (e.g., family) or pointed out why our understanding of a topic is more limited (e.g., asexuality). Textbooks did not speak of this as conceptualizing a variable, but it provides a rudimentary foundation that books and instructors could build upon.
The most common data shared in textbooks were basic descriptive statistical data from think tanks and governments. This finding presents an opportunity to teach students from where data come because many of these statistics are accessible to students (e.g., the census). Instructors can teach students how to access these data, make sense of them, and explain how sociologists use data from think tanks and government sources (see Wilder 2010). Regardless of how interesting attitudinal and demographic data are, they do not tell us why or how something is. Students must learn how to interpret these data and how to ask sociological questions about them. Textbooks offered limited guidance on data interpretation or how to ask sociological questions.
Sociology textbooks should demonstrate how sociologists know what they know because public understanding of the contours of how experts know what they know is critical and our responsibility as scholars and teachers. Shedding light on the research process benefits students and instructors because textbooks are a means of cognitive socialization into the discipline (see Fleck 1981; Zerubavel 1997) and uphold disciplines (Lynch and Bogen 1997; see also Dunham et al. 2004; Fitzgerald 2012; Perrucci 1980; Tischler 1988). Textbooks should consistently use scientific principles as the basis of claims and reflect how sociologists do sociology. Moreover, textbooks should explain how quantitative and qualitative data are analyzed, reduce the use of shortcuts in establishing scientific credibility, and consistently use credible sources and fair arguments that reflect contemporary sociological knowledge.
Scholars have analyzed texts’ section headers, indices, reference lists, or all three. My analysis demonstrates that qualitative approaches to the texts’ body are necessary. For example, quantitative analyses of reference lists would miss the positioning of anecdotes from in-class activities as credible sources. Moreover, studies looking at reference quality miss how textbooks use these sources in the body of the text. For example, reference lists with more peer-reviewed sources over other types of sources make a text appear more scholarly than may be warranted.
Strengths and Limitations
This study’s strengths are that I analyzed textbooks across the undergraduate sociology curriculum and my sampling procedure. This research is the first study incorporating social problems textbooks, and intermediate elective courses, alongside books for introduction to sociology. By sampling across publishers, I decreased publisher-induced issues such as a publisher using the same copyeditor across texts. I included the popular open education resource from OpenStax, which to my knowledge is the first time sociologists have analyzed this text.
My sampling procedure, however, has three limitations. First, it was a convenience sample. Although I compiled a complete list of possible textbooks, I could not access all books identified for inclusion. Second, by focusing on standard texts (see Appendix B), many textbooks were excluded. It is unknown if nonstandard books better model sociological and scientific thinking. Third, I did not code the reference sections of these textbooks. I based conclusions about references on the in-text citations. Problem citations remain hidden in textbooks that used endnotes or used the standard citation practice of using the author’s last name (instead of a Web site).
These textbooks were developed before the publication of Ferguson and Carbonaro’s (2016) SLF. Therefore, I expected the SLF to not explicitly be visible in textbooks. Still, the SLF reflects agreement among leading scholars of teaching and learning in sociology about what skills students should develop in our courses and programs. The SLF provides a foundation to examine how texts explain how sociologists know what they know, which texts should be doing regardless of the SLF.
Future Research
This study repeats the question raised by others about why some topics remain in our textbooks. We have greater access to information today than when a substantial number of these texts were initially conceived. Therefore, textbooks should include topics because they support the development of sociological and scientific thinking or help students understand topics in more advanced courses. Moreover, space devoted to one topic comes at the expense of developing research skills.
Sociology textbooks also include research from other disciplines (e.g., psychology, anthropology, and biology). Do other fields report findings from sociology? And if they do, how do they describe sociological results? Scholars should examine introductory textbooks across disciplines to identify overlap. Researchers could investigate chapters on socialization, race, and gender across disciplines to explore this issue.
My sample was printed or PDF sociology textbooks. What publishers include in electronic interactive books, course cartridges for learning management systems, or companion Web sites remains unknown. As texts become available as digital-only products (e.g., Pearson’s catalog), it is more critical to understand what is in them, as instructors and institutions may become more dependent on the whole package of learning activities and resources the publisher provides. Digital products may be harder to modify or edit. Scholars should also analyze the content of popular supplementary materials such as the Crash Course Sociology and Khan Academy videos for sociology.
Conclusions
Overall, sociology textbooks do not show how sociologists know what they know. Instead, they model asociological and unscientific thinking. Despite the call for incorporating research methods instruction throughout the sociology curriculum since at least the 1980s, textbooks have not sufficiently responded. Some texts highlight the research process in a stand-alone methods chapter or insets, but this arrangement makes research instruction optional. Textbooks must illuminate the research methods process often and in the main body of the text. It must be inescapable. However, it is a big ask for textbooks to emphasize developing student research skills if instructors are not asking for it.
Textbooks play an important role in constructing sociological knowledge. Therefore, authors and publishers should take care to minimize asociological and nonscientific thinking (e.g., false equivalence practices, reinforcement of commonsense explanations, using less credible sources or vague citation). It is unrealistic to expect textbooks to eliminate all shortcuts to scientific credibility (i.e., textbooks can have only so many pages). However, textbooks could (on occasion) explain why a particular shortcut matters. Textbooks provide instructors with an entry point for learning about conceptualizing variables and understanding basic descriptive statistics. Instructors, authors, and publishers could build on this foundation already present in these textbooks.
Furthermore, the American Sociological Association and regional associations need to revisit developing resources and guidance for successfully and sustainably incorporating research methods throughout the curriculum (see Howery and Rodriguez 2006). The Teaching Resources and Innovation Library for Sociology (TRAILS) is a start, but individuals submit a resource they have used successfully (and fills a gap) rather than resources that reflect significant gaps in the body of work on TRAILS (see Medley-Rath 2023 for a compilation of TRAILS resources on the research methods course). In other words, sociologists should collectively address the challenge of incorporating research methods instruction rather than relying on individual instructors, courses, or textbooks.
Most people who learn sociology do so in introduction to sociology, social problems, or intermediate elective courses. Most students will not major in sociology or take another sociology course. Any sociology course should support the development of research consumption skills, which are critical for careers and everyday life. Textbooks have the potential to reinforce these skills. Therefore, these courses and their textbooks are critical to helping the public understand what we do and how we know what we know.
Supplemental Material
sj-doc-1-srd-10.1177_23780231221138728 – Supplemental material for How Do Sociologists Know What They Know? An Examination of Sociology Textbooks for Evidence of Sociological and Scientific Thinking
Supplemental material, sj-doc-1-srd-10.1177_23780231221138728 for How Do Sociologists Know What They Know? An Examination of Sociology Textbooks for Evidence of Sociological and Scientific Thinking by Stephanie Medley-Rath in Socius
Footnotes
Supplemental Material
Supplemental material for this article is available online.
Author Biography
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
