Say our name (and say it right)! Extending Walton et al. on the evolution of race in political science scholarship

Abstract

To explore the evolution of political-science research on race, Walton et al., have done a systematic review of more than a century of publications appearing in the discipline’s oldest and most prestigious journals: Political Science Quarterly and the American Political Science Review, respectively. Walton and his colleagues uncover “dual traditions” of race scholarship: an “African American Politics” (AAP) paradigm emphasizing empowerment and Blacks’ cultural distinctiveness, and a “Race Relations Politics” (RRP) approach that focuses on Blacks’ socio-political status vis-à-vis Whites. Using computer-assisted text analyses, we introduce a measure of racial dialogue that is informed by theory and has suitable empirical properties. We replicate and extend Walton’s research by adding a third periodical (the Journal of Politics) and demonstrating that, while race conversations are becoming more frequent over time, the dialogues taking place in mainstream journals typically fit Walton’s RRP (rather than AAP) tradition. Following our analyses, we offer guidelines for researchers seeking to apply our measure to alternative contexts.

Introduction

Race scholarship ¹ has been particularly enlightened by political empowerment, a topic that is an important area of research (Bobo and Gilliam, 1990) as well as an essential feature of racial politics itself (Perry, 2013). How scholars perceive African-American political life is investigated in the seminal works of Walton et al. (1995) and Walton (1997, Chapter 4). Empowerment, in their formulation, is about symbolism: when Blacks move politically—or when moves are made against them—what do race scholars observe (and, by extension, write about)? To answer these questions, the authors examine over 100 years of publications in the discipline’s oldest and most prestigious journals, Political Science Quarterly (PSQ) and the American Political Science Review (APSR), respectively, and they uncover a scholarly viewpoint split into two camps: a side that treasures Black empowerment; and one that does not. They name the side that treasures empowerment “African American Politics” (AAP); the other side of the coin is dubbed “Race Relations Politics” (RRP). Walton et al. (1995) acknowledge that RRP authors making Black–White comparisons can do so on behalf of racial subjugation or acquiescence—for instance, by holding Whites up as standard bearers, blaming Blacks for delayed racial progress (or overemphasizing the importance of White benevolence in ameliorating race problems), advocating strategies for uplift that do not upset the status quo (and might even reinforce hierarchies), and so on. Likewise, Walton (1997) reminds readers that conversations about empowerment are often linked to demands for Black liberation: publications in the AAP tradition can celebrate Black life, point out the inadequacy of mainstream efforts for racial progress, and champion strategies that advance the race. ²

Walton and his colleagues offer an artful analysis based on nuances in language and theory. We generalize from the authors’ meta-study, making the process by which publications are deemed AAP or RRP transparent for readers. Specifically, we characterize political-science race research as a process by which authors “say Blacks’ name” (making African Americans prevalent in the document) and “say Blacks’ name right” (by striking a favorable tone when discussing this group of citizens). Combined, prevalence and tone help us to analyze the centrality of race to our discipline’s discourse. By building an instrument that gets at what Walton and his colleagues observe, we re-analyze their corpus of text in novel ways and apply our instrument to current articles in top-tier political science journals. This replication-and-extension confirms not only that ours and Walton’s operationalizations are empirically comparable but also that the former can serve as a proxy for the latter when reviewing the race literature. Consistent with McClerking and Walton (2010), we discovered that race conversations are becoming more frequent over time, but the dialogues taking place in mainstream journals typically fit Walton’s RRP (rather than AAP) paradigm.

Saying Blacks’ name, and saying their name right

Walton et al. lay a rich foundation for studying the evolution of race politics scholarship. Philpot and White (2010: 5–6) summarize the logic of Walton’s “dual traditions” argument:

Two major research veins have appeared in the study of African American politics—a race-relations [politics] literature, which focuses on African Americans’ place in American political culture vis-à-vis Whites, and an African-American politics literature whose emphasis is more on political empowerment and cultural distinctiveness.

Walton’s concept of interest is the centrality of conversations about race. Implied in this notion of centrality are considerations of both the prevalence and tone of racial dialogue: authors can prioritize or de-emphasize discussing race, and those conversations can be uplifting or denigrating in connotation. Beyond merely mentioning Blacks by name, papers from the AAP tradition also say the right things about African Americans by characterizing them in a complimentary manner. Centrality is perhaps too narrow to encompass the full scope of empowerment and its accompanying challenges. But the concept is adequate for exploring how/why some scholars mention African Americans “vis-à-vis Whites” in their writing while others promote the edification of Black life.

To assess prevalence, Walton et al. ask: when race is being discussed, are the political empowerment and/or cultural distinctiveness of African Americans a central feature of the publication? To capture more variation, prevalence is best conceived of as a continuum. Therefore, we ask: when race is being talked about in a publication, how central a topic is political empowerment and/or cultural distinctiveness? Documents can refer exclusively to African-American empowerment, they could never mention it, or they may fall anywhere between these extremes. We operationalize the prevalence component of our centrality index as the percentage of a document’s in-group racial references

$$centrality=tone\times \left[\left(\frac{ingroup}{ingroup+outgroup}\right)\times 100\right]$$

Prevalence is typically measured by tallying documents’ references to the in-group (“Blacks” and its various synonyms). The rationale is that larger tallies signal a stronger commitment among authors to having race dialogues (Alexander-Floyd, 2014; Wilson, 1985; Wilson and Frasure, 2007). The denominator, often implied by this approach, is a paper’s total word count. We propose a more relevant denominator. The portion of the formula in square brackets calculates the percentage of in-group mentions relative to other race words, defined here as allusions to both the in-group and out-group (derivations of the term “Whites”). For example, a paper with six in-group mentions out of 100 total words is referring to Blacks 6% of the time. Assuming the document contains 10 race words, it is more useful to know that African Americans make up 60% of the racial dialogue and 40% of that conversation is about Whites.

One weakness of our prevalence measure is that it can yield false positives. Papers that mention African Americans frequently, while making few references to Whites, could be discussing racial subjugation rather than uplift. The authors’ prevalence score could still classify the document as AAP, but a study about subjugation would be incompatible with that scholarly tradition; ³ therefore, we address this issue by considering a document’s “tone.” Linguistic psychologists infer tone by comparing the frequency of positive-affect and negative-affect words (Kahn et al., 2007; Pennebaker et al., 2015a; Tausczik and Pennebaker, 2010). Following this precedent, we conceive of centrality as a function of prevalence, weighted by tone. The weighting variable (located to the left of the square brackets) is structured so that higher values suggest a tone that is more favorable toward Blacks. Our centrality index, therefore, generates a “weighted percentage” in which papers focusing primarily on Blacks (i.e. earning higher marks for prevalence), while criticizing this racial group (receiving lower values for tone), get reduced centrality scores.

Analysis and discussion

Walton replication

We collect data from JSTOR, an online archive of scholarly writings. Our timeframe (1885–1990) covers the period analyzed by Walton et al., and we include the same 54 publications the authors analyzed. ⁴ Linguistic Inquiry and Word Count (LIWC), an automated text-mining program, scans documents for patterns in word use, punctuation, etc. (Pennebaker et al., 2015b). LIWC extracts tone metrics by matching a document’s text to a pre-compiled “dictionary” of terms representing positive and negative emotions (Pennebaker et al., 2015a). ⁵ We re-scale tone to range from zero to one, with higher values indicating an upbeat writing style, and texts with lower values feature words connoting anxiety, sadness, or hostility. As noted earlier, prevalence gauges the in-group racial references in a paper. LIWC’s default dictionaries do not follow race words, so we customized our own search routine (Appendix 2). We multiply prevalence and tone to create our index of centrality. The average centrality score is 28.43, with a range from 2.99 to 54.63 and a standard deviation of 13.58. So a typical article among the 54 is less than one-third of the way up the scale, and only 3.70% of those papers are above 50% on this measure.

We adopt the authors’ typology for distinguishing RRP and AAP papers, and readers should consult the reference lists in Walton et al. (1995: Appendix 1) for details. As Figure 1 shows, publications with centrality scores at the higher end of the scale are likely to fall within the AAP tradition, and papers scoring at the lower end are often classified as RRP. The horizontal axis divides scores into cut-points: the lower threshold ranges from the minimum value to 15%, which is approximately one standard deviation below the mean (28.43%–13.58% ≈ 15%). The upper threshold contains values that are at least a standard deviation above the mean (28.43% + 13.58% ≈ 42%), and the middle threshold captures scores between 15% and 42%. The vertical axis records the proportion of publications meeting these thresholds, while grey (black) bars distinguish RRP (AAP) papers. If “greater than 42%” of the weighted race words refer to Blacks, then nearly 8 of 10 articles fulfilling these criteria are AAP, and less than one-quarter of these documents are RRP. The inverse of this pattern emerges when the cut-point is “less than 15%:” nearly 9 of every 10 publications are RRP, while 1 in 10 papers are AAP. It makes sense that articles with the highest centrality scores meet the AAP standard, and that publications with lower scores are overrepresented in the RRP category. More importantly, while noting the limits of his admittedly rigid classification system, Walton (1997: 61) concedes that RRP and AAP papers can overlap. Our continuous measure addresses this challenge by allowing for finer gradations of centrality levels.

Figure 1.

Across-tradition differences in the centrality scores of papers meeting various thresholds.

Scholarly attention paid to Blacks often reflects watershed racial events. Given the timeframe covered here, there is no event more pertinent than the Modern Civil Rights Movement—hereafter referred to simply as the “Civil Rights Movement”—which is viewed by many historians as having occurred between 1955 and 1968 (Smith, 1996). Following McClerking and Philpot (2008), we classify articles published before 1956 as appearing before the height of the Civil Rights Movement, and the papers coming afterward should be shaped by these events. Like Walton (1997: Table 4.3), we trace racial discourse over time and across journals. Figure 2 contains separate scatterplots for each journal. The vertical axes record the average centrality level, while horizontal axes track the decades covered (1890 through 1990 for PSQ, and 1900 to 1990 for APSR). Each dot signifies a mean score for that particular decade, and its size is proportionate to the number of papers published during that period. Vertical dashed lines distinguish articles written before and after 1956. Mean-difference tests show that centrality levels in the APSR tend not to vary with time (before 1956: mean = 29.26%; after 1956: mean = 24.77%; F = 0.44, p > 0.05). There is more contrast in the “before” (mean = 35.37%) and “after” (mean = 27.60%) scores in the PSQ, but the difference is non-significant (F = 2.28, p > 0.10). Although centrality does not change much over time, papers about race become more numerous after the Civil Rights Movement. As the size disparity in the dots before and after 1956 illustrate, the average frequency of race papers per decade rises from 4.38 to 12.53 in the APSR (F = 15.89, p = < 0.01), and from 3.91 to 11 in PSQ (F = 20.14, p < 0.01).

Figure 2.

Centrality scores across decades, according to frequency, and by journal (part 1).

Extension analyses

Aside from its distribution resembling that of Walton’s, our index exhibits construct validity (Cronbach and Meehl, 1955) because centrality correlates in predictable ways with other variables. We now assess centrality’s robustness by: (a) expanding our viewpoint with articles from a third general-interest journal; and (b) extending the time span from the 1990s to 2015. Like APSR and PSQ, the Journal of Politics (JOP) is a leading periodical whose editors and reviewers serve as research “gatekeepers” (Mohammadreza et al., 2003). JOP is published by the Southern Political Science Association. Given the widely-documented connection between southern politics and racial identity ⁶ adding JOP helps us to compare race discourse across mainstream journals of similar pedigree while also exploring such dialogue in an outlet that is known for showcasing identity-based research (for a similar justification, see McClerking and Walton 2010). We supplemented JSTOR searches with scans of journals’ online archives (Appendix 2) to locate 151 additional articles: 35 more from APSR, an extra 15 from PSQ, and 101 from JOP. This new dataset merges the “replication” and “extension” stages of data collection and contains a total sample of 205 papers (Table A1 in Appendix).

Centrality levels in the extension dataset concentrate along the bottom two-fifths of the scale. These scores have a mean of 21.73%, and minimum and maximum scores of 0.92% and 57.94%. Compared to the average of 28.43% for the papers appearing in Walton’s study, articles from our updated collection tend to get lower scores (F = 10.14, p < 0.01). But the variability in these scores are similar across datasets: the standard deviations for the replication and extension studies were 13.16 and 13.58, respectively, and an equality of variance test shows that these difference are non-significant (f = 0.94, p > 0.10). In Figure 3, we illustrate the distribution of centrality scores in the extension dataset using kernel density plots. The horizontal axis contains placeholders for every possible score that publications can earn, while the vertical axis records how frequently such scores appear in the data. Most publications fall within the 13% to 35% range. Documents that get centrality scores of 40% or higher are rare, as are papers scoring 10% or lower. The look of these distributions varies; this is particularly true for the bimodal appearance of the JOP scores. That said racial dialogues are empirically similar across journals. Mean-difference tests confirm that the centrality scores of papers appearing in the APSR (22.57%), JOP (22.59%), and PSQ (27.05%) are comparable (F = 1.83, p > 0.10), and equality-of-variance tests show that the standard deviations for these journals (12.30%, 14.06%, and 13.89%, respectively) are similar. ⁷

Figure 3.

Between-journal comparisons of the distribution of centrality scores.

The organization of information in Figure 4 is similar to that of Figure 2, and the results across both sets of analyses are largely consistent. Using bivariate correlation to explore changes in average centrality levels by decade and between journals, we see that there is no statistically-significant time trend in the centrality scores of papers from the APSR (r = 0.22, p > 0.05). Yet, centrality decreased significantly over time in the JOP (r = −0.33, p < 0.01) and PSQ (r = −0.29, p = 0.06), which suggests that publications befitting Walton’s RRP tradition are becoming increasingly more common in these journals. Judging from the fluctuations in the sizes of the dots, we see that, regardless of journal, the number of articles focusing on race has been increasing (APSR: r = 0.72, p < 0.01; JOP: r = 0.49, p < 0.01; PSQ: r = 0.88, p < 0.01). Overall, dialogues about race in these top-tier journals have become more frequent (measured as the number of publications), while the focus on Blacks within these dialogues (operationalized by our centrality index) has declined in two of the three journals analyzed here.

Figure 4.

Centrality scores across decades, according to frequency, and by journal (part 2).

The above findings are clearer when we explore centrality scores and publication frequencies across racial periods. Mean difference tests of publication frequencies demonstrate that, compared to the pre-Civil Rights Movement era, the APSR (F = 61.03, p < 0.01), JOP (F = 16.88, p < 0.01), and PSQ (F = 44.22, p < 0.01) featured more race research after 1956. Although references to African Americans in the text of articles appearing in the APSR (F = 2.8, p > 0.05) and JOP (F = 0.62, p > 0.05) did not change much after the Civil Rights Movement, the centrality of racial dialogue in PSQ articles swelled significantly (F = 6.01, p < 0.05). Additional analyses (not shown) comparing racial dialogues before and after 2008 suggest an “Obama effect.” While it is not the case for the APSR (F = 1.71, p > 0.05), evidence from the JOP (F = 13.62, p < 0.05) and PSQ (F = 8.80, p < 0.05) reveals that articles written after Obama declared his presidential candidacy get higher centrality scores than those published before 2008. Moreover, race papers are significantly more common in the JOP (F = 3.09, p = 0.06) and PSQ after 2008 (F = 3.09, p < 0.05), but there is less evidence of an Obama-induced increase in the frequency of race papers being published in the APSR (F = 0.09, p > 0.10). We explore these and related patterns in Figure A1 of the Appendix. ⁸

Conclusion

Political-science discourse draws its inspiration from real-world events. Walton and his colleagues argue that the key variable in the study of race politics research is the centrality of empowering and/or culturally-distinctive dialogues within broader conversations about group relations. In their comprehensive literature review, the authors explore the distribution of racial centrality, as well as its correlates, in publications from two flagship journals. Following this lead, we introduce a theoretically-grounded measure of racial centrality. Our analyses capture some aspects of what Walton et al. (1995) discuss, and, given the necessity of replication (King 1995), it is encouraging—but hardly surprising—that Walton’s work not only stands the test of time but also holds up to rigorous re-examination. We hope that others will expand upon the work Walton started, for our approach lends itself to an exploration of race dialogue in other political science journals, and, ultimately, to comparative analyses (not unlike Wilson, 1985) that chronicle discourse in other social-science and humanities disciplines. Although we focus on African Americans here, we recognize that insights from our work might inform analyses of the evolution of ethnic (e.g. Latino and Asian), gender, and sexuality politics scholarship.

Economists distinguish between “positive” and “normative” research, characterizing the former as data-driven and the latter as value-based. RRP papers are often considered to be positive political science, while AAP publications are billed as normative works. Admittedly, the distinction between positive and normative is unclear, reflecting social hierarchies and academic norms (see Caplin and Schotte, 2008). Students of Black politics will agree that the work of eminent scholars like Mack Jones, Ron Walters, Jewel Prestage, Nick Nelson and Linda Faye Williams, are just as much about advancing knowledge as they are about racial advocacy. By revisiting Walton’s work, we hope to enliven the ongoing conversation about race in American politics, and we invite others to join in.