An Insufficiently Substantiated Claim Based on a Confirmation Strategy: Comment on Bartels’ “Indoctrination in Introduction to Psychology”

Abstract

In his target article on “Indoctrination in Introduction to Psychology,” Bartels proposes that in introductory textbooks of psychology studies and their results are systematically presented in such a way that they tend to correspond to left-liberal political positions and that the state of psychological knowledge is reflected in a correspondingly distorted way. In our commentary, we clarify that the evidence Bartels presents for this claim is insufficient. At first, he takes a purely hypothesis-confirming approach based on selective sampling. Second, he draws an invalid causal inference from a supposed liberal majority in the psychological community to their representation of psychological content in textbooks. And third, he assigns introductory textbooks a function that we believe they do not have. Nonetheless, we welcome the discussion of how best to teach critical reflective thinking in psychology courses.

In his article “Indoctrination in Introduction to Psychology” Bartels reviews the presentation of selected topics (three examples after disqualifying another set of three examples for story-telling purposes) in introductory textbooks in psychology. Based on this set of examples and the accompanying literature he comes to the conclusion that introductory textbooks not only misrepresent “numerous topics” but more so that this is the result of decades of liberal bias in psychological science and textbook authors labeled as “indoctrination.” We certainly agree with some fundamental claims: The necessity to separate political orientation from and keep political activism out of scientific practice, the need to include and discuss methodologically sound findings that do not agree with one's own position or preferred narrative, and the importance of critical reflection and teaching critical thinking in psychology courses (and in teaching in general). However, we are not convinced that the presented evidence in Bartels’ article does support his strong claims of liberal indoctrination, instead, it deserves more critical reflection in itself.

Bartels uses the notion of biases in human cognition as an argument for the presence of ideological bias in textbooks that can therefore seemingly not be refuted. He implies that since scientists are not immune to confirmation bias their work is necessarily biased by their political orientation as well. However, Bartels himself presents his entire argumentation without testing hypotheses that might disprove his narrative. Neither his argumentation nor most of his referenced literature is based on representative samples (e.g., specifically selected set of topics that supports the narrative). Even if those were representative topics, in our opinion the presented reasoning does not sufficiently explain why the selected topics would necessarily imply liberal biased reports and neglects methodological critiques that are not politically framed. Finally, the focus on introductory textbooks and their function in psychological curricula needs to be taken into consideration when discussing potential misrepresentation of psychological topics and their critical reflection. Consequently, in this comment, we will address the issue of confirmatory hypotheses testing and biased sampling, political framing, as well as the question of what the function of introductory textbooks is and how to adequately include critical reflection.

Confirmatory Hypotheses Testing and Biased Sampling

In a critical evaluation of the Stanford Prison Experiment, Bartels and Griggs (2019) write that students should reflect on “how Zimbardo could have minimized the impact of his beliefs on the study. For example, he could have structured the study to attempt to falsify his beliefs rather than verify them” (p.301). This basic notion of critical thinking to avoid confirmation bias is probably the biggest shortcoming of Bartels now-published article on ideological bias in the introduction to psychology. At no point is there any serious attempt to falsify the severe allegation of indoctrination. On the contrary, a set of three example topics is used as proof. The selection of those topics at best represents a biased sample if not an intentionally selected set to confirm the hypothesis and narrative. One of the references that he uses to support his claims, Ferguson et al. (2018), states that they selected the topics they investigated based on familiarity with the authors or being subject to public debate. They explicitly “make no claim that these are any form of randomly selected set of potential mythmaking areas” (p.576). In addition to the nonrandomly selected topics the selection of the investigated textbooks in the presented studies varies dramatically without trying to retrieve a representative set of textbooks or conducting a systematic search that clearly defines search and inclusion criteria (like would be standard in literature reviews or meta-analyses). Ferguson et al. (2018) stress for example that “this sample of books is not comprehensive, of course” (p.578) without providing a source that verifies they are amongst the most popular books. In addition, it does not become clear how representative the findings for selected textbooks are compared to introductory textbooks to psychological subdisciplines (e.g., personality psychology). If such a bias would only be present in textbooks that introduce psychology in general, then this would question the presence of an ideological bias (Why should it only be present at a specific level?). On top of this, even the survey by Inbar and Lammers (2012) presented to introduce the politically biased demographics of the field has received criticism for biased sampling and lacking representativeness (Jussim et al., 2015; Skitka, 2012).

While presenting different viewpoints based on contrasting, sound empirical findings is an important element of enabling perspective taking and critical evaluation, presenting opposing views just for its own sake can be highly detrimental. In parts of his conclusion, Bartels creates the impression that a balanced presentation of diverse viewpoints is always beneficial and should be the ultimate goal. However, if the only goal is to provide divers political viewpoints one opens up the opportunity of introducing a misrepresentation of viewpoints and evidence in itself. If there is ample evidence for one side and only few for the other, creating a balanced representation leads to the false impression that both sides are equally well founded, which might not be the case. For example, inviting two alleged experts to present a pro and a contra perspective on the existence of climate change results in a skewed sample of the actual distribution of evidence (Newell et al., 2014). Thus, creating a balanced representation just for its own sake can never be the goal. It always has to be based on the actual distribution of evidence in order to not create a biased sample of the environment which will otherwise most likely lead to biased outcomes (Fiedler, 2000).

Political Framing

Aside from the biased samples, the presented topics and the reported misrepresentation are attributed to an ideological bias, resulting in a mere political framing of reported research in textbooks. The question arises why such misrepresentation should be associated with political orientation and even further, why the presented examples attest to a liberal bias? So far, the author only tried to show that there is a liberal majority in the psychological community (which in itself is nothing new, neither is the debate about why this is the case and of potential impacts; see the debate on a speech by Jonathan Haidt, Brockman, 2011) and that three example topics are not represented accurately or critically enough in introductory textbooks. However, this warrants more of a pseudocontingency (Fiedler & Freytag, 2004; Kutzner & Fiedler, 2017) than an actual contingency. Even if there were a correlation, the causal chain remains up in the air as none of the presented literature takes the political orientation of the textbook author(s) into account, which would warrant a causal relation. In addition, the literature on cognitive biases does not necessarily support the accused mechanism which states that scientists with a liberal bias produce biased outcomes in the expected direction. Classic findings show that biases can lead to overcorrection (Kahneman, 2003), especially when one is aware of that bias (which should be assumed to be the case with the longstanding discussion of a liberal majority), thereby ending up further on the opposite end of the spectrum.

Unfortunately, instead of focusing on relevant methodological criticism of the presented examples, Bartels drifts toward politicized argumentation for ideologically motivated misrepresentation. Thereby he leaves the scientific and methodological debate and omits relevant research that for example criticizes the validity of the Implicit Association Test against faking (Fiedler & Bluemke, 2005; Fiedler et al., 2006).

The Function of Introductory Textbooks and Critical Reflection

Finally, the question arises what the function of introductory textbooks is. Bartels only addresses the aspect of “selling psychology.” However, if we are talking about a basic introduction to psychology textbooks it should become clear that their main goal is to provide a comprehensive overview of psychology, its history, and the major milestones of its various subdisciplines. This sheer amount alone shows that it is almost impossible to cover every single topic with adequate, in-depth criticism that even comes close to doing justice to the state of the research landscape. This of course does not mean or excuse blindly reporting false or highly questionable findings. However, it must be clear that introductory textbooks cannot encompass extensive scientific debates on various topics—and quite frankly, that is not what they are made for. For this purpose, there exist accompanying materials like workbooks, worksheets, and potential online resources. In addition, this of course could be addressed to some extend in the lecture but more likely seminars and tutorials later on in the curriculum are more appropriate contexts for more in-depth critical reflection. Not only due to time constraints, but also because if you want to have profound methodological and theoretical debates and induce critical thinking in your students, they need to have the tools to critically evaluate the presented research. It is very unlikely that freshmen students in introductory courses have enough knowledge to effectively engage in a well-founded scientific debate of specific studies without adequate training in methodology and without knowing about the theoretical backgrounds of the field. Otherwise, you only present students with a study accompanied by a critique and limitations which they are also not able to critically evaluate. This means that you can basically present any critique you want for any study you present because your audience will not have enough training to argue with you. This is quite the opposite of teaching critical thinking (Beaulac & Kenyon, 2014; Maynes, 2015).

Concluding Thoughts

The author of the original paper raises strong accusations of liberal indoctrination in introductory textbooks in psychology, but, in our opinion, those claims are based on a very weak and inconclusive argumentation. We reviewed major issues of the argumentation: Positive testing, sampling bias, political framing, misattribution of the purpose of introductory textbooks, and an adequate setting for critical reflection. Based on these issues, Bartels uses rather harsh language when he already convicts textbook authors of indoctrination, a term that is commonly associated with harmful goals or used synonymous to brainwashing. All of this is based on a rather narrow evaluation that itself lacks attempts to critically evaluate its claims. Even further, the claims are broadly generalized across the entire field of psychological science, despite showing a clear US-centrism, mostly without taking the international community into account and focusing on US-based political topics and categorization. Additionally, for such a broad claim the reviewed references have not only methodological issues themselves (as laid out before), but are also very narrowly focused on a very limited range of specifically selected psychological topics that do not constitute a representative sample of the entire field. Unfortunately, the author failed to apply the same standards he requests from textbook authors to his own work—a critical reflection and evaluation with the presentation of contrasting evidence.

Nevertheless, the measures Bartels proposes at the end of his article (e.g., providing supplementary reading and background information to be able to critically discuss and evaluate specific findings) are valuable steps for a well-founded psychological education. They must be located at the right time in the curriculum, at which students have appropriate prior knowledge and skills. However, we would be cautiously optimistic that these straight-forward measures are already widely implemented and assumed as the standard for teaching psychology.