Abstract
Introduction to the Special Issue
In the Chinese language, the word “crisis” is composed of two characters, one representing danger and the other, opportunity. – John F Kennedy, 1959
These reflections seem an apt way to begin this special issue on teaching in the new era of psychological science. Our field has experienced epistemic upheaval, where important findings of the past have been challenged: brain–behavior relationships have failed to replicate (Boekel et al., 2015), ego-depletion effects have become suddenly elusive (Hagger et al., 2016), and the famous effect of red on physical attraction has declined, perhaps into non-existence (Lehmann, Elliot, & Calin-Jageman, 2018). Much of our “textbook” knowledge may actually be widespread misconception based on poor research practices. As one prominent researcher wrote, “I feel like the ground is moving from underneath me and I no longer know what is real and what is not” (Inzlicht, 2016).
Yet, even though specific findings and theories may now need revision, psychological science retains its broad utility and productivity. In fact, the “replication crisis” has provided a true opportunity, fueling a parallel methodological upheaval to reify the aspects of our science that are most ethical and transparent and that lead us to more trustworthy results. Some have even termed the shift a revolution, “not about the content of our science” but “about the values we hold as we conceptualize, implement, analyze, and share our science” (Spellman, 2015, p. 886).
In response to calls for sweeping change, psychological scientists have engaged in large-scale collaborations that have enabled sample sizes and research questions that were previously out of reach (e.g., Klein et al., 2014, 2018). They have spearheaded changes in statistical reporting to better calibrate research conclusions to the evidence at hand (Cumming, 2012; Levitt et al., 2018). And they have introduced badge systems to reward transparency (e.g., Eich, 2014), new publication models to help make peer review more fruitful (e.g., Nosek & Lakens, 2014), and new tools to foster collaboration, transparency, and a clear demarcation between exploratory and confirmatory research (Center for Open Science, 2017).
We envisioned this special issue of Psychology Learning and Teaching after our experiences presenting on this topic at several conferences. Session attendees frequently requested additional information about open-science-related pedagogy and classroom-ready resources, signaling a need for more dissemination of why and how to teach open science. We hope that this special issue will help to address this need; it features reviews, articles, and reports related to perceptions of open science and the implementation and integration of related topics in the classroom and across the curriculum. In several cases, the authors provide resources and tools for instructors looking to integrate open science into their courses. The contributions come from authors who have seized the opportunity that the open-science movement presents, and are working to train new generations as thoughtful and ethical psychological scientists and citizens.
The Current Issue
The current special issue includes two reviews, three articles, and three reports. In addition, this issue includes abstracts of the current issues of Psychology Teaching Review (PTR25(2)) and Teaching of Psychology (ToP46(4)).
Reviews
Grahe, Cuccolo, Leighton, and Cramblet Alvarez explore open-science initiatives that promote diversity, justice, and sustainability in psychological science. The authors provide questions that might spur researchers in these efforts, and demonstrate how open science makes our discipline more diverse, efficient, just, and sustainable, with greater access to participation in research efforts. This review spurred us, as guest editors, to wonder if the approach these authors outlined might also be a way to combat an anti-science climate. If a greater number of students are actually engaged in science (e.g., in replications) rather than just class projects, they may view themselves as part of a larger scientific community, and, in turn, be less likely to have an overall distrust of science.
Next, Pek and van Zandt explore ways to challenge students’ statistical thinking by elevating process over procedure. They review both frequentist and Bayesian approaches, and provide concrete examples, including classroom-ready supplementary material (i.e., data and R code). Importantly, they encourage statistical thinking rather than the application of a particular rote process, a critical distinction given that the latter is part of what brought us to the replication crisis in the first place.
Articles
van Doorn, Matzke, and Wagenmakers also address Bayesian inference, describing an in-class demonstration; they gave the classic “Lady Tasting Tea” experiment a makeover, presenting a hands-on beer-tasting experiment (that, they observe, could easily be swapped out for cola or coffee) to help students understand Bayesian inference.
Together, Pek and van Zandt’s review and van Doorn and colleagues’ article on Bayesian inference provide an approachable primer—a welcome balance of background and application—for those of us with less experience in these areas. We particularly encourage those new to these approaches to read this pair of contributions. Indeed, van Doorn and colleagues note that it is not necessary to understand the math behind Bayesian belief-updating in order to understand the basics of Bayesian hypothesis testing.
Sarafoglou, Hoogeveen, Matzke, and Wagenmakers then offer additional practical guidance for instructors. They outline a student-centered research course they have developed based on Chambers’ book, The 7 Deadly Sins of Psychology, that offers practical open-science instruction. Such a course could be appropriate either at the master’s or advanced-undergraduate level. Moreover, instructors could repurpose individual modules from the course and embed them in existing statistics or methods courses as a stepping stone toward a wholesale curricular change.
Anglin and Edlund’s article may be the one you want to read first. They set the stage for the timeliness and importance of this special issue. They report findings from a survey of undergraduate and graduate instructors about the frequency of their inclusion of topics related to replication and reform of research across the curriculum. The bad news? Most instructors are teaching these topics only minimally, or not at all. The good news? A number of instructors, particularly in upper-level classes including statistics and research methods and among those who supervise undergraduate research, are doing this work. And most instructors perceive that we need to do more, suggesting that many of us are ready to shift our teaching practices.
Reports
Finally, the special issue includes three reports that, as a group, might serve as responses to the problems outlined by Anglin and Edlund. All three point out how open-science methods can be taught within the framework of existing coursework or curricula, and offer concrete suggestions for doing so.
Button, Chambers, Lawrence, and Munafò offer a training model that complements the student-centered research course described by Sarafoglou and her colleagues. They outline the development of a consortium that disrupts the current paradigm of how undergraduate final-year research projects are carried out; in many countries, standard practice involves one student engaged in one original research study, with all the limitations that implies—think low statistical power and less-than-creative projects. Why not, they argue, combine forces? They present a model in which multiple undergraduate students from multiple institutions collaborate under the tutelage of professors, doctoral students, and post-doctoral fellows; together, they conduct one large study, but with various hypotheses that can fuel independent research projects under the consortium umbrella. Importantly, this approach to training supports the acquisition of employable skills; indeed, being able to work in teams reflects the future, not just of science, but of the workplace more generally.
Jekel, Fiedler, Allstadt Torras, Mischkowski, Dorrough, and Glöckner then describe another creative way to provide research experiences to multiple students. The Hagen Cumulative Science Project is an innovative initiative in which undergraduate and master’s students conduct replications of research, rather than original research. This project thwarts some of the same limitations addressed by Button and her colleagues in the consortium approach. The side-by-side comparison of these reports may spur even more variations of learning experiences for students that center open science and avoid underpowered, less-than-original projects.
Finally, Blincoe and Buchert detail a preregistration activity for use in a capstone research course, along with data on their students’ perceptions of preregistration. For faculty members who are thinking of revamping their curricula, or even developing new thesis models à la the consortium model or the Hagen Project described above, Blincoe and Buchert’s clear instructions and clear-eyed examination of the pros and cons of preregistration templates provide an entry point for injecting open science in a way that promotes critical thinking.
The contributions to the special issue work particularly well when considered in their entirety. Anglin and Edlund’s article outlined the current gulf between the relative scarcity of open-science instruction in psychology courses and the much higher perceived need for such instruction, starkly underscoring the importance of more guidance on how to teach these topics. Grahe and colleagues further highlight the importance of open-science initiatives in fostering diversity, justice, and sustainability. The remainder of the special issue answers these calls by providing guidance on how Bayesian inference can fit in, how we can teach open science across the curriculum through hands-on and collaborative research activities and projects, and what an entire student-centric, open-science-focused research course can look like.
We hope this special issue will inspire you to embed more open science teaching and learning in all of your classes!