Open Science Promotes Diverse,Just,and Sustainable Research and Educational Outcomes

Abstract

Open science initiatives, which are often collaborative efforts focused on making research more transparent, have experienced increasing popularity in the past decade. Open science principles of openness and transparency provide opportunities to advance diversity, justice, and sustainability by promoting diverse, just, and sustainable outcomes among both undergraduate and senior researchers. We review models that demonstrate the importance of greater diversity, justice, and sustainability in psychological science before describing how open science initiatives promote these values. Open science initiatives also promote diversity, justice, and sustainability through increased levels of inclusion and access, equitable distribution of opportunities and dissemination of knowledge, and increased sustainability stemming from increased generalizability. In order to provide an application of the concepts discussed, we offer a set of diversity, justice, and sustainability lens questions for individuals to use while assessing research projects and other organizational systems and consider concrete classroom applications for these initiatives.

Keywords

Diversity justice sustainability open science reproducibility scientific transparency

Introduction

A cascading series of open science initiatives has emerged since 2011. Many of these initiatives offer opportunities for students; other initiatives facilitate storage and widespread sharing of research plans, materials, data, and manuscripts. These initiatives adopt open science principles, are free, and available to present and future generations of scholars. Though researchers working alone can practice open science, many initiatives explicitly encourage greater collaboration across researchers. These collaborative initiatives particularly provide opportunities to teach about the value of diversity, justice, and sustainability, and provide diverse, just, and sustainable outcomes to undergraduate and advanced researchers. We assert that researchers should adopt a diverse, just, and sustainable lens when choosing and developing projects. When critically focusing a diverse, just, and sustainable lens on our science, we see research and researchers that are not sufficiently diverse, voices and perspectives that are not equally or equitably supported, and outcomes with limited sustainability and insufficient concern for advancing environmental sustainability or other practical concerns. We suggest that by inviting students into open science initiatives in the classroom, long-term outcomes will increasingly yield diverse, socially just, and sustainable research outcomes and human communities.

Finding Diversity, Justice, and Sustainability in Psychological Sciences

Diversity

The importance of diversity in science has long been recognized (Fausto-Sterling, 1985; Intemann, 2009; Longino, 1990, 2016). Diversity often refers to identity or demographic diversity: the experiences and perceptions that are a product of our sex, gender, religion, age, ethnicity, or other aspects of our background or identities (e.g., Page, 2007). Despite the numerous benefits of diverse perspectives (Fiske, 2010; Stahl, Maznevski, Voigt, & Jonsen, 2010) the pool of researchers and participants is incredibly homogeneous, as is the composition of faculty (McFarland et al., 2018). Likewise, student perspectives, especially those from underserved populations, have been repressed (Markus, Steele, & Steele, 2000), which significantly hinders the field’s ability (and ethical responsibility) to serve those persons.

Also important for diversity is cognitive diversity: the difference in patterns of thought, problem-solving, perspectives, inferences, etc. (Page, 2007). Identity and demographic differences are systematically related to cognitive diversity. For example, in studies of individuals from Asian and Western cultures, differences exist in categorization of objects, salience of background context versus foreground objects, and even causal relations (Nisbett & Miyamoto, 2005). Science is a process that requires cognitive diversity to foster unique and innovative research ideas and interpretation. We propose that implementing open science initiatives promotes multiple forms of diversity (identity, demographic, and cognitive) because open science initiatives and methods enlarge psychological science beyond its traditional base of high productivity researchers to include researchers and students less traditionally represented (Eagly & Miller, 2016; Jones, 2019; Vazire, 2017).

Justice

Through a diverse, just, and sustainable lens, the view should include distributive and procedural justice in psychological science. Distributive justice (e.g., Deutsch, 1985) requires that the means and benefits of research accrue fairly to all participants; specifically, the distribution of assets such as scientific education (i.e., preparation for, and access to, baccalaureate and postgraduate education), training (i.e., mentorship, research experiences), and benefits (i.e., the products of research should be distributed broadly, especially to the subjects of the research). A just psychological science would also include procedural justice (e.g., Lind & Tyler, 1988; Thibaut & Walker, 1978), whereby all participants have voice in, and control over, the scientific process.

Situating both distributive and procedural justice within the broader context of human rights means everyone is entitled to take part in and reap the benefits of the scientific process (American Association for the Advancement of Science, 2018). For example, psychology can be used to advance human rights agendas (i.e., acknowledging and reducing racial biases) through culturally sensitive research, uplifting and enabling marginalized voices, and advocacy that informs government policy (American Association for the Advancement of Science, 2018).

Equipping laypersons with critical thinking and reasoning skills provides a base for individual empowerment to produce informed social change. However, this empowerment largely depends on both justly fostering scientific literacy and the just diffusion of scientific knowledge (American Association for the Advancement of Science, 2018). A just psychological science means that both privileged and marginalized populations have access to, and are agents of, research conceptualization, education, training, dissemination, and outcomes (Intemann, 2009). Finally, including both distributive and procedural justice ensures that persons’ rights to enjoyment of scientific progress and its benefits are being met (UN General Assembly, 1948).

Sustainability

Sustainability is often conceptualized as a process of environmental or economic growth that does not outstrip available resources (e.g., Basiago, 1995; Hardin, 1968; Schiebinger, 1997). In the context of sustainable science, sustainability represents a shift from a seventeenth-century science of, and for, the cultural and academic elite to a twenty-first-century science that asks critical questions such as:

“Science for whom? How is our knowledge influenced by who is included in science and who is excluded, which projects are pursued and which ignored, whose experiences are validated and whose are not, and who stands to gain in terms of wealth or well-being and who does not? And for how long?” (Schiebinger, 1997, p. 212).

These questions imply a sustainable psychological science requires both justice and diversity, which are necessary, not sufficient, conditions for sustainability. That is, psychological science can be both just and diverse with involvement and benefits of science distributed widely, but sustainable science also requires goals, processes, and outcomes of research to benefit future generations as much as the present one (Schiebinger, 1997). We argue that open science initiatives help serve a sustainable psychological science through diversity, justice, and methodological changes (e.g., open data, open materials, preregistration, and replication) which ensure present advancements remain valid and reliable.

Considering Psychological Sciences from a Diverse, Just and Sustainable Lens

A useful model for an interactive diverse, just, and sustainable lens (see Seghezzo, 2009) originated to address questions of ecological and developmental sustainability. This model delineates three factors of sustainability: persons, place, and permanence. Persons represents individuals with their unique perspectives, goals, and ideas. Place represents the sources of facts, identities, and behaviors we engage in. Permanence represents the future effects of our present actions (or inactions), and the changes and improvements we make to serve these considerations. Diversity is embodied in the persons factor, justice is embodied in the place factor, and sustainability is embodied in the permanence factor. In this model, sustainability through open science initiatives exists at the intersection of these three factors (see Figure 1).

Figure 1.

The Diversity, Justice, and Sustainability Model for Psychological Science. Adapted from Seghezzo (2009).

Improving psychological science through a diverse, just, and sustainable lens focuses our efforts on preparation, training, and inclusion of diverse individuals (the persons factor), support from institutions and infrastructure in the form of distributive and procedural justice (the place factor), and scientific methods and procedures that change and adapt to ensure that future generations will benefit (the permanence factor). With these three factors in place, the synergistic result should be a sustainable psychological science. Arguably, the open science initiatives we reference here are critical to creating a diverse, just, and sustainable psychological science by enabling synergy between persons, place, and permanence factors.

Focusing a Diverse, Just, and Sustainable Lens

Evaluating whether to adopt a research project for classroom or professional purposes requires consideration of the match between resources and potential benefits. It is also critical to examine the larger values that influenced the development and intended use of the research project. We provide brief descriptions of open science initiatives that we discuss in this paper on our project page (https://osf.io/rh7ta). These initiatives can be broadly categorized into crowdsourcing projects (where various lab resources and researchers are pooled into a single study) and tools that promote transparency (online repositories, Open Science Badges). In order to extend this evaluation to these and other projects, initiatives, or practices, we introduce 12 diverse, just, and sustainable lens questions that might be considered (Grahe, Cuccolo, Leighton, & Cramblet Alvarez, 2019; see https://osf.io/qv8nt/wiki/home/).

Persons: Open Science Initiatives Promote Diversity

Increased inclusion at both the level of individual contribution and the open-access public dissemination of findings invites greater diversity of voices in the scientific conversation, including research collaborators, participants, and the consumers of the research. Cundiff (2012) suggests that gains have been made in the representation of racial-ethnic minority groups in senior author and editor roles, while women remain underrepresented as senior authors but not as editors. With regard to diverse representation as subjects of research, racial-ethnic minorities are underrepresented in data, as are other historically oppressed groups (Cundiff, 2012). Other evidence echoes concerning trends in publication bias in which males typically inhabit first or last author positions (indicating a leadership role) and that reviewers report a preference for male-sounding names (West, Jacquet, King, Correll, & Bergstrom, 2013).

The inclusion of diverse voices is critical to the vertical movement of psychology as a science for several reasons. First, diverse collaborators representative of the population being studied allows for a more intimate understanding and application of cultural relativism to the effect of interest (Fisher, et al., 2002; Jones, 2010). Second, diverse participants increases generalizability, allowing for results to be applied to more persons, moderating/mediating effects of culture/demographics to be understood, and the likelihood of replicability to increase (Jones, 2010; Pratto & Stewart, 2012; Sue, 1999). Third, diverse consumers of research increases the ways in which that knowledge can be utilized.

Open science initiatives are also critical in supporting diversity through access to the scientific research process; specifically, students benefit from their inclusion as diverse voices in preparation, training, and research. For instance, open science initiatives might invite students to prepare study materials, pursue as-yet-unanswered research questions using extant data sets, run replications, or conduct their own extensions to completed studies. This prepares and trains students for their careers as psychologists by promoting cultural sensitivity, critical thinking, and scientific inquiry, while simultaneously allowing them to make a contribution to the field via publication.

Additionally, several open science initiatives promote the inclusion of diverse voices in research as both leaders/scientists and participants. Open science initiatives that include diverse voices in the inputs of the system (research design) and implementation (participant pool) enable students to network. This promotes students’ professional growth and cultivates scientific inquiry (Kniffen & Hanks, 2018). Diverse inputs may help to prevent ethnocentric research designs and expand research beyond WEIRD (Western, Educated, Industrialized, Rich, Democratic) samples currently cluttering psychological science. Inclusion of diverse voices and implementation further assists in generalizability. These initiatives are relatively low cost (due to collaboration and sharing of materials), so access to the conversation is not only inviting, but possible, regardless of resource availability.

Place: Open Science Initiatives Promote Social Justice

The hierarchical academic research model with few top-producing (Eagly & Miller, 2016) researchers dominating the conversation limits emerging academics and marginalized populations. Science is intended as a meritocracy, with quality work being rewarded. As the open science movement has demonstrated, and generations of female and minoritized researchers can attest, credit is easily stolen, and eminence at times a carefully constructed mirage. Transparency, as argued by Vazire (2017), is critical for quality science and diminishing the veneer of protection afforded by eminence which perpetuates publication bias. Those with high status tend to be white, male, able, heterosexual, etc., and thus maintain the status quo of those voices dominating the conversation. Open science initiatives offer mechanisms and opportunities for diverse (and relatively new) groups of researchers to demonstrate novel and important science.

Emerging academics at the level of student and early career faculty face significant barriers to research productivity (e.g., Cohen, Morgan, DiLillo, & Flores, 2003; Galassi, Brooks, Stoltz, & Trexler, 1986). Further, other traditionally marginalized groups (i.e., people of color, women, first-generation and non-traditional students, student parents, LGBTQ, and persons with intersectional identities) face disadvantages in terms of both opportunities and resources (e.g., Evans & Cokley, 2008). Early career researchers often struggle to balance the need for personal/family time against the demands of teaching, research, service, and fulfilling tenure requirements (Good et al., 2013; Sharobeam & Howard, 2002). Additional time demands such as inequitable teaching and service loads for female and minoritized faculty systematically detract from scholarly efforts (Gasser & Shaffer, 2014). Taken together, these systemic barriers may hinder students and researchers from achieving their full academic potential.

Open science initiatives provide both resources and opportunities for these groups. First, two main tenets of open science initiatives are openness and transparency. As such, open science initiatives often provide materials to contributors at no cost and make their datasets publicly available for researchers to download and ask as-yet-unanswered research questions. This mitigates both the financial burden and time constraints of conducting research. Collaborative projects allow researchers to pool their expertise and overcome the barriers of an underresourced lab. These projects lessen the burden on any one researcher or institution to collect a large sample that would be necessary to power desired/planned analyses. Instead, many collaborators can collect smaller sample sizes and pool the participants for adequate power (Moshontz et al., 2018; Wagge et al., 2019a, 2019b). Moreover, collaborative projects allow for broad sharing of expertise (Moshontz et al., 2018) and researchers need not be deterred by a lack of familiarity with advanced or specialized statistical techniques, data collection platforms, or statistical packages. The ability of open science initiatives to lessen both the financial and time constraints of research may facilitate research among groups such as students, early career, and minoritized researchers who may experience systemic barriers which affect scholarly productivity.

Another social justice issue is the dissemination of knowledge—what is being distributed and by whom. Transparency tools such as the Transparency and Openness Promotion guidelines, Open Science Badges, data and materials resource repositories, and open access publishing provide an infrastructure that allow for a just distribution of research. These tools, in particular repositories such as the Open Science Framework (OSF), provide the “place” for justice in science. These structures may also assist with ameliorating publication bias, which extends beyond gender and ethnicity to journals’ obsession with improving impact via “sexy” findings (Vazire, 2017), a practice that has historically perpetuated the file drawer phenomenon (Rosenthal, 1979). In other words, only significant results are received by the scientific community, and who obtains “significant” is limited. Individuals can circumvent this bias via several open science initiatives such as preprints, online data/materials repositories, and PsyArXiv.

The benefit to a more equitable dissemination of findings is that one can have a more realistic expectation of an effect/phenomenon. If only significant results are published, researchers may operate under a misconception that a certain effect is “guaranteed,” thus leading them to undertake research that is based on a biased representation of an effect (Giner-Sorolla, 2012). As such, faculty and students can make more informed research decisions about which projects to pursue, and can also make a meaningful impact in psychological science by sharing their findings publicly.

Likewise, replications are traditionally undervalued in psychological science despite their epistemological value. Replications help combat Type I errors, identify boundary conditions and moderators, and validate the credibility/robustness of a finding (Schmidt, 2009; Simons, 2014; Zwaan, Etz, Lucas, & Donnellan, 2018). Open science initiatives, particularly the student-friendly crowdsourcing projects, can serve as a means of conducting and archiving replication studies as initially proposed by Grahe et al. (2012). Such projects not only increase the credibility of psychological findings and ensure accurate representation of effects, but are also great projects for students to undertake to learn about research methods and scientific inquiry.

Permanence: Open Science Initiatives Promote Sustainability

Compared with traditional research models, open science initiatives provide research outcomes that can generalize to a larger portion of humanity, increasing external validity, and spreading the benefits of the research beyond the sample. Collaboration enables more diverse samples to be recruited (with contributors from different geographical locations and cultural settings). Indeed, a diverse, just, and sustainable benefit of all open science initiatives is to derive more resources to conduct research that is not dominated by WEIRD samples. WEIRD samples differ from non-WEIRD samples in domains such as education, income, and physical health known to affect psychological outcomes (Arnett, 2008). Further, it is inappropriate to assume effects observed in one population will generalize to another distinct population. For example, what is perceived to be “fair” and “cooperative” differs between industrialized and small-scale societies (Henrich, Heine & Norenzayan, 2010).

Reliance on WEIRD samples severely restricts the assumptions that can be made about human behavior, or effects applicable to all humans, as it cannot rule out the chance that the result was obtained due to specific sample characteristics such as demographic, regional, or cultural factors. Moving away from reliance on strictly WEIRD samples will increase the generalizability of psychological research (Sears, 1986; Stolp, 2017) and as such the accuracy and detail of the conclusions one can draw about the effect, the boundary conditions under which it may exist, and potential moderators. Increased generalizability also increases the chances that a study will be replicated in independent samples because the original effect was observed among participants with differing characteristics. Indeed, using a diverse sample in one’s research contributes to a broader goal of psychological science: understanding mechanisms and factors that influence the effect of context on a particular finding (Bennis & Medin, 2010; Machery, 2010). In turn, the research is more likely to hold up over time when replicated or conducted elsewhere as the effect was originally observed among many different persons (Munafò et al., 2017). In order to have a healthy world and a credible psychological science, we need to have sustainable outcomes, such that results apply to humanity broadly, and not only a subset of the broader population.

Open science initiatives are resource-efficient projects that create more sustainable research. Academic faculty, especially women, commonly experience burnout (Lackritz, 2004). Women and minoritized faculty may have significant time pressure and barriers due to increased teaching loads, etc. (Gasser & Shaffer, 2014). Given that an individual’s workload and its intensity, time demands, and complexity are all factors underlying burnout, open science initiatives that reduce these factors may help reduce feelings of burnout while sustaining productivity. For instance, open science initiatives that promote collaboration, open sharing of materials, and sharing of resources reduce the time, effort, and cost to conducting research. For instance, NICE contributors are provided with all materials needed to run the project (free of cost), are provided with mentorship and assistance regarding ethical review board document preparation, and must only collect around 100 participant cases to be considered for authorship. However, researchers are still making a meaningful contribution to the field as these datasets are often large, diverse, and can result in a series of publications (see Reifman & Grahe, 2016, for an example).

Open Science Through Collaboration

Technological advances and novel methodological tools more often empower professional researchers than students. Infrastructure developments such as internet-based repositories facilitate collaboration, which fosters the development of open-invitation, open science research projects (Hesse, 2018). A number of crowdsourcing projects were developed for professional researchers and have garnered much deserved media and scientific attention (see the Reproducibility Project: Psychology (RP:P; Open Science Collaboration, 2015); Many Labs 1 (Klein et al., 2014); Many Labs 2 (Klein et al., 2018); Many Labs 3 (Ebersole et al., 2016); Many Labs 4 (Klein et al., 2019); Many Labs 5 (Ebersole et al., 2018); Psychological Science Accelerator (Moshontz et al., 2018)). In many cases, undergraduates contributed to these projects in a supportive or assistant role for a professor at a given location. Here we direct our focus toward psychology projects which explicitly include students in meaningful roles. Notably, other disciplines have developed similar student-focused projects (see Auchincloss et al., 2014; Harrison, Dunbar, Ratmansky, Boyd, & Lopatto, 2011).

Students as research assistants, collaborators, even co-authors, is not novel, though the level of participation and recognition seems to have increased greatly in the past 15–20 years (Grobman, 2009; Kyvik & Smeby, 1994). The projects highlighted here are distinctive by expanding students’ experience of science beyond the walls of their local laboratory. Researchers are no longer bound to the limits of their small, inherently biased sample, or perhaps the limits of their own expertise. We provide an overview of open science initiatives that are designed directly to support undergraduate researchers (for the complete overview, see https://osf.io/rh7ta/) such as: the Collaborative Replications and Education Project (CREP; http://osf.io/wfc6u/), the Network for International Collaborative Exchange (NICE; http://osf.io/juupx/), and the Emerging Adulthood Measured at Multiple Institutions 2 (EAMMi2; http://osf.io/te54b/). Additionally, the complete overview at https://osf.io/rh7ta/ describes open science initiatives that offer students opportunities, but are not explicitly for students, such as: Study Swap; https://osf.io/view/StudySwap/), PsychFileDrawer (http://psychfiledrawer.org/), Registered Replication Reports (Simons, Holcombe, & Spellman, 2014), Pipeline Projects (Schweinsburg et al., 2016), and ManyBabies (http://osf.io/rpw6d/).

Implementing student-friendly open science initiatives into the classroom provides transformative experiences (Karukstis, 2010) for the instructor and the student. Indeed, from critical reflection of engagement in open science initiatives, individuals can become aware of how their unique cultural perspectives color their perceived experiences (Brown, 2004). Student training can become synonymous with scholarship by replacing canned protocols for classroom exercises with research experiences that might become published. Previously, Grahe (2017) described authentic research experiences as where the goal of the research is to generate publishable research findings, distinguishing these from learning-based research, which yields authentic learning experiences but does not result in publishable outcomes. In addition to considering the conditions necessary to conduct authentic research in the classroom, Grahe (2017) identified a set of distinct characteristics such as how the questions are determined and resource requirements of these projects which should be considered by instructors or students interested in answering one of these open invitations. We offer Supplementary Table 1 (https://osf.io/pcmk7/), which highlights differences among these student-friendly projects. By considering one’s goals and resources, these questions can help match researchers to ideal open invitation projects.

Whereas open invitation crowdsourcing projects can be categorized into student-friendly, student-tolerant, and non-student options, innovation in open science related to transparency increases access for all scientists regardless of status or role. We review innovations relative to transparency at https://osf.io/rh7ta/, such as the Transparency and Openness Promotion guidelines (Nosek et al., 2015), Open Science Badges (Blohowiak et al., 2018), Open Science Framework (https://osf.io/), and the PsyArXiv Preprint service (https://psyarxiv.com/discover). These resources are beneficial for any type of research.

Overall, open invitation projects invite students as researchers into conversations regarding current directions in psychological science. Concurrently, students experience mentorship that promotes their academic growth. Through these initiatives students gain insight into the professional world of psychology, while learning how to engage in high-quality research efforts which have the ability to positively impact the field. There is ample evidence of academic and professional benefits of undergraduate research experiences both in and outside the classroom (Linn, Palmer, Baranger, Gerard, & Stone, 2015; Lopatto, 2008; Rodenbusch, Hernandez, Simmons, & Dolan, 2016; Russell, Hancock, & McCullough, 2007). Further, faculty can embrace open science initiatives in their own research to enhance teaching, and advance research objectives. Next, we consider how open science initiatives can be integrated into the classroom and the learning outcomes that can be achieved through their use.

Open Science Initiatives in the Classroom

Open science initiatives are a vehicle for authentic research while simultaneously serving to meet the American Psychological Association’s Guidelines for the Undergraduate Psychology Major (2013). Though there are many connections to the APA learning goals, one of the most obvious is to Goal 2: Scientific Inquiry and Critical Thinking. Open science initiatives provide unique opportunities to engage with the research process and facilitate the inclusion of sociocultural factors (Goal 2.5, 2013). For example, students may engage in the entire research process as a NICE Crowd collaborator and submit IRB documents, collect data, and even assist with manuscript preparation under the guidance of a mentor. Likewise, students participating in CREP can engage in similar research-oriented tasks. These projects can be implemented in a group format by having students work together to develop IRB documents, administer measures, examine data, and write up results.

For courses where a full research project, including data collection, may be inappropriate or impractical, instructors can facilitate the development of skills related to scientific inquiry and authentically engage students using crowdsourced datasets. Publicly available datasets (such as those from NICE, EAMMi2, Many Labs, the Reproducibility Project) allow students to examine a wide variety of variables across the breadth of the discipline. Class discussions can center on concepts including hypothesis formation, testing, families of statistical tests, interpretation, and cogent dissemination.

Goal 3 deals explicitly with diversity, justice, and (arguably) sustainability, “Ethical and social responsibility in a diverse world.” Open science initiatives encourage both transparency and ethics in research practice and provide a framework for engaging in the application of “ethical standards to evaluate psychological science and practice.” Open Science Badges provide a clear example of accountability for ethical behavior. If students are engaging in research that results in Open Science Badges, we easily meet that standard.

Moreover, open science initiatives provide fodder for classroom discussions about ethics. Instructors may, for example, show students, and subsequently discuss, the biases in dissemination of knowledge by comparing and contrasting results on PsychFileDrawer.org to those in peer-reviewed journals. Open science initiatives that promote transparency (i.e., preregistration templates, data and materials repositories) can also be used to highlight the discrepancy between what is taught and what is often practiced. For example, educators could provide students with (redacted) completed preregistration templates and have students critique the research question and subsequent hypotheses, the sample size and rationale, study design, and analysis plan. The student critiques could take the form of a debate, written assignment, or verbal discussion.

Further, open science initiatives support Goals 3.2 and 3.3, “build and enhance interpersonal relationships,” and “adopt values that build community at local, nation, and global levels,” respectively. Open science initiatives allow for cross-institutional collaborations which may be across the state or the world. Projects like NICE Crowd and Connect, among others with an international scope (Klein et al., 2018; Schweinsburg et al., 2016), provide access for students at universities with limited human and fiscal resources to cultivate broad collaborative connections, and build community globally. An added benefit is that open science initiatives serve to broaden access for historically marginalized or underresourced students. Through open science initiatives, students and faculty at smaller institutions without access to lab or participant pools can contribute diverse voices to scientific discourse.

These collaborative open science initiatives also assist in diversifying students’ mentoring pool. While smaller schools may have less diverse faculty in regard to expertise, they may also be less diverse demographically. Cross-institutional collaboration may benefit minoritized students by expanding access to demographically similar mentors. Though the literature in this area is complex, women and students of color report that working with a similar mentor is important (Blake-Beard, Bayne, Crosby, & Muller, 2011).

Lastly, Goal 5: Professional Development may be addressed in a unique way by including open science initiatives in the undergraduate classroom. Collaborative projects such as the CREP invite students to interact with professionals beyond their home institution during the research process, as opposed to only during dissemination. Teamwork skills applicable to distance collaboration (not to mention communication skills that are practiced along the way) are fostered by open science initiative involvement as well. Employers value the skills developed when students complete research projects (Hart Research Associates, 2018; Kuther, 2013; National Association of Colleges and Employers, 2012), and the benefits of research accumulate with increased frequency of experiences (Taraban & Logue, 2012).

Open Science and Diversity, Justice, and Sustainability

We live in an increasingly globalized academic community that promotes and values exchange and collaboration across the world, and that simultaneously struggles with emerging nationalistic, xenophobic elitism worldwide that limits the opportunities of the most disadvantaged members of our community. Using open science initiatives in teaching helps remove barriers, providing traditionally underrepresented students access, and at the same time promoting the value of engaging with diversity, justice, and sustainability to relatively privileged students. Arguably, curricula could be greatly improved if projects and tasks included intentional consideration of the nature of diversity, justice, and sustainability. And, arguably, open science initiatives offer opportunities to resolve this gap.

Successfully navigating the landscape of psychological science demands a consideration of diversity, justice, and sustainability. Implementing open science initiatives may assist in the promotion of these principles in our classrooms and improve our students’ experiences and our science. Further work should seek to apply a diverse, just, and sustainable lens framework to the evaluation of other systems and organizations, even personal perspectives. A diverse, just, and sustainable lens should consider both inputs and outcomes and should search for spaces that maximize diversity, justice, and sustainability simultaneously, while being forthright regarding costs of implementation.

Although there is much concern about topics of diversity, justice, and sustainability within psychology, a Special Issue of Perspectives of Psychological Science (volume 14, issue 1), titled “How Can Psychological Science Cultivate a Healthier, Happier, and More Sustainable World?”, represents a rare collection addressing diversity, justice, and sustainability together (see Gruber, Saxbe, Bushman, McNamara, & Rhodes, 2019, for introduction). Though not explicitly titled as such, they describe their attempt to improve that world at the individual, group, organizational, and system level. Remarkably, the one manuscript specifically devoted to teaching explicitly demonstrates our main thesis. Mendoza-Denton (2019) reports taking advantage of authentic data from a local repository to increase student engagement when learning statistics and methods. He argues that this benefits diversity because all students are in the classroom, so all are invited to engage in the practice of science. Above, we referenced 12 diversity, justice, and sustainability questions that may be utilized for a critical evaluation of educational initiatives, institutions, and organizations (Grahe, Cuccolo, Leighton, & Cramblet Alvarez, 2019; see https://osf.io/qv8nt/). We assert that compared with traditional teaching and research methods, open science initiatives provide for a more diverse, socially just, and sustainable future for psychological science, and by teaching diversity, justice, and sustainability to our students, they can carry diversity, justice, and sustainability into the world around them.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Author biographies

Jon E Grahe is a Professor of Psychology at Pacific Lutheran University. He is the Managing Executive Editor of The Journal of Social Psychology and serves as chair of the Open Science Badges committee, and a former President of Psi Chi. He was the Executive Director of both the Collaborative Replications and Education Project (CREP) and the Emerging Adulthood Measured at Multiple Institutions (EAMMi2) project. He primarily teaches the year-long statistics/methods combined course, but also enjoys guiding students through capstone, social, and personality courses. His scholarship of teaching and learning produced a co-authored book, Designing and Teaching Undergraduate Capstone Courses.

Kelly Cuccolo is a doctoral candidate and lecturer at the University of North Dakota. Her main research interests lie in understanding the effects of various dietary habits on psychological well-being, and cognitive performance. She is additionally interested in social justice in the classroom. She served as the Chair of Psi Chi’s Network for International Collaborative Exchange for two years, and is currently working on submitting several manuscripts for publication and grants for evaluation.

Dana C Leighton is an Assistant Professor of Psychology at Texas A&M University – Texarkana. He has published on his teaching methods in eBooks from the Society for the Teaching of Psychology, presented his methods at regional and national conferences, and received grants to develop teaching methods. His teaching interests are in undergraduate research, writing in psychology, and online student engagement. He is co-author of the open textbook, Research Methods in Psychology, 4th American Edition.

Leslie D Cramblet Alvarez is a Professor of Psychology, Vice President for Psi Chi for the Rocky Mountain Region, and a former President of the Rocky Mountain Psychological Association. She also serves as the Founding Director of the Center for Teaching, Innovation, & Research at Adams State University in Alamosa, Colorado. In her current role she enjoys both helping faculty grow as educators, and nurturing undergraduate psychology majors as they develop as scholars. She teaches Research Methods, Theories of Learning, Cognitive and Behavior Modification, Child Development, and gets to dabble occasionally in Psychopathology and Positive Psychology of Yoga.

References

AAAS Science and Human Rights Coalition. Defining the Right to Enjoy the Benefits of Scientific Progress and Its Applications: American Scientists' Perspectives. (Report prepared by Margaret Weigers Vitullo and Jessica Wyndham), October 2013. DOI: 10.1126/srhrl.aaa0028. Washington DC.

American Psychological Association. (2013). APA guidelines for the undergraduate psychology major: Version 2.0. Retrieved from http://www.apa.org/ed/precollege/undergrad/index.aspx.

Arnett

J. J.

(2008) The neglected 95%: Why American psychology needs to become less American. American Psychologist 63(7): 602.

Auchincloss

L. C.

Laursen

S. L.

Branchaw

J. L.

Eagan

Graham

Hanauer

D. I.

Towns

(2014) Assessment of course-based undergraduate research experiences: A meeting report. CBE—Life Sciences Education 13(1): 29–40.

Basiago

A. D.

(1995) Methods of defining “sustainability”. Sustainable Development 3: 109–119.

Bennis

W. M.

Medin

D. L.

(2010) Weirdness is in the eye of the beholder. Behavioral and Brain Sciences 33(2–3): 85–86.

Blake-Beard

Bayne

M. L.

Crosby

F. J.

Muller

C. B.

(2011) Matching by race and gender in mentoring relationships: Keeping our eye on the prize. Journal of Social Issues 67(3): 622–643.

Blohowiak, B. B., Cohoon, J., de-Wit, L., Eich, E., Farach, F. J., Hasselman, F., … DeHaven, A. C. (2018, November 14). Badges to acknowledge open practices. Retrieved from osf.io/tvyxz.

Brown

K. M.

(2004) Leadership for social justice and equity: Weaving a transformative framework and pedagogy. Educational Administration Quarterly 40(1): 77–108.

10.

Cohen

L. M.

Morgan

R. D.

DiLillo

Flores

L. Y.

(2003) Why was my major professor so busy? Establishing an academic career while pursuing applied work. Professional Psychology: Research and Practice 34(1): 88.

11.

Cundiff

J. L.

(2012) Is mainstream psychological research “womanless” and “raceless”? An updated analysis. Sex Roles 67: 158–173.

12.

Deutsch

(1985) Distributive justice: A social-psychological perspective, New Haven, CT: Yale University Press.

13.

Eagly

A. H.

Miller

D. I.

(2016) Scientific eminence: Where are the women? Perspectives on Psychological Science 11: 899–904.

14.

Ebersole, C. R., Andrighetto, L., Casini, E., Chiorri, C., Dalla Rosa, A., Domaneschi, F., … Vianello, M. (in principle acceptance). Many Labs 5: Registered Replication Report of Payne, Burkley, & Stokes (2008), Study 4. Advances in Methods and Practices in Psychological Science.

15.

Ebersole

C. R.

Atherton

O. E.

Belanger

A. L.

Skulborstad

H. M.

Allen

J. M.

Banks

J. B.

Nosek

B. A.

(2016) Many Labs 3: Evaluating participant pool quality across the academic semester via replication. Journal of Experimental Social Psychology 67: 68–82.

16.

Evans

G. L.

Cokley

K. O.

(2008) African American women and the academy: Using career mentoring to increase research productivity. Training and Education in Professional Psychology 2(1): 50.

17.

Fausto-Sterling

(1985) The new research on women: How does it affect the natural sciences. Women’s Studies Quarterly 1985: 30–32.

18.

Fisher

C. B.

Hoagwood

Boyce

Duster

Frank

D. A.

Grisso

Trimble

J. E.

(2002) Research ethics for mental health science involving ethnic minority children and youths. American Psychologist 57(12): 1024.

19.

Fiske

S. T.

(2010) Venus and Mars or down to Earth: Stereotypes and realities of gender differences. Perspectives on Psychological Science 5(6): 688–692.

20.

Galassi

J. P.

Brooks

Stoltz

R. F.

Trexler

K. A.

(1986) Research training environments and student productivity: An exploratory study. The Counseling Psychologist 14(1): 31–36.

21.

Gasser

C. E.

Shaffer

K. S.

(2014) Career development of women in academia: Traversing the leaky pipeline. The Professional Counselor 4: 332–352.

22.

Giner-Sorolla

(2012) Science or art? How aesthetic standards grease the way through the publication bottleneck but undermine science. Perspectives on Psychological Science 7(6): 562–571.

23.

Good

J. J.

Keeley

J. W.

Leder

Afful

S. E.

Stiegler-Balfour

J. J.

(2013) Supporting our junior faculty: Assessing the concerns and needs of early career psychologists. Teaching of Psychology 40(4): 340–345.

24.

Grahe, J. E. (2017). Authentic research projects benefit students, their instructors, and science. In R. Obeid, A. Schwartz, C. Shane-Simpson, & P. J. Brooks (Eds.) How we teach now: The GSTA guide to student-centered teaching. Retrieved from http://teachpsych.org/ebooks/howweteachnow.

25.

Grahe, J. E., Cuccolo, K., Leighton, D. C., & Cramblet Alvarez, L. D. (2019, July 23). Open Science for Diversity, Justice, and Sustainability. Retrieved from osf.io/qv8nt.

26.

Grahe

J. E.

Reifman

Herman

Walker

Oleson

Nario-Redmond

Wiebe

(2012) Harnessing the undiscovered resource of student research projects. Perspectives on Psychological Science 7(6): 605–607.

27.

Grobman

(2009) The student scholar: (Re)negotiating authorship and authority. College Composition and Communication 61(1): W175–W196.

28.

Gruber

Saxbe

Bushman

B. J.

McNamara

Rhodes

(2019) How can psychological science contribute to a healthier, happier, and more sustainable world? Perspectives on Psychological Science 14(1): 3–6.

29.

Hardin

(1968) The tragedy of the commons. Science, New Series 162(3859): 1243–1248.

30.

Harrison

Dunbar

Ratmansky

Boyd

Lopatto

(2011) Classroom-based science research at the introductory level: Changes in career choices and attitude. CBE—Life Sciences Education 10(3): 279–286.

31.

Hart Research Associates (2018). Fulfilling the American dream: Liberal education and the future of the work. Association of American Colleges and Universities. Retrieved from: https://www.aacu.org/sites/default/files/files/LEAP/2018EmployerResearchReport.pdf.

32.

Henrich

Heine

S. J.

Norenzayan

(2010) The weirdest people in the world? Behavioral and Brain Sciences 33(2–3): 61–83.

33.

Hesse

B. W.

(2018) Can psychology walk the walk of open science? American Psychologist 73(2): 126.

34.

Intemann

(2009) Why diversity matters: Understanding and applying the diversity component of the National Science Foundation’s Broader Impacts Criterion. Social Epistemology 23(3–4): 249–266.

35.

Jones

J. M.

(2010) I’m white and you’re not: The value of unraveling ethnocentric science. Perspectives on Psychological Science 5(6): 700–707.

36.

Jones

J. M.

(2019) Doing diversity: Creating spaces, opportunities, and change. Perspectives on Psychological Science 14(1): 43–48.

37.

Karukstis, K. K. (2010). Multiple approaches to transformative research. In K. K. Karukstis & N. H. Hensel (Eds.), Transformative research at predominately undergraduate institutions (pp. 21–34). Washington, D.C.: Council on Undergraduate Research.

38.

Klein, R. A., Ebersole, C. R., Cook, C. L., Nosek, B. A., Redford, L., Vaughn, L. A., … Harton, H. C. (2019, May 22). Many Labs 4: Investigating Effects of Researcher Expertise on Replication Outcomes. Retrieved from osf.io/8ccnw.

39.

Klein

R. A.

Ratliff

K. A.

Vianello

Adams

R. B.

Jr Bahník

Š.

Bernstein

M. J.

Cemalcilar

(2014) Investigating variation in replicability. Social Psychology 45: 142–152.

40.

Klein, R. A., Vianello, M., Hasselman, F., Adams, B. G., Adams, R. B., Alper, S., … Nosek, B. A. (2018). Many Labs 2: Investigating Variation in Replicability Across Samples and Settings. Advances in Methods and Practices in Psychological Science, 1(4), 443–490. DOI: 10.1177/2515245918810225.

41.

Kniffin

K. M.

Hanks

A. S.

(2018) The trade-offs of teamwork among STEM doctoral graduates. American Psychologist 73(4): 420.

42.

Kuther, T. L. (2013, September). What employers seek in job applicants: You’ve got what they want. Psychology Student Network. Retrieved from: https://www.apa.org/ed/precollege/psn/2013/09/job-applicants.

43.

Kyvik

Smeby

J. C.

(1994) Teaching and research: The relationship between the supervision of graduate students and faculty research performance. Higher Education 28(2): 227–239.

44.

Lackritz

J. R.

(2004) Exploring burnout among university faculty: Incidence, performance, and demographic issues. Teaching and Teacher Education 20(7): 713–729.

45.

Lind, E. A., & Tyler, T. R. (1988). The social psychology of procedural justice. Plenum Press, New York, NY.

46.

Linn

M. C.

Palmer

Baranger

Gerard

Stone

(2015) Undergraduate research experiences: Impacts and opportunities. Science 347(6222): 1261757.

47.

Longino

H. E.

(1990) Science as social knowledge: Values and objectivity in scientific inquiry, Princeton, N.J.: Princeton University Press.

48.

Longino, H. (2016). The social dimensions of scientific knowledge. Retrieved from https://plato.stanford.edu/entries/scientific-knowledge-social/.

49.

Lopatto, D. (2008). Exploring the benefits of undergraduate research experiences: The SURE survey. In R. Taraban & R. L. Blanton (Eds.), Creating effective undergraduate research programs in science (pp. 112–132). New York, NY, US: Teachers College Press.

50.

Machery

(2010) Explaining why experimental behavior varies across cultures: A missing step in “The weirdest people in the world?”. Behavioral and Brain Sciences 33(2–3): 101–102.

51.

Markus

Steele

(2000) Colorblindness as a barrier to inclusion: Assimilation and nonimmigrant minorities. Daedelus 129(4): 233–263.

52.

McFarland, J., Hussar, B., Wang, X., Zhang, J., Wang, K., Rathbun, A., & Bullock Mann, F. (2018). The condition of education 2018 (NCES 2018-144). U.S. Department of Education. Washington, D.C.: National Center for Education Statistics. Retrieved from https://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2018144.

53.

Mendoza-Denton

(2019) I also teach. Perspectives on Psychological Science 14(1): 101–104.

54.

Moshontz

Campbell

Ebersole

C. R.

IJzerman

Urry

H. L.

Forscher

P. S.

Chartier

(2018) The psychological science accelerator: Advancing psychology through a distributed collaborative network. Advances in Methods and Practices in Psychological Science 1(4): 501–515.

55.

Munafò

M. R.

Nosek

B. A.

Bishop

D. V.

Button

K. S.

Chambers

C. D.

Du Sert

N. P.

Ioannidis

J. P.

(2017) A manifesto for reproducible science. Nature Human Behaviour 1(1): 0021.

56.

National Association of Colleges and Employers (2012) Job outlook 2013 survey, Washington, D.C.: National Association of Colleges and Employers.

57.

Nisbett

R. E.

Miyamoto

(2005) The influence of culture: Holistic versus analytic perception. Trends in Cognitive Science 9: 467–473.

58.

Nosek

B. A.

Alter

Banks

G. C.

Borsboom

Bowman

S. D.

Breckler

S. J.

Contestabile

(2015) Promoting an open research culture. Science 348(6242): 1422–1425.

59.

Open Science Collaboration (2015) Estimating the reproducibility of psychological science. Science 349(6251): aac4716.

60.

Page

(2007) The difference: How the power of diversity creates better groups, firms, schools, and societies, Princeton, NJ: Princeton University Press.

61.

Pratto

Stewart

A. L.

(2012) Group dominance and the half-blindness of privilege. Journal of Social Issues 68(1): 28–45.

62.

Reifman

Grahe

J. E.

(2016) Introduction to the special issue of emerging adulthood. Emerging Adulthood 4: 135–141.

63.

Rodenbusch

S. E.

Hernandez

P. R.

Simmons

S. L.

Dolan

E. L.

(2016) Early engagement in course-based research increases graduation rates and completion of science, engineering, and mathematics degrees. CBE—Life Sciences Education 15(2): ar20.

64.

Rosenthal

(1979) The file drawer problem and tolerance for null results. Psychological Bulletin 86(3): 638.

65.

Russell

S. H.

Hancock

M. P.

McCullough

(2007) Benefits of undergraduate research experiences. Science 316(5824): 548–549.

66.

Schiebinger

(1997) Creating sustainable science. Osiris, 12, 201–216. Schmidt, S. (2009). Shall we really do it again? The powerful concept of replication is neglected in the social sciences. Review of General Psychology 13(2): 90.

67.

Schweinsberg

Madan

Vianello

Sommer

S. A.

Jordan

Tierney

Uhlmann

E. L.

(2016) The pipeline project: Pre-publication independent replications of a single laboratory’s research pipeline. Journal of Experimental Social Psychology 66: 55–67.

68.

Sears

D. O.

(1986) College sophomores in the laboratory: Influences of a narrow data base on social psychology’s view of human nature. Journal of Personality and Social Psychology 51(3): 515–530.

69.

Seghezzo

(2009) The five dimensions of sustainability. Environmental Politics 18(4): 539–556.

70.

Sharobeam

M. H.

Howard

(2002) Teaching demands versus research productivity. Journal of College Science Teaching 31(7): 436.

71.

Simons

D. J.

(2014) The value of direct replication. Perspectives on Psychological Science 9(1): 76–80.

72.

Simons

D. J.

Holcombe

A. O.

Spellman

B. A.

(2014) An introduction to registered replication reports at perspectives on psychological science. Perspectives on Psychological Science 9(5): 552–555.

73.

Stahl

G. K.

Maznevski

M. L.

Voigt

Jonsen

(2010) Unraveling the effects of cultural diversity in teams: A meta-analysis of research on multicultural work groups. Journal of International Business Studies 41(4): 690–709.

74.

Stolp, M. (2017). Who are we studying?: Reliance on WEIRD and/or student subjects in cross-cultural, social/personality and evolutionary psychology. (Master's thesis, Universiteit Leiden, Leiden, Netherlands). Retrieved from https://openaccess.leidenuniv.nl/bitstream/handle/1887/60846/Stolp%2C%20Marieke-s1232266-MA%20Thesis%20SOP-2017.pdf?sequence=1.

75.

Sue

(1999) Science, ethnicity, and bias: Where have we gone wrong? American Psychologist 54(12): 1070.

76.

Taraban

Logue

(2012) Academic factors that affect undergraduate research experiences. Journal of Educational Psychology 104(2): 499.

77.

Thibaut, J., & Walker, L. (1978). A theory of procedure. California Law Review, 66, 541–566. Retrieved from http://scholarship.law.berkeley.edu/cgi/viewcontent.cgi?article=2369&context=californialawreview.

78.

UN General Assembly (1948) Universal declaration of human rights, New York, NY: UN General Assembly.

79.

Vazire, S. (2017, April 18). Against eminence. Retrieved from https://osf.io/n37g9.

80.

Wagge

J. R.

Baciu

Banas

Nadler

J. T.

Schwarz

Weisberg

, et al.(2019a) A demonstration of the collaborative replication and education project: Replication attempts of the red-romance effect. Collabra: Psychology 5: 5.

81.

Wagge

J. R.

Brandt

M. J.

Lazarevic

L. B.

Legate

Christopherson

Wiggins

Grahe

J. E.

(2019b) Publishing research with undergraduates via replication work: The Collaborative Replications and Education Project. Frontiers of Psychology 10: 247. doi: 10.3389/fpsyg.2019.00247.

82.

West

J. D.

Jacquet

King

M. M.

Correll

S. J.

Bergstrom

C. T.

(2013) The role of gender in scholarly authorship. PLOS ONE 8: 1–6.

83.

Zwaan

R. A.

Etz

Lucas

R. E.

Donnellan

M. B.

(2018) Making replication mainstream. Behavioral and Brain Sciences 41: e120.