Exploring the Great Schism in the Social Sciences: Confirmation Bias and the Interpretation of Results Relating to Biological Influences on Human Behavior and Psychology

Abstract

The nature–nurture debate is one that biologists often dismiss as a false dichotomy, as all phenotypic traits are the results of complex processes of gene and environment interactions. However, such dismissiveness belies the ongoing debate that is unmistakable throughout the biological and social sciences concerning the role of biological influences in the development of psychological and behavioral traits in humans. Many have proposed that this debate is due to ideologically driven biases in the interpretation of results. Those favoring biological approaches have been accused of a greater willingness to accept biological explanations so as to rationalize or justify the status quo of inequality. Those rejecting biological approaches have been accused of an unwillingness to accept biological explanations so as to attribute inequalities solely to social and institutional factors, ultimately allowing for the possibility of social equality. While it is important to continue to investigate this topic through further research and debate, another approach is to examine the degree to which the allegations of bias are indeed valid. To accomplish this, a convenience sample of individuals with relevant postgraduate degrees was recruited from Mechanical Turk and social media. Participants were asked to rate the inferential power of different research designs and of mock results that varied in the degree to which they supported different ideologies. Results were suggestive that researchers harbor sincere differences of opinion concerning the inferential value of relevant research. There was no suggestion that ideological confirmation biases drive these differences. However, challenges associated with recruiting a large enough sample of experts as well as identifying believable mock scenarios limit the study’s inferential scope.

Keywords

confirmation bias academia evolutionary studies cognitive bias MTurk

The nature–nurture debate is one that biologists often dismiss as a false dichotomy, as all phenotypic traits are the results of complex processes of gene and environment interactions playing out across numerous dimensions. Therefore, all traits might be considered to be due to 100% nature and 100% nurture. However, such dismissiveness belies the long-standing debate that is unmistakable throughout the biological and social sciences concerning the role of biological influences in the development of psychological and behavioral traits in humans (e.g., Buller, 2006; Horowitz & Yaworsky, 2014; Marks, 2012; McKinnon & Silverman, 2005b; Pinker, 2002; Rose & Rose, 2001). Indeed, this debate divides many disciplines and has at times proven quite contentious (Segerstrale, 2000).

Much of the theory and methods employed by biologically oriented social scientists flows from the animal literature. When human studies suggest homologous or analogous features, however, explanations invoking biological pathways or adaptive functions tend to be less readily accepted for human traits than for animal traits (e.g., for incest avoidance, see Chapais, 2008; El-Guindi & Read, 2012; Kushnick & Fessler, 2011; Leavitt, 2007). This is reasonable, given that humans exhibit an unprecedented level of behavioral plasticity, driven by uniquely elevated capacities for fluid reasoning and social learning (i.e., culture). Many have argued that such characteristics make it impossible to disentangle the roles of culture and biology in the phylogenetic story of humans and the ontogenetic stories of individuals (Fuentes, 2012; Marks, 2012; McKinnon & Silverman, 2005a). Under this argument, it is not simply that such a goal is logistically unattainable, but that such endeavors are meaningless: In humans, the biological is inextricably cultural and the cultural biological. In other academic fields, extreme forms of social constructionism have eliminated the consideration of biological factors entirely (Sayer, 1997).

Obviously, complex phylogenetic and ontogenetic processes cannot be captured by the single dimension of nature versus nurture, genetic versus environment, and so on. For many phenomena, such a dimension would prove nonsensical in its mutually exclusive, linear form. However, the debate that divides the social sciences is more specific.

When biologists refer to gene–environment interactions, many types of environmental factors can influence outcomes: intrauterine hormonal levels, salinity of interstitial fluids, energy availability, and so on. However, as the debate has played out in academic journals and popular publications, many of these factors (such as intrauterine hormonal levels) would likely be conceptually placed on the biological side of the ledger. The environmental factors that are relevant are typically those relating to experiential influences—those derived from what an individual experiences through his or her senses through time. This includes one’s upbringing, schooling, and cultural environment. Even when limiting the discussion to these factors, one’s experiences and biological processes continue to interact to define an intertwined developmental trajectory. However, the overarching spectrum of possibilities becomes limited, and we can more meaningfully characterize the debate in a more limited question: To what degree, do biological processes constrain humans’ remarkable cultural and behavioral responsiveness to experiential variance? And the question explored here, therefore, is what explains the remarkable variation in academics’ interpretations of data relating to this question?

For those who still question the meaningfulness of suggesting that a trait is more or less biologically constrained (i.e., environmentally canalized) than another, perhaps a look at the tails would be helpful. There are a number of traits for which there is less doubt that selection played a role in shaping their expression and that genetically determined biologically processes play a role in constraining their development. These include the aversion to the smell of vomit on one’s food, preferring to be exposed to room temperatures rather than arctic temperatures, perceiving the act of sex to feel more pleasant than being stabbed, and the consequent motivation to pursue sex more than being stabbed. On the other hand, there are many traits for which there is little doubt that biological processes do not play a large role in constraining their development such as the sound used to represent the concept of “tree,” the aesthetic preference for a certain hair length on men, and the feeling of embarrassment when one’s bare legs are seen by others. Obviously, there are some biological constraints such as the fact that humans must be able to produce and hear the sound representing “tree,” grow hair to a certain length, and be able to internalize norms and feel shame when they are broken. But within those broad constraints, the possibility that any particular preference might emerge through a cultural process of historical happenstance is indeed real.

Inferential Power (IP) of Different Study Designs

If there is fair agreement at the tails, this would suggest that there is some degree of evidence and/or cogency of argument that satisfies most parties and that the debate likely rests on variance in the inferential thresholds demanded before inferring biological constraints. This raises the question of how we infer a biological influence (Andrews, Gangestad, & Matthews, 2002). A preference to not be exposed to the arctic cold is cross-culturally universal; it appears early in development, can be seen in other animals, and serves a clear adaptive purpose. None of those things can be said for long hair on men. Thus, despite not knowing the genetic or biological etiology of a cognitive motivation to seek warmth—a common critique of biological explanations—most are comfortable with assigning a biological foundation to a preference for warmth over freezing. There are a number of research designs (RDs) that are used to examine the role of biology in the development of a particular trait. Referenced above, these include explorations of cross-cultural universality (Buss, 1989; Eagly & Wood, 1999; Henrich et al., 2004), early developmental onset (Campos, Bertenthal, & Kermoian, 1992; Langlois et al., 1987; Thompson & Newton, 2013), animal homologs/analogs (Plavcan, 2012; Proctor, Williamson, de Waal, & Brosnan, 2013; Sussman, 2013; Wrangham & Glowacki, 2012), and adaptive design (Gould & Lewontin, 1979; Oaten, Stevenson, & Case, 2009; Winking, Gurven, Kaplan, & Stieglitz, 2009) but also allelic associations (Hamer, Hu, Magnuson, Hu, & Pattatucci, 1993; Munafò, Yalcin, Willis-Owen, & Flint, 2008), hormonal associations (Gettler, McDade, Feranil, & Kuzawa, 2011; Mehta, Jones, & Josephs, 2008), and heritability (Polderman et al., 2015).

There likely exists disagreement concerning the IP of these different designs and this might explain some of the variance in the degree to which researchers believe biology plays a role in the development of human traits. Unfortunately, no study to my knowledge has explored how researchers differ in their interpretations according to their outlook on biological influences. This is one of the goals of this study.

Confirmation Bias

In addition to epistemological differences, researchers on both sides of the debate have alleged that the disagreements stem from ideological biases. Those who believe biological influences to be unimportant often allege that the other side is motivated to rationalize at best, and justify at worst, current and historical patterns of hegemony, social dominance, and inequality. This argument harkens back to the beginning of the sociobiology debates, summarized in the collective response of an early Harvard study group which evaluated the merits of sociobiology, and which included intellectual giants like Stephen J. Gould and Richard Lewontin, “[T]hey consistently tend to provide a genetic justification of the status quo and of existing privileges for certain groups according to class, race, or sex” (Allen et al., 1975). The argument continues today and is evident in writings across multiple disciplines:

Many researchers use [evolutionary approaches to the study of international relations] to justify the status quo in the guise of science. (Lebow, 2013, p. 2)

The totality [of sociobiology and evolutionary psychology] is a myth of origin that is compelling precisely because it resonates strongly with Euro American presuppositions about the nature of the world. (McKinnon & Silverman, 2005a, p. 4)

…in the social sciences (with the exception of primatology and psychology) sociobiology appeals most to right-wing social scientists. (Pavelka, 2002, p. 42)

Conversely, those who favor biological approaches have argued that those on the other side are rendered incapable of objective assessment by their ideological promotion of equality. They are alleged to erroneously reject evidence of biological influences because such evidence suggests that social outcomes are partially explained by biology, and this might inhibit the realization of equality. Their critiques of biological approaches are therefore often blithely dismissed as examples of the moralistic/naturalistic fallacy. This line of reason is exemplified in the quote by biologist Jerry Coyne

If you can read the [major Evolutionary Psychology review paper] and still dismiss the entire field as worthless, or as a mere attempt to justify scientists’ social prejudices, then I’d suggest your opinions are based more on ideology than judicious scientific inquiry. (Coyne, 2012)

Again, such claims are quite common:

For critics, the heart of the intellectual problem remains an ideological adherence to the increasingly implausible view that human behavior is strictly determined by socialization.…Should [social] hierarchies result strictly from culture, then the possibilities for an egalitarian future were seen to be as open and boundless as our ever-malleable brains might imagine. (Horowitz & Yaworsky, 2014, p. 490)

Like the Church, a number of contemporary thinkers have also grounded their moral and political views in scientific assumptions about…human nature, specifically that there isn’t one. (Hagen, 2005, p. 166)

Although we academics like to believe that we approach debates with an open mind, that we objectively weigh the evidential support for competing arguments, and that we are always prepared to change our beliefs upon the presentation of sufficient evidence, a great deal of research suggests that the cognitive game is fixed (Nickerson, 1998; Snyder & Swann, 1978; Stanovich, West, & Toplak, 2013). Our reasoning is often clouded by a confirmation bias—in its broad definition, confirmation bias is the tendency to advantage preexisting beliefs through many forms of inferential acrobatics (Nickerson, 1998). We actively seek confirmatory evidence and avoid disconfirmatory evidence (Jonas, Schulz-Hardt, Frey, & Thelen, 2001; Taber & Lodge, 2006), and we recall this confirmatory evidence more readily (Stanovich et al., 2013). More importantly here, we accept confirmatory evidence more readily and weight its inferential value more heavily, while being overly critical or dismissive of disconfirmatory evidence (often referred to as a “disconfirmation bias”; Lehner, Adelman, Cheikes, & Brown, 2008; Nickerson, 1998; Taber & Lodge, 2006).

While it is important to engage in research and debate that advances our understanding of how biological factors influence the development of behavioral and psychological outcomes, it is also important to examine the validity of these alleged biases. Previous research has shown that those who self-identify as political radicals or feminists are more likely to reject biological influences on human traits (Horowitz & Yaworsky, 2014; Sanderson & Ellis, 1992), while others have found academic proponents of biological approaches to be no less liberal on political issues (with both groups being far more liberal than the population at large; Lyle & Smith, 2012; Tybur, Miller, & Gangestad, 2007). However, while a finding of an association with political outlook might suggest the possibility of an ideological bias in interpretations, it does not shed light on which side, if either or if both, is indeed biased. To answer this question, we must examine the interpretations of results that differ solely on the degree to which they support or fail to support the preexisting ideological beliefs that allegedly drive these biases. I attempt this here.

Hypotheses

It is entirely possible that confirmation bias plays no role in driving this disagreement and that the overarching debate in academia is driven by sincere disagreements concerning the inferential value of the RDs informing the debate. Although academics might believe that inferential logic is universal and abundantly clear, biological systems are inconceivably complex, and it is unlikely that we will ever reach a consensus on the meaningfulness of large bodies of research. While no predictions can be made concerning which RDs should be perceived as holding more or less IP, we can at least predict that those who believe biology to play a more important role (for convenience, we will dichotomize this belief and label this group “bioists”) should report that positive results, across a range of RDs, would carry more inferential weight than that reported by participants who believe biology to be less relevant (“skeptics”; Hypothesis 1a; Table 1). Similarly, skeptics should report that null results carry more inferential weight than bioists report (Hypothesis 1b). Such effects should be evident even when reviewing the IP of generic RD descriptions, without viewing any specific results.

Table 1.

Hypotheses.

Research Design	Description
Hypothesis 1a	Bioists will assign a higher influential value than skeptics to positive results of studies exploring biological influences
Hypothesis 1b	Skeptics will assign a higher influential value than bioists to null results of studies exploring biological influences
Hypothesis 2	Bioists will exhibit a more conservative political orientation than skeptics
Hypothesis 3a	Bioists will rate results suggesting a biological antecedent to a socially undesirable trait higher when that biological antecedent is reported to exist in a higher frequency in a nonprivileged group than those that do not mention frequencies
Hypothesis 3b	Bioists will rate results suggesting a biological antecedent to a socially undesirable trait higher when that biological antecedent is reported to exist in a higher frequency in a nonprivileged group than those that report the frequency higher in a privileged group
Hypothesis 4a	Skeptics will rate results suggesting a biological antecedent to a socially undesirable trait lower when that biological antecedent is reported to exist in a higher frequency in a nonprivileged group than those that do not mention frequencies
Hypothesis 4b	Skeptics will rate results suggesting a biological antecedent to a socially undesirable trait lower when that biological antecedent is reported to exist in a higher frequency in a nonprivileged group than those that report the frequency higher in a privileged group

If ideological biases are driving this debate, then it must be the case that bioists and skeptics differ in their ideological outlook. Based on the alleged biases, we can predict that compared to skeptics, bioists should report a more conservative orientation, particularly with regard to causes of social stratification and efforts to reverse it (Hypothesis 2). That is to say, compared to skeptics, bioists will report that social outcomes are based more on inherent differences as opposed to social forces and less should be done to reduce inequalities.

Ideological confirmation biases would be suggested if the inferential value assigned to the results were associated with the degree to which those studies supported a particular ideology, holding all else constant. For instance, results suggesting that a socially undesirable trait was associated with a biological antecedent that occurred more frequently in a disadvantaged population should evoke a positive ideological confirmation bias in bioists and a disconfirmation bias in skeptics. Thus, bioists should rate studies suggesting such a scenario as having a higher inferential value than those that report such an association but do not mention relative frequencies in advantaged versus disadvantaged groups (Hypothesis 3a), as well as studies that mention that such antecedents occur in higher frequencies in advantaged groups (Hypothesis 4a). The opposite effects should occur among skeptics (Hypotheses 3b and 4b, respectively).

RD and Methods

A convenience sample was recruited to take part in online Qualtrics surveys through Amazon’s MTurk and through social media sites. Only those who had received a master’s degree or higher in the biological sciences, social sciences, or relevant humanities were included. Two separate surveys were conducted, an RD survey and IP (IP) survey. In both, those entering through MTurk were offered US$0.10 to answer a series of eligibility questions. If they were deemed eligible, they were allowed to continue and received a US$1.00 bonus for their participation. Social media announcements were made via Facebook pages of relevant groups (Biological Anthropology Society, Evolutionary Anthropology Society, and Evolutionary Psychology). E-mails were also sent out to eligible acquaintances with the instructions to forward the e-mail to anyone they thought might be eligible. This was done in part because of the lower number of biologically oriented researchers on MTurk. Social media recruits were offered to take part in a lottery in which 1 in 50 would win a US$50 Amazon gift certificate. The RD survey was conducted in June 2015, and the IP survey was conducted from August 2015 to September 2016.

Inclusion Criteria

Only participants who reported a master’s or PhD in a relevant field were included. These fields included the social sciences, biological sciences, gender studies, and philosophy. A US$1.00 bonus trends toward the high side of MTurk payouts and therefore might lead to higher participation rate of “professional” MTurkers—those who seek the most lucrative “hits” and minimize the time they spend engaged per hit (Berinsky, Margolis, & Sances, 2014; Wu, Corney, & Grant, 2014). Furthermore, recruitment materials advertised the fact that only master’s or PhD holders were eligible so as to minimize the number of ineligible participants needlessly beginning the survey. Therefore, in addition to the degree and discipline requirements, a number of additional inclusion criteria were used to ensure that participants were completing the survey appropriately.

Participants were asked two timed questions (30 s) regarding knowledge specific to academia to ensure honesty concerning reported educational level. One question focused on the final research product for a master’s degree (a thesis) and the other on the meaning of “curriculum vitae.” In a previous survey of 256 MTurk participants who were not told of any education requirements, 22 of the 24 (92%) master’s and PhD holders answered both of these questions correctly, while only 92 of the 232 (40%) participants of other educational levels answered correctly. Only those who answered both questions correctly were included in the surveys used here.

Furthermore, MTurk participants were excluded if they finished the survey in under 6 min for the RD survey and under 9 min for the IP survey. This was based on the fastest times (6.7 min for RD and 9.3 min for IP) for completion among the social media sample, which was expected to not include “professional” survey takers. An attention check question was included; however, 17% of participants who answered incorrectly did not significantly differ in the amount of time spent taking the survey, and it was therefore not used as an exclusionary factor. Finally, Reddit’s “HitsWorthTurkingFor” page was regularly searched to ensure that inclusion criteria were not being discussed. The IP survey was discussed on this site 4 times, resulting in drastic increases in the frequency of participants. Participants who completed the survey directly after such postings were excluded.

Surveys

The RD and IP surveys (Supplementary Figures S1 and S2) begin with four identical sections. These include questions covering participants’ prior beliefs concerning the role of biology in influencing human behavior and psychology, political beliefs, epistemological approaches, and statistical knowledge. Specifically, participants were asked where they place themselves on the “general spectrum that exists in academia concerning the role of biological factors in determining psychological and behavioral patterns in humans.” Provided with a 4-point scale, participants were forced to choose which side of the spectrum they placed themselves. For all other questions with ordinal responses, participants chose from a 7-point scale for scales that had a meaningful center (e.g., extremely liberal–extremely conservative); otherwise, a 5-point scale was used.

In the RD survey, participants were then presented with eight RDs and asked to rate the inferential value of significant results regarding the role of biology in influencing psychological and behavioral traits and the influential value of null results regarding the lack of a role of biology. Participants used a sliding scale ranging from 0 to 100.

In the IP survey, participants were instead presented with three mock data plots that were designed to appear, as if they were scanned from a legitimate academic journal or conference poster (due to reasons described in the section below, data from only one of these proved useful). Participants then read a brief synopsis of the findings and conclusions of the study and were asked to rate the inferential value of the study regarding the role of biology in influencing the outcome of the trait in question. Participants used a sliding scale from 0 to 100 as in the RD survey. Participants were also asked to rate the psychological complexity of each trait and how socially preferred they believed the trait to be. At the end of the survey, participants were asked whether they were suspicious that the results were forged and whether they believed this influenced their responses.

Mock Results

The original study design included tests of both positive and null results. However, the mock null results included psychological traits that participants found too different to be meaningfully comparable and are not used here. Similarly, one condition employing mock positive results included a psychological trait that was deemed to be significantly more complex than the others and was excluded from the study. A full description of these can be found in the Online Supplemental Material. The remaining three conditions are described below.

The creation of the mock results was based on a number of criteria. Psychological traits were considered based on (1) degree of social preference and (2) familiarity to participants while not being the subject of a large, well-known bodies of literature focusing on their genetic and biological associates (e.g., general intelligence, schizophrenia). Based on the results from the RD survey, mock results were created using three RDs that were deemed to have high IP. The designs were presented pseudorandomly (16 sequences). As described above and in the Online Supplementary Material, data from only one RD are reported on here.

The mock results suggesting a biological influence were based on a genetic allele design, which was rated as having the third highest IP out of eight. Participants were presented with a plot showing a substantial association (Cohen’s d = .86) between a particular allele and appetitive aggression. In the first condition, the allele was described as being found in higher frequency in populations with African ancestry. In the second condition, this language was absent, and in the third condition, it was described as being found in higher frequency in populations with European ancestry. Appetitive aggression was described as “pro-active and predatory aggression…associated with things such as likelihood of committing violence and positive attitudes towards warfare.” By including references to both “violence” and “warfare,” such a trait can be reasonably aligned with negative stereotypes or perceptions associated with African American culture (Devine, 1989; Dixon, 2008) and European history (Wolfe, 2006).

After participants rated the IP of each set of results, they were asked how socially preferred they believed the relevant trait to be on a scale ranging from −3 to 3. They were also asked how psychologically complex they believed it to be using a 5-point scale.

Analysis

Simple comparisons of bioist and skeptic measures are analyzed using Wilcoxon rank tests for ordinal data and t tests for continuous ratings. Multivariate analysis of variance (MANOVA) tests are employed for tests involving multiple dependent ratings. All analyses were conducted in SAS Version 9.3.

Results

Of the 327 participants who began the RD survey, 60 were excluded for not having the appropriate educational level, 124 were excluded for working in a field that was not included, 36 were excluded for not answering the academia questions correctly, 15 were excluded for taking the survey too quickly, and 21 were excluded for not finishing the survey, leaving 71 included. Of the 2,631 participants who began the IP survey, 248 were excluded for education, 848 for their field, 235 for the academia questions, 231 for taking it too quickly, 610 for having started the survey after a Reddit post, and 94 for not completing the survey, leaving 365 included. Table 2 presents the descriptive results for the two data sets. The RD survey included only one condition and thus required a much smaller sample size. Most participants could thus be recruited from social media. The sample also has a higher proportion who have doctorates, are anthropologists, and are faculty.

Table 2.

Descriptive Statistics.

Factor	Research Design		Inferential Power
	Bioist	Skeptic	Bioist	Skeptic
n	32	39	140	225
Male (%)	15 (47)	20 (51)	91 (65)	149 (66)
Female	17 (53)	19 (49)	49 (35)	76 (34)
Mean age	35.8	34.8	39.0	39.5
MTurk	7 (22)	23 (59)	120 (86)	215 (96)
Social media	25 (78)	16 (41)	20 (14)	10 (4)
Master’s	15 (47)	20 (51)	106 (76)	194 (86)
PhD	17 (53)	19 (49)	34 (24)	31 (14)
Anthropology	21 (66)	14 (36)	37 (14)	17 (8)
Biological sciences	4 (13)	8 (21)	40 (29)	22 (10)
Economics	0	3 (8)	21 (15)	42 (19)
Gender studies	0	0	1 (1)	3 (1)
Philosophy	0	2 (5)	2 (1)	5 (2)
Political science	0	4 (10)	7 (5)	19 (8)
Psychology	5 (16)	3 (8)	16 (11)	43 (19)
Sociology	0	2 (5)	2 (1)	6 (3)
Social science, other	2 (6)	3 (8)	31 (22)	68 (30)
Graduate student	6 (19)	7 (18)	11 (8)	19 (8)
Faculty	15 (47)	9 (23)	13 (9)	22 (10)
Research	4 (13)	2 (5)	19 (14)	28 (12)
Academia, other	2 (6)	5 (13)	12 (9)	9 (4)
Nonacademia/research	5 (16)	16 (41)	85 (61)	147 (65)

When asked to place themselves on the broad spectrum in academia regarding their opinion on the role of biology in influencing the development of behavioral and psychological traits, participants appear to have segregated themselves correctly. Across the four options, there is a linear increase in the reported degree to which participants believe biology compared to experience to be important in the development of both general traits and in sex differences (Figure 1, from IP survey). Because of the small sample sizes at the tails, the four groups were collapsed into two groups for both surveys: bioists and skeptics.

Figure 1.

Orientation.

Exploring only the IP survey, there was no difference in the reported familiarity with statistical tests across four areas (general statistical knowledge, t tests, correlations, and p values). Bioists did, however, report a stronger quantitative/empirical orientation as opposed to a qualitative/interpretive orientation compared to skeptics (on a 5-point scale, bioists $\bar{x}$ = 2.74, skeptics $\bar{x}$ = 2.30, Wilcoxon Z = −3.66, p < .001).

IP of RDs

Figure 2 presents the reported IP of eight different RDs in descending order of pooled average for both significant (a) and null (b) results. The eight RDs are described in Table 3. There was good internal agreement among ratings within each participant, with a Cronbach’s α of .823 for positive results and .827 for null results. For seven of the eight RDs, bioists’ ratings were higher than those of skeptics for positive results. The effect was not significant, however, using either a sum of standardized ratings (bioists $\bar{x}$ = 0.093, skeptics $\bar{x}$ = −0.077, t = 1.06, p = .292) or an MANOVA of standardized ratings (F = 0.91, p = .517). Similarly, for seven of the eight RDs, skeptics’ ratings were higher than those of bioists for null results. The effect attained one-tailed significance as a test of the summed standardized ratings (bioists $\bar{x}$ = −0.194, skeptics $\bar{x}$ = 0.113, t = −1.85, p = .069) but did not as an MANOVA of standardized ratings (F = 1.59, p = .151). Therefore, while both effects are in the predicted direction, Hypothesis 1a was not supported and there is only tentative support for Hypothesis 1b.

Figure 2.

Research designs.

Table 3.

Research Designs.

Research Design	Description
Cross-cultural universality	A trait or pattern exists universally among cultures…
Twin studies	Identical twins reared together are…compared to identical twins reared apart…[and] are also compared to adopted siblings reared together.
Early developmental onset	A trait appears very early in the development of infants…
Genetic variants	A particular genetic variant is found to be associated with a trait…
Sex hormones within the uterus	Fetuses of a particular genetic sex are exposed to levels of sex hormones within the uterus that are closer to the exposure levels typical of the other genetic sex…
Adaptive design	[A trait that] increases an individual’s ability to survive and/or reproduce.
Hormone levels	A trait is found to be associated with individuals’ circulating levels of a particular hormone.
Animal comparisons	A trait is found to have close analogs in other animal species.

Political Orientation

The four questions referencing political orientation exhibited fair agreement (α = .79, n = 365). These included the importance of reducing inequality, whether social position was determined by inherent traits versus social forces, and where participants placed themselves on the social and the economic political spectrum. Bioists and skeptics did not differ substantially in an index consisting of the sum of these responses (bioists, n = 139, $\bar{x}$ = 19.36; skeptics, n = 225, $\bar{x}$ = 19.64; t = −0.53, p = .596). Nor do the two groups differ along any individual response (Figure 3). Thus, Hypothesis 2 is not supported.

Figure 3.

Political orientations.

Confirmation Bias

In all three conditions, participants were shown the same figure. Bioists, however, did not rate the inferential value of the result higher when accompanied with the information that the allele was found in higher frequencies in populations with African ancestry (Table 4). The difference actually approached significance in the opposite direction than what was predicted (n = 70, t = −1.77, p = .081). Similarly, bioists did not rate the inferential value of the result differently when the allele was reported to occur in higher frequencies in populations of European ancestry (n = 72, t = −0.49, p = .310). Thus, Hypotheses 3a and 3b are not supported.

Table 4.

Ratings of Inferential Values.

Condition	n	Mean	SD	Direction	p Value (Different From Previous)
Bioists
Appetitive aggression higher in nonprivileged	37	53.84	32.45
No mention of frequency	35	71.09	26.01	Opposite	.081
Appetitive aggression higher in privileged	35	57.31	27.26	Opposite	.310
Skeptics
Appetitive aggression higher in nonprivileged	72	57.60	27.58
No mention of frequency	66	60.36	28.87	Predicted	.653
Appetitive aggression higher in privileged	48	49.29	29.51	Opposite	.119

Skeptics did not rate the results in the third condition as having a higher IP when there was no mention of frequencies (n = 111, t = 0.45, p = .653), nor did they do so when it was reported that the allele occurred in higher frequencies in populations with European ancestry (n = 120, t = 1.57, p = .119). Thus, Hypotheses 4a and 4b were not supported.

It should be noted that the ratings were quite variable with an average coefficient of variation of 51%. As reference, the coefficient of variance for height in U.S. men is 8.5% and for weight, it is 38% (McDowell, Fryar, Ogden, & Flegal, 2008). Sample sizes ranged from 33 to 72 for single groups, and consequently, effect sizes that could be detected with 80% likelihood tended to be of moderate size (ranging from 0.43 to 0.61). Many of the reported relationships, however, were often in the opposite direction of those predicted, making Type II errors less likely.

Discussion

The only hypothesis to receive even marginal support was Hypothesis 1b that bioists and skeptics differ in their interpretation of null results of studies exploring biological influences on human psychological and behavioral traits. Similarly, for seven of the eight study designs, bioists rated the IP of positive results as being more inferentially powerful, but the effect was not significant and Hypothesis 1a was not supported. In the IP survey, bioists also rated the specific (positive) results as having a higher inferential value overall, after controlling for condition, adding some weight to Hypothesis 1a (albeit post hoc weight). While such effects are indicative of a true disagreement in the inferential value of such results, it is possible that they result from ontological confirmation biases, as well—that is to say, confirmation biases that exist due to preexisting beliefs, irrespective of ideology (e.g., that flossing prevents gingivitis).

Similar to findings reported in related studies (Lyle & Smith, 2012; Tybur et al., 2007), bioists and skeptics were not found to differ in their political views, failing to support Hypothesis 2. The remaining hypotheses (Hypotheses 3a, 3b, 4a, and 4b) were not supported either. Participants’ perceptions of inferential value did not vary by the degree to which results supported a particular ideology, suggesting that ideological confirmation bias is not affecting participant perceptions of inferential value.

There are a number of potential shortcomings that could be addressed in future research. The high variance in reported inferential values and the modest sample sizes lead to relatively low power in these tests, and thus, a null effect cannot be declared with great confidence. For most tests, only modest effect sizes could be detected with an 80% probability. Recruiting a sufficiently large sample of academics is not easy, and while all participants in the present study had earned an advanced degree in a relevant field, the majority were not working in academia. Those who pursue a career path in academia might follow the literature more closely and have stronger opinions on the role of biology and on equality, such that ideological confirmation biases become more significant. In the IP sample, those working in research or teaching reported being more aware of the research relevant to the debate (5-point scale, academics: $\bar{x}$ = 3.14, n = 112, other: $\bar{x}$ = 2.38, n = 253, Wilcoxon Z = 6.02, p < .001) and were more likely to choose one of the “extreme” options on the 4-point scale (1 or 4) concerning biological versus experiential orientation (academics: extreme = 22.32%, other: extreme = 12.65%, χ² = 5.51, p = .019). A similar RD relying solely on engaged academics might therefore produce different results. For instance, e-mail mailing lists from relevant academic societies could be sampled. These lists tend to number in the thousands, allowing for the creation of a list large enough to overcome the low response rates associated with cold requests.

As described in the Online Supplementary Material, the survey included assessments of mock results aimed at exploring interpretations of null effects as well. However, participants did not rate the complexity or the social desirability of the associated psychological traits in ways that allowed for meaningful comparisons. Future studies could rely on a pilot study which would include the ratings of a battery psychological traits so that comparable traits could be paired. It would be necessary to take into account the many factors, described earlier, that would make a trait more or less appropriate for a such a study—psychological complexity, social desirability, presence of existing literature on the topic, believability, and so on. Fortunately, the tests that remained meaningful, those focusing on appetitive aggression, were those that provided the cleanest tests of ideological bias. In these, the only differences between treatments were small amounts of text that greatly altered the degree to which the results conformed to the alleged ideological biases.

The presentation of the mock results was perhaps more successful. Twenty-nine percent of participants were completely unaware that the results were fake, and an additional 26% were only a little suspicious. Despite this, only 15% of participants believed that their responses were affected more than a little by these suspicions. Fewer than 1% believed their suspicions affected their responses “very much.”

Ultimately, this study did not find any evidence of confirmation bias, despite the common allegations that have been lodged by both sides for some time. Regardless of one’s position, it is important to remind scholars that if they believe a group of intelligent and informed academics could be so unknowingly blinded by ideology that they wholeheartedly subscribe to an unquestionably erroneous interpretation of an entire body of research, then they must acknowledge they themselves are equally as capable of being so misguided. And while it is likely that biases do to some degree impact researchers’ interpretation of individual results and collective literatures, it is also very likely that the overarching debate is also driven by true, lucid, well-informed disagreements about the nature by which complex traits develop.

Supplemental Material

Supplemental Material, sj-pdf-1-evp-10.1177_1474704917752691 - Exploring the Great Schism in the Social Sciences: Confirmation Bias and the Interpretation of Results Relating to Biological Influences on Human Behavior and Psychology

Supplemental Material, sj-pdf-1-evp-10.1177_1474704917752691 for Exploring the Great Schism in the Social Sciences: Confirmation Bias and the Interpretation of Results Relating to Biological Influences on Human Behavior and Psychology by Jeffrey Winking in Evolutionary Psychology

Supplemental Material

Supplemental Material, sj-pdf-2-evp-10.1177_1474704917752691 - Exploring the Great Schism in the Social Sciences: Confirmation Bias and the Interpretation of Results Relating to Biological Influences on Human Behavior and Psychology

Supplemental Material, sj-pdf-2-evp-10.1177_1474704917752691 for Exploring the Great Schism in the Social Sciences: Confirmation Bias and the Interpretation of Results Relating to Biological Influences on Human Behavior and Psychology by Jeffrey Winking in Evolutionary Psychology

Supplemental Material

Supplemental Material, sj-pdf-3-evp-10.1177_1474704917752691 - Exploring the Great Schism in the Social Sciences: Confirmation Bias and the Interpretation of Results Relating to Biological Influences on Human Behavior and Psychology

Supplemental Material, sj-pdf-3-evp-10.1177_1474704917752691 for Exploring the Great Schism in the Social Sciences: Confirmation Bias and the Interpretation of Results Relating to Biological Influences on Human Behavior and Psychology by Jeffrey Winking in Evolutionary Psychology

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The open access publishing fees for this article have been covered by the Texas A&M University Open Access to Knowledge Fund (OAK Fund), supported by the University Libraries and the Office of the Vice President for Research.

Supplemental Material

Supplementary material for this article is available online.

References

Allen

Beckwith

Chorover

Culver

Duncan

… Schreier

(1975). Against “Sociobiology” (pp. 43–44). The New York Review of Books.

Andrews

P. W.

Gangestad

S. W.

Matthews

. (2002). Adaptationism, exaptationism, and evolutionary behavioral science. Behavioral and Brain Sciences, 25, 534–553. Retrieved from <Go to ISI>://WOS:000183687600056

Berinsky

A. J.

Margolis

M. F.

Sances

M. W.

(2014). Separating the shirkers from the workers? Making sure respondents pay attention on self-administered surveys. American Journal of Political Science, 58, 739–753.

Buller

(2006). Adapting minds: Evolutionary psychology and the persistent quest for human nature. Cambridge: MIT Press.

Buss

D. M.

(1989). Sex differences in human mate preferences: Evolutionary hypotheses tested in 37 cultures. Behavioral and Brain Sciences, 12, 1–14.

Campos

J. J.

Bertenthal

B. I.

Kermoian

(1992). Early experience and emotional development: The emergence of wariness of heights. Psychological Science, 3, 61–64.

Chapais

(2008). Primeval kinship: How pair-bonding gave birth to human society. Cambridge, MA: Harvard University Press.

Coyne

(2012). Is evolutionary psychology worthless?. Why Evolution is True. Retrieved from https://whyevolutionistrue.wordpress.com/2012/12/10/is-evolutionary-psychology-worthless/.

Devine

P. G.

(1989). Stereotypes and prejudice: Their automatic and controlled components. Journal of Personality and Social Psychology, 56, 5.

10.

Dixon

T. L.

(2008). Crime news and racialized beliefs: Understanding the relationship between local news viewing and perceptions of African Americans and crime. Journal of Communication, 58, 106–125.

11.

Eagly

A. H.

Wood

(1999). The origins of sex differences in human behavior—Evolved dispositions versus social roles. American Psychologist, 54, 408–423.

12.

El-Guindi

Read

(2012). Westermarck hypothesis reconsidered. Current Anthropology, 53, 134–135. doi:10.1086/663576

13.

Fuentes

(2012). Race, monogamy, and other lies they told you: Busting myths about human nature. Berkeley: University of California Press.

14.

Gettler

L. T.

McDade

T. W.

Feranil

A. B.

Kuzawa

C. W.

(2011). Longitudinal evidence that fatherhood decreases testosterone in human males. Proceedings of the National Academy of Sciences, 108, 16194–16199.

15.

Gould

S. J.

Lewontin

R. C.

(1979). The spandrels of San Marco and the Panglossian paradigm: A critique of the adaptationist programme. Proceedings of the Royal Society of London B: Biological Sciences, 205, 581–598.

16.

Hagen

(2005). Controversial issues in Evolutionary Psychology. The Handbook of Evolutionary Psychology (pp. 145–176). Hoboken, NJ: John Wiley & Sons.

17.

Hamer

D. H.

Magnuson

V. L.

Pattatucci

A. L.

(1993). A linkage between DNA markers on the X chromosome and male sexual orientation. Science, 261, 321–327.

18.

Henrich

Boyd

Bowles

Camerer

Fehr

Gintis

(Eds.). (2004). Foundations of human sociality: Economic experiments and ethnographic evidence from fifteen small-scale societies. New York, NY: Oxford University Press.

19.

Horowitz

Yaworsky

(2014). Whither the blank slate? A report on the reception of evolutionary biological ideas among sociological theorists. Sociological Spectrum, 34, 489–509.

20.

Jonas

Schulz-Hardt

Frey

Thelen

(2001). Confirmation bias in sequential information search after preliminary decisions: An expansion of dissonance theoretical research on selective exposure to information. Journal of Personality and Social Psychology, 80, 557.

21.

Kushnick

Fessler

D. M. T.

(2011). Karo Batak cousin marriage, cosocialization, and the Westermarck hypothesis. Current Anthropology, 52, 443–448. doi:10.1086/659337

22.

Langlois

J. H.

Roggman

L. A.

Casey

R. J.

Ritter

J. M.

Rieser-Danner

L. A.

Jenkins

V. Y.

(1987). Infant preferences for attractive faces: Rudiments of a stereotype? Developmental Psychology, 23, 363.

23.

Leavitt

G. C.

(2007). The incest taboo? A reconsideration of Westermarck. Anthropological Theory, 7, 393–419. doi:10.1177/1463499607083427

24.

Lebow

R. N.

(2013). You can’t keep a bad idea down: Evolutionary biology and international relations. International Politics Reviews, 1, 2–10.

25.

Lehner

P. E.

Adelman

Cheikes

B. A.

Brown

M. J.

(2008). Confirmation bias in complex analyses. IEEE Transactions on Systems, Man, and Cybernetics Part A: Systems and Humans, 38, 584–592.

26.

Lyle

H. F.

Smith

E. A.

(2012). How conservative are evolutionary anthropologists? Human Nature, 23, 306–322.

27.

Marks

(2012). The biological myth of human evolution. Contemporary Social Science, 7, 139–165.

28.

McDowell

M. A.

Fryar

C. D.

Ogden

C. L.

Flegal

K. M.

(2008). Anthropometric reference data for children and adults: United States, 2003–2006. National Health Statistics Reports, 10.

29.

McKinnon

Silverman

(2005a). Introduction. In Complexities: Beyond nature and nurture (pp. 1–20). Chicago, IL: University of Chicago Press.

30.

McKinnon

Silverman

(Eds.). (2005b). Complexities: Beyond nature & nurture. Chicago, IL: The University of Chicago Press.

31.

Mehta

P. H.

Jones

A. C.

Josephs

R. A.

(2008). The social endocrinology of dominance: Basal testosterone predicts cortisol changes and behavior following victory and defeat. Journal of Personality and Social Psychology, 94, 1078.

32.

Munafò

M. R.

Yalcin

Willis-Owen

S. A.

Flint

(2008). Association of the dopamine D4 receptor (DRD4) gene and approach-related personality traits: Meta-analysis and new data. Biological Psychiatry, 63, 197–206.

33.

Nickerson

R. S.

(1998). Confirmation bias: A ubiquitous phenomenon in many guises. Review of General Psychology, 2, 175–220.

34.

Oaten

Stevenson

R. J.

Case

T. I.

(2009). Disgust as a disease-avoidance mechanism. Psychological Bulletin, 135, 303.

35.

Pavelka

M. M.

(2002). Resistance to the cross-species perspective in anthropology. Cambridge Studies in Biological and Evolutionary Anthropology, 25–44.

36.

Pinker

(2002). The blank slate. New York, NY: Viking.

37.

Plavcan

J. M.

(2012). Sexual size dimorphism, canine dimorphism, and male–male competition in primates. Human Nature, 23, 45–67.

38.

Polderman

T. J.

Benyamin

De Leeuw

C. A.

Sullivan

P. F.

Van Bochoven

Visscher

P. M.

Posthuma

(2015). Meta-analysis of the heritability of human traits based on fifty years of twin studies. Nature Genetics, 47, 702–709.

39.

Proctor

Williamson

R. A.

de Waal

F. B.

Brosnan

S. F.

(2013). Chimpanzees play the ultimatum game. Proceedings of the National Academy of Sciences, 110, 2070–2075.

40.

Rose

(2001). Alas, poor Darwin: Arguments against evolutionary psychology. London, England: Cape.

41.

Sanderson

Ellis

(1992). Theoretical and political perspectives of American sociologists in the 1990s. The American Sociologist, 23, 26–42.

42.

Sayer

(1997). Essentialism, social constructionism, and beyond. The Sociological Review, 45, 453–487.

43.

Segerstrale

(2000). Defenders of the truth: The battle for science in the sociobiology debate and beyond. New York, NY: Oxford University Press.

44.

Snyder

Swann

W. B.

(1978). Hypothesis-testing processes in social interaction. Journal of Personality and Social Psychology, 36, 1202.

45.

Stanovich

K. E.

West

R. F.

Toplak

M. E.

(2013). Myside bias, rational thinking, and intelligence. Current Directions in Psychological Science, 22, 259–264.

46.

Sussman

R. W.

(2013). Why the Legend of the Killer Ape Never Dies. War, Peace, and Human Nature: The Convergence of Evolutionary and Cultural Views (pp. 97).

47.

Taber

C. S.

Lodge

(2006). Motivated skepticism in the evaluation of political beliefs. American Journal of Political Science, 50, 755–769.

48.

Thompson

R. A.

Newton

E. K.

(2013). Baby altruists? Examining the complexity of prosocial motivation in young children. Infancy, 18, 120–133.

49.

Tybur

J. M.

Miller

G. F.

Gangestad

S. W.

(2007). Testing the controversy—An empirical examination of adaptationists’ attitudes toward politics and science. Human Nature—An Interdisciplinary Biosocial Perspective, 18, 313–328. doi:10.1007/s12110-007-9024-y

50.

Winking

Gurven

Kaplan

Stieglitz

(2009). The goals of direct paternal care among a South Amerindian population. American Journal of Physical Anthropology, 139, 295–304.

51.

Wolfe

(2006). Settler colonialism and the elimination of the native. Journal of Genocide Research, 8, 387–409.

52.

Wrangham

R. W.

Glowacki

(2012). Intergroup aggression in chimpanzees and war in nomadic hunter-gatherers. Human Nature, 23, 5–29.

53.

Corney

Grant

(2014). Relationship between quality and payment in crowdsourced design. Proceedings of the 2014 IEEE 18th International Conference on Computer Supported Cooperative Work in Design (CSCWD).

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB

0.99 MB

0.20 MB