Abstract
Four experiments were conducted to re-examine the recently proposed “existence bias,” according to which people think that the status quo is good simply because it exists. In Experiment 1, 56 psychology undergraduates read a scenario that described the status quo and a proposal for change in institutional degree requirements. Contrary to the existence bias, in which the status quo is preferred, participants gave a more positive evaluation of the option with more credits. In Experiment 2, with 110 undergraduates from a variety of disciplines, the reasons given for the status quo and the proposed change were manipulated. The existence bias was confirmed, but the effect size was small. Finally, in Experiment 3 (77 psychology students) and Experiment 4 (145 students from various disciplines), in which the students considered the requirements for their own major, participants showed a clear preference for the status quo (with moderate and large effect sizes, respectively). Together, these experiments provide a systematic replication of the existence bias. Suggestions are offered for future research.
Although replication is important for science (Amir & Sharon, 1991; Lamal, 1991), and although procedures for replication have been set out (Rosenthal, 1991), journal editors in psychology have not accorded it high priority in the past (Neuliep & Crandall, 1991). However, there has recently been a renewal of interest in the issue of replication in science in general and in psychology in particular. In particular, there is concern that initially strong effects have become weaker or have even disappeared over time (the “decline effect”; Schooler, 2011; see also Lehrer, 2010, for a popular account that has received attention in academic circles). More seriously, there is concern that many effects might actually be false positives (Ioannidis, 2005; Coyne, 2009; Simmons, Neilson, & Simonsohn, 2011). In addition, it has been shown that if a single paper contains multiple experiments with only moderate power, it is unlikely that all of them will show statistically significant results (Schimmack, 2012). This casts doubt on apparent replications. As part of their attempts to account for these errors, Simmons, et al., (2011) and Schimmack (2012) argue that when planning and conducting experiments, and when analyzing the results, researchers often have many options to choose from (i.e., there are many degrees of freedom), and that they sometimes make decisions that favor their hypotheses (a confirmation bias). Writers have advocated more publication of replications than has commonly been the case, particularly if they are conducted by different research teams (Ioannidis, 2005; Simmons, et al., 2011), and have recommended that these replications should be published even if the null hypothesis is not rejected (Coyne, 2009; Ozonoff, 2010, 2011; Schimmack, 2012). The purpose of the present paper is to report a systematic replication of a recently reported cognitive illusion: the existence bias (Eidelman, Crandall, & Pattershall, 2009, 2010).
Cognitive Biases
Although people can function rationally when making judgments, they are also prone to cognitive biases that lead to systematic errors (Galotti, 2004, p. 434). For example, when making numerical estimates, people employ three major heuristics (representativeness, availability, and anchoring and adjustment; Tversky & Kahneman, 1974). Although these strategies may be economical and effective, they can also mislead. Other kinds of cognitive bias include the hindsight bias (Fischhoff, 1975, 2007; Guilbault, Bryant, Brockway, & Posavac, 2004; McKelvie, 2012), the confirmation bias (Hart, Albaracin, Eagly, Brechin, Lindberg, & Merrill, 2009), the overconfidence effect (Koriat, Lichtenstein, & Fischhoff, 1980; Dunning, Griffin, Milojkovic, & Ross, 1990; Kahneman & Tversky, 1996), and the spotlight effect (Gilovich, Medvec, & Savistsky, 2000). These heuristics and biases (also known as cognitive illusions), have been demonstrated both in the laboratory and in everyday life (e.g., Nicholls, 1999; Fischhoff, Gonzalez, Small, & Lerner, 2005; Hergovich, Schott, & Burger, 2010), and are most likely to occur when people make intuitive predictions and judgments under conditions of uncertainty (Kahneman & Tversky, 1996).
As noted above, the existence bias has recently been identified. Here, people infer that something is good simply because it exists, leading them to overvalue the present state of affairs (Eidelman, et al., 2009). This is different from previous cognitive biases where people might, for example, overvalue their own performance (the overconfidence effect) or overvalue the accuracy of their predictions (the hindsight bias). According to Eidelman, et al. (2009) the tendency to see “what is” as right was noted by the philosopher David Hume over 200 years ago. However, Eidelman, et al. formally captured the phenomenon in five experiments with different scenarios (a company moving location, a university changing credit requirements, and with choices involving Presidential candidates, aesthetic judgments, and taste judgments). In all cases, Eidelman, et al. showed that undergraduates systematically favored the status quo. In a second series of experiments (Eidelman, et al., 2010), these results were generally confirmed and extended with four new scenarios (the same institutional requirement, and with choices involving acupuncture, art, nature and chocolate). This constitutes a “systematic replication” (McBurney & White, 2010, p. 209) or a “partial replication” (Hendrick, 1991; Shaughnessy, Zechmeister, & Zechmeister, 2012, p. 214), in which features such as participants, stimuli or measures (and in this case, the scenarios) of the original experiment are changed and others are left intact.
As both McBurney and White (2010, p. 209) and Shaughnessy, et al. (2012, p. 214) observe, this kind of replication tests external validity. However, with one scenario (the institutional requirement for the number of credit hours required for the major), there was one inconsistent result. The existence bias occurred in the first study (Eidelman, et al., 2009, Experiment 2) and was replicated with a second sample (see their General Discussion). In contrast, in one condition in the second study, there was a trend towards favoring the status quo, but it was not statistically significant (Eidelman, et al., 2010, Experiment 1). Consequently, the present studies were designed to re-examine the existence bias with an institutional degree requirement. If for some reason this kind of bias is becoming weaker over time, it might not occur in the present study. However, if Eidelman, et al.'s (2010) second result is an anomaly, or perhaps a consequence of reporting multiple experiments with moderate power (Schimmack, 2012), then the bias should be replicated.
Comparing Methodological Features of the Two Experiments
To introduce the present research, Eidelman, et al.'s (2009, 2010) procedures and results were considered in some detail, in an attempt identify the most important factors that might contribute to the existence bias. This analysis reflects the concern that researchers have many degrees of freedom in their procedures and that some of their methodological choices may influence their results (Simmons, et al., 2011; Schimmack, 2012).
In the first study (Experiment 2, Eidelman, et al., 2009), undergraduates read about a proposed change in degree requirements that would occur after they were scheduled to graduate. They were told that the present requirement of 32 credit hours might change to 38, or that the requirement of 38 might change to 32. They were informed that 32 hours would provide greater breadth and that 38 hours would provide greater expertise. They rated each option on three evaluative dimensions (“good,” “right,” “the way things ought to be”), scores for which were combined. Under these circumstances, it was found that ratings were statistically significantly higher for the present state of affairs than for the proposed change, with an effect size of η2 = 0.13 that converts to a standardized effect size d = 0.77. According to Cohen's (1977) guideline that 0.20, 0.50, and 0.80 indicate small, medium, and large effects respectively, this effect is large.
In the second study (Eidelman, et al., 2010), the main purpose was to investigate time in existence, to discover if the bias towards the status quo would be stronger if it had been in place for a longer period of time. In their Experiment 1, undergraduates read about the same proposed change in degree requirements as in the previous study and were told that any change would not occur for 10 years. To examine time in existence, the researchers also manipulated how long the present requirement had been in place: 10 years or 100 years. Under these circumstances, ratings were again generally higher for the present state of affairs than for the proposed change, but there was a statistically significant interaction between this effect and the longevity of the present requirement. When the status quo had been in place for 100 years, there was a very large preference for it over the proposed change (η2 = 0.58, which converts to d = 1.42). However, when the status quo had been in place for 10 years, there was what the authors term a “weak” preference for it over the proposed change (η2= 0.24). This converts to d = 0.50, which is actually “medium” by Cohen's standards. These results show that the existence bias was stronger when the status quo had been in place for a longer time. However, and of most importance for the present argument, the smaller effect for 10 years in existence was not statistically significant (p < .11).
A statistically non-significant effect that is medium in size suggests that the experiment may have lacked power. Alternatively, there may be something special about being in existence for 10 years. It is also possible that some of the methodological variations between the two experiments might account for the inconsistent results in this condition. Certain procedural differences (in class or in small groups, questionnaires part of a larger package or not, respectively) seem incidental. However, three other differences seem more substantial: sex distribution (more women or unknown, respectively), timing of the proposed change (soon but probably after they graduate or in 10 years, respectively), and time in existence (unspecified or 10 years, respectively). The main goal of Experiment 1 was to re-examine the existence bias using Eidelman, et al.'s (2009) original values for these three variables (more women, change to occur soon, unspecified time in existence). Assuming that the existence bias is not declining over time, it was hypothesized that the effect would be replicated.
Procedure for Experiment 1
With regard to the timing of the change, another factor that may have played a role in the different results in the two experiments is the direct personal relevance of the proposal. People might prefer the status quo if they thought there was a possibility that their own programme requirements might change, but they might care less about it if they were certain that they would not be affected by the change. In the first paper (Eidelman, et al., 2009), participants were told that the proposed change would occur after the students were scheduled to graduate. However, although the date when these data were gathered was not given, the paper was published in 2009 and it is likely that the year given for the change (2010) was not far off. Consequently, some participants might not have been sure when they would graduate, and therefore might have thought that they could be affected by the proposal. This may have biased them towards the status quo. In the second paper, in which the change would take place in 10 years, everyone would have graduated, so that they would not be affected. Under these circumstances, they may not have cared so much about the status quo and consequently may not have exhibited the existence bias, at least in the condition where the status quo had been in place for 10 years. For Experiment 1, it was decided to present the change as occurring in the near future, as Eidelman, et al. (2009) did, but to remove any ambiguity about personal relevance by making clear that the change would only apply to new students. This controls for the possibility that Eidelman, et al.'s (2009) participants favored the status quo because they thought that they might have been personally affected by the change. Notably, assurance that the change would only take place for new students is consistent with the policy of the university that the present participants were attending, because students have the right to be governed by the regulations in the calendar under which they initially enrolled. If direct personal relevance is an important factor in the existence bias, the effect will not occur in Experiment 1 because it is clear that the students would not be affected by any changes.
One of the interesting features of the institutional credit requirement scenario is that the issue is generally relevant to the lives of students, and more relevant than most of the other scenarios that were employed in the two experiments (business location, presidential candidates, aesthetics, taste, acupuncture, art, and nature). Indeed, as noted above, the participants in these two experiments were led to believe that the proposed change might actually occur (in the first case, in the near future but after they were scheduled to graduate; in the second case, not for 10 years). Moreover, I consulted the website of the university where the students were tested (University of Maine) and found that for a number of major disciplines (English, history, political science, mathematics, psychology, sociology), the required number of credits ranged from 30 to 40 credit hours. In particular, for psychology it was 33 credit hours. This is similar to the numbers (32, 38) mentioned in the scenarios, giving them credibility. However, it is also likely that some students would have been aware of the requirement for their major. If so, unless 32 credit hours or 38 credit hours happened to be the status quo in their discipline, they may have doubted that the scenario was real. Even if the number that they were given were the status quo in their discipline, they may have speculated that there would be other people in the room to whom it would not apply.
These considerations indicate that it is not clear what the mindset of Eidelman, et al.'s (2009) participants was. They may have thought that the scenario was real and would occur at their university, they may have thought that it was imaginary, or they may have been unsure. In Experiment 1, uncertainty was minimized by indicating that the proposal was not occurring at the participants' institution. More specifically, it was stated that the scenario was being considered at “a” university, which implies that it is not the one that the students were attending. This not only removes doubt about whether the scenario was real, it also reinforces the message that the students would not be affected by the change. However, in order for the credit requirements in the scenario to appear reasonable to the participants, they were changed to resemble those in place in the Québec university system in which the students studied, where the requirement for the major in most disciplines ranges from 42 credits to 48 credits in a three-year, 90-credit programme.
Given the usual population of students who take introductory psychology, it is likely that the sample tested by Eidelman, et al. (2009) was composed of students from different disciplines, with psychology the most frequent. In Experiment 1, it was decided to focus on this group. This decision also controlled for the various assumptions about majors associated with having students from different disciplines. In particular, the students were enrolled in a course in research methods. Although such students are likely to be more aware than most about the choices that are made when planning research, they were tested in the second week of the semester before these technicalities were discussed and were asked to carefully follow the instructions. In addition, although testing such students may limit the generality of the results, there is no indication in Eidelman, et al.'s (2009, 2010) work of any restrictions to whom the existence bias applies. As Eysenck (1975) argues in a defense of constrained samples, if a theory does not specify exceptions, it is appropriate to test it with any group.
One aspect of Eidelman et al.'s (2009) original procedure was altered. Whereas their participants were given reasons for requiring more credit hours and for requiring fewer credit hours, the present participants were only given the argument for the proposed change. This might create a demand characteristic to favor the change, thereby weakening the existence bias, perhaps removing it or even reversing it. Consequently, any bias under these circumstances would be robust. On the other hand, only providing an argument for change might be perceived as an attempt to sway opinion, and might encourage students to think of their own argument for defending the status quo. This process resembles psychological reactance, which occurs when freedom is threatened (Brehm, 1966). If participants developed their own counterarguments supporting the status quo, reasons for change and reasons for maintaining the status quo would be balanced, as in Eidelman, et al.'s (2009) work. Given that they found the existence bias under these conditions in their 2009 study, this implies that the existence bias would be replicated.
The remainder of the procedure followed the description given by Eidelman, et al. (2009) (e.g., ratings on the same three evaluative dimensions). If a lack of personal relevance was a factor in Eidelman, et al.'s (2010) statistically non-significant result, then the existence bias would not occur in the present experiment. However, it that is not the case, then, given that the choices for the other three major variables (sex distribution, the time of the proposed change, and the time in existence for the status quo) were also like those of Eidelman, et al.'s (2009) experiment, then the existence effect should be replicated. That is, students for whom the status quo was 48 credits would evaluate 48 credits more positively than 42 credits, and students for whom the status quo was 42 credits would evaluate 42 credits more positively than 48 credits.
Experiment 1
Method
Participants
Fifty-six undergraduate students (13 men, 43 women; 77% women) enrolled in a psychology research methods course at Bishop's University received 1% course credit for their participation. They were assigned randomly to two groups, but with proportional matching for sex. There were six men and 20 women (77% women) in the 48-credit status quo group and seven men and 23 women (77%) in the 42-credit status quo group.
Materials
The questionnaire was modeled after Eidelman, et al.'s (2009) description of their materials with the exception that the number of credits was modified as noted above. The scenario read as follows:
“Imagine that you are a student at a university where the requirements to graduate in your 3-year degree programme are 90 credits in total (30 courses), 42 credits of which (14 courses) [or, 48 credits (16 courses)] must be taken in your major area of study. For new students starting the following year, the university is proposing to change the number of credits for the major to 48 (16 courses) [or, 42 credits (14 courses)]. This would give a student more courses in the major, leading to more expertise in the student's discipline [or, This would give a student more choice outside of the major, leading to more breadth in the student's education]. Use the following rating scale, please give your answer to the questions that follow. Write a number beside each question.”
The rating scale had anchors of 1: Do not agree and 9: Agree strongly. There were six questions, presented in the following order: “I think that the requirement of X credits is good,” “I think that the requirement of X credits is right,” and “I think that the requirement of X credits is the way things ought to be.” For half of the participants in each condition, X was 42 for these three questions and then 48 for the next three (“good, right, ought to be”). For the other half of the participants, this order was reversed.
Procedure
Participants were tested in a group in two class sessions. They were placed so that they could only see their own questionnaire. The four versions of the questionnaire (42 or 48 credits as the status quo; first three questions 42 and second three question 48 credits, or vice versa), were distributed randomly throughout the class, with care taken to ensure than men and women were represented proportionally in each condition. Participants were told to read the scenario and answer the questions that followed it. Informed consent was obtained and participants were debriefed.
Results
Alpha was set at .05. Ratings from the three scales were combined and the results are shown in Table 1.
Evaluation of Credit Requirements for Each Programme in Experiments 1 and 2
The independent variables were status quo (42 or 48 credits), question order (first three 42 and second three 48 or vice versa), sex (women, men), and credits being judged (42, 48). A 2 (Status Quo) × 2 (Question Order) × 2 (Sex) × 2 (Credits) mixed model analysis of variance (ANOVA) with status quo, question order, and sex as between- subjects variables, and credits as a within-subjects variable showed that two effects were statistically significant: credits (F1,48 = 36.01, p < .001, η2 = 0.37), and the interaction between status quo and credits (F1,48 = 5.69, p = .02, η2 = 0.06). Participants generally rated the 48-credit major more positively than the 42-credit major (d = 1.47). The interaction occurred because this effect was stronger when 48 credits was the status quo (d = 1.97) than when 42 credits was the status quo (d = 1.09). In addition, the preference scores for the 48-credit programme were similar when either 48 credits or 42 credits was the status quo. However, the preference scores for the 42-credit programme were higher when 42 credits was the status quo (t54 = 3.01, p = .004, d = 0.80).
Discussion
Eidelman, et al. (2009, 2010) found that participants rated the credit requirement for the status quo (32 or 38 credit hours) as higher on “good”, “right,” and “the way things ought to be” than the credit requirement for the proposal for change (38 or 32 credit hours respectively). In addition, effect sizes were large. However, when the present regulations had been in place for 10 years, the effect size was moderate but the result was not statistically significant (Eidelman, et al., 2010).
It was speculated that one reason for the latter result might be that participants in this condition were also told that any changes would not take place for 10 years, and that they may have been less likely to favor the status quo if they knew they would not be affected personally. The results of Experiment 1, in which participants did not favor the status quo, are consistent with this possibility, because it was clear that the change would never apply to students who were currently enrolled.
It was suggested that three factors (sex distribution, timing of the proposed change, and time in existence for the status quo) might be related to the existence bias effect. In the present experiment, the values of these three factors (more female than male participants, change occurring after the students would have graduated and an unspecified time in existence), were similar to those in Eidelman, et al.'s (2009) experiment in which the existence bias was found. Despite this similarity, the effect was not replicated. Rather, participants very strongly preferred the 48-credit programme to the 42-credit programme, and this effect occurred both when 48 credits was the status quo and when 42 credits was the status quo. The only hint of an existence bias was that the preference for 48 credits was smaller when 42 credits was the status quo. This was mainly due to the fact that the 42-credit programme was rated more favourably (an effect that was large) when 42 credits was the status quo than when 48 credits was the status quo.
One reason for the strong preference for the higher number of credits may be that the participants were all majors from one discipline (psychology) taking an important course (research methods) towards their degree. In particular, the psychology research methods students may have shared the assumption that more knowledge in the discipline is an advantage. Furthermore, the actual number of credits required in the programme at this university was 54, which is closer to 48 than to 42. Consequently, even though the participants were instructed to imagine that they were a student at “a” university with the stated requirement, their preference for 48 credits over 42 credits may have been influenced by a preference for their own programme, which has a fairly high number of required credits. This may mean that they preferred greater expertise over greater choice, i.e., more credits, but it may also mean that they simply preferred their own existing requirements. If so, their response might actually represent a true existence bias (as distinct from one based on an experimental manipulation) for this specific group of students. To explore this matter further, Experiment 2 was conducted with a broader sample of participants from different programmes in the university. This was more similar to the population in Eidelman et al.'s experiments because students taking introductory psychology are usually drawn from a variety of disciplines. Furthermore, most programmes at the university require 42 to 48 credits.
In Experiment 1, another change was made to Eidelman, et al.'s (2009) procedure. Whether participants were exposed to the change from 32 to 38 credit hours or from 38 to 32 credit hours, Eidelman, et al. (2009) gave them reasons both for requiring 32 credit hours (more breadth outside the major) and for requiring 38 credit hours (more expertise within the major). In contrast, in Experiment 1, participants were only given a reason for the proposed change (48 represents more depth/expertise when the status quo was 42 and 42 represents more breadth when the status was 48). Although it was argued that only giving reasons for the change may have been balanced by participants creating their own reasons for the status quo, it was also observed that the reasons may have biased participants towards favoring change, which is the opposite of the existence bias. Clearly, this did not occur because participants preferred a higher number of credits.
In Experiment 2, this issue was examined by including conditions with three sets of reasons: firstly, Eidelman et al.'s (2009, 2010) balanced reasons for the status quo and for change; secondly, the reasons for change alone (as in Experiment 1 here); and thirdly, a control condition in which no reasons were given. Ideally, if the existence bias occurred in Experiment 2, it would be unaffected by the different reasons. However, if the reasons did have an effect, the existence bias would probably be weaker with the reasons for change alone.
After Experiment 1 was completed, Dr. Eidelman shared his materials for the institutional credit study from his 2009 paper, but indicated (personal communication) that they might not be exactly correct because he had employed a number of slightly different sets of materials in various studies. In fact, they stated that the proposed change in credit requirements would take place in 10 years, which was the time for implementation in his second experiment with the scenario involving institutional credits (Eidelman, et al., 2010).
The rating scale in Eidelman's materials had anchors 1: Strongly disagree and 9: Strongly agree. In the 2009 and 2010 papers, only the numbers were reported.
This was not the same as in the present Experiment 1, where the scale had anchors 1: Do not agree and 9: Agree strongly. Moreover, the format of the six questions was also slightly different. In Experiment 1 here, the questions for “good”, “right”, and “ought to be” were presented in this order for 48 credits then for 42 credits or for 42 credits then for 48 credits. In Eidelman, et al.'s (2009) materials, the questions were presented in pairs, as follows: two “right” questions, one for 32 credit hours then one for 38 credit hours, two “good” questions (32 credit hours then 38 credit hours), and two “ought to be” questions (32 credit hours then 38 credit hours). In addition, the order “32, 38” was constant throughout the experiment (Eideleman, personal communication). Experiment 2 employed the format of Eidelman's questionnaire, but tightened control over any order effect by counterbalancing the order in which the two sets of credit requirements were rated. Finally, after the six questions just described, Eidelman's materials had three additional questions that were not analyzed in either of his papers. To ensure that the questionnaire in Experiment 2 was identical to Eidelman's materials, they were included. Although these methodological details may seem minor, they again reflect the large number of degrees of freedom that researchers have when planning their experiments and which may contribute to the results in subtle ways (Schimmack, 2012).
In Eidelman, et al.'s (2009) paper, and in Experiment 1 here, most of the participants were women, whereas in Eidelman, et al.'s (2010) paper the sex distribution was not specified. While the sex of the participant was not a statistically significant factor in Experiment 1, equal numbers of men and women were included in Experiment 2. This provided a better test of any sex difference in the existence bias.
In summary, compared to Experiment 1, the methodology of Experiment 2 (materials, participants) was more similar to the methodology in Eidelman, et al.'s (2009) experiment. However, the order of the questions was counterbalanced, the sex distribution of participant was equated, and reasons for the status quo and proposal for change were manipulated. On the basis of these considerations, it was hypothesized that the existence bias would be replicated. At the same time, it was again made clear that students in the scenario would not be affected by the proposed changes. If personal relevance were a factor in the existence bias, this feature of the experiment would undermine the existence bias.
Experiment 2
Method
Participants
Participants were 110 (55 women, 55 men) undergraduates enrolled as majors in a variety of programmes at Bishop's University. They were assigned randomly to six experimental groups (three sets of reasons, two status quos) with matching for sex.
Materials
The scenario and questionnaire were based on the materials supplied by Eidelman. However, the credit requirements for the major programmes were modified as in Experiment 1. The scenario read as follows for the condition in which Eidelman, et al.'s (2009, 2010) balanced set of reasons were given:
“Imagine that you are a student at a university that is considering a change in degree requirements for graduating students. The existing rule states that the student must complete a total of 48 credits (16 courses) [or (42 credits, 14 courses)] within their major in order to graduate. Recently there has been a proposal to change this rule, such that students would need to complete 42 credits (14 courses) [or 48 credits (16 courses)] within their major in order to graduate. Those in favor of keeping the existing requirement argue that students will be more competitive on the job market because they will have greater expertise in their area of interest. Those who want to change the requirements maintain that the greater breadth of knowledge gained from taking more non-major courses will increase students' attractiveness to potential employers. [These reasons were presented in the opposite order when the status quo was 42 credits.] Either way, the total number of credit hours needed to graduate would remain the same, so there would be no financial cost to students. If adopted, this change would take place in ten (10) years. For each of these questions, circle the number that corresponds to your opinion.”
In the other two reasons conditions, participants received either the same reasons as in Experiment 1 (i.e., reasons for the proposed change) or no reasons at all.
The rating scale had anchors 1: Strongly disagree and 9: Strongly agree. The six main questions were presented in the following order: “I think that the requirement of X credit hours within one's major is right” (with the question repeated for the other number of credits), “I think that the requirement of X credit hours within one's major is good” (with the question repeated for the other number of credits), and “I think that the requirement of X credit hours within one's major is the way things ought to be” (with the question repeated for the other number of credits). For half of the participants in each condition, X was 48 then 42. For the other half of the participants, this order was reversed. As in Eidelman, et al.'s (2009) questionnaire, there were three additional questions at the end, but they were not scored.
Procedure
Participants were recruited on an ad hoc basis around the campus and tested individually at a variety of campus and off-campus locations. They were told to read the scenario and to answer the questions that followed it. Informed consent was obtained and participants were debriefed.
Results
Alpha was set at .05. Ratings from the three scales were combined and the results are shown in Table 1.
The independent variables were reasons (reasons for the both status quo and for change, reasons for change alone, no reasons), status quo (48 or 42 credits), question order (48 then 42 for pairs of questions or vice versa), sex of participant (female, male), and credits being judged (48, 42). In the corresponding 3 (Reasons) × 2 (Reasons) × 2 (Status Quo) × 2 (Question Order) × 2 (Sex) × 2 (Credits) mixed model ANOVA with reasons, status quo, question order and sex as between-subjects variables and credits as a within-subjects variable, two interactions were statistically significant: status quo × credits (F1,86 = 4.54, p = .036, η2 = 0.04), and status quo × order × sex (F1,86 = 9.85, p = .002, η2 = 0.09). Because the second interaction did not involve credits, it was not considered further. Collapsing over the statistically non-significant effects, a 2 (Status Quo) × 2 (Credits) ANOVA was conducted. Here, there was one statistically significant interaction, status quo × credits (F1,108 = 4.85, p = .03, η2 = 0.04). Table 2 shows that when 48 credits was the status quo, 48 credits was preferred over 42 credits (d = 0.19) and when 42 credits was the status quo, 42 credits was preferred over 48 credits (d = 0.24). Averaging these two values, the overall effect size was d = 0.21.
Evaluation of Credit Requirements for Each Programme in Experiments 3 and 4
To confirm these results, and to investigate more directly whether the existence bias was related to any predictor variables, an existence bias score was calculated for each participant by subtracting their rating for the changed credits from their rating for the status quo credits. This difference score was examined with a Reasons × Status Quo × Question Order × Sex between groups ANOVA. None of the effects were statistically significant. However, for all 110 participants combined, the mean existence bias score was 2.20 (SD = 10.43), which was statistically significantly different from 0 (t109= 2.10, p = .029, d = 0.21).
Discussion
In Experiment 2, the existence bias was successfully replicated. Participants rated the status quo, whether it was 48 credits or 42 credits, as higher on being right, good and the way that things ought to be.
Although the standardized effect size of 0.21 was small, it provides some external validity for Eidelman, et al.'s (2009) original demonstration of the existence bias with institutional requirements. Firstly, it occurred with university students in a different country (Canada rather than the USA). Secondly, it occurred with both women and men, with both orders of presentation of the ratings for 48 and 42 credits, and with all three sets of reasons for the proposed change. Moreover, the equal number of women and men and the counterbalancing provide greater control over sampling and procedure compared to the studies of Eidelman, et al. (2009, 2010). Indeed, these controls may have contributed to the smaller effect. Notably, the existence bias was not smaller for the reasons that were potentially leaning against it (giving only a justification for change and not for the status quo). Together, these results suggest that the existence effect may be small, but it is robust.
Also notable is the fact that the effect was obtained when students in the scenario would clearly not be affected by the changes. This contradicts the suggestion that the difference between Eidelman, et al.'s 2009 and 2010 results was due to personal relevance, i.e., that the statistically significant effect in 2009 occurred because students thought they might be affected and the statistically non-significant effect in 2010 occurred because students knew they would not be affected.
Given that the existence bias was found in Experiment 2, the preference of psychology students in Experiment 1 for a higher number of credits (48 over 42) may indeed reflect a preference for the actual programme in which they were enrolled, which had 54 credits. As observed above, this may be a naturally occurring existence bias. To examine this issue further, a second experiment was conducted with psychology research methods students. However, in Experiment 3, the scenario was altered to apply directly to their university. The students were told to imagine that their university was considering a change to the degree requirements in psychology. Notably, Eidelman, et al. (2009) also explicitly asked students engage in an imagination exercise in their scenario with presidential candidates (Experiment 3). Participants in the present experiment were reminded that the existing requirement was 54 credits (18 courses). They were then told that the proposed alternative was 60 credits (six credits more) or 48 credits (six credits less). In addition, Eidelman, et al.'s (2009) balanced set of reasons were given. Under these conditions, it was expected that the existence bias would be replicated.
Experiment 3
Method
Participants
Participants were 77 (62 women, 15 men; 85% women) psychology major students who were enrolled in a research methods course at Bishop's University. They were assigned randomly to two experimental groups (proposal to increase credits, proposal to decrease credits) with proportional matching for sex.
Materials and Procedure
The scenario and questionnaire were again based on the materials supplied by Eidelman. However, it was stated that the psychology department was considering a change in the credit requirements for the major. The scenario read as follows:
“Consider this scenario: You are a psychology student at Bishop's University and the department is thinking about a change in degree requirements for graduating students. Under the existing rule, you must complete 54 credits (18 courses) for the general major in order to graduate. We would like you to consider your reaction to a proposal to change this rule, such that students would need 60 credits (20 courses) [or 48 credits, (16 courses) in order to graduate.”
The remainder of the instructions, the format of the questions, and the rating scale were the same as in Experiment 2. In addition, for half of the participants in each condition, the ratings were presented for 54 credits followed by 48 credits (or 60 credits) for each pair of questions (right, good, ought to be). For the other half of the participants, this order was reversed. As in Experiment 2, there were three additional questions at the end that were not scored.
As in Experiment 1, participants were tested in a group in two class sessions. The four versions of the questionnaire (alternative proposal of 60 credits, alternative proposal of 48 credits; 54 credits rated first, alternative credits rated first) were distributed randomly throughout the class. Participants were told to read the scenario and answer the questions that followed it. Informed consent was obtained and participants were debriefed.
Results
Alpha was set at .05. Ratings from the three scales were combined and the results in each condition are shown in Table 2.
The independent variables were change (60 credits, 48 credits), question order [54 then 60 (48) for pairs of questions or vice versa] and credits (present rule, alternative proposal). In the corresponding 2 (Change) × 2 (Question Order) × 2 (Credits) mixed-model ANOVA with change and question order as between-subjects variables and credits as a within-subjects variable, there were four statistically significant effects: the main effect of change (F1,73 = 12.46, p = .001, η2 = 0.04), the main effect of credits (F1,73 = 23.81, p < .001, η2 = 0.19), the interaction between change and credits (F1,73 = 7.34, p = .008, η2 = 0.06), and the interaction between order and credits (F1,73 = 19.12, p < .001, η2 = 0.16).
For the main effect of credits, preference scores were higher for the status quo than for the alternative proposal (d = 0.46). However, although this effect was statistically significant when the proposal was to decrease credits from 54 to 48 (t36 = 4.17, p < .001, d = 0.68), it was not statistically significant when the proposal was to increase credits from 54 to 60 (t39 = 1.56, p = .13, d = 0.25). In addition, it was statistically significant when the status quo was rated first (t36 = 6.91, p < .001, d = 1.14) but not when it was rated second (t39 = 0.32, p = .75). For the main effect of change, ratings were generally higher for the proposal to increase credits than for the proposal to decrease credits. However, this effect only occurred for the ratings of the proposed change (t75 = 3.50, p = .001, d = 0.05), not for the ratings of the status quo (54 credits) (t75 = 0.35, p = .73).
To dissect these two interactions further, the effect of credits was examined for the four change/order combinations. When the status quo (54 credits) was rated first, ratings were higher for the status quo than for the alternative for both the 60 credit alternative (t = 4.05, p = .001, d = 0.91), and for 48 credit alternative (t16 = 6.15, p < .001, d = 1.49). However, when the status quo was rated second, neither of these comparisons was statistically significant (t19 = 1.34, p = .20, d = 0.30; 60 credits alternative) and (t19= 1.35, p = .19, d = 0.30; 48 credits alternative).
Discussion
The most important finding in Experiment 3 is that, for all participants combined, the existence bias was replicated, with a medium effect size (d = 0.46). Because this result was obtained with psychology students drawn from the same population as in Experiment 1, it suggests that the earlier finding in which participants favored 48 credits over 42 credits was due to a preference for the students' own status quo, which was 54 credits. In the present experiment, when similar students were asked to consider the change relative to that status quo of 54 credits, they preferred 54 credits.
However, the overall existence bias was moderated by both the alternative (an increase to 60 credits or a decrease to 48 credits) and the order in which the status quo and the alternative were judged. In particular, it seems that there was both an existence bias, with a preference for the present state of affairs and an order effect, with higher ratings for the credits that were rated first. When the alternative was a decrease to 48 credits and 54 credits was rated first, the two effects combined to produce a much higher rating for 54 credits than for 48 credits (very large effect, d = 1.49). However, when 54 credits was rated second, the existence bias and the order effect cancelled each other out to produce similar ratings for 54 credits and for 48 credits. When the alternative was an increase to 60 credits and 54 credits was rated first, the two effects combined to produce a higher rating for 54 credits than for 60 credits. However, although this effect was large (d = 0.91), it was not as large as for the decrease to 48 credits. In addition, when 54 credits was rated second, the existence bias and the order effect combined to produce similar ratings for 54 credits and for 60 credits. Notably, in this case, the mean rating for 60 credits was actually slightly higher, although not statistically significantly higher, than the rating for 54 credits. Together, these results show both an existence bias and an order effect, but the existence bias appears to be greater for the proposed decrease to 48 credits than for the proposed increase to 60 credits. This suggests that psychology students prefer the status quo of 54 credits, but are less opposed to the alternative of 60 credits than to the alternative of 48 credits. That is, they favor a high number of credits.
In the final experiment, an attempt was made to generalize the existence bias found in Experiment 3 to a wider sample of university students (as in Experiment 2). Similar numbers of male and female participants were recruited from a variety of disciplines and asked to rate the status quo in their own major against the proposed alternatives of six more credits or six fewer credits. It was predicted that the existence bias would again be replicated.
Experiment 4
Method
Participants
Participants were 145 undergraduates (78 women, 67 men) enrolled as majors in a variety of programmes (natural sciences, social sciences [but not psychology], humanities, business, and education] at Bishop's University. They were assigned randomly to four experimental groups (two kinds of change, two orders) with matching for sex.
Materials and Procedure
The scenario and questionnaire resembled those in Experiment 3, with the modification that the reference to psychology was replaced by a reference to the participant's major. The instructions read as follows:
“Consider this scenario: The department in which you enrolled for your major is thinking about a change in degree requirements for graduating students. We would like you to consider your reaction to a proposal to change the rules, such that students would need two more courses (or two fewer courses) in order to graduate.”
The remainder of the instructions and the questionnaire were the same as in Experiment 3. As in Experiment 2, participants were recruited on an ad hoc basis and tested individually at a variety of campus and off-campus locations. They were told to read the scenario and to answer the questions that followed it. Informed consent was obtained and participants were debriefed.
Results
Alpha was set at .05. Ratings from the three scales were combined and the results are shown in Table 2.
As in Experiment 3, the independent variables were change (60 credits, 48 credits), question order [54 then 60 (48) for pairs of questions or vice versa] and credits (present rule, alternative proposal). In the corresponding 2 (Change) × 2 (Sex) × 2 (Question Order) × 2 (Credits) mixed model ANOVA with change, sex, and question order as between-subjects variables and credits as a within-subjects variable, there were two statistically significant main effects: change (F1,137 = 3.94, p = .049, η2 = 0.03) and credits (F1,137 = 107.96, p < .001, η2 = 0.43). When the data were collapsed over sex and order, the only statistically significant effect in the 2 (Change) × 2 (Credits) mixed ANOVA was credits (F1,143 = 108.99, p < .001, η2 = 0.43). Ratings were higher for the status quo than for the alternative (d = 0.87).
General Discussion
In Experiment 4, male and female participants from a variety of disciplines demonstrated a large preference for the present state of affairs in their own discipline over the proposals to increase or decrease the number of required credits. This confirms the results of Experiment 3, where the existence bias occurred with psychology students who were thinking about the requirements in psychology. Although the existence bias also occurred with a varied sample of students in Experiment 2, that effect was small. One reason for the stronger effect in Experiment 4 may be that the students were explicitly thinking of their own major, whereas those in Experiment 2 were not.
Together, the four experiments reported here provide further support for the existence bias, particularly with the scenario involving institutional requirements, which is relevant to students. Although participants in Experiment 1 preferred more credits than fewer credits, it is likely that they were influenced by the programme in which they were enrolled, which required a fairly high number of credits. When another sample from the same population (psychology research methods students) was tested in Experiment 3, they favored that requirement over the alternatives of more or fewer credits, although the effect was weaker with the proposal for more. This indicates that these students favored the present state of affairs, but were also inclined towards a higher number of required credits.
Why, then, did Eidelman, et al. (2010) find only a statistically non-significant trend in favor of the status quo when it had been in place for 10 years? Earlier, it was suggested that this might have been a consequence of students being certain that they would not be affected personally by proposed changes. However, this possibility was contradicted by the presence of the existence bias in Experiment 2 where this condition held. The results of Experiments 3 and 4 support this argument because the existence bias again occurred under this condition.
In view of Eidelman, et al.'s other work (2009, 2010) and of the present results, Eidelman, et al.'s (2010) outcome may have been a sampling error. On the other hand, given Schimmack's (2012) claim that a multiexperiment paper is likely to have low total power, with some statistically non-significant results, this outcome may have been one of them. Another possibility, perhaps more likely, is that time in existence is an important factor in the existence bias. In Eidelman, et al.'s (2009) Experiment 2 and in the present work with the institutional requirement scenario, time in existence was not specified. If participants took this to mean that the present regulation had not been in place for long, people may have favored it because they thought it was too early for a re-evaluation. In addition, when the present regulation had been in place for 100 years (Eidelman, et al., 2010, Experiment 1), participants may have favored it because any regulation that had survived for such a long time must be good. By these arguments, the non-significant existence bias for the condition in which the regulation had been in place for 10 years implies that 10 years was not perceived as a long time. Although 10 years may seem like a long time in some situations (e.g., for the life of a smart phone), it may not be long for degree requirements at a well-established university. 2 Perhaps being in existence for 10 years is just the right amount of time for a re-evaluation, meaning that participants were more open to alternative proposals, leading them away from the status quo. These considerations imply that the relationship between time in existence and the existence bias might be curvilinear: a short time in existence and a very long time in existence are both viewed positively, leading to the existence bias, whereas a regulation existing for an intermediate time might be viewed as being ready for re-appraisal, removing the existence bias. This could be examined by manipulating time in existence over a number of periods from, say, 1 year to 100 years.
It was noted earlier that some effects seem to be declining over time (Lehrer, 2010; Schooler, 2011). With the institutional requirement scenario, the three effect sizes for the existence bias have been 0.77 (large, Eidelman, et al., 2009, Study 2), 0.50 (medium and not statistically significant, Eidelman, et al., 2010, Study 1, when the status quo had been in place for 10 years), 1.42 (very large, Eidelman, et al., 2010, Study 1, when the status quo had been in place for 100 years), 0.21 (small, Experiment 2 here), 0.46 (medium, Experiment 3 here), and 0.87 (large, Experiment 4 here). These results do not show a decline.
Overall, the present study provides additional support for the existence bias with the institutional requirements scenario. The replication is also strengthened by the fact that it increased methodological control over the studies by Eidelman, et al. (2009, 2010): there were similar numbers of women and men, the order of presentation of the judgments was counterbalanced, and it was clear that the students would not be affected by the proposed change. It might be objected that the participants in the present experiments were given an imaginary scenario, whereas Eidelman, et al.'s (2009, 2010) experiments were more realistic because the proposed change was described as actually happening at the students' university. However, it was argued that some of these students may have doubted that the scenario was real. This matter was dealt with in Experiments 3 and 4 by asking participants to consider their reaction to a hypothetical change to their own programme (which was akin to Eidelman, et al.'s (2009) imagination exercise in their Experiment 3). Although this does not mean that people would behave in the same way if the change was actually proposed, it is interesting that the existence bias was successfully replicated with a scenario that was relevant. As noted earlier, Eidelman, et al.'s (2009, 2010) other scenarios were generally of little relevance to the immediate lives of students. The present results show that the existence bias generalizes across this dimension.
Conclusion
Eidelman, et al. (2009, 2010) reported that, in a number of different scenarios, participants rated pre-existing states as higher than other options on the extent to which they were “good,” “right,” and “the way things ought to be.” In other words, “people infer goodness from existence” (Eidelman, et al., 2009, p. 773). However, in one case, in which the institutional requirements for a university major programme had been in place for 10 years, this “existence bias” was not significant. In view of the recent concern that many effects in science may decline in size over time (Lehrer, 2010) or may even be false positives (e.g., Simmons, et al., 2011), which has given rise to a call for more replications (Simmons, et al., 2011), the present study was designed to reexamine the existence bias with the institutional requirement scenario.
In Experiment 1, there was no existence bias: participants did not evaluate the status quo more positively than the proposed change. However, they did show a strong preference for the programme option that required more credits. It was suggested that this might indicate that these participants thought that a programme was better if it offered more depth, leading to greater expertise. On the other hand, because these students were themselves enrolled in a psychology programme that required more credits than the higher of the two options in the experiment, it was also suggested that their choice may have reflected a preference for their own existing requirements, which could be interpreted as a naturally occurring existence bias. This possibility received support from Experiment 3 where similar group of psychology students favored the existing requirements of their own programme over the proposed change.
In Experiment 2, the sample of students was larger and more varied than in Experiments 1 and 3, but was also more similar to Eidelman, et al.'s (2009) original sample. In addition, it included additional controls (counterbalancing, equal number of women and men) compared to the original study by Eidelman, et al. (2009). An existence bias was found but, in contrast to the large effect obtained by Eidelman, et al. (2009), and perhaps because of the tighter control over variables, the effect size was small.
Finally, Experiment 4 provided the clearest replication of the existence bias. With the same methodological controls as in Experiments 2 and 3, students drawn from a number of disciplines strongly favored the status quo in their own discipline over proposals to increase or decrease the required number of credits. Taken together, the present results provide a successful systematic replication of the effect with a scenario that is relevant to the participants.
The existence bias is an interesting new addition to the literature on cognitive illusions. One factor that should be explored further is the time that the present state of affairs has been in place, particularly if it has a curvilinear relationship to the size of the existence bias. Because a preference for the status quo may also vary from person to person, future research might also examine whether it is related to personality traits. For example, the existence bias might be greater in people who are more conservative.
Footnotes
2
I thank an anonymous reviewer for this suggestion.