The Workings of Choosing and Rejecting: Commentary on Many Labs 2

The Many Labs 2 project included a finding from an article of mine (Shafir, 1993) and obtained a pattern in the direction opposite the one I reported (see Klein et al., 2018, this issue). In the original study, people were presented with descriptions of two hypothetical parents: One, the “enriched” parent, was described by several positive as well as negative features, and the other, the “impoverished” parent, had a description composed of largely neutral features. The task was to decide either which parent should be awarded custody or which parent should be denied custody. In the original study, the enriched parent was more likely than the impoverished parent to be both chosen and rejected. That is, the enriched parent’s share of choice and rejection exceeded the expected 100%, whereas in the Many Labs 2 replication, this parent’s share was less than 100%. The divergent findings are thought provoking—some of those thoughts being potentially more interesting than others.

First, the less interesting. As is often the challenge with replication projects of this kind, of interest is a type of finding, whereas the test is confined to a single token. The test item that was used in Many Labs 2 was created nearly 30 years ago for American undergraduates. The parents’ descriptions were composed of features—frequent work-related travel, health problems, high income, and so forth—chosen to represent negative and positive parental attributes. They were also designed not to indicate which parent was the father or the mother. In early exchanges, the Many Labs 2 Editor and I discussed the fact that “the value / meaning of certain features might not be the same [and might be] geographically sensitive in ways that are hard to gauge without [pilot testing] in advance” (personal communication to D. Simons, September 3, 2014). Because pilot testing was not feasible, it was agreed to proceed without it, but that leaves the issue unresolved. For one thing, some of the features used three decades ago seem to have lost some of their valence. For example, “lots of work-related travel,” which was rated negatively then, now is rated as largely neutral. Furthermore, attributes such as “high income” and “lots of work-related travel” might signal “father” (as opposed to “mother”) in many places, and that would drastically change things: Instead of choosing between two parents, and showing the anticipated malleability, participants in those places would be deciding whether custody should go to the father or the mother, a decision about which they may have clear opinions.

Another “less interesting” account has to do with the presentation of the materials. Whereas the original study counterbalanced the options’ order of presentation, the Many Labs 2 replication did not. The impoverished option—the one surprisingly selected more often—always appeared first. And several studies (Carney & Banaji, 2012; Mantonakis, Rodero, Lesschaeve, & Hastie, 2009; see also Li & Epley, 2009) have documented a first-is-best effect, namely, a proclivity for the first of two items presented in quick succession to be chosen more often than the second.

Now to the more interesting possibilities. Assuming the Many Labs 2 data are not subject to the possible distortions I have just described, what might they teach us? Wedell (1997) replicated the results I obtained for a couple of items from my original study (including the item used in Many Labs 2) but also documented new items that showed patterns in the opposite direction. To account for these differences, Wedell postulated an “accentuation” hypothesis, according to which the greater demands for justification in a choice task compared with a rejection task lead to accentuation of the differences between the alternatives in the former. According to the accentuation hypothesis, because people are more discriminating when choosing than when rejecting, the sum of choice plus rejection for the enriched option can be less 100% (as observed in the Many Labs 2 replication study) if the enriched option is unpopular in the choice task and the rejection rate is relatively regressive.

The experimental findings suggest that there are at least two distinct influences on simple binary choices of the kind considered here. Compatibility, the effect I attributed my findings to, raises the weights of attributes compatible with the task at hand; accentuation, as proposed by Wedell (1997), leads to greater weighting of attribute differences in choice than in rejection. Meloy and Russo (2004) investigated these influences, with supplemental process measures, both in choice and in rejection. The choice process, they concluded, is most fluent when selection is the natural approach, and less fluent when rejection is appropriate. (Applied to the Many Labs 2 replication, this suggests that choice may be fluent when divorce and custody decisions are common, whereas rejection may become more fluent when divorce is frowned upon.) Meloy and Russo concluded that “Wedell’s assertion of a main [accentuation] effect was supported, but only when restricted to non-negatively valenced attributes” (p. 123) and that “the results consistently support a compatibility effect” (p. 114; see also Laran & Wilcox, 2011).

There is, nonetheless, an instructive, if not humbling, aspect to these findings. When I devised the binary problems reported in my 1993 article, I created enriched options that were preferred in choice, and then calibrated negative attributes so that those options would also be rejected. This was a reasonable strategy to show compatibility, but it happened not to yield enriched options that were unpopular in choice, and thus could receive a smaller share of choice plus rejection than the corresponding impoverished options, such as those later created by Wedell (1997). In follow-up work, my student Nathan Cheek and I have run several studies intended to further clarify these apparently conflicting patterns. In a working paper (Cheek & Shafir, 2018), we show that the original pattern is easy to replicate and that the pattern predicted by Wedell and observed in Many Labs 2 is also easy to document, and we further consider the cognitive mechanisms that might account for both.