Abstract
Gerd Gigerenzer is a man with a mission, and a mission that has some point to it. He wants to show that people are rational decision makers, most of the time. To understand why Gigerenzer is on this mission we have to have a bit of background.
Modern ideas about rationality are very largely based on Von Neumann and Morgenstern's (1947) Theory of Games and Economic Behavior, a masterful analysis of decision making. Von Neumann and Morgenstern assumed that life consists of choices between lotteries, where if you take action A you will receive award R1 with probability p1, award R2 with probability p2, and so forth. This depiction can be applied very widely. You can choose to invest your money in one stock or another. That is clearly a choice between lotteries. While walking on the shady side of the street you can choose to cross to the sunny side or not, and there's a chance you might be hit by a car. You are trading “coolness for sure” against “sunny walk, probability p1 or hit by a car, probability 1-p1.”
Von Neumann and Morgenstern postulated a small number of statements that they believed described rational behavior in such situations. To gain a flavor for these axioms here are informal descriptions of three of them.
Transitivity. If A is preferred to B, and B is preferred to C, then A is preferred to C. Example: If you prefer Chocolate to Vanilla, and Vanilla to Strawberry, you should prefer Chocolate to Strawberry.
Irrelevance of unpreferred alternatives: Suppose that a person is offered a choice between rewards A and B, and prefers B. If the choice is repeated with the addition of a new reward, C, the person should choose either A the original choice, or C, the new choice. Suppose that you are offered a choice between chocolate and vanilla, and choose chocolate. Then the waiter says “Oh, we also have strawberry.” It would be rational to remain with chocolate, or to choose strawberry, but it would not be rational to switch from chocolate to vanilla just because strawberry was present.
To give a final (but not exhaustive) example of the Von Neumann and Morgenstern axioms, Every lottery has its price: Suppose that you prefer A to C and you are offered a lottery where you win A with probability p and C with probability (1-p) (written (A,p),(C,1-p)). Then there is some reward, B, where you prefer A to B and B to C, such that you are indifferent between playing the lottery or simply receiving C. Indeed, that is the way a lottery works; there is a price that you will pay for a ticket. The argument works the other way, too. To go back to an earlier example, suppose you prefer walking in the Sun to walking in the shade, and walking in the shade to being hit by a car. If the traffic is dense enough so that the probability of being hit is high enough, then you will stay in the shade instead of crossing to the sunny side.
Von Neumann and Morgenstern proved that such a rational decision maker can be described as assigning utility to each reward, u(A), u(B), etc. (which need not be monetary value, even if the rewards are money), that the utility function is unique up to a linear transformation, and that the utility of the lottery ((A, p),(B,1-p)) is
In words, the utility of a lottery is equal to its expected value in utility units. When offered a choice between two lotteries a rational person (as defined by von Neumann and Morgenstern) should choose the lottery with the highest expected utility.
There is a hidden assumption here; that the probability measure, p, has to conform to the definition of a probability measure. Notice that I did not say that the measure the decision maker is using has to be an accurate measure in the world. That is, I could be rational in the von Neumann sense if I used the “wrong probabilities”, providing that my subjective probability estimates were all non-zero, summed to one, and otherwise followed the rules of the probability calculus.
This distinction, which Von Neumann and Morgenstern did not stress (nor does Gigerenzer) leads to two further definitions, which are mine but are needed to discuss Gigerenzer's work. I will call a person rational if he or she conforms to the Von Neumann and Morgenstern model using some probability measure. I will call a decision maker informed if the decision maker's probability assignments are identical to those that operate in “the real world,” i.e. the decision maker knows the real odds. Finally, a decision maker is adaptive to a particular environment if the decisions they make result in the highest possible rewards. The relation between these concepts is important. A decision maker would be adaptive in any environment in which he or she was both rational and informed. However, as Gigerenzer argues in detail, there are other ways to be adaptive.
Von Neumann and Morgenstern's analysis has had a huge effect on theoretical economics. In psychology it served as a starting point for a descriptive theory of decision making, a program that was first spearheaded by Ward Edwards, and then by Daniel Kahneman, Amos Tversky, and many related decision theorists. (Kahneman received the Nobel Prize for his work in 2003). The general conclusion behind the psychological research is that people do not act as rational decision makers for two reasons. First, instead of analyzing decisions as choices amongst lotteries people use a variety of short cuts in reasoning (‘heuristics’) that can lead them into inconsistent decision making, and to error in estimating probabilities. Second, people's decision making choices can be altered by describing a situation to emphasize gains or losses. To illustrate, would you be more likely to by from Store A, which advertised “Discount for cash” or Store B, which advertised “Charge for credit?”
This description of psychological research can be generalized beyond the study of decision making. Descriptive studies of deduction and logical reasoning also start by setting a normative standard for rational behavior, in this case the laws of logic, and then studying how human behavior differs from this ideal.
The typical decision theory study presents people with a decision or probability estimation problem, identifies a strategy that people use, and shows that this strategy is incompatible with either rational decision making or inconsistent with the rules of a probability calculus, or both.
The experimental strategy can be illustrated by the much used Linda problem. Experimental participants (usually university students) are presented with a description of “Linda” as a young woman who, in college, was neat, precise, good with mathematical arguments, and also a political activist on the extreme liberal side. Participants are then asked which is more likely, that (a) Linda is a feminist, (b) Linda is an accountant, or (c) Linda is a feminist and an accountant. A substantial number of participants choose answer (c). Such a selection violates one of the axioms of the probability calculus, which states that for any two events, a and b,
implying that
People who choose alternative (c) are said to have committed the conjunction fallacy.
Why might someone commit the conjunction fallacy? The usual argument is that instead of answering the question by recourse to the probability calculus people apply a representativeness strategy. They ask “Is Linda's description typical of a person who might be an accountant and a feminist.” More formally, let E be the evidence (here the description of Linda) and H be the hypothesis (here, that she is an accountant (Ha) or a feminist (Hf). People applying the representativeness heuristic are said to respond “accountant and feminist” because
In doing so they are answering the wrong question, for the question required evaluating statements of the form p(H|E), not p(E|H).
Analogs of the Linda problem can be developed for many other settings, e.g. as a problem in personnel selection.
Representativeness can also lead to a rather different type of error, called base rate neglect. Imagine a physician whose patient presents with a headache and extreme nausea. This is typical of West Nile fever, a rather rare disease, and slightly atypical (because of the headache) of common food poisoning. The physician would be said to have committed the representativeness error of he or she diagnosed West Nile fever. As in the case of the Linda problem the physician has confused “typicality,” defined as p(symptoms| disease) with “diagnosticity,” p(disease|symptoms). Diagnosticity requires a consideration of both the typicality of the symptom, given the disease, and the a priori probability, before the symptom was observed, of possible diseases.
Suppose that prior to observing a piece of evidence it is possible to assign prior probabilities to possible hypotheses. In the physician's case these probabilities would be the relative frequencies of food poisoning and West Nile fever in the relevant population. Let Hi be the ith possible hypothesis, and let p(Hi) be its probability. (This is the base rate of Hi) Now, let E be the observed evidence, and let p(E|Hi) be the probability that a particular piece of evidence would be observed, given that hypothesis Hi is, in fact, correct. What we seek is p(Hi|E), the a posteriori probability of each hypothesis after the evidence has been observed.
The necessary reasoning is encapsulated in Bayes' theorem, an 18th century theorem on evidence evaluation that is a consequence of the definition of a probability calculus. Bayes' theorem is
Neglect of base rates is often illustrated by the cab problem;
In a certain city 80% of the cabs are painted blue and 20% painted green. An accident occurs, and a witness to the accident states that the cab involved was green. Experiments show that under the conditions of visibility at the accident site the witness is able to identify the color of a cab correctly 80% of the time. What is the probability that the cab was green?
Most people who attempt this problem give answers in the .8 to .6 range. However the probability according to Bayes theorem is only .5 As I have indicated, it is easy to construct a medical diagnostic problem analogous to the cab problem. Similar confusions occur, even when the participants are physicians.
The next fallacy to be discussed is the overconfidence fallacy. Suppose that an experimental participant is asked a series of questions, such as “Is a baseball heavier than a tennis ball?” or “Who made George Washington's false teeth?” In addition to answering the question the participant is asked to indicate the probability that his/her answer was correct. (I suspect that most American readers would assign a high probability of being correct to their answer for the first question, and a low probability for the second question.) Call this the Confidence Level. At the end of answering the questions we would have a set of questions with confidence level .8, .7, .6, etc. If a person has an accurate calibration of his/her knowledge we would expect 80% of the questions assigned a confidence level of .8 to be answered correctly, 70% of the questions assigned a confidence level of .7 to be correct, and so forth.
In fact, this is not what happens. The frequency of correct answers for a set of questions at confidence level x (0 ≤ x ≤ 1) is almost always below the confidence level. This is said to indicate overconfidence. Clearly if overconfidence is characteristic of human performance we are likely to be a bit more rash than we should be.
The fallacies just cited are believed to operate in the world outside of the laboratory. Gigerenzer cites a particularly egregious problem due to base rate neglect; counseling heterosexual, non-drug using individuals who, on a routine screening, test positively for human immune deficiency virus (HIV). Suppose that the counselor who reports this test to the affected individual confuses the probability that the test is positive, given that a person has HIV (p(positive test|HIV)) with what the patient wants to know, p(HIV|positive test). Gigerenzer cites figures, based on German data for low-risk individuals, in which the first number is .99 but the second is only .50. Field surveys show that many health professionals do make this confusion, and as a result transmit a terrifying message to their clients.
Findings like this are the reason that Kahneman, Tversky, and many others have concluded that people are often irrational and uninformed, in the sense defined above, and that as a result they are not adaptive in the normal world. Gigerenzer believes that this conclusion is wrong, wrong, wrong. In fact, he offers three arguments against modern decision theory, one for each of my wrongs. He then generalizes his argument by presenting two overarching reasons why decision theorists have gotten into what Gigerenzer perceives as a mess. I present and comment on each of the wrongs, and then deal with the overarching reasons.
Wrong 1. Virtually all the conclusions of modern decision theory are based upon laboratory experiments rather than observations of behavior in natural environments. This can be extremely misleading. Gigerenzer offers an extensive paean to Egon Brunswik, who made the same point in the 1950s. The Brunswik-Gigerenzer argument is that the heuristics people use (e.g. representativeness) are very close to optimal in the natural environment, because they exploit regularities in that environment. The strategies only look bad in the laboratory because the controlled laboratory setting destroys these regularities. Broadening the argument somewhat, Gigerenzer makes the valid point that the rationality of a cognitive strategy should be determined by its usefulness in a natural environment (adaptability) rather than by rationality.
Gigerenzer illustrates his point by considering some experiments on knowledge of cities. Suppose that an American student is asked “Does New York have a larger population than Los Angeles?” The student will probably be able to answer this directly, because US education contains numerous references to New York as the largest US city. But what about a comparison between St. Louis, Missouri. and Toledo, Ohio? The student is unlikely to know the size of either city on an explicit basis. On the other hand, as Gigerenzer observes, the student will know things about these cities that are correlated with size. For one, St. Louis will simply be more familiar, for (outside of Ohio!) the student is likely to have encountered the name “St. Louis” more often than “Toledo.” St. Louis is associated with more cultural icons (the St. Louis Blues, a jazz theme) and sporting events (the St. Louis Cardinals, a major league baseball team.). I doubt that anyone ever played the Toledo Blues, and the Toledo baseball team (the Mudhens) is definitely in the minor leagues. Accordingly, it is reasonable to decide, on the basis of sheer familiarity and correlated cues, that St. Louis is larger than Toledo. More generally, if you recall facts about two cities that are correlated with size, and stop when the preponderance of evidence indicates that one city is larger than another, you will be right more often than not.
The strategy Gigerenzer proposes falls into a class of decision procedures referred to as decision making by lexicographic ordering. It is one of a variety of strategies designed to get a reasonable number of decisions right without committing oneself to extensive computation to make any one of them. The technique is called “satisficing,” as opposed to “optimizing.” Shortly after Von Neumann and Morgenstern's publication Herbert Simon (1957), in work for which he received the Nobel Prize (!) argued that satisficing is more descriptive of natural decision making than is optimizing. And like Gigerenzer, Simon stressed that satisficing could lead to adaptive behavior in the world outside of the laboratory.
For those who dislike the mathematical argument, I offer the following example. The time is the 12th century, and the place is England. Two groups of travelers are going through Sherwood Forest, on separate paths. Each group consists of a few rich people and a larger number of ordinary individuals. The resident bandit of Sherwood Forest, Robin Hood, has only enough Merry Men to rob one of the groups. Which one should Robin pick? In order to find an optimal solution Robin Hood should estimate the wealth of all individuals in each group, and then accost the wealthiest group. But that's a lot of work. It is easier to estimate the wealth of the one or two richest individuals in each group, and attack the group that appears to have the fattest cats. This solution may be right often enough so that it is satisfactory. In Simon's terms, Robin Hood satisfices rather than optimizes.
To put the argument mathematically, consider two alternative rewards, X and Y, each of which consists of a bundle of N comparable subrewards, {xi} and {yi}. Let these rewards be considered in order, i.e. the decision maker first evaluates x1 and y1, the x2 and y2, and so forth. A value maximizer will compute the total rewards for each alternative, choosing X over Y if and only Σi=,N xi ≥ Σi=,N yi. A satisficer will either only consider the first M, where M < N, subrewards, deciding for X if Σ i=,M xi ≥ Σi=M yi, or will keep a running average of the differences, D = Σ i=,N xi − Σi=,N yi and decide for X (or Y) if the absolute value of D reaches a threshold of satisfaction.
It is fairly easy to show that satisficing can lead to “irrational” decision making as conventionally defined. Robin Hood's decision could be wrong, because the group with the wealthiest one or two individuals might not have the greatest total wealth. More generally, the value of the satisficing strategy depends upon the structure of the environment. Satisficing works to the extent that the relative values of the attributes considered predict the relative values of the attributes not considered. In Robin Hood's case, he and his band will attack the right group of travelers if the group with the greatest average wealth also contains the wealthiest individual.
Gigerenzer argues that, in fact, the environment is structured in such a way that conditions like this do exist. He offers an interesting experimental example. German students were asked to judge the relative size of US cities (e.g. “Is St. Louis larger than Toledo?“) The German students, in general, knew only a very little about each city. Nevertheless, their judgments were nearly as accurate as those of US students, who (presumably) knew more about each city.
Gigerenzer claimed that this was because the environment is so tightly structured that good guesses could be made on the basis of impoverished information. In his terms, “fast and frugal” algorithms work because things are so highly correlated. One of the simplest is just to ask “Given any two cities, have I heard of one and not the other?” If so, the city that had been heard of was likely to be the bigger city, providing that the cities are either chosen randomly or exhaustively from the set of US (or German) cities that exceed a certain size. On the other hand, it is possible to design an experiment in which selected cities are compared, that will fool this ‘fast and frugal’ algorithm. Las Vegas, Nevada, is well known as a gambling resort and Boston as a historical site. Indianapolis is not as well known, but (as of 2000) it was about 40% larger than either Las Vegas or Boston.
Gigerenzer also objects to studies showing that people make errors in logical reasoning. Many laboratory studies have shown that people tend to believe that if a implies b, then b implies a. A famous and much studied example is the Wason card selection task. A participant is first told that a set of cards was to be printed according to the rule “If there is a vowel on one side there must be an odd number on the other.” In logical terms, (vowel) implies (odd number). The participant is then shown four cards with one face displayed, as in:
and is asked which cards have to be turned over to see if the rule is being followed. The correct answer is A and 4. However a substantial number of people also turn over the 7, erroneously thinking that if there is an odd number on one side, then the other side must have a vowel.
Gigerenzer objects to this sort of experiment on the grounds that it has no ecological validity. He points out (as have others before him, notably Cheng and Holyoak (1985)) that the card sorting task is easy if it is presented as a task in detecting whether or not people are doing things they are not permitted to do. Suppose that you are a liquor-law inspector, enforcing the law “No drinking is permitted unless you are twenty one.” This is logically equivalent to “Drinking implies person is over twenty one.” Which of these people must be further investigated to see if the law is being followed?
A young woman, apparently a college student, drinking beer.
An obvious teenager with an unidentified drink.
A young man, also a college student, with a coca cola.
An elderly professor with an unidentified drink.
This problem is quite easy. Nevertheless, it is formally identical to the card sorting task, for “drinking alcohol” maps to “vowel” and “over 21” to “odd number.”
More generally, Gigerenzer argues that people are quite able to reason logically about social situations, such as those just described. Therefore poor performance in abstract tasks, such as Wason's card selection task, may considerably underestimate people's capability.
Gigerenzer's more general argument is that experimenters have drawn erroneous conclusions about the irrationality of various ‘fast and frugal’ algorithms because they conduct studies that are like the Las Vegas, Boston, Indianapolis comparison; intentionally designed to illustrate fallacious reasoning. In general, I agree. Experimental designs that balance variables do destroy the natural structure that would be found by representative sampling (Brunswik's point) and Gigerenzer has a point when he says that decision theorists have often chosen situations designed to expose errors in human reasoning, regardless of how representative these situations are in practice.
Wrong 2. The experiments used to demonstrate faulty reasoning about probability and statistics are based upon an erroneous interpretation of probability. When the experiments are redesigned to correct this error erroneous reasoning virtually disappears.
Consider the ‘Linda’ and the ‘Cab’ problems. Gigerenzer criticizes them on two grounds. First, all information is presented to the participant. In the normal case people acquire frequency information directly from their experience. Furthermore, there is good reason to believe that frequencies derived from experience are automatically encoded. Ecological validity is lost when frequencies are presented as facts, rather than being experienced.
Second, in both the Linda and the Cab problems (and their analogs) the participant is asked to reason about the probability of a single event, Linda's being an accountant or the cab's having a certain color, without being told, explicitly, that the event was randomly sampled from some defined universe.
Both of these objections refer to experimental details. Gigerenzer and his colleagues have conducted experiments that show, quite convincingly, that in situations in which people acquire statistical information directly fallacious reasoning disappears. His results certainly limit the generalizations that have been drawn from the research he attacks.
Whether or not this demonstrates ‘lack of ecological validity,’ though, is a trickier question. People are frequently asked to reason about statistical issues in situations where they have not, themselves, observed the relevant frequencies. Consultations with physicians about alternative medical treatments are a good case in point. We are also often asked to use probabilistic information to deal with patently non-random events. This leads us to consider Gigerenzer's third objection to the research on probabilistic reasoning.
Wrong 3. Gigerenzer claims that virtually all American students of decision making, including Kahneman and Tversky (both Israelis, but who did most of their work in the US), simply do not understand probability.
Gigerenzer is a strong proponent of the frequentist view of probability, in which the probability of an event is equal to the relative frequency with which an event will appear in an infinite sequence of random selections from the relevant universe of possible events. Thus the probability of rolling 7 in dice is the relative frequency with which a 7 would be rolled in a (hypothetical) infinite set of throws of a pair of dice. This view is either implicit or explicit in many texts on applied statistics, and is natural in the context in which it is usually presented. It becomes problematical when we consider single events that are not drawn from any clearly defined universe.
What is the probability that there will be a terrorist attack on a United States city in the next five years? Or that the Seattle Mariners baseball team will win the baseball World Series in 2010? These questions sound meaningful, but neither event can reasonably be thought of as a random sample from a defined universe of possible events.
Many statistical texts, and certainly many experiments on probability estimation, simply ignore this point. But it is important. What is the probability that Linda is a feminist accountant? What is the probability that I am correct in believing that Vienna is a larger city than Prague? In both cases the statements are either true or false. Probability would apply only I knew that Linda had been chosen, randomly, from some population with subpopulations of feminists and accountants, or if the question about Vienna and Prague had been drawn, randomly, from some larger set of comparisons that I had made.
Gigerenzer concludes that when people are asked to judge “probabilities” in situations like these they rightly refuse to adopt a frequentist interpretation. Instead they interpret “probability” to mean the more vaguely defined “Is this statement to be believed?” Therefore their answers need not conform to the probability calculus and they cannot be said to be behaving irrationally.
He buttresses his case with a clever series of experiments in which the base rate, overconfidence, and conjunction fallacies disappear (i.e. people's judgments conform to the laws of probability) when a frequentist interpretation is clearly appropriate. For instance, overconfidence appears if a person is asked to give a judgment of the probability of a correct answer each time a question is asked. But suppose that instead of being asked “What is the probability that the answer to this question is correct” the participant is asked “Of the last 10 questions that you answered, how many do you think you answered correctly?” Gigerenzer's results show that, at least in the city size judgment situation, frequentist responses are well calibrated.
This finding has important practical implications. Gigerenzer reports that German AIDS counselors often present risk information in the confusing probabilistic form, in which p(Positive test|Disease) is confused with p(Disease|Positive test). Indeed, because the relevant information was presented to the counselors in terms of probabilities, they may not understand the distinction themselves. Gigerenzer then points out that there is a simple way in which people can behave in accordance with Bayes' theorem. As this is important I will first present the strategy in a medical setting, and then generalize.
Let D be the statement “A patient has the disease of interest” and let S be the statement “The patient displays the symptom of interest.” (Symptom here includes a positive result of a diagnostic test.) Suppose that a physician keeps track of the number of his or her patients who display the symptom and either do or do not have the disease. Let these figures be N(D|S) and N(~D|S). Suppose a new patient comes in, who in fact displays the symptom. What is the probability that the patient has the disease? Taking a frequentist view of probability, this is
which incorporates base rates and is in agreement with Bayes' theorem.
Gigerenzer builds on this finding in two ways. First he points out that if people (and other animals) act this way they will be informed, in the sense defined above, and thus prepared to act adaptively. This is so, but Gigerenzer offers no substantive attempt to work his observation into actual findings and theories on the evolution of behavior. Second, Gigerenzer cites the psychological literature that shows that people are capable of recording the frequency of outcomes of different sorts, and they do so in an apparently automatic fashion. This argument links Gigerenzer's views to a well established experimental literature. In fact, Gigerenzer does not go far enough.
Both Gigerenzer's work and most other studies assume that people should treat information about relative frequencies as if they were observing a succession of random samples from a distribution that is stationary over time. A good deal of research has shown that in this situation people behave suboptimally, because they are more sensitive than they should be to short term phenomenon, such as a run of several heads in a row, in a coin tossing game, or several baskets in a row (“the hot hand”) in a game of basketball. Excessive sensitivity to recent runs of events has been taken as evidence of irrationality, and would be non adaptive if the assumption of stationarity can be defended. On the other hand, if the generating parameters are changing over time it is both adaptive and rational to give more weight to the most recent observations. This would certainly be the case in many medical situations. Think of patients who present with fever and muscular aches. The probability of influenza goes up in the winter and down in the summer, the probability of the (much rarer) West Nile fever goes up in summer and down in winter.
Although Gigerenzer's main argument is with theories of decision making, at various places in his book he broadens the attack to include arguments about the very way that theories are generated in modern social sciences. He points out, as have many others, that scientific theories (especially in the social sciences) reflect of the social situation in which scientists operate and, interestingly, the technologies that they use. Two of the technologies that concern him are so closely allied that I will consider them as one. They are the use of computer models and the pervasive acceptance of formal models of rationality, such as the Von Neumann model and Bayesian reasoning, as the appropriate description of ideal behavior. The reason that the two are mixed up is that, according to Gigerenzer, when we program a computer model of cognition we start with the “rational solution” and then distort the model from that solution in order to make it more descriptive. Gigerenzer (once again echoing many others) also objects to what he sees as the unvarying formalistic approach to thought that computer modeling is said to engender.
I disagree strongly. However others have made this argument, so the counter-argument is worth restating. The argument against computer program is based on a misunderstanding of what computer modeling is. Computer modeling refers to the investigation of any theory by constructing a special Turing machine (i.e. a program). The fact that the computer on which the program runs is itself a general Turing machine is irrelevant. The act of programming ensures that the original theory does not contain inconsistencies or vagueness, such as an appeal to “intuition” without defining intuition. ‘Illogical’ rules of reasoning can be built into a computer program, but undefined rules cannot. But do undefined rules have any place in any scientific model of thought?
The claim that computer models are unresponsive to experience or that they require exact matches between descriptions internal to a model and events in the external world is simply false. The whole field of machine learning deals with programs that change their responses as a function of experience. Exact matches between an external stimulus and a memory stored in a machine are features of some programs, but matching by analogy or similarity measures are permitted in others.
Gigerenzer claims that the fact that the brain has modules of “special purpose computers,” e.g. for spatial vs. verbal reasoning, is evidence against the general purpose computer as a model of the mind. The logic is again faulty. First, all the “special modules” of the brain are realized by neural networks of the same basic computing unit, so it is not clear that the brain isn't a general purpose computer, in the sense that neurons could be organized in a lot of different ways. Second, modules are routinely contained in computer simulations of reasoning. See, for instance, Meyer and Kieras' (1997) EPIC model of human performance.
Gigerenzer's attack on models of rational behavior similarly fails, because of a misunderstanding of what these models are. The attack is based on thought problems that are claimed to illustrate the limitations of the Von Neumann and Morgenstern approach. I will consider two of them, to illustrate the problems in his reasoning.
The Von Neumann and Morgenstern approach, and of its more modern variants, such as prospect theory (Kahneman and Tversky, 1979) assume that a decision maker should maximize his/her utility, which is determined by the decision maker's perception of the value of various outcomes. (The basic difference between the two is whether the decision maker considers the utility of the absolute value of a reward or considers the utility of the gains and losses, with present wealth taken as a zero point.) Gigerenzer offers the example of a species of bird that may have either dark or light plumage. Dark plumage increases the probability that the bird will fall to predators except when snow is on the ground. At that point light plumage is favored. Given appropriate parameters for survival as a function of plumage and season, it is possible to construct a situation in which an individual bird has its best chance of survival with dark plumage, but the species has its best chance for survival if some birds are dark and some white.
Gigerenzer argues that this is an example in which the individual alternative of having white plumage is never rational, in the sense that this minimizes probability of survival, but nevertheless some birds have white plumage.
I cannot see how this is relevant to decision making. The most obvious point is that an individual bird does not decide on its plumage. So let me strengthen Gigerenzer's example with an exactly analogous one taking from the early days of operations research, that does involve human reasoning. The example involves strategic decisions taken by the British and American (“allied”) air raids on Nazi targets in World War II.
Allied bombing squadrons were safest from fighter attack, but most vulnerable to anti-aircraft artillery, if they flew in tight formations, with one bomber relatively close to another. If the bombers flew in loose formations they were relatively safe from the anti-aircraft artillery of the time but were vulnerable to fighter attack. Finally, it was always more dangerous for an individual aircraft to fly in a loose formation. The questions are (a) how should the Allied forces fly as a force and (b) how should individual aircraft commanders behave? The solution was that the Allied commanders should follow a mixed strategy; generally flying tight formations but sometimes flying in loose ones. The example is an exact analog of Gigerenzer's bird-plumage example, but has the advantage of describing human reasoning. This forced the Nazi commanders to split their resources, and minimized losses to Allied aircraft overall. But what about the individual aircraft commander?
Von Neumann and Morgenstern's analysis of games against a rational opponent covers this situation exactly. Individual aircraft commanders knew that they were at high risk if they flew in loose formations. Why did they do so? The answer is that the individual aircrew had no choice in the matter, they were ordered to fly in a particular way. They accepted this decision because the alternative, failing to follow a lawful order during combat, would carry very serious consequences, both legally and in terms of self esteem. One can argue whether or not the warrior ethic is correct, in some moral sense, but following it certainly does not violate decision theory.
In the bombing example the rational decision making was carried out at the level of the high commands of the combatants, not at the level of individual pilots or anti-aircraft commanders. If we force an analogy somewhat, the bird species and the predator species can be thought of as playing a ‘game’ against a rational opponent. If such species exist it would be interesting to know whether their strategies result in an optimal mix of dark and light birds. If they do, then this would be an interesting illustration of how genetic selection can produce the same strategy that would be produced by a conscious, rational decision maker. However all this shows is how two different solution processes might arrange at the same solution in a particular case.
Gigerenzer's second thought experiment attacks the axiom of the irrelevance of non-preferred axioms. Recall that this is the axiom that can be exemplified by saying that if you prefer chocolate to vanilla, alone, then you cannot choose vanilla from the set {chocolate, vanilla, strawberry}. Gigerenzer argues that there are social situations where this is violated. Suppose that at a formal dinner you are passed a tray with a single dessert, a piece of cake, on it. One more person remains, so if you take the cake your companion will have to go without. Given the choice {cake, nothing} in these circumstances you take nothing. But suppose that there is an ice cream on the tray. You can now take the cake and still leave something for your companion. So you choose cake from the set {cake, ice cream, nothing}. According to Gigerenzer this violates the Von Neumann and Morgenstern axiom.
It does not. The first choice was {(cake + appearance of selfishness),(nothing + appearance of generosity)}. The second choice was {cake, ice cream, nothing}). These are simply different reward sets, so the choices cannot be compared.
All in all, I did not find Gigerenzer's attack on either models of rationality or computer modeling to be convincing.
Gigerenzer's next technological target startled me, then I realized that he had a point. The target was the analysis of variance, and the villain was Sir Ronald Fisher. Gigerenzer observes, correctly, that Fisher's ideas on the logic of experimental design have pervaded Psychology. Gigerenzer's argument is that Fisher's emphasis on experimenter-controlled, balanced designs, rather than Brunswik's emphasis on environmentally representative designs, has had a deleterious effect.
To the extent that one accepts Fisher-style experimental designs as the only, or perhaps a superior, way to establish a scientific conclusion is lead to study those variables that can, in fact, be placed in the design. Scientists emphasize the study of controllable variation in independent variables, possibly at the expense of studying uncontrollable variables that exert a great deal of influence outside of the laboratory. To use an example of my own, rather than Gigerenzer's, the National Science Foundation has invested a great deal of money in studies of how in-class variations in teaching styles and the use of technology can influence the performance of students in the public K-12 system. In general, the results have been meager. Experimental studies work, in the sense that the null hypothesis of ‘no effect’ can be rejected, but the findings do not seem to scale up to improvement in the school system. Why not? There is evidence that variables outside the classroom, such as parental and student attitudes toward schooling, are far more important variables than those being studied (Steinberg, 1996). However it is hard to study these variables if you are fixated on a Fisher-style experimental design.
Gigerenzer has a point. However he overstates it, because he does not consider the fact that there are middle grounds. People do conduct field studies. Indeed, the whole field of sociology is closely bound up with such designs. The methods used by modern mathematical sociologists go well beyond the simple regression models that Brunswik advocated over fifty years ago. Even with these statistical tools in hand, though, the representative design has limitations. They are just different limitations than those found in the classic experimental design. Hammond's (1996) discussion of this issue is far more sophisticated than Gigerenzer's. But on one point (unaddressed by Hammond) Gigerenzer is absolutely correct.
The undergraduate curriculum in psychology (not sociology) virtually ignores the logic of Brunswik's representative design. The concept of correlation has almost totally dropped out of undergraduate statistics. About the only place it occupies in courses on experimental methods is as part of a mantra, “Correlation is not causation.” The reality is far more subtle, especially when a person has to deal with the decidedly non-random, unbalanced environment that exists outside of the laboratory. I grant that the concepts involved are subtle, but that is not an excuse for not teaching them.
And this brings me to Gigerenzer's final target; the modern (largely American) social organization of science in general, and psychology in particular. Gigerenzer argues that the practice of peer review and the importance of publication counts in promotion have the effect of setting the Fisher-style approach to science in concrete. If you want to have lots of publications you do not want to get into an argument with reviewers about methodology, so the wise young Ph.D. is advised to go along with the accepted style of experimentation. An unambiguous but trivial result is to be preferred (and published, and counted) in preference to a somewhat ambiguous finding about an important topic. (Gigerenzer does not actually say this, but I infer that he thinks this from his other remarks.)
Once again, such remarks are not entirely new. Many people have deplored the conservatism of peer-review and the perceived overemphasis on publication counts as a way to evaluate faculty. Government granting agencies will get into far more trouble with their political superiors if they fund a bad idea, and especially an unconventional bad idea, than if they fail to fund a good, albeit unconventional idea. Gigerenzer's connection of such conservatism to the limitations of the analysis of variance is certainly a creative connection!
But what do you do about this? Several passages in Gigerenzer's book speak somewhat nostalgically of the interplay of ideas said to occur in the well-read, highly cultured intellectuals in “circles” and “schools” that flourished in continental Europe from the 1850s until shortly before World War II. Gigerenzer seems to think that considered debate between these savants would permit a slower but better guided development of scientific psychology.
There is no way to disprove this, but I have my doubts. While it may well be that the “schools” approach lead to the development of a few very bright individuals, I doubt that their contribution would match the total contribution of the very large number of competent scientists, very widely distributed, that we have today. The distinction between ‘elite’ and ‘provincial’ universities is rapidly breaking down. Admittedly this leads to an information glut, biases toward overproduction of small papers, and perhaps conservatism in the spread of ideas. Gigerenzer has done us a service in highlighting some of the resulting problems, so that we may deal with them. But he fails to consider the benefits; more people, less beholden to single individuals, working on more different approaches to problems. There is a great deal to be said for decentralized problem solving accompanied by centralized review. As for turning the business of science over to a small number of philosopher-professors? Ain't gonna happen, and it probably should not.
There are a lot of interesting ideas here, so Gigerenzer is worth reading. Just take his arguments with a grain of salt.