A series of recent debates in experimental economics have associated demand effects with the artificiality of the experimental setting and have linked it to the problem of external validity. In this paper, we argue that these associations can be misleading, partly because of the ambiguity with which “artificiality” has been defined, but also because demand effects and external validity are related in complex ways. We argue that artificiality (understood as unfamiliarity of the experimental environment) may be directly as well as inversely correlated with demand effects. We also distinguish between the demand effects of experimentation and the reactions that they may trigger and that might endanger experimental validity. We conclude that economists should pay more attention to the way in which subjects construe the experimental task and learn to exploit subjects’ reactivity to expectations in their experiments
AdairJ. G.1984. “The Hawthorne Effect: A Reconsideration of the Methodological Artifact.” Journal of Applied Psychology69:334-45.
2.
AndreoniJ.MillerJ.2002. “Giving According to GARP: An Experimental Test of the Consistency of Preferences for Altruism.” Econometrica70:737-53.
3.
BardsleyN.2005. “Experimental Economics and the Artificiality of Alteration.” Journal of Economic Methodology12:239-51.
4.
BardsleyN.2008. “Dictator Game Giving: Altruism or Artifact?”Experimental Economics11:122-33.
5.
BardsleyN.CubittR.LoomesG.MoffattP.StarmerC.SugdenR.2010. Experimental Economics: Rethinking the Rules. Princeton: Princeton University Press.
6.
BarrA.BurnsJ.MillerL.ShawI.2015. “Economic Status and Acknowledgement of Earned Entitlement.” Journal of Economic Behavior & Organization118:55-68.
7.
BerkowitzL.DonnersteinE.1982. “External Validity Is More than Skin Deep: Some Answers to Criticisms of Laboratory Experiments.” American Psychologist37:245-57.
8.
BicchieriC.2006. The Grammar of Society. Cambridge: Cambridge University Press.
9.
Branas-GarzaP.2007. “Promoting Helping Behavior with Framing in Dictator Games.” Journal of Economic Psychology28:477-86.
10.
CamererC.2003. Behavioral Game Theory. Princeton: Princeton University Press.
11.
CherryT. L.FrykblomP.ShogrenJ. F.2002. “Hardnose the Dictator.” American Economic Review92:1218-21.
12.
CronkL.2007. “The Influence of Cultural Framing on Play in the Trust Game: A Maasai Example.” Evolution & Human Behavior28:352-58.
13.
DanaJ.WeberR.KuangJ. K.2007. “Exploiting Moral Wriggle Room: Experiments Demonstrating an Illusory Preference for Fairness.” Economic Theory33:67-80.
14.
EckelC.GrossmanP. J.1996. “Altruism in Anonymous Dictator Games.” Games and Economic Behavior16:181-91.
15.
EngelC.2011. “Dictator Games: A Meta Study.” Experimental Economics14:583-610.
16.
EnsmingerJ.2004. “Market Integration and Fairness: Evidence from Ultimatum, Dictator, and Public Goods Experiments in East Africa.” In Foundations of Human Sociality, edited by HenrichJ.BoydR.BowlesS.CamererC.FehrE.GintisH.356-81. Oxford: Oxford University Press.
17.
FehrE.FischbacherU.2002. “Why Social Preferences Matter—The Impact of Non-selfish Motives on Competition, Cooperation and Incentives.” Economic Journal112:C1-C33.
18.
FehrE.SchmidtK.2006. “The Economics of Fairness, Reciprocity and Altruism—Experimental Evidence and New Theories.” In Handbook of the Economics of Giving, Reciprocity and Altruism, edited by KolmS.YthierJ. M., 615-91. Amsterdam, The Netherlands: Elsevier.
19.
FlemingP.ZizzoD.2015. “A Simple Stress Test of Experimenter Demand Effects.” Theory and Decision78:219-31. doi:10.1007/s11238-014-9419-2.
20.
ForsytheR.HorowitzJ. L.SavinN. E.SeftonM.1994. “Fairness in Simple Bargaining Experiments.” Games and Economic Behavior6:347-69.
21.
GerkeyD.2013. “Cooperation in Context: Public Goods Games and Post-Soviet Collectives in Kamchatka, Russia.” Current Anthropology54:144-76.
22.
GualaF.2005. The Methodology of Experimental Economics. Cambridge: Cambridge University Press.
23.
GualaF.MittoneL.2010. “Paradigmatic Experiments: The Dictator Game.” Journal of Socio-Economics39:578-84.
24.
HeyJ. D.1991. Experiments in Economics. Oxford: Blackwell.
25.
JakielaP.2015. “How Fair Shares Compare: Experimental Evidence from Two Cultures.” Journal of Economic Behavior & Organization118:40-54.
26.
Jimenez-BuedoM.MillerL. M.2010. “Why a Trade-Off? The Relationship between the External and Internal Validity of Experiments.” Theoria69:301-21.
27.
JonesM. K.2011. “External Validity and Libraries of Phenomena: A Critique of Francesco Guala’s Methodology of Experimental Economics.” Economics & Philosophy27:247-71.
28.
KagelJ.KimC.MoserD.1996. “Fairness in Ultimatum Games with Asymmetric Information and Asymmetric Payoffs.” Games and Economic Behavior13:100-10.
29.
KahnemanD.KnetschJ.ThalerR.1986. “Fairness and the Assumptions of Economics.” Journal of Business59:S285-S300.
30.
KrupkaE. L.WeberR.2013. “Identifying Social Norms Using Coordination Games: Why Does Dictator Game Sharing Vary?”Journal of the European Economic Association11:495-524.
31.
LesorogolC. K.2007. “Bringing Norms In: The Role of Context in Experimental Dictator Games.” Current Anthropology48:920-26.
32.
LevittS. D.ListJ. A.2005. “What Do Laboratory Experiments Tell Us about the Real World?” NBER Working Paper.
33.
LevittS. D.ListJ. A.2007. “What Do Laboratory Experiments Measuring Social Preferences Reveal about the Real World?”Journal of Economic Perspectives21:153-74.
34.
ListJ. A.2007. “On the Interpretation of Giving in Dictator Games.” Journal of Political Economy115:482-93.
35.
LoomesG.1989. “Experimental Economics.” In Current Issues in Microeconomics, edited by HeyJ. D.152-78. New York: St. Martin’s Press.
36.
LucasJ. W.2003. “Theory-Testing, Generalization, and the Problem of External Validity.” Sociological Theory21:236-53.
37.
MookD. G.1983. “In Defense of External Invalidity.” American Psychologist38:379-87.
38.
MoscatiI.2007. “Early Experiments in Consumer Demand Theory: 1930-1970.” History of Political Economy39:359-401.
39.
OrneM. T.1962. “On the Social Psychology of the Psychological Experiment: With Particular Reference to Demand Characteristics and Their Implications.” American Psychologist17:776-83.
40.
OrneM. T.1969. “Demand Characteristics and the Concept of Quasi-Controls.” In Artifact in Behavioral Research, edited by RosenthalR.RosnowR., 143-79. New York: Academic Press.
41.
PlottC. R.1991. “Will Economics Become an Experimental Science?”Southern Economic Journal57:901-19.
42.
SchramA.2005. “Artificiality: The Tension between Internal and External Validity in Economic Experiments.” Journal of Economic Methodology12:225-37.
43.
SmithV. L.1982. “Microeconomic Systems as an Experimental Science.” American Economic Review72:923-55.
44.
SmithV. L.2008. Rationality in Economics: Constructivist and Ecological Forms. Cambridge: Cambridge University Press.
45.
StarmerC.1999. “Experiments in Economics … (Should We Trust the Dismal Scientists in White Coats?).” Journal of Economic Methodology6:1-30.
46.
StrohmetzD.RosnowR.2004. “Artifacts in Research Process.” In The Sage Encyclopedia of Social Science Research Methods. Vol. 1, edited by Lewis-BeckM.BrymanA. E.LiaoT. F., 26-8. Thousand Oaks, CA: Sage.
47.
ThurstoneL. L.1931. “The Indifference Function.” Journal of Social Psychology2:139-67.
48.
WallisW. A.FriedmanM.1942. “The Empirical Derivation of Indifference Functions.” In Studies in Mathematical Economics and Econometrics, edited by LangeO.McIntyreF.YntemaT. O., 175-89. Chicago: University of Chicago Press.
49.
WiessnerP.2009. “Experimental Games and Games of Life among the Ju/’hoan Bushmen.” Current Anthropology50:133-38.
50.
WildeL. L.1981. “On the Use of Laboratory Experiments in Economics.” In Philosophy in Economics, edited by PittJ. C.137-48. Dordrecht, The Netherlands: Reidel.
51.
ZizzoD.2010. “Experimenter Demand Effects in Economic Experiments.” Experimental Economics13:75-98.
