It is often argued that experiments are strong on causal identification (internal validity) but weak on generalizability (external validity). One widely accepted way to limit threats to external validity is to incorporate as much variation in the background conditions and in the covariates as possible through replication. Another strategy is to make the theoretical foundations of the experiment more explicit. The latter requires that we develop trajectories of experiments that are consistent with a theoretical argument. In other words, new experiments should not simply consist of changing the context of old ones, but do so in ways that explicitly test various aspects of a theory in a coherent way.
Albright, Linda, and Thomas E. Malloy.2000. Experimental validity: Brunswik, Campbell, Cronbach, and enduring issues. Review of General Psychology4 (4): 337-53.
2.
Angrist, Joshua D., Guido W. Imbens, and Donald B. Rubin .1996. Identification of causal effects using instrumental variables. Journal of the American Statistical Association91 (434): 444-55.
3.
Baron, Reuben M., and David A. Kenny .1986. The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology51 (6): 1173-82.
4.
Berger, James O.1985. Statistical decision theory and Bayesian analysis . 2nd ed. New York: Springer .
5.
Bullock, John G., Donald P. Green, and Shang E. Ha.2008. Experimental approaches to mediation: A new guide for assessing causal pathways. Unpublished manuscript, Yale University. Accessed online at http://www.ipeg.org.uk/events/field_experiments/Documents/A%20Critique%20of%20Conventional%20Mediation%20Analyses%20--%20Bullock%20Green%20and%20Ha.pdf
6.
Castro, Rui M., Charles Kalish, Robert Nowak, Ruichen Qian, Timothy J. Rogers, and Xiaojin Zhu .2008. Human active learning. Technical report. Columbia University, New York.
7.
Das, Jishnu, Quy-Toan Do, and Berk Ozler.2005. Reassessing conditional cash transfer programs. World Bank Research Observer20 (1): 57-80.
8.
de Sá e Silva, Michelle Morais.2008. Opportunity NYC: A performance-based conditional cash transfer programme. A qualitative analysis. Working Paper 49, International Poverty Centre, Brasilia , and Columbia University, New York.
9.
Druckman, James N., Donald P. Green, James H. Kuklinski, and Arthur Lupia.2006. The growth and development of experimental research in political science. American Political Science Review100 (4): 627-35.
10.
Ekman, Paul, and Wallace V. Friesen .1971. Constants across cultures in the face and emotion. Journal of Personality and Social Psychology17:124-29.
11.
Frangakis, Constantine E., Donald B. Rubin, and Xiao-Hua Zhou.2002. Clustered encouragement design with individual noncompliance: Bayesian inference and application to advance directive forms. Biostatistics3:147-64.
12.
Gelinas, Nicole.2006. New York isn’t Mexico. City Journal . Accessed from http://www.city-journal.org/html/eon2006-10-20ng.html .
13.
Gerber, Alan S.2004. Does campaign spending work? Field experiments provide evidence and suggest new theory. American Behavioral Scientist47 (5): 541-74.
14.
Gerber, Alan S., and Donald P. Green .2000. The effects of canvassing, telephone calls, and direct mail on voter turnout: A field experiment. American Political Science Review94 (3): 653-63.
15.
Gerber, Alan S., Donald P. Green, and Edward H. Kaplan .2002. The illusion of learning from observational research. Institution for Social and Policy Studies Working Paper, Yale University, New Haven, CT.
16.
Granger, Clive W. J., and Mark J. Machina .2006. Forecasting and decision theory. In Handbook of economic forecasting, vol. 1, 81-98 . Amsterdam: Elsevier.
17.
Green, Donald P., and Alan S. Gerber .2008. Get out the vote: How to increase voter turnout. 2nd ed. Washington, DC: Brookings Institution.
18.
Guala, Francesco, and Luigi Mittone.2005. Experiments in economics: External validity and the robustness of phenomena. Journal of Economic Methodology12 (4): 495-515.
19.
Heckman, James J.2005. The scientific model of causality. Sociological Methodology35 (1): 1-98.
20.
Heckman, James J.2008. Econometric causality. Social Science Research Network Electronic Library.
21.
Horiuchi, Yusaku, Kosuke Imai, and Naoko Taniguchi .2007. Designing and analyzing randomized experiments: Application to a Japanese election survey experiment . American Journal of Political Science51 (3): 669-87.
22.
Kraemer, Helena C., G. Terence Wilson, Christopher G. Fairburn, and Stewart W. Agras .2002. Mediators and moderators of treatment effects in randomized clinical trials. Archives of General Psychiatry59 (10): 877-83.
23.
Levitt, Steven D., and John A. List.2007. What do laboratory experiments measuring social preferences reveal about the real world?Journal of Economic Perspectives21 (2): 153-74.
24.
Lucas, Jeffrey W.2003. Theory-testing, generalization, and the problem of external validity. Sociological Theory21:236-53.
25.
Lynch, John G., Jr.1999. Theory and external validity. Journal of the Academy of Marketing Science27 (3): 367-76.
26.
Mahoney, James, and Gary Goertz.2006. A tale of two cultures: Contrasting quantitative and qualitative research. Political Analysis14 (3): 227-49.
27.
Manski, Charles F.2008. Identification for prediction and decision. Cambridge, MA: Harvard University Press .
28.
McDermott, Rose.2002. Experimental methodology in political science. Political Analysis10 (4): 325-42.
29.
Moffitt, Robert A.2004. The role of randomized field trials in social science research: A perspective from evaluations of reforms of social welfare programs . American Behavioral Scientist47 (5): 506-40.
30.
Mook, Douglas G.1983. In defense of external invalidity. American Psychologist38 (4): 379-87.
31.
Nickerson, David W.2006. Volunteer phone calls can increase turnout: Evidence from eight field experiments. American Politics Research34 (3): 271-92.
32.
Przeworski, Adam.2007. Is the science of comparative politics possible? In The Oxford handbook of comparative politics (Oxford handbooks of political science), ed. Carles Boix and Susan C. Stokes, 147-71. Oxford, UK: Oxford University Press.
33.
Rawlings, Laura B.2005. Evaluating the impact of conditional cash transfer programs . World Bank Research Observer20 (1): 29-55.
34.
Rosenbaum, Paul R.2002. Observational studies. New York : Springer.
35.
Rubin, Donald B.1996. Multiple imputation after 18+ years. Journal of the American Statistical Association91 (434): 473-489.
36.
Schram, Arthur.2005. Artificiality: The tension between internal and external validity in economic experiments. Journal of Economic Methodology12:225-37.
37.
Shadish, William R., Thomas D. Cook, and Donald T. Campbell .2002. Experimental and quasi-experimental designs for generalized causal inference. 2nd ed. New York: Houghton Mifflin.
38.
Sims, Christopher A.1980. Macroeconomics and reality. Econometrica48 (1): 1-48.
39.
Wantchekon, Leonard.2003. Clientelism and voting behavior: Evidence from a field experiment in Benin. World Politics55:399-422.
40.
Wantchekon, Leonard.2008. Expert information, public deliberation and electoral support for good governance: Experimental evidence from Benin. Mimeo. Paper presented at the Freeman Spogli Institute for International Studies on December 2nd 2008, in Palo Alto, California. Accessed online at http://fsi.stanford.edu/events/expert_information_public_deliberation_and_electoral_support_for_good_governance_experimental_evidence_from_benin/
