Problems with inferring causal relationships from nonexperimental data are briefly reviewed, and four broad classes of methods designed to allow estimation of and inference about causal parameters are described: panel regression, matching or reweighting, instrumental variables, and regression discontinuity. Practical examples are offered, and discussion focuses on checking required assumptions to the extent possible.
AbadieA., DrukkerD., Leber HerrJ., and ImbensG. W.2004. Implementing matching estimators for average treatment effects in Stata. Stata Journal4: 290–311.
2.
AbadieA., and ImbensG. W.2006. On the failure of the bootstrap for matching estimators. NBER Technical Working Paper No. 325. http://www.nber.org/papers/t0325/.
3.
AndersonT., and RubinH.1949. Estimation of the parameters of a single equation in a complete system of stochastic equations. Annals of Mathematical Statistics20: 46–63.
4.
AngristJ. D., ImbensG. W., and RubinD. B.1996. Identification of causal effects using instrumental variables. Journal of the American Statistical Association91: 444–472.
5.
AngristJ. D., and KruegerA. B.1991. Does compulsory school attendance affect schooling and earnings?Quarterly Journal of Economics106: 979–1014.
6.
AutorD. H., KatzL. F., and KerneyM. S.2005. Rising wage inequality: The role of composition and prices. NBER Technical Working Paper No. 11628. http://www.nber.org/papers/w11628/.
7.
AzevedoJ. P.2005. dfl: Stata module to estimate DiNardo, Fortin, and Lemieux coun-terfactual kernel density. Statistical Software Components S449001, Boston College Department of Economics. Downloadable from http://ideas.repec.org/c/boc/bocode/s449001.html.
8.
BakerM., BenjaminD., and StangerS.1999. The highs and lows of the minimum wage effect: A time-series cross-section study of the Canadian law. Journal of Labor Economics17: 318–350.
BaumC. F., SchafferM., and StillmanS.2007. Enhanced routines for IV/GMM estimation and testing. Stata Journal7: 465–506.
11.
BaumC. F., SchafferM., StillmanS., and WigginsV.2006. overid: Stata module to calculate tests of overidentifying restrictions after ivreg, ivreg2, ivprobit, ivtobit, and reg3. Statistical Software Components S396802, Boston College Department of Economics. Downloadable from http://ideas.repec.org/c/boc/bocode/s396802.html.
12.
BeckerS., and CaliendoM.2007. Sensitivity analysis for average treatment effects. Stata Journal7: 71–83.
13.
BeckerS. O., and IchinoA.2002. Estimation of average treatment effects based on propensity scores. Stata Journal2: 358–377.
14.
BlackS.1999. Do better schools matter? Parental valuation of elmentary education. Quarterly Journal of Economics114: 577–599.
15.
BlinderA. S.1973. Wage discimination: Reduced form and structural estimates. Journal of Human Resources8: 436–455.
16.
BoundJ., JaegerD., and BakerR.1995. Problems with instrumental variable estimation when the correlation between the instruments and the endogenous explanatory variables is weak. Journal of the American Statistical Association90: 443–450.
17.
CardD. E.1995a. Using geographic variation in college proximity to estimate the return to schooling. In Aspects of Labour Economics: Essays in Honour of John Vanderkamp, ed. ChristofidesL., GrantE. K., and SwindinskyR.Toronto, Canada: University of Toronto Press.
18.
CardD. E.1995b. Earnings, schooling, and ability revisited. Research in Labor Economics14: 23–48.
19.
CardD. E.1999. The causal effect of education on earnings. Handbook of Labor Economics3: 1761–1800.
20.
CardD. E.2001. Estimating the return to schooling: Progress on some persistent econometric problems. Econometrica69: 1127–1160.
21.
ChengM., JianqingF., and MarronJ. S.1997. On automatic boundary corrections. Annals of Statistics25: 1691–1708.
22.
CochranW., and RubinD. B.1973. Controlling bias in observational studies. Sankhyā35: 417–446.
DiNardoJ., FortinN. M., and LemieuxT.1996. Labor market institutions and the distribution of wages, 1973–1992: A semiparametric approach. Econometrica64: 1001–1044.
25.
DiNardoJ., and LeeD.2002. The impact of unionization on establishment closure: A regression discontinuity analysis of representation elections. NBER Technical Working Paper No. 8993. http://www.nber.org/papers/w8993/.
26.
DiPreteT., and GanglM.2004. Assessing bias in the estimation of causal effects: Rosenbaum bounds on matching estimators and instrumental variables estimation with imperfect instruments. Sociological Methodology34: 271–310.
27.
FanJ., and GibelsI.1996. Local Polynomial Modelling and Its Applications.New York: Chapman & Hall.
28.
FisherR. A.1918. The causes of human variability. Eugenics Review10: 213–220.
29.
FisherR. A.1925. Statistical Methods for Research Workers.Edinburgh: Oliver & Boyd.
30.
GlazermanS., LevyD. M., and MyersD.2003. Nonexperimental versus experimental estimates of earnings impacts. Annals of the American Academy of Political and Social Science589: 63–93.
31.
GoldbergerA. S., and DuncanO. D.1973. Structural Equation Models in the Social Sciences.New York: Seminar Press.
32.
GomulkaJ., and SternN.1990. The employment of married women in the United Kingdom, 1970–1983. Econometrica57: 171–199.
33.
GrilichesZ., and HausmanJ. A.1986. Errors in variables in panel data. Journal of Econometrics31: 93–118.
34.
GutierrezR. G., LinhartJ. M., and PitbladoJ. S.2003. From the help desk: Local polynomial regression and Stata plugins. Stata Journal3: 412–419.
35.
HahnJ., ToddP., and van der KlaauwW.2001. Identification and estimation of treatment effects with a regression-discontinuity design. Econometrica69: 201–209.
36.
HallA. R., RudebuschG. D., and WilcoxD. W.1996. Judging instrument relevance in instrumental variables estimation. International Economic Review37: 283–298.
37.
HardinJ. W., SchmiedicheH., and CarrollR. J.2003. Instrumental variables, bootstrapping, and generalized linear models. Stata Journal3: 351–360.
38.
HeckmanJ., IchimuraH., and ToddP.1997. Matching as an econometric evaluation estimator: Evidence from evaluating a job training program. Review of Economic Studies64: 605–654.
39.
HeckmanJ. J., and VytlacilE.2004. Structural equations, treatment effects, and econometric policy evaluation. Econometrica73: 669–738.
40.
HollandP. W.1986. Statistics and causal inference. Journal of the American Statistical Association8: 945–960.
41.
ImaiK., and van DykD. A.2004. Causal inference with general treatment regimes: Generalizing the propensity score. Journal of the American Statistical Association99: 854–866.
42.
ImbensG.2004. Nonparametric estimation of average treatment effects under exogene-ity: A review. Review of Economics and Statistics86: 4–29.
43.
ImbensG. W., and LemieuxT.2007. Regression discontinuity designs: A guide to Practice. NBER Technical Working Paper No. 13039. http://www.nber.org/papers/w13039/.
44.
JannB.2005a. jmpierce: Stata module to perform Juhn–Murphy-Pierce decomposition. Statistical Software Components S448803, Boston College Department of Economics. Downloadable from http://ideas.repec.org/c/boc/bocode/s448803.html.
45.
JannB.2005b. oaxaca: Stata module to compute decompositions of outcome differentials. Statistical Software Components S450604, Boston College Department of Economics. Downloadable from http://ideas.repec.org/c/boc/bocode/s450604.html.
46.
JuhnC., MurphyK. M., and PierceB.1991. Accounting for the slowdown in black-white wage convergence. In Workers and Their Wages: Changing Patterns in the United States, ed. KostersM., 107–143. Washington, DC: American Enterprise Institute.
47.
JuhnC.1993. Wage inequality and the rise in returns to skill. Journal of Political Economy101: 410–442.
48.
KleibergenF., and SchafferM.2007. Ranktest: Stata module to test the rank of a matrix using the Kleibergen–Paap rk statistic.Boston College Department of Economics, Statistical Software Components S456865. Downloadable from http://ideas.repec.org/c/boc/bocode/s456865.html.
49.
LeeD. S.2001. The electoral advantage to incumbency and voters’ valuation of politicians’ experience: A regression discontinuity analysis of elections to the U.S. House. NBER Technical Working Paper No. 8441. http://www.nber.org/papers/w8441/.
50.
LeeD. S.2005. Training, wages, and sample selection: Estimating sharp bounds on treatment effects. NBER Technical Working Paper No. 11721. http://www.nber.org/papers/w11721/.
51.
LeeD. S., and CardD.2006. Regression discontinuity inference with specification error. NBER Technical Working Paper No. 322. http://www.nber.org/papers/t0322/.
52.
LeibbrandtM., LevinsohnJ., and McCraryJ.2005. Incomes in South Africa since the fall of apartheid. NBER Technical Working Paper No. 11384. http://www.nber.org/papers/w11384/.
53.
LeuvenE., and SianesiB.2003. psmatch2: Stata module to perform full Mahalanobis and propensity score matching, common support graphing, and covariate imbalance testing.Boston College Department of Economics, Statistical Software Components. Downloadable from http://ideas.repec.org/c/boc/bocode/s432001.html.
54.
LudwigJ., and MillerD. L.2005. Does head start improve children's life chances? Evidence from a regression discontinuity design. NBER Technical Working Paper No. 11702. http://www.nber.org/papers/w11702/.
55.
MachadoJ., and MataJ.2005. Counterfactual decompositions of changes in wage distributions using quantile regression. Journal of Applied Econometrics20: 445–465.
56.
ManskiC.1995. Identification Problems in the Social Sciences.Cambridge, MA: Harvard University Press.
57.
McCraryJ.2007. Manipulation of the running variable in the regression discontinuity design: A density test. NBER Technical Working Paper No. 334. http://www.nber.org/papers/t0334/.
58.
MikushevaA., and PoiB. P.2006. Tests and confidence sets with correct size when instruments are potentially weak. Stata Journal6: 335–347.
59.
MorganS. L., and HardingD. J.2006. Matching estimators of causal effects: Prospects and pitfalls in theory and practice. Sociological Methods and Research35: 3–60.
60.
NanniciniT.2006. sensatt: A simulation-based sensitivity analysis for matching estimators.Boston College Department of Economics, Statistical Software Components. Downloadable from http://ideas.repec.org/c/boc/bocode/s456747.html.
61.
NelsonC., and StartzR.1990. Some further results on the exact small sample properties of the instrumental variable estimator. Econometrica58: 967–976.
62.
NeymanJ.1923. Roczniki Nauk Roiniczych (Annals of Agricultural Sciences) Tom X: 1–51 [In Polish]. Translated as “On the application of probability theory to agricultural experiments. Essay on principles. Section 9,” by D. M. Dabrowska and T. P. Speed (Statistical Science 5: 465–472, 1990).
NicholsA., and RaderK.2007. Spending in the districts of marginal incumbent victors in the House of Representatives. Unpublished working paper.
65.
NicholsA., and SchafferM. E.2007. Cluster–robust and GLS corrections. Unpublished working paper.
66.
OaxacaR.1973. Male–female wage differentials in urban labor markets. International Economic Review14: 693–709.
67.
PoiB. P.2006. Jackknife instrumental variables estimation in Stata. Stata Journal6: 364–376.
68.
RosenbaumP. R.2002. Observational Studies. 2nd ed. New York: Springer.
69.
RosenbaumP. R., and RubinD. B.1983. The central role of the propensity score in observational studies for causal effects. Biometrika70: 41–55.
70.
RothsteinJ.2007. Do value-added models add value? Tracking fixed effects and causal inference. Unpublished working paper.
71.
RubinD. B.1974. Estimating causal effects of treatments in randomised and non-randomised studies. Journal of Educational Psychology66: 688–701.
72.
RubinD. B.1986. Statistics and causal inference: Comment: Which ifs have causal answers. Journal of the American Statistical Association81: 961–962.
73.
RubinD. B.1990. Comment: Neyman (1923) and causal inference in experiments and observational studies. Statistical Science5: 472–480.
74.
SchafferM., and StillmanS.2006. xtoverid: Stata module to calculate tests of overidentifying restrictions after xtreg, xtivreg, xtivreg2, and xthtaylor. Statistical Software Components S456779, Boston College Department of Economics. Downloadable from http://ideas.repec.org/c/boc/bocode/s456779.html.
75.
SchafferM.2007. xtivreg2: Stata module to perform extended IV/2SLS, GMM and AC/HAC, LIML, and k-class regression for panel-data models. Statistical Software Components S456501, Boston College Department of Economics. Downloadable from http://ideas.repec.org/c/boc/bocode/s456501.html.
76.
ShadishW. R., CookT. D., and CampbellD. T.2002. Experimental and Quasi-Experimental Designs for Generalized Causal Inference.Boston: Houghton Mifflin.
77.
SimpsonE. H.1951. The interpretation of interaction in contingency tables. Journal of the Royal Statistical Society, Series B13: 238–241.
78.
SpenceM.1973. Job market signaling. Quarterly Journal of Economics87: 355–374.
79.
StockJ. H., and YogoM.2005. Testing for weak instruments in linear IV regression. In Identification and Inference for Econometric Models: Essays in Honor of Thomas Rothenberg, ed. AndrewsD. W. K., and StockJ. H., 80–108. Cambridge: Cambridge University Press.
80.
StoneM.1974. Cross-validation and multinomial prediction. Biometrika61: 509–515.
81.
StoneM.1977. Asymptotics for and against cross-validation. Biometrika64: 29–35.
82.
StuartE. A., and RubinD. B.2007. Best practices in quasiexperimental designs: Matching methods for causal inference. In Best Practices in Quantitative Social Science, ed. OsborneJ.Thousand Oaks, CA: Sage.
83.
van der KlaauwW.2002. Estimating the effect of financial aid offers on college enrollment: A regression discontinuity approach. International Economic Review43: 1249–1287.
84.
WooldridgeJ. M.2002. Econometric Analysis of Cross Section and Panel Data.Cambridge, MA: MIT Press.
85.
YuleG. U.1903. Notes on the theory of association of attributes in statistics. Biometrika2: 275–280.
86.
YunM.-S.2004. Decomposing differences in the first moment. Economics Letters82: 275–280.
87.
YunM.-S.2005a. Normalized equation and decomposition analysis: Computation and inference. IZA Discussion Paper No. 1822. ftp://ftp.iza.org/dps/dp1822.pdf.
88.
YunM.-S.2005b. A simple solution to the identification problem in detailed wage decompositions. Economic Inquiry43: 766–772.