We address the task of determining, from statistical averages alone, whether a population under study consists of several subpopulations, unknown to the investigator, each responding to a given treatment markedly differently. We show that such determination is feasible in three cases: (1) randomized trials with binary treatments, (2) models where treatment effects can be identified by adjustment for covariates, and (3) models in which treatment effects can be identified by mediating instruments. In each of these cases, we provide an explicit condition which, if confirmed empirically, proves that treatment effect is not uniform but varies appreciably across individuals.
AngristJ. D.ImbensG.RubinD.. 1996. “Identification of Causal Effects Using Instrumental Variables (with Comments).” Journal of the American Statistical Association91:444–72.
2.
AngristJ. D.1998. “Estimating the Labor Market on Voluntary Military Service Using Social Security Date on Military Applicants.” Econometrica66:249–88.
3.
AngristJ. D.KruegerA. B.. 1999. “Handbook of Labor Economics.” Pp. 1277–366 in Causality: Statistical Perspectives and Applications, 1st ed., vol. 3, edited by AshenfelterO.CardD.. Amsterdam, the Netherlands: Elsevier.
4.
AngristJ. D.PischkeJ.-S.. 2010. “The Credibility Revolution in Empirical Economics: How Better Research Design Is Taking the Con out of Econometrics.” Journal of Economic Perspectives24:3–30.
5.
BalkeA.PearlJ.. 1994a. “Counterfactual Probabilities: Computational Methods, Bounds, and Applications.” Pp. 46–54 in Uncertainty in Artificial Intelligence 10, edited by de MantarasR. L.PooleD.. San Mateo, CA: Morgan Kaufmann.
6.
BalkeA.PearlJ.. 1994b. “Probabilistic Evaluation of Counterfactual Queries.” Pp. 230–37 in Proceedings of the Twelfth National Conference on Artificial Intelligence, vol. I, edited by Hayes-RothB.KorfR. E.. Menlo Park, CA: MIT Press.
7.
BlythC.1972. “On Simpson’s Paradox and the Sure-thing Principle.” Journal of the American Statistical Association67:364–66.
8.
BrandJ. E.HalabyC. N.. 2005. “Regression and Matching Estimates of the Effects of Elite College Attendance on Educational and Career Achievement.” Social Science Research35:749–70.
9.
BrandJ. E.ThomasJ. S.. 2013. “Causal Effect Heterogeneity.” Pp. 189–213 in Handbook of Causal Analysis for Social Research, chap. 11, edited by MorganS. L.. Dordrecht, the Netherlands: Springer.
10.
BrandJ. E.XieY.. 2010. “Who Benefits Most from College? Evidence for Negative Selection in Heterogeneous Economic Returns to Higher Education.” American Sociological Review75:273–302.
11.
ChalakK.WhiteH.. 2012. “An Extended Class of Instrumental Variables for the Estimation of Causal Effects.” Canadian Journal of Economics44:1–31.
ElwertF.WinshipC.. 2010. “Effect Heterogeneity and Bias in Main-effects-only Regression Models.” Pp. 327–36 in Heuristics, Probability and Causality: A Tribute to Judea Pearl, edited by DechterR.GeffnerH.HalpernJ.. Milton Keynes, U.K.: College Publications.
14.
GallesD.PearlJ.. 1998. “An Axiomatic Characterization of Causal Counterfactuals.” Foundation of Science3:151–82.
15.
GreenlandS.1991. “On the Logical Justification of Conditional Tests for Two-by-two Contingency Tables.” The American Statistician45:248–51.
16.
HalpernJ.1998. “Axiomatizing Causal Reasoning.” Pp. 202–10 in Uncertainty in Artificial Intelligence, edited by CooperG.MoralS.. San Francisco, CA: Morgan Kaufmann; Journal of Artificial Intelligence Research12:17–37, 2000.
17.
HeckmanJ.1992. “Randomization and Social Policy Evaluation.” Pp. 201–30 in Evaluations: Welfare and Training Programs, edited by ManskiC.GarfinkleI.. Cambridge, MA: Harvard University Press.
18.
HeckmanJ.RobbR.. 1985. “Alternative Methods for Evaluating the Impact of Interventions.” Pp. 156–245 in Longitudinal Analysis of Labor Market Data, edited by HeckmanJ.SingerB.. New York: Cambridge University Press.
19.
HeckmanJ.RobbR.. 1986. “Alternative Methods for Solving the Problem of Selection Bias in Evaluating the Impact of Treatments on Outcomes.” Pp. 63–107 in Drawing Inference from Self Selected Samples, edited by WainerH.. New York: Springer-Verlag.
20.
HeckmanJ.UrzuaS.VytlacilE.. 2006. “Understanding Instrumental Variables in Models with Essential Heterogeneity.” The Review of Economics and Statistics88:389–432.
21.
MorganS. L.WinshipC.. 2007. Counterfactuals and Causal Inference: Methods and Principles for Social Research (Analytical Methods for Social Research). 2nd ed. New York: Cambridge University Press.
22.
MorganS. L.WinshipC.. 2015. Counterfactuals and Causal Inference: Methods and Principles for Social Research (Analytical Methods for Social Research). 2nd ed. New York: Cambridge University Press.
23.
MorganS. L.ToddJ. J.. 2008. “A Diagnostic Routine for the Detection of Consequential Heterogeneity of Causal Effects.” Sociological Methodology38:231–81.
24.
PearlJ.1993. “Comment: Graphical Models, Causality, and Intervention.” Statistical Science8:266–69.
25.
PearlJ.1995. “Causal Diagrams for Empirical Research.” Biometrika82:669–710.
26.
PearlJ.2009. Causality: Models, Reasoning, and Inference. 2nd ed. New York: Cambridge University Press.
27.
PearlJ.2013. “The Curse of Free-will and the Paradox of Inevitable Regret.” Journal of Causal Inference1:255–257.
28.
PearlJ.2014. “Understanding Simpson’s Paradox.” The American Statistician68:8–13.
29.
PearlJ.2015. “Trygve Haavelmo and the Emergence of Causal Calculus.” Econometric Theory31:152–79. Special issue on Haavelmo Centennial.
30.
RosenbaumP.RubinD.. 1983. “The Central Role of Propensity Score in Observational Studies for Causal Effects.” Biometrika70:41–55.
31.
ShpitserI.PearlJ.. 2006. “Identification of Conditional Interventional Distributions.” Pp. 437–44 in Proceedings of the Twenty-second Conference on Uncertainty in Artificial Intelligence, edited by DechterR.RichardsonT.. Corvallis, OR: AUAI Press.
32.
ShpitserI.PearlJ.. 2009. “Effects of Treatment on the Treated: Identification and Generalization.” Pp. 514–21 in Proceedings of the Twenty-fifth Conference Annual Conference on Uncertainty in Artificial Intelligence, edited by BilmesJ.NgA.Corvallis, OR: AUAI Press.
33.
SimonH.RescherN.. 1966. “Cause and Counterfactual.” Philosophy and Science33:323–40.
34.
SimpsonE.1951. “The Interpretation of Interaction in Contingency Tables.” Journal of the Royal Statistical Society, Series B13:238–41.
35.
VanderWeeleT.RobinsJ.. 2007. “Four Types of Effect Modification: A Classification Based on Directed Acyclic Graphs.” Epidemiology18:561–68.
36.
WinshipC.MorganS. L.. 1999. “The Estimation of Causal Effects from Observational Data.” Annual Review of Sociology25:659–706.
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
