Researchers conducting field experiments are sometimes faced with the challenge of analyzing field experiment results when the unit of assignment does not coincide with the unit of outcome measurement. For example, in electoral research, election results may be reported at a level of geography defined by electoral law, while the assignment of treatment can be made only at a level of geography different from this. Using examples from field experiments conducted in Canada and Mexico, we describe this problem and its consequences for analysis and interpretation of field experiment data and results. We also offer a number of practical solutions analysts can employ when faced with non-overlapping units of assignment and outcome measure in field experiments.
Bernstein, Robert, Anita Chadha, and Robert Montjoy.2001. Overreporting voting: Why it happens and why it matters. Public Opinion Quarterly65 (1): 22-44.
2.
Brunell, Thomas L., and John DiNardo.2004. A propensity score reweighting approach to estimating the partisan effects of full turnout in American presidential elections. Political Analysis12 (1): 28-45.
3.
De La O, Ana.2008. Do poverty relief programs affect electoral behavior? Evidence from a randomized experiment in Mexico. Unpublished manuscript, Yale University, New Haven, CT.
4.
Diamond, Alexis, and Jasjeet S. Sekhon .2008. Genetic matching for estimating causal effects: A general multivariate matching method for achieving balance in observational studies. Unpublished manuscript, University of California, Berkeley.
5.
Green, Donald P., and Alan S. Gerber .2004. Get out the vote. Washington, DC: Brookings Institution .
6.
Hahn, Jinyong.1998. On the role of the propensity score in efficient semiparametric estimation of average treatment effects. Econometrica66 (2): 315-31.
7.
Hastings, Justine S., Thomas J. Kane, Douglas O. Staiger, and Jeffrey M. Weinstein .2007. The effect of randomized school admissions on voter participation. Journal of Public Economics91:915-37.
8.
Hirano, Keisuke, Guido W. Imbens, and Geert Ridder.2003. Efficient estimation of average treatment effects using the estimated propensity score. Econometrica71:1161-89.
9.
Ho, Daniel E., Kosuke Imai, Gary King, and Elizabeth A. Stuart .2007. Matching as nonparametric preprocessing for reducing model dependence in parametric matching as nonparametric preprocessing for reducing model dependence in parametric causal inference . Political Analysis15:199-236.
10.
Imbens, Guido W.2000. The role of propensity score in estimating dose-response functions. Biometrika87 (3): 706-10.
11.
Imbens, Guido W.2004. Nonparametric estimation of average treatment effects under exogenity: A review. Review of Economics and Statistics86 (1): 4-29.
12.
Karp, Jeffrey A., and David Brockingtona .2005. Social desirability and response validity: A comparative analysis of overreporting voter turnout in five countries . Journal of Politics67:825-40.
13.
Loewen, Peter John, and Daniel Rubenson .2008. Both sides now: A field experiment with competing messages. Unpublished manuscript, Ryerson University, Toronto, Canada.
14.
Morgan, Stephen L., and Christopher Winship .2007. Counterfactuals and causal inference: Methods and principles for social research. Cambridge: Cambridge University Press.
15.
Rosenbaum, Paul R.1987. Model-based direct adjustment. Journal of the American Statistical Association82:387-94.
16.
Rosenbaum, Paul R., and Donald B. Rubin .1983. The central role of the propensity score in observational studies for causal effects. Biometrika70 (1): 41-55.
17.
Rubin, Donald B.1980. Comment on "Randomization analysis of experimental data in the Fisher randomization test" by Basu. Journal of the American Statistical Association75:591-93.
18.
Rubin, Donald B.1986. Which ifs have causal answers?Journal of the American Statistical Association81:961-62.
19.
Sekhon, Jasjeet S.2004. The varying role of voter information across democratic societies. Unpublished manuscript, University of California , Berkeley.
20.
Wooldridge, Jeffrey M.1999. Asymptotic properties of weighted M-estimators for variable probability samples. Econometrica67:1385-1406.
21.
Wooldridge, Jeffrey M.2004. Estimating average partial effects under conditional moment independence assumptions. Unpublished manuscript, Michigan State University, East Lansing.
