Defining causal effects as comparisons between marginal population means, this article introduces marginal mean weighting through stratification (MMW-S) to adjust for selection bias in multilevel educational data. The article formally shows the inherent connections among the MMW-S method, propensity score stratification, and inverse-probability-of-treatment weighting (IPTW). Both MMW-S and IPTW are suitable for evaluating multiple concurrent treatments, and hence have broader applications than matching, stratification, or covariance adjustment for the propensity score. Furthermore, mathematical consideration and a series of simulations reveal that the MMW-S method has incorporated some important strengths of the propensity score stratification method, which generally enhance the robustness of MMW-S estimates in comparison with IPTW estimates. To illustrate, the author applies the MMW-S method to evaluations of within-class homogeneous grouping in early elementary reading instruction.
Cochran W. G..Analysis of covariance: Its nature and uses.Biometrics. 1957;13:261-281
2.
Cochran W. G..The effectiveness of adjustment by subclassification in removing bias in observational studies.Biometrics. 1968;24:295-313
3.
Correnti R..An empirical investigation of professional development effects on literacy instruction using daily logs.Educational Evaluation and Policy Analysis. 2007;29:262-295
4.
Drake C..Effects of misspecification of the propensity score on estimators of treatments effects.Biometrics. 1993;49:1231-1236
5.
Holland P. W..Statistics and causal inference.Journal of the American Statistical Association. 1986;81:945-960
6.
Holt D.,Smith T. M. F..Post stratification.Journal of the Royal Statistical Society, Series A. 1979;142:33-46
7.
HongG. (2004). Causal inference for multi-level observational data with application to kindergarten retention. Unpublished doctoral dissertation, University of Michigan.
8.
Hong G.,Corter C.,Hong Y.,Pelletier J...
Paper presented at the Annual Meeting of the American Educational Research Association; 2009San Diego, California, U.S; 2009. .
9.
Hong G.,Hong Y..Reading instruction time and homogeneous grouping in kindergarten: An application of marginal mean weighting through stratification.Educational Evaluation and Policy Analysis. 2009;31 (1): 54-81
10.
Hong G.,Raudenbush S. W..Effects of kindergarten retention policy on children's cognitive growth in reading and mathematics.Educational Evaluation and Policy Analysis. 2005;27 (3): 205-224
11.
Hong G.,Raudenbush S. W..Evaluating kindergarten retention policy: A case study of causal inference for multi-level observational data.Journal of the American Statistical Association. 2006;101 (475): 901-910
12.
Hong G.,Raudenbush S. W..Causal inference for time-varying instructional treatments.Journal of Educational and Behavioral Statistics. 2008;33 (3): 333-362
13.
Hong G.,Yu B..Early grade retention and children’s reading and math learning in elementary years.Educational Evaluation and Policy Analysis. 2007;29 (4): 239-261
14.
Hong G.,Yu B..Effects of kindergarten retention on children’s social-emotional development: An application of propensity score method to multivariate multi-level data.Special Section on New Methods in Developmental Psychology. 2008;44 (2): 407-421
15.
Horvitz D. G.,Thompson D. J..A generalization of sampling without replacement from a finite universe.Journal of the American Statistical Association. 1952;47:663-685
16.
Huang I.-C.,Frangakis C.,Dominici F.,Diette G. B.,Wu A. W..Approach for risk adjustment in profiling multiple physician groups on asthma care.Health Services Research. 2005;40:253-278
17.
Imbens G. W..The role of the propensity score in estimating dose-response functions.Biometrika. 2000;87:706-710
18.
Imbens G. W..Nonparametric estimation of average treatment effects under exogeneity: A review.Review of Economics and Statistics. 2004;86:4-29
19.
Joffe M. M.,Rosenbaum P. R..Invited commentary: Propensity scores.American Journal of Epidemiology. 1999;150:327-333
20.
Kish L.Survey sampling. New York: John Wiley; 1965:
21.
Little R. J. A..Models for nonresponse in sample surveys.Journal of the American Statistical Association. 1982;77:237-250
22.
Little R. J. A..Survey nonresponse adjustments.International Statistical Review. 1986;54:139-157
23.
Little R. J. A.,Vartivarian S..On weighting the rates in non-response weights.Statistics in Medicine. 2003;22:1589-1599
24.
Little R. J. A.,Vartivarian S..Does weighting for nonresponse increase the variance of survey means?.Survey Methodology. 2005;31:161-168
25.
Loveless T.The Fordham Report. Washington, DC: The Fordham Foundation; 1998:
26.
Morgan S..Counterfactuals, causal effect heterogeneity, and the Catholic school effect on learning.Sociology of Education. 2001;74:341-374
27.
Oh H. L.,Scheuren F. S.Incomplete data in sample surveys, Vol. 2, theory and bibliographies. Madow W. G.Olkin I.Rubin D. B., ed. New York: Academic Press; 1983:143-184.
28.
Pfefferman D.,Skinner C. J.,Homes D. J.,Goldstein H.,Rasbash J..Weighting for unequal selection models in multilevel models.Journal of the Royal Statistical Society, Series B. 1998;60:23-40
29.
Robins J. M..A new approach to causal inference in mortality studies with a sustained exposure period—Application to control of the healthy worker survivor effect.Mathematical Modeling. 1986;7:1393-1512
30.
Robins J. M.Statistical Models in Epidemiology, the Environment, and Clinical Trials. Halloran M. E.Berry D., ed. New York: Springer; 2000:95-134.
31.
Robins J. M.,Hernan M.,Siebert U..Effects of multiple interventions.Population Health Metrics. 2003;2:2191-2230
32.
Robins J. M.,Rotnitzky A.,Zhao L. P..Analysis of semiparametric regression models for repeated outcomes in the presence of missing data.Journal of the American Statistical Association. 1995;90 (429): 106-121
33.
Rosenbaum P. R..Model-based direct adjustment.Journal of the American Statistical Association. 1987;82:387-394
34.
Rosenbaum P. R.,Rubin D. B..The central role of the propensity score in observational studies for causal effects.Biometrika. 1983;70:41-55
35.
Rosenbaum P. R.,Rubin D. B..Reducing bias in observational studies using subclassification on the propensity score.Journal of the American Statistical Association. 1984;79:516-524
36.
Rubin D. B..Bayesian inference for causal effects: The role of randomization.The Annals of Statistics. 1978;6:34-58
37.
Rubin D. B..Statistics and causal inference: Comment: Which ifs have causal answers.Journal of the American Statistical Association. 1986;81 (396): 961-962
38.
Rubin D. B..[On the application of probability theory to agricultural experiments. Essay on principles. Section 9.] Comment: Neyman (1923) and causal inference in experiments and observational studies.Statistical Science. 1990;5 (4): 472-480
39.
Rubin D. B..Estimating causal effects from large data sets using propensity score.Annuals of Internal Medicine. 1997;127:757-763
40.
Sobel M. E..What do randomized studies of housing mobility demonstrate? Causal inference in the face of interference.Journal of the American Statistical Association. 2006;101:1398-1407
41.
Stuart E..Estimating causal effects using school-level data sets.Educational Researcher. 2007;36:187-198
