This article describes a simple way to assess the statistical power of experimental designs. The
approach presented is based on the concept of a minimum detectable effect, which, intuitively,
is the smallest true impact that an experiment has a good chance of detecting. The article
illustrates how to compute minimum detectable effects and how to apply this concept to the
assessment of alternative experimental designs. Applications to impact estimators for both
continuous and binary outcome measures are considered
Get full access to this article
View all access options for this article.
References
1.
Bloom, Howard S., Larry L. Orr, George Cave, Stephen H. Bell, and Fred Doolittle.1993. The national JTPA study: Title IIA impacts on earnings and employment at 18 months. Bethesda, MD: Abt Associates.
2.
Bloom, Howard S., Larry L. Orr, Fred Doolittle, Joseph Hotz, and Burt Barnow.1990. Design of the national JTPA study . Bethesda, MD: Abt Associates .
3.
Cohen, Jacob.1977. Statistical power analysis for the behavioral sciences . New York: Academic Press.
4.
Corson, Walter S., Paul T. Decker, Terry R. Johnson, and Daniel H. Klepinger .1994. Job search assistance demonstration design report. Princeton, NJ: Mathematica Policy Research.
5.
Glass, Gene V, Barry McGraw, and Mary Lee Smith.1981. Meta-analysis in social research. Beverly Hills, CA: Sage.
6.
Greenberg, David, Robert H. Meyer, and Michael Wiseman .1992. Prying the lid from the black box: Plotting evaluation strategy for employment and training programs . Paper presented at the Fourteenth Annual Research Conference of the Association for Public Policy Analysis and Management, 29-31 October, Denver.
7.
Kraemer, Helena Chmura, and Sue Thiemann.1987. How many subjects? Statistical power analysis in research. Newbury Park, CA: Sage.
8.
Lipsey, Mark W.1990. Design sensitivity: Statistical powerfor experimental research. Newbury Park, CA: Sage .
9.
Metcalf, Charles.1974. Alternative approaches to optimal sample assignment in the supported work evaluation. Working paper no. E-6, Mathematica Policy Research, Princeton, NJ.
10.
Rosenthal, R., and D.B. Rubin.1982. A simple, general purpose display of the magnitude of experimental effect. Journal of Educational Psychology74:166-9.
11.
Sechrest, L., and W.H. Yeaton.1982. Magnitudes of experimental effects in social science research. Evaluation Review6:579-600.
12.
Woodbury, Stephen A., and Robert G. Spiegelman .1987. Bonuses to workers and employers to reduce unemployment: Randomized trials in Illinois. American Economic Review77 (4): 513-30.
