Significance tests have certain normative implications that are not commonly recognized
and present serious complications for evaluation research. One of the conclusions reached
in this article is that evaluators often make normative decisions, albeit unintentionally,
when they use the traditional .05 significance level in studies with small numbers of cases.
Another conclusion, however, is that evaluators cannot abandon significance tests, but
should use and interpret them differently than in conventional social science research.
Get full access to this article
View all access options for this article.
References
1.
Bakan, D. (1970) "The test of significance in psychological research ," pp. 231-251 in D. E. Morrison and R. E. Henkel (eds.) The Significance Test Controversy. Chicago: Aldine .
2.
Berk, R.A. and P.H. Rossi (1977) "Doing good or worse evaluation research politically re-examined." Social Problems23: 337-349.
3.
Blalock, H.M., Jr. ( 1972) Social Statistics. New York: McGraw-Hill.
4.
Campbell, D.T. and J.C. Stanley (1963) Expenmental and Quasi-Experimental Designs for Research on Teaching. Chicago: Rand McNally.
5.
Cohen, J. (1977) "Statistical Power Analysis for the Behavioral Sciences." New York: Academic.
6.
Cook, T.D. and D.T. Campbell (1979) "Quasi-Experimentation: Design and Analysis Issues for Field Settings." Chicago: Rand McNally.
7.
Cronbach, L.J. (1982) Designing Evaluations of Educational and Social Programs. San Francisco: Jossey-Bass .
8.
———( 1980) Toward Reform of Program Evaluation. San Francisco: Jossey-Bass.
9.
Daniels, M. and R. Darcy (1983) "Notes of the use and interpretation of discriminate analysis." Amer. J. of Pol. Sci.27: 359-380.
10.
Fisher, R.A. (1970) The Design of Experiments. London: Oliver and Boyd.
11.
Gold, D. (1970) "Comment on 'a critique of tests of significance ,' pp. 107-108 in D. E. Morrison and R. E. Henkel (eds.) The Significance Test Controversy. Chicago: Aldine .
12.
Hagood, J. (1970) "The notion of a hypothetical universe,"pp. 65-78 in D. E. Mornson and H. E. Henkel (eds.) The Significance Test Controversy. Chicago : Aldine.
13.
Haveman, R.H. and J. Margolis (1983) Public Expenditure and Policy Analysis. Boston: Houghton Mifflin.
14.
Kelling, G. (1972) The Kansas City Preventive Patrol Experiment. Washington, DC: The Police Foundation.
15.
Medler, J.F., P.R. Schneider, and A.L. Schneider (1981) "Statistical power analysis and experimental field research: some examples from the National Juvenile Restitution Evalution." Evaluation Rev.5, 6: 834-850.
16.
Meehl, P.E. (1970) "Theory testing in psychology and physics: a methodological paradox," pp. 252-266 in D. E. Morrison and R. E. Henkel (eds.) The Significance Test Controversy. Chicago: Aldine .
17.
Meier, K.J. and J.L. Brudney (1981) Applied Statistics for Public Administration. Boston: Duxbury.
18.
Morrison, D.E. and R.E. Henkel (1970) The Significance Test Controversy. Chicago. Aldine.
19.
Nagel, S.S. and M. Neef (1979) Policy Analysis in Social Science Research. Beverly Hills, CA: Sage.
20.
Neyman, J. and E.S. Pearson (1970) "On the problem of the most efficient tests of statistical hypothesis," in D. E. Morrison and R. E. Henkel (eds.) The Significance Test Controversy. Chicago: Aldine.
21.
Pitkin, H.F. (1972) Wittgenstein and Justice. Berkeley, CA: Univ. of California Press.
22.
Riecken, H.W. and R.F. Boruch (1974) Social Experimentation. New York: Academic.
23.
Schneider, A., P. Schneider, L.A. Wilson, and W. Griffith (1978) Handbook of Research for Criminal Justice Evaluators . Washington, DC: Government Printing Office.
24.
Schneider, P.R. and A.L. Schneider (1982) The National Evaluation of Juvenile Restitution Programs: A Proposal. Eugene, ORInstitute of Policy Analysis.
25.
Wald, A. (1970) Selected Papers in Statistics and Probability. New York: McGraw-Hill.
26.
Welch, S. and J.C. Comer (1983) Quantitative Methods for Public Administration . Homewood, IL: Dorsey.
27.
Wilson, M. (1982) "Power analysis: comparisons of ARIMA and ANCOVA time series analysis. Ph.D. dissertation. University of Oregon.
28.
Winch, R.F. and D.T. Campbell (1970) "Proof? No evidence? Yes. The significance of tests of significance," in D. E. Morrison and R. E. Henkel (eds.) The Significance Test Controversy. Chicago: Aldine .
