Formal, systematic social scrence research on the effectiveness of public policy has
become increasingly resorted to in the last two decades as policy-makers have become
increasingly skeptical about the effectiveness of public policy, especially in the social
welfare areas A survey of existing research designs and accompanying techniques is
presented along with brief assessments of their usefulness for programs of different types.
The authors stress particularly the use of field experiments for the assessment ofprospective social policies and programs
Get full access to this article
View all access options for this article.
References
1.
Anderson, A.B. (1976) "Policy experiments: selected analytic issues," pp. 17-34 in I. N. Bernstein (ed.) Validity Issues in Evaluative Research. Beverly Hills, Calif.: Sage.
2.
Armor, D.J., J.M. Polich, and H.B. Stanbul (1976) Alcoholism and Treatment. Santa Monica, Calif.: Rand Corporation, R-1739.
3.
Ball, S. and G.A. Bogatz (1973) Reading with Television. An Evaluation of The Electric Company. Princeton, N.J.: Educational Testing Service.
4.
——— (1970) The First Year of Sesame Street: An Evaluation . Princeton, N.J.: Educational Testing Service.
5.
Barnow, B.S. (1973) "Evaluating Project Head Start." Madison University of Wisconsin, Institute for Research on Poverty, Discussion Paper 189-73.
6.
Battelle Columbus Laboratories (1972) Final Report on the Office of Economic Opportunity Experiment in Educational Performance Contracting. Columbus, Ohio: Battelle Memorial Institute.
7.
Bennett, C. A. and a. A. Lumsdaine [eds.] (1975) Evaluation and Experiment . New York: Academic Press.
8.
Berk, R.A. and P.H. Rossi (1976) "Doing good or worse evaluation research politically re-examined." Social Problems23 (February): 337-349.
9.
Bernstein, I. N. [ed.] (1976) Validity Issues in Evaluative Research. Beverly Hills, Calif.: Sage.
10.
——— and H.F. Freeman (1975) Academic and Entrepreneurial Research. New York. Russell Sage
11.
Bice, T.W. and R.L. Eichhorn (1975) "Evaluation of public health programs," pp. 605-620 in M. Guttentag and E. L. Struening (eds.) Handbook of Evaluation Research, Vol. 2 Beverly Hills, Calif.: Sage.
12.
Blau, P.H. and O.D. Duncan (1967) The American Occupational Structure. New York: John Wiley.
13.
Bogatz, G.A. and S. Ball (1971) The Second Year of Sesame Street: A Continuing Evaluation. Princeton, N.J.Educational Testing Service
14.
Bowles, S. and H.M. Levin (1968) "The determinants of scholastic achievement-an appraisal of some recent evidence " J. of Human Resources3 (Winter): 3-24.
15.
Cain, G.G. (1975) "Regression and selection models to improve nonexperimental comparisons," pp 297-317 in C. A. Bennett and A. A. Lumsdaine (eds.) Evaluation and Experiment. New York. Academic Press.
16.
--- ( 1968) "Benefit/cost estimates for Job Corps." Madison. University of Wisconsin, Institute for Research on Poverty, Discussion Paper 9-68.
17.
——— and R.G. Hollister (1972) "The methodology of evaluating social action programs ," pp. 109-137 in P. H Rossi and W. Williams (eds.) Evaluating Social ProgramsNew York: Seminar Press.
18.
Cain, G.G. and H.W. Watts (1970) "Problems in making policy inferences from the Coleman Report." Amer. Soc Rev.35 (April): 228-242.
19.
Campbell, D.T. (1969) "Reforms as experiments." Amer. Psychologist24 (April): 228-242.
20.
--- and R.F. Boruch (1975) "Making the case for randomized assignment to treatments by considering the alternatives: six ways in which quasi-experimental evaluations in compensatory education tend to underestimate effects," pp. 195-296 in C. A. Bennett and A. A. Lumsdaine (eds.) Evaluation and Experiment. New York: Academic Press.
21.
Campbell, D.T. and A. Erlebacher (1970) "How regression artifacts in quasi-experimental evaluations can mistakenly make compensatory education look harmful," pp. 185-210 in J. Helmuth (ed.) The Disadvantaged Child, Vol. 3: Education : A National Debate. New York: Brunner/ Mazel.
22.
Campbell, D.T. and J.C. Stanley (1966) Experimental and Quasi-Experimental Designs for Research. Chicago: Rand-McNally .
23.
Cicirelli, V.G. et al. (1969) The Impact of Head Start. Athens, Ohio: Westinghouse Learning Corporation and Ohio University.
24.
Cline, M.G. et al. (1975) Education as Experimentation: Evaluation of the Follow Through Planned Variation Model. Cambridge, Mass.: Abt Associates.
25.
Coleman, J.S. (1975) "Problems of conceptualization and measurement in studying policy impacts," pp. 19-40 in K. M. Dolbeare (ed.) Public Policy Evaluation. Beverly Hills, Calif.: Sage.
26.
——— ( 1970) "Reply to Cain and Watts." Amer. Soc. Rev.35 (April): 242-249.
27.
———, S.D. Kelly, and J.A. Moore (1975) Trends in School Desegregation, 1968-73. Washington, D.C.: Urban Institute.
28.
Coleman, J.S. et al. (1966) Equality of Educational Opportunity. Washington, D.C.: Government Printing Office .
29.
Comptroller General ( 1973) Evaluation of the Office of Economic Opportunity's Performance Contracting Experiment. Washington, D.C.. General Accounting Office.
30.
Cook, T.C. (1975) Sesame Street Revisited. New York: Russell Sage.
31.
——— and D.T. Campbell (1975) "The design and conduct of quasi-experiments and true experiments in field settings," pp. 223-326 in M. D. Dunnette (ed.) Handbook of Industrial and Organizational Research. Chicago: Rand-McNally .
32.
Dacy, D. and H. Kunreuther (1969) The Economics of Natural Disasters. New York: Free Press.
33.
Diaz Guerrero, R. (n.d.) Experimental Studies of Plaza Sesamo by CIPAC. Mexico City: Centro de Investigaciones Psicopedagogicas .
34.
Dorfman, R. [ed.] (1965) Measuring Benefits of Government Investments. Washington, D.C.: Brookings.
Fechter, A. (1974) "Public employment programs: an evaluation study ." Washington, D.C.: Urban Institute , Working Paper 963-41.
37.
Fisk, D.M. (1970) The Indianapolis Police Fleet Plan: An Example of Program Evaluation for Local Government." Washington, D.C.: Urban Institute.
38.
Freeman, H. and C.C. Sherwood (1970) Social Research and Social Policy. Englewood Cliffs, N.J.: Prentice-Hall.
39.
Friesema, P. (1976) "Preliminary reports: NSF-RANN project on economic effects of natural disasters." Evanston, Ill.: Northwestern University. (mimeo)
40.
Garfinkel I. and E.M. Gramlich (1973) "A statistical analysis of the OEO experiment in performance contracting." J. of Human Resources8 (Summer): 275-305.
41.
Garms, W.I. (1971) "A benefit-cost analysis of the Upward Bound program ." J. of Human Resources6 (Spring): 206-220.
42.
Gilbert, J.P., R.J. Light, and F. Mosteller (1975) "Assessing social innovations : an empirical base for policy," pp. 39-193 in C. A. Bennett and A. A. Lumsdaine (eds.) Evaluation and Experiment. New York: Academic Press.
43.
Glennan, T.K., Jr. (1972) "Evaluating federal manpower programs: notes and observations," pp 187-220 in P. H. Rossi and W. Williams (eds.) Evaluating Social Programs . New York: Seminar Press.
44.
Goldstein, J. (1972) "The effectiveness of manpower training programs. a review of research on the impact on the poor." Washington, D.C.: U.S Congress, Joint Economic Committee, Studies in Public Welfare, Paper 3.
45.
Gramlich, E.M. and P.P. Koshel (1975) "Is real-world experimentation possible? The case of Educational Performance Contracting." Policy AnalysisI (Summer): 511-530.
46.
Greeley, A.M. and P.H. Rossi (1966) The Education of Catholic Americans. Chicago: Aldine.
47.
Greenleigh Associates (1970) Upward Bound 1965-1969: A History and Synthesis of Data on the Program in the Office of Economic Opportunity. New York. Greenleigh Associates.
48.
Grosse, R.N. (1972) "Cost-benefit analysis of health services." Annals of the Amer. Academy of Pol. & Social Sci.399 (January): 89-99.
49.
Guttentag, M. and E. L. Struening [eds.] (1975) Handbook of Evaluation Research , Vol. 2. Beverly Hills, Calif. : Sage.
50.
Hansen, M.H., W.N. Hurwitz, and W.G. Madow (1953) Sample Survey Methods and Theory. New York: John Wiley.
51.
Hardin, E. and M.E. Borus (1971) The Economic Benefits and Costs of Retraining. Lexington, Mass.D. C. Heath.
52.
Haveman, R.H. et al. [eds.] (1974) Benefit-Cost and Policy Analysis 1973. Chicago: Aldine.
53.
Hibbs, D.A., Jr. (1976) "On analyzing the effects of policy interventions. Box-Jenkins and Box-Tiao vs structural equation models," in D. L Heise (ed ) Sociological Methodology . San Francisco: Jossey-Bass.
54.
Hilton, E.T. and A.A. Lumsdaine (1975) "Field trial designs in gauging the impact of fertility planning programs," pp 319-408 in C. A Bennett and A A. Lumsdaine (eds ) Evaluation and Experiment. New YorkAcademic Press.
55.
Jencks, C. et al. (1972) Inequality. New York . Basic Books.
56.
Kassebaum, G. , D. Ward, and D. Wilner (1971) Prison Treatment and Parole Survival. New York. John Wiley
57.
Kelling, G. et al. (1974) The Kansas City Preventive Patrol ExperimentWashington, D.C.Police Foundation .
58.
Kish, L. (1965) Survey Sampling. New York: John Wiley.
59.
Levin, H.M. (1975) "Cost-effectiveness analysis in evaluation research ," pp. 89-122 in M. Guttentag and E. L Struening (eds ) Handbook of Evaluation Research, Vol 2. Beverly Hills, CalifSage.
60.
Main, E.D. (1968) "A nationwide evaluation of the MDTA institutional job training " J. of Human Resources3 (Spring): 159-170.
61.
Martinson, R. (1974) "What works?Questions and answers about prison reform " Public Interest35 (Spring). 22-54
62.
McDill, E.L., M.S. McDill, and J.T. Sprehe (1972) "Evaluation in practice: compensatory education ," pp. 141-185 in P. H. Rossi and W. Williams (eds ) Evaluating Social Programs. New York: Seminar Press .
63.
McLaughlin, M.W. (1975) Evaluation and Reform: The Elementary and Secondary Education Act of 1965/ Title I. Cambridge, Mass.: Ballinger.
64.
Mosteller, F. and D. P. Moynihan [eds.] (1972) On Equality of Educational Opportunity . New York: Vintage .
65.
Murphy, J.T. (1976) "Title V of ESEA: the impact of discretionary funds on state education bureaucracies," in W. Williams and R. F. Elmore (eds.) Social Program ImplementationNew York: Academic Press.
66.
Nelson, C.R. (1973) Applied Time Series Analysis. San Francisco: Holden-Day.
67.
Niskanen, W.A. et al. [eds.] (1973) Benefit-Cost and Policy Analysis 1972. Chicago: Aldine.
68.
Nunnally, J.C. and R.L. Durham (1975) "Validity, reliability, and special problems of measurement in evaluation research," pp. 289-352 in E. L. Struening and M. Guttentag (eds.) Handbook of Evaluation Research, Vol. 1. Beverly Hills, Calif.: Sage.
69.
Nunnally, J.C. and W.H. Wilson (1975) "Method and theory for developing measures in evaluation research," pp. 227-289 in E. L Struening and M Guttentag (eds ) Handbook of Evaluation Research, Vol. 1. Beverly Hills, Calif.: Sage.
70.
Palmer, E. (1969) "Can television really teach?" Amer. Education (August-September) : 1-5.
71.
——— and S. Gibbon (1970) Pre-Reading on Sesame Street. New York: Children's Television Workshop
72.
Pechman, J. and P.M. Timpane (1975) Work Incentives and Income Guarantees. Washington, D.C.: Brookings.
73.
Pincus, J. (1976) "Incentives for innovation in the public schools," in W Williams and R. F. Elmore (eds ) Social Program Implementation . New York: Academic Press.
74.
Ribich, T.I. (1968) Education and Poverty. Washington, D.C.: Brookings
75.
Riecken, H.W. and R.F. Boruch (1974) Social ExperimentationNew YorkAcademic Press.
76.
Rivlin, A. (1971) Systematic Thinking for Social Action. Washington, D C.. Brookings.
77.
——— and P.M. Timpane (1975) Planned Variation in Education. Washington, D.C.: Brookings
78.
Ross, H.L. (1973) "Law, science, and accidents the British Road Safety Act of 1967." J. of Legal Studies2: 1-75.
79.
Rossi, P.H. (1976) "Review of Milbrey W. McLaughlin, Evaluation and Reform" Harvard Educ. Rev.46 (May): 263-267.
80.
--- ( 1972) "Testing for success and failure in social action," pp 11-49 in P H Rossi and W Williams (eds ) Evaluating Social Programs. New YorkSeminar Press.
81.
——— and K.C. Lyall (1976) Reforming Public WelfareNew YorkRussell Sage.
82.
Rossi, P. H and W. Williams [eds.] (1972) Evaluating Social ProgramsNew York: Seminar Press
83.
Rossi, P.H. , R.A. Berk, and B.K. Eidson (1974) The Roots of Urban Discontent. New York. John Wiley
84.
Rothenberg, J. (1975) "Cost-benefit analysis: a methodological exposition ," pp. 55-88 in M. Guttentag and E. L. Struening (eds.) Handbook of Evaluation Research, Vol. 2. Beverly Hills, Calif.: Sage.
85.
——— (1967) Economic Evaluation of Urban Renewal. Washington, D.C.: Brookings.
86.
Scriven, M. (1967) "The methodology of evaluation," in R. W. Tyler et al. (eds.) Perspectives on Curriculum Evaluation. Chicago: Rand-McNally.
87.
Sewell, D.O. (1971) Training the Poor: A Benefit-Cost Analysis of Manpower Programs in the U S. Antipoverty Program. Kingston, Ontario : Queen's University, Industrial Relations Center .
88.
Sewell, W.H. and R.M. Hauser (1975) Education, Occupation and Earnings. New York: Academic Press.
89.
Smith, M.S. and J.S. Bissell (1970) "Report analysis: the impact of Head Start." Harvard Educ. Rev.40 (Winter): 51-104.
90.
Somers, G.G. and E.W. Stromsdorfer (1970) A Cost-Effectiveness Study of the In-School and Summer Neighborhood Youth Corps. Madison. University of Wisconsin, Industrial Relations Research Institute.
91.
Stromsdorfer, E.W. (1972) "Review and synthesis of cost-effectiveness studies of vocational and technical education." Columbus: Ohio State University (mimeo)
92.
Stanley, J. (1972) "Controlled field experiments as a model for evaluation ," pp. 67-71 in P. H. Rossi and W. Williams (eds.) Evaluating Social Programs. New York. Seminar Press.
93.
Struening, E.L. and M. Guttentag (1975) Handbook of Evaluation Research, Vol. 1. Beverly Hills, Calif.: Sage.
94.
Suchman, E. (1967) Evaluative Research. New York : Russell Sage.
95.
Sudman, S. (1976) Applied Sampling. New York: Academic Press.
96.
Vanecko, J.J. and B. Jacobs (1970) Reports from the 100-City CAP Evaluation: The Impact of the Community Action Program on Institutional Change. Chicago: University of Chicago, National Opinion Research Center.
97.
Weikart, D.P. and B.A. Banet (1975) "Planned variation from the perspective of a model sponsor." Policy Analysis1 (Summer): 485-510.
98.
Weinstein, A.S. (1975) "Evaluation through medical records and related information systems," pp. 397-481 in E. L. Struening and M. Guttentag (eds.) Handbook of Evaluation Research, Vol. 1. Beverly Hills, Calif.Sage.
99.
Weiss, C. H. [ed.] (1972a) Evaluating Action Programs. Boston. Allyn & Bacon ——— (1972b) Evaluation Research. Englewood Cliffs, N.J.: Prentice-Hall.
100.
Weiss, R.S. and M. Rein (1972) "The evaluation of broad-aim programs difficulties in experimental design and an alternative," in C. H. Weiss (ed.) Evaluating Action Programs. BostonAllyn & Bacon.
101.
Williams, W. (1975) "Implementation analysis and assessment." Policy Analysis1 (Summer). 531-566
102.
——— and R. F. Elmore [eds.] (1976) Social Program Implementation. New York: Academic Press.
