Increased use of randomized experiments to evaluate social programs throughout the world has
been a major advance in evaluation research. This article focuses on determining which program
evaluations are appropriate or feasible for randomized experiments. These threshold conditions
include: (1) the present practice must need improvement; (2) the efficacy of the proposed
intervention(s) must be uncertain; (3) there should be no simpler alternatives; (4) the results
must be potentially important for policy; and (5) the design must meet the ethical standards of
both the researchers and the service providers. To illustrate the issues involved and examine
some of the innovative research designs for addressing them, experiments from Barbados, China,
Colombia, Kenya, India, Israel, Nicaragua, Pakistan, Taiwan, and the U.S. are reviewed.
Get full access to this article
View all access options for this article.
References
1.
Baja, T.S. (1984) "Evaluation methods and techniques in Nigeria," pp. 89-118 in R. F. Boruch (ed.) Project Evaluation: Problems of Methodology. Paris: UNESCO.
2.
Berk, R.A., R.F. Boruch, D.L. Chambers, P.H. Rossi, and A.D. Witte (1985) "Social policy experimentation: a position paper ." Evaluation Rev.9 (4): 387-429.
3.
Bertrand, J.T., P. Russell-Brown, E. Landry, D. Murray, S. Norville, and Y. Rotschell (1986) A Test of Two Strategies for Delaying a Second Pregnancy in Teenage Mothers in Barbados: Final Report (contract AID/DPE-0632-0632-C-00-2007-00 . New Orleans, LA: Tulane University Medical Center, School of Public Health and Tropical Medicine, Applied Health Sciences Department.
4.
Boruch, R.F. (1984a) "Ideas about social research, evaluation, and statistics in medieval Arabic literature: Ibn Khaldun and Al-Biruni." Evaluation Rev.8 (6): 823-842.
5.
Boruch, R. F. [ed.] (1984b) Project Evaluation: Problems of Methodology. Paris: UNESCO.
6.
Bryk, A. S. [ed.] (1983) Stakeholder Based Evaluation. New Directions for Program Evaluation, 17. San Francisco: Jossey-Bass.
7.
Campbell, D.T. (1969) "Reforms as experiments." Amer. Psychologist24: 409-429.
8.
Campbell, D.T. (1980) Models of Experiments in Social Psychology and Applied Research (M. I. Bobneva and G. M. Andreeva, eds.). Moscow: Progress.
9.
Campbell, D.T. and J.C. Stanley (1979) Delineamentos experimentais e quase-experimentais de pesquisa. (Traducao de Renato Alberto T.D. Dio). Sao Paulo: Editora Pedagogica e Universitaria Ltda, Editora da Universidade de Sao Paulo.
10.
Campbell, D.T. and J.C. Stanley (1963) "Experimental and quasi-experimental designs for research on teaching." In N. L. Gage (ed.) Handbook of Research on Teaching. Chicago : Rand McNally. (Also published in 1966 as Experimental and Quasi-Experimental Designs for Research.)
11.
Chipeta, C., K.L. Bell, L. Engberg, and B.F. Kandoole (1984) "Guidelines for improving evaluation methods and techniques in Malawi," pp. 55-88 in R. F. Boruch (ed.) Project Evaluation: Problems of Methodology. Paris: UNESCO.
12.
Cuca, R. and C.S. Pierce (1977) Experiments in Family Planning: Lesson from the Developing World. Baltimore and London: Johns Hopkins Univ. Press.
13.
Davis, D.J. (1981) "Evaluating the family as an educational framework , " pp. 443-454 in A. Lewy and D. Nevo (eds.) Evaluation Roles in Education . New York and London: Gordon & Breach.
14.
Federal Judicial Center (1981) Experimentation in the Law: Report of the Federal Judicial Center Advisory Committee on Experimentation in the Law. Washington, DC : Federal Judicial Center.
15.
Fraker, T. and R. Maynard (1985) The Use of Comparison Group Designs in Evaluations of Employment Related Programs. Princeton, NJ: Mathematica Policy Research.
16.
Fraker, T. and R. Maynard (1987) "Evaluating comparison group designs with employment-related programs." J. of Human Resources22 (2): 195-227.
17.
Gomez, F. (1985) "Community-based distribution: the case of Colombia ," pp. 203-213 in M. Warner, S. Huffman, D. Cebula, and R. Osborn (eds.) Health and Family Planning in Community-Based Distribution Programs. Boulder, CO: Westview.
18.
Hendricks, M. (1981) "Service delivery assessment." New Directions for Program Evaluation12: 4-24.
19.
Issawi, C. [trans. and ed.] (1950) An Arab Philosophy of History: Selections from the Prolegomena of Ibn Khaldun of Tunis, (1332-1406). London: John Murray .
20.
Kaliappan, K.V. and R.K. Kaliappan (1984) "Effect of behavior therapy techniques on state and anxiety of juvenile delinquents." Social Defense Q.29 (75): 16-21.
21.
Kidd, J.R., L.T. Khoi, B. Hammiche, A.L. Correa, and A. Rudra (1976) The Experimental World Literacy Programme: A Critical Assessment. Paris: UNDP.
22.
Klein, R. E., M. S. Read , H. W. Riecken, J. A. Brown, A. Pradilla, and C. H. Daza [eds.] (1979) Evaluating the Impact of Nutrition and Health Programs. New York: Plenum.
23.
Landrum, J., S. Miles, R. Nett, T. Pritchard, J. Roebuck, E. Wells Parker, and G. Windham (1982) Mississippi DUI Probation Follow-Up Project (final report to NHTSA, contract no. DOT-HS-5-10098. Washington, DC: NTIS.
24.
Lewy, A. and D. Nevo (1981) Evaluation Roles in Education. Paris, London, and New York : Gordon & Breach.
25.
McKay, H., A. McKay, L. Sinisterra, H. Gomez, and P. Lloreda (1978) "Improving cognitive ability in chronically deprived children." Science200: 270-278.
26.
Meier, P. (1978) "The biggest public health experiment ever: the 1954 field trial of the Salk poliomyelitis vaccine," in J. M. Tanur, F. Mosteller , W. B. Kruskal, R. S. Link, R. S. Pieters, and G. Rising (eds.) Statistics: A Guide to the Unknown. San Francisco: Holden-Day .
27.
Middleton, J. (1985) "The Philippine radio education project. a case study of decision design and interactions," pp 7-72 in B. Searle (ed.) Evaluation in Three World Bank Education Projects: Lessons From Three Case Studies (Report No. EDT5, Operations Policy Staff. Washington, DC: World Bank .
28.
Norton andBenoliel, S.P. (1987) Guidelines for Data Collection, Monitoring, and Evaluation Plans for AID-Assisted Projects (AID program design and evaluation methodology report No. 9. Washington, DC: U.S. Agency for International Development.
29.
Parashar, R.K. (1984) "Evaluating social action projects in India: guidelines for improving methods and techniques," pp. 15-54 in R. F. Boruch (ed.) Project Evaluation: Problems of Methodology . Paris: UNESCO.
30.
Population Council (1986) An Experimental Study of the Efficiency and Effectiveness of an IUD Insertion and Back-Up Component (English summary of first six month report, PC PES86 . Lima, Peru: Population Council .
31.
Population Information Program (1986) "Operations research: lessons for policy and programs" (Population Reports, Series J 31. Baltimore, MD: Population Information Program, Johns Hopkins University.
32.
Riecken, H.W. and A.F. Boruch (1974) Social Experimentation: A Method for Planning and Evaluating Social Programs. New York: Academic Press.
33.
Riecken, H.W. and A.F. Boruch (1978) "Social experiments." Annual Rev. of Sociology4: 511-532.
34.
Sack, R. (1985) "Lessons from an evaluation/monitoring system for Haitian education reform," pp. 107-138 in B. Searle (ed.) Evaluation in World Bank Education Projects: Lessons from Three Case Studies. Washington, DC: World Bank.
35.
Schneider, A.L. (1980) "Effects of status offender deinstitutionalization in Clark County, Washington." Presented at the American Society of Criminology Annual Convention, San Francisco, November.
36.
Schwartz, P.A. (1985) "Evaluation of the Pakistan primary education project: a methodological case study," pp. 73-106 in B. Searle (ed.) Evaluation in World Bank Education Projects: Lessons From Three Case Studies (Report No. EDT5, Operations Policy Staff. Washington, DC: World Bank.
37.
Searle, B. [ed.] (1985) Evaluation in World Bank Education Projects: Lessons from Three Case Studies (Report No. EDT5, Operations Policy Staff. Washington, DC: World Bank.
38.
Searle, B., P. Matthews, P. Suppes, and J. Friend (1978) "Formal evaluation of the radio mathematics instructional program, Nicaragua-Grade 1, 1976," pp. 651-672 in T. D. Cook and associates (eds.) Evaluation Studies Rev. Annual, Vol. 3. Beverly Hills, CA: Sage.
39.
Sieber, J. (1982) The Ethics of Social Research: Surveys and Experiments . New York: Springer-Verlag.
40.
U.S. General Accounting Office (1986) Teenage Pregnancy: 500,000 Births a Year But Few Tested Programs. Washington, DC: Author.
41.
Watson, K.F. (1986) "Programs, experiments, and other evaluations: an interview with Donald Campbell." Canadian J. of Program Evaluation1 (1): 83-86.
42.
White, L.G. (1986) An Approach to Evaluating the Impact of AID Projects (AID program design and evaluation methodology Report No. 5. Washington, DC: U.S. Agency for International Development .
43.
Wholey, J.S. (1983) Evaluation and Effective Public Management. Boston: Little, Brown.
44.
World Health Organization Special Programme of Research, Development and Research Training in Human Reproduction (1985) Fourteenth Annual Report. Geneva: Author.
45.
World Health Organization Special Programme of Research, Development and Research Training in Human Reproduction (1986) Impact of Integrated Family Planning Clinics on Acceptance and Continuation in Rural Areas of Kenya: Phase 1 (HRP-TF. Nairobi, Kenya: Nairobi Center for Research in Reproduction.
