The author examines the role of mediators and moderators in the evaluation of programs for children. The terms are defined and examples of each are presented. Using bibliometric analysis, the author examines how evaluators use mediators and moderators in treatment studies in education, juvenile justice, health care, child protection, and mental health. The use of mediators and moderators is sporadic and vague at best. An agenda for improvement is outlined that includes greater use of program theory and intensive case studies to find out why researchers in prevention and health promotion incorporate mediators and moderators more effectively in their evaluations.
Get full access to this article
View all access options for this article.
References
1.
Adams, Stuart
. 1970. The PICO project. In The sociology of punishment and correction, edited by Norman B. Johnston, Leonard Savitz, and Marvin Wolfgang, 548-561. New York: Wiley.
2.
Baron, Rueben M.
, and David A. Kenny. 1986. The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology51 (6): 1173-1182.
3.
Bickman, Leonard
, ed. 1987. Using program theory in evaluation. New directions in evaluation. Vol. 33. San Francisco: Jossey-Bass.
4.
Bickman, Leonard
, ed. 1990. Advances in program theory. New directions for program evaluation. Vol. 47. San Francisco: Jossey-Bass.
5.
Bickman, Leonard
, Craig Anne Heflinger, Denine Northrup, Susan Sonnichsen, and Stephen Schilling. 1998. Long term outcomes to family caregiver empowerment. Journal of Child & Family Studies7 (3): 269-282.
6.
Boruch, Robert F.1997. Randomized experiments for planning and evaluation: A practical guide. Thousand Oaks, CA: Sage.
7.
Boruch, Robert F.
, Anthony Petrosino, and Iain Chalmers. 1999. The Campbell Collaboration: A proposal for multinational, continuous and systematic reviews of evidence. Paper presented at the International Meeting on Systematic Reviews of the Effects of Social and Educational Interventions, 16 July, University College, London.
8.
Chandler, Michael J.1973. Egocentrism and anti-social behavior: The assessment and training of social perspective-taking skills. Developmental Psychology9 (3): 326-332.
9.
Chen, Huey-tsyh
, and Peter Rossi, eds. 1992. Using theory to improve program and policy evaluations. New York: Greenwood.
10.
Cook, Thomas D.1999. Considering the major arguments against random assignment: An analysis of the intellectual culture surrounding evaluation in American schools of education. Paper presented at the American Academy of Arts & Sciences, 12 May, Cambridge, MA.
11.
Davies, Philip
, Anthony Petrosino, and Iain Chalmers. 1999. Proceedings of the International Meeting on Systematic Reviews of the Effects of Social and Educational Interventions, 15-16 July, University College, London.
12.
Donaldson, Stewart I. Forthcoming. Mediator and moderator analysis in program development. In Handbook of program development in health behavior research and practice.
13.
Duff, Alistair S.1995. The “information society” as paradigm: A bibliometric inquiry. Journal of Information Science21 (5): 390-395.
14.
Eddy, J. Mark
, Thomas J. Dishion, and Michael Stoolmiller. 1998. The analysis of intervention change in children and families: Methodological and conceptual issues embedded in intervention studies. Journal of Abnormal Child Psychology26:53-69.
15.
Eisen, Marvin
, Gail Zellman, and Alfred McAlister. 1992. A health belief model-social learning theory approach to adolescents' fertility control: Findings from a controlled field trial. Health Education Quarterly19 (2): 249-262.
16.
Fassbender, Pantaleon
. 1997. Parapsychology and the neurosciences: A computer-based content analysis of abstracts in the database “MEDLINE” from 1975-1995. Perceptual and Motor Skills84 (2): 452-454.
17.
Gilham, Steven A.
, Wayne L. Lucas, and David Sivewright. 1997. The impact of drug education and prevention programs: Disparity between impressionistic and empirical assessments. Evaluation Review21 (5): 589-613.
18.
Gortmaker, Steven
, and Jane Wiecha. 1999. Presentation to the Harvard Children's Initiative Evaluation Task Force, 16 March, Cambridge, MA.
19.
Hall, Judith A.
, Linda Tickle-Degnan, Robert Rosenthal, and Frederick Mosteller. 1994. Hypotheses and problems in research synthesis. In Handbook of research synthesis, edited by Harris Cooper and Larry Hedges, 17-28. New York: Russell Sage.
20.
Harvard Project on Schooling & Children
. 1997. Evaluation Task Force. Summary report for the academic year, 1996-1997. Cambridge, MA: Author.
21.
Holmbeck, Grayson N.1997. Toward terminological, conceptual and statistical clarity in the study of mediators and moderators: Examples from the child-clinical and pediatric psychology literatures. Journal of Consulting and Clinical Psychology65 (4): 599-610.
22.
Lipsey, Mark W.1988. Practice and malpractice in evaluation research. Evaluation Practice9(4): 5-24.
23.
Lipsey, Mark W.1990. Design sensitivity. Statistical power for experimental research. Newbury Park, CA: Sage.
24.
Lipsey, Mark W.1997. What can you build with thousands of bricks? Musings on the cumulation of knowledge in program evaluation. New Directions for Evaluation76:7-23.
25.
Lipsey, Mark W.
, Scott Crosse, Jan Dunkle, John Pollard, and Gordon Stobart. 1985. Evaluation:The state of the art and the sorry state of the science. New Directions for Program Evaluation27:7-28.
26.
Lipsey, Mark W.
, and John A. Pollard. 1989. Driving toward theory in program evaluation: More models to choose from. Evaluation and Program Planning12:317-329.
27.
Lipsey, Mark W.
, and David Wilson. 1993. The efficacy of psychological, educational and behavioral treatment. Confirmation from meta-analysis. American Psychologist48 (12): 1181-1209.
28.
MacKinnon, David P.
, Mark D. Weber, and Mary A. Pentz. 1988. How do school-based drug prevention programs work and for whom?Drugs & Society3 (1-2): 125-143.
29.
Mark, Melvin M.
, David A. Hofmann, and Charles S. Reichardt. 1992. Testing theories in theory-driven evaluations: (Tests of) moderation in all things. In Using theory to improve program and policy evaluations, edited by Huey-tsyh Chen and Peter H. Rossi, chap. 4. New York: Greenwood.
30.
Miech, Edward
, Bill Nave, and Frederick Mosteller. 1998. A rare design: The role of field trials in evaluating school practices. Preliminary presentation to the Harvard Children's Initiative Evaluation Task Force, October, Cambridge, MA.
31.
Muehrer, Peter
. 1997. Introduction to the special issue: Mental health prevention science in rural communities and contexts. American Journal of Community Psychology25 (4): 421-421.
32.
Pawson, Ray
, and Nick Tilley. 1994. What works in evaluation research?British Journal of Criminology34 (3): 291-309.
33.
Petrosino, Anthony J.1997. “What works?” Revisited again: A meta-analysis of randomized field experiments in rehabilitation, deterrence and prevention. Ann Arbor, MI: University Microfilms.
34.
Petrosino, Anthony J.1998. Building better information: The potential mutual benefits of theory-based evaluation and meta-analysis. Paper presented at the American Evaluation Association annual meeting, November, Chicago.
35.
Petrosino, Anthony J.
n.d. Outcome studies—especially RCTs—just do them! Manuscript in preparation.
36.
Petrosino, Anthony J.
, Kathleen Boudett, and Pamela Perry. n.d. Evaluating children's programs: Unique challenges in the conduct and utilization of evaluation. Manuscript in preparation.
37.
Shadish, William R.
, and Rebecca B. Sweeney. 1991. Mediators and moderators in meta-analysis: There's a reason we don't let dodo birds tell us which psychotherapies should have prizes. Journal of Consulting and Clinical Psychology59 (6): 883-893.
38.
Tonry, Michael
, Lloyd E. Ohlin, and David P. Farrington. 1991. Human development and criminal behavior. New ways of advancing knowledge. New York: Springer-Verlag.
39.
Weiss, Carol H.
(1995). Nothing as practical as good theory: Exploring theory-based evaluation for comprehensive community initiatives for children and families. In New approaches to evaluating community initiatives. Concepts, methods, and contexts, edited by James P. Connell, Anne C. Kubisch, Lisbeth B. Schorr, and Carol H. Weiss, 65-92. Aspen, CO: Aspen Foundation.
40.
Weiss, Carol H.1997. Theory-based evaluation: Past, present, and future. New Directions for Evaluation76:41-55.
41.
Weiss, Carol H.1998. Evaluation: Methods for studying programs and policies. Upper Saddle River, NJ: Prentice Hall.
42.
Weiss, Carol H.1999. What to do until the random assigner comes. Paper presented to the American Academy of Arts & Sciences, May, Cambridge, MA.
43.
Weissberg, Roger P.
, and Mark T. Greenberg. 1998. Prevention science and collaborative community action research: Combining the best from both perspectives. Journal of Mental Health7 (5): 479-479.
44.
White, Jay D.1986. Dissertations and publications in public administration. Public Administration Review46 (3): 227-234.
45.
Wilson, James Q.1980. What works revisited: New findings on criminal rehabilitation. Public Interest6:3-17.
46.
Zaslavsky, Alan
. 1995. Supplementary notes on research design and data collection. Unpublished notes, Harvard University, Department of Statistics.
