Although the need for evaluation of prevention programs is clear, the implementation of evaluation, particularly for large, complex projects with multiple investigators, poses numerous challenges. Program evaluators in these circumstances do not have the same degree of control as, say, experimental psychologists and must find ways to obtain data while being sensitive to local environmental influences. The purpose of this article is to identify for social work evaluators some of the ways program evaluation can differ from true experiments and to discuss issues that may affect the planning and conduct of evaluation efforts. The Appalachia Community Cancer Network is used as a case example, and illustrations of its approach using formative and process evaluation are presented.
Appalachian Regional Commission. (2003). Online resource center. Regional data & research: High school and college completion rates, poverty rates in Appalachia. Retrieved May 18, 2007, from http://www.arc.gov/index.do?nodeID=56
2.
Arcury, T.A., Preisser, J.S., Gesler, W.M., & Powers, J.M. (2005). Access to transportation and health care utilization in a rural region. Journal of Rural Health, 21, 31-38.
3.
Bauman, L.J., Stein, R.K., & Ireys, H.T. (1991). Reinventing fidelity: The transfer of social technology among settings. American Journal of Community Psychology, 19, 619-639.
4.
Bureau of Labor Statistics. (2006). Occupational outlook handbook, 2006-2007. Retrieved August 1, 2007, from http://www.bls.gov/oco/ocos060.htm
5.
Centers for Disease Control and Prevention. ( 2002). Cancer death rates-Appalachia, 1994-1998. Morbidity and Mortality Weekly Report, 51(24), 527-529.
6.
Corbie-Smith, G., Thomas, S.B., Williams, M.V., & Moody-Ayers, S. (1999). Attitudes and beliefs of African-Americans toward participation in medical research. Journal of General Internal Medicine, 14, 537-546.
7.
Deskins, S., Harris, C.V., Bradlyn, A.S., Cottrell, L., Coffman, J.W., Olexa, J., et al. (2006). Preventive care in Appalachia: Use of the theory of planned behavior to identify barriers to participation in cholesterol screenings among West Virginians. Journal of Rural Health , 22, 367-374.
8.
Donaldson, S.I., & Gooler, L.E. (2003). Theory-driven evaluation in action: Lessons from a $20 million statewide work and health initiative. Evaluation and Program Planning, 26, 355-366.
Fetterman, D.M. (2002). Empowerment evaluation: Building communities of practice and a culture of learning. American Journal of Community Psychology, 30, 89-102.
11.
Friedell, G.H., Rubio, A., Maretzki, A., Garland, B., Brown, P., Crane, M., et al. (2001). Community cancer control in a rural, underserved population; The Appalachian Leadership Initiative on Cancer Project. Journal of Health Care for the Poor and Underserved, 12(1), 5-19.
12.
Kellam, S.G., & Langevin, D.J. (2003). A framework for understanding "evidence" in prevention research. Prevention Science, 4, 137-153.
13.
Kilbourne, A.M., Switzer, G., Hyman, K., Crowley-Matoka, M., & Fine, M.J. (2006). Advancing health disparities research within the health care system: A conceptual framework. American Journal of Public Health, 96, 2113-2121.
14.
Larson, S.L., & Hill, S.C. (2005). Rural-urban differences in employment-related health insurance. Journal of Rural Health, 21(1), 21-30.
15.
Lengerich, E.J., Tucker, T.C., Powell, R.K., Colsher, P., Lehman, E., Ward, A.J., et al. (2005). Cancer incidence in Kentucky, Pennsylvania, and West Virginia: Disparities in Appalachia. Journal of Rural Health, 21(1), 39-47.
16.
Lengerich, E.J., Wyatt, S.J., Rubio, A., Beaulieu, J., Coyne, C.A., Fleisher, L., et al. (2004). The Appalachia Cancer Network: Cancer control research among a rural, medically underserved population. Journal of Rural Health, 20, 181-187.
17.
Lipsey, M.W., Crosse, S., Dunkle, J., Pollard, J.U., & Stobart, G. (1985). Evaluation: The state of the art and the sorry state of the science. In D. S. Cordray (Ed.), Utilizing prior research in evaluation planning (pp. 7-28). San Francisco: Jossey-Bass.
18.
Mason, S.E. (2005). Offering African-Americans opportunities to participate in clinical trials research: How social workers can help. Health and Social Work, 30, 296-304.
19.
Maynard, R.A. (2000). Whether a sociologist, economist, psychologist or simply a skilled evaluator: Lessons from evaluation practice in the United States. Evaluation, 6, 471-480.
Royse, D. (2007). Research methods in social work. Belmont, CA: Brooks/Cole.
26.
Royse, D., Thyer, B.A., Padgett, D.K., & Logan, T.K. (2006). Program evaluation: An introduction. Belmont, CA: Wadsworth.
27.
Rubin, A. (1997). The family preservation evaluation from hell: Implications for program evaluation fidelity. Children and Youth Services Review, 19, 77-99.
28.
Schoenberg, N.E., Hopenhayn, C., Christian, A., Knight, E.A., & Rubio, A. (2005). An in-depth and updated perspective on determinants of cervical cancer screening among central Appalachian women. Women and Health, 42, 89-105.
29.
Strickland, J., & Strickland, D.L. (1996). Barriers to preventive health services for minority households in the rural south. Journal of Rural Health, 12, 206-217.
30.
Sue, S., & Dhindsa, M.K. (2006). Ethnic and racial health disparities research: Issues and problems. Health Education and Behavior, 33, 459-469.
31.
Walker, R., Lucas, W., & Crespo, R. (1994). The West Virginia Rural Cancer Prevention Project . Cancer Practice, 2, 421-426.
32.
Weinbach, R.W. (2005). Evaluating social work services and programs. Boston: Allyn & Bacon.
