Evaluators are often faced with many challenges in the design and implementation of a program's evaluation. Because programs are entangled in complex networks of structures and stakeholders, they can be challenging to understand, and they often pose issues of competing and conflicting goals. However, by using a systems mapping approach to evaluation design, evaluators can identify and diagram the relationships between different variables and learn about their potential impact and unintended consequences on a program's processes. In this article, we describe how using a systems approach and causal loop mapping approach during an evaluation of a demonstration program for people with developmental disabilities helped evaluators to better understand the program's implementation and the variation in the program's outcomes.
Center for Policy Research, University at Albany. (2002a). DDD evaluation study: Staff survey report. Albany, NY: Author.
2.
Center for Policy Research, University at Albany. (2002b). DDD evaluation study: Site visit report. Albany, NY: Author.
3.
Center for Policy Research, University at Albany. (2003a). DDD evaluation study: A summary of the project's implementation to date. Albany, NY: Author.
4.
Center for Policy Research, University at Albany. (2003b). DDD evaluation study: A preliminary summary of the DDD project's findings. Albany, NY: Author.
5.
Center for Policy Research, University at Albany. (2005). DDD evaluation study: Project outcomes, a final report. Albany, NY: Author.
6.
Coyle, G. (2000). Qualitative and quantitative modeling in system dynamics: Some research questions. System Dynamics Review , 16(3), 225-244.
7.
Goodman, M.R. (1974). Study notes in system dynamics. Portland, OR: Productivity Press.
Forrester, J. (1971). Counterintuitive behavior of social systems. Technology Review, 73(3), 52-68.
10.
Forrester, J. (1994). Policies, decisions and information sources for modeling. European Journal of Operational Research, 59(1), 42-63.
11.
Fredericks, K.A. (2005). Network analysis of a demonstration program for the developmentally disabled. New Directions for Evaluation , 107, 55-67.
12.
Harris, B., and Williams, B. (2005). System dynamics methodology. Retrieved September 13, 2005, from http://www.facilitatedsystems.com/weblog
13.
Hirsch, G.B., Levine, R., & Miller, R.L. (2007). Using system dynamics modeling to understand the impact of social change initiatives. American Journal Community Psychology, 39(3-4), 239-253. Retrieved February 1, 2008, from http://file://C:\documents and settings\user\locals settings\temp\xpgrpwise\millerarticle.htm
14.
Holburn, S. (2002). How science can evaluate and enhance person-centered planning. Research and Practice for Persons With Severe Mental Retardation, 27(4), 250-260.
15.
Holburn, S., Jacobson, J.W., Vietze, P.M., Schwartz, A.A., & Sersen, E. (2000). Quantifying the process and outcomes of person-centered planning. American Journal on Mental Retardation, 105, 402-416.
16.
Holburn, S., Pfadt, A., Vietze, P.M., Schwartz, A.A., & Jacobson, J.W. (1996). Person centered planning quality of life indicators. Albany, NY: State Office of Mental Retardation and Developmental Disabilities.
17.
Homer, J., & Oliva, R. (2001). Diagrams and models in system dynamics: A response to Coyle. System Dynamics Review, 17(4), 347-355.
18.
Levin, G., & Roberts, E. (1976). The dynamics of human service delivery. Cambridge, MA: Ballinger.
19.
MacDonald, R. (2002). The dynamics of federal deposit insurance: A feedback view of system behavior. Unpublished doctoral dissertation, University at Albany, Albany, NY.
20.
Morris, D. (2005, July 17-21). Causal inference in the social sciences: Variance theory, process theory, and system dynamics. Paper presented at the 23rd International Conference of the System Dynamics Society, Boston.
21.
Richardson, G.P. (1999). Feedback thought in social science and systems theory. Waltham, MA: Pegasus Communications .
22.
Richardson, G.P. (2002, July). Misattribution in welfare dynamics: The puzzling dynamics of recidivism. Paper presented at the International System Dynamics Conference, Palermo , Italy. Retrieved on February 29, 2008, from http://systemdynamics.org/conferences/2002/papers/Richard1.pdf
23.
Richardson, G.P., & Pugh, A.L. (1981). Introduction to system dynamics modeling with DYNAMO. Cambridge, MA: Productivity Press.
24.
Senge, P. (1990). The fifth discipline. New York : Doubleday.
25.
Sterman, J.D. (2000). Business dynamics: Systems thinking and modeling for a complex world. Boston: Irwin / McGraw-Hill.
26.
Wolstenholme, E. (1990). System enquiry: A system dynamics approach. New York: John Wiley.
