Design and Measurement Considerations in Evaluating Integrated Human Service Delivery Systems

Abstract

This article calls attention to the fact that evaluators are increasingly being called upon to evaluate integrated human service delivery systems. This is a complex policy and program context which creates a number of potential difficulties for evaluators. This article draws upon the experiences of a group of evaluators who have recently completed a four-year implementation evaluation of a multi-site integrated human service delivery system. The following "pre-implementation" Shea issues are discussed: assisting program implementors in the specification of program parameters and negotiables; assessing the "preimplementation environment" of multiple sites; choice of research design; and, separating service delivery from program management. In addition, this article also discusses a number of "implementation phase" evaluation activities including: measuring the extent of implementation of key features of service integration; identifying transfers of authority and the locus of decision making; the use of both quantitative and qualitative data collection and analysis; and, reporting considerations.