The User-Oriented Evaluator’s Role in Formulating a Program Theory: Using a Theory-Driven Approach

Abstract

Program theory plays a prominent role in many evaluations, not only in theory-driven evaluations. This paper presents a case study of the process of developing and refining a program’s theory within a user-oriented evaluation. In user-oriented (or utilization-focused) evaluations, primary users can play a role in defining their own program theory. This is different, however, from the typical process by which a program theory is developed when using theory-driven evaluation framework. This case study concerns a university’s academic outreach program with three local school districts. The program’s objective is to increase the number of University of California eligible and specifically University of California, Los Angeles (UCLA) admissible students from targeted local public schools. The authors helped develop and refine outreach staff’s program theory. The evaluation procedures are described and results of the theory building process presented.

Get full access to this article

View all access options for this article.

References

Adelson, M. , Alkin, M. C. , Carey, C. , & Helmer, O. (1967). Planning education for the future: Comments on a pilot study. American Behavioral Scientist, 10(7), 1-29.

Alkin, M.C. (1991). Evaluation theory development II. In M.McLaughlin & D.Phillips (Eds.), Evaluation and education at quarter century (pp. 91-112). Chicago, IL: University of Chicago Press.

Chen, H. (1990). Theory-driven evaluations. Newbury, CA: Sage.

Chen, H. , & Rossi, P. H. (1980). The multi-goal, theory-driven approach to evaluation: A model linking basic and applied social science. Social Forces, 59, 106-122.

Chen, H. , & Rossi, P. H. (1983). Evaluating with sense: The theory-driven approach. Evaluation Review, 7, 283-302.

Chen, H. , & Rossi, P. H. (1992). Introduction: Integrating theory into evaluation practice. In H. Chen & P. H. Rossi (Eds.), Using theory to improve program and policy evaluations (pp. 2-11). Westport, CT: Greenwood Press.

Christie, C. A. (2003). What guides evaluation? A study of how evaluation practice maps onto evaluation theory. In C. A. Christie (Ed.), The practice-theory relationship in evaluation. New directions for evaluation, No. 97 (pp. 7-35). San Francisco, CA: Jossey-Bass.

Dalkey, N. , Rourke, D. L. , Lewis, R. , & Snyder, D. (1972). Studies in the quality of life: Delphi and decision-making. Lexington, MA: Lexington Books.

Datta, L.-e. (2001). Coming attractions. American Journal of Evaluation, 22, 403-408.

10.

Donaldson, S. I. (2002). Theory-driven program evaluation in the new millennium. In S. I. Donaldson & M. Scriven (Eds.), Evaluating social programs and problems (pp. 109-141). Mahwah, NJ: Lawrence Erlbaum.

11.

Mark, M. (2001). Evaluation’s future: Furor, futile or fertile. American Journal of Evaluation, 22, 457-479.

12.

Mark, M. (2002). Toward a integrative view of the theory and practice of program and policy evaluation. In S. I. Donaldson & M. Scriven (Eds.), Evaluating social programs and problems (pp. 183-204). Mahwah, NJ: Lawrence Erlbaum.

13.

Oakes, J. (2001). Six conditions for college going: Research guiding UCLA outreach partnership work. UC ACCORD Technical Report.

14.

Patton, M. Q. (1997). Utilization-focused evaluation: A new century text (3rd ed.). Thousand Oaks, CA: Sage.

15.

Rogers, P. J. , Petrosino, A. , Huebner, T. A. , & Hacsi, T. A. (2000). Program theory evaluation: Practice, promise, and problems. In P. J. Rogers , T. A. Hacsi , A. Petrosino , & T. A. Huebner (Eds.), Program theory in evaluation: Challenges and opportunities. New directions for evaluation, No. 87 (pp. 5-13). San Francisco, CA: Jossey-Bass.

16.

Scriven, M. (1991). Evaluation thesaurus (4th ed.). Newbury Park, CA: Sage.

17.

Stufflebeam, D. (2000). Evaluation models. New directions for evaluation, No. 89. San Francisco, CA: Jossey-Bass.

18.

Weiss, C. H. (1998). Evaluation: Methods for studying programs and policies (2nd ed.). Upper Saddle River, NJ: Prentice Hall.