Two studies were conducted to determine the stability of self-assessment of evaluation performance and use over a six-week period. Two instruments, the Evaluation Performance Questionnaire (EPQ) and the Evaluation Utilization Questionnaire (EUQ) were developed to assess internal evaluation of community mental health centers (CMHCs). One hundred one evaluators of
CMHCs were surveyed in Study 1, and 175 directors and evaluators were surveyed in Study 2;
response rates were 54% and 60%, respectively. Test-retest reliability coefftcients were calculated for total and subscale scores, and scale means were compared across time. Results show
that the EPQ and EUQ provide reliable information about the level and scope of evaluation
activity and the use of evaluation information within community mental health centers. The value
of these tools for research, administration, evaluation, and practice is discussed
Get full access to this article
View all access options for this article.
References
1.
Attkisson, C.C., and W.A. Hargreaves .1979. A conceptual model for program evaluations in health organizations In Program evaluation in the health fields. Vol. 2, edited by H. C. Schulberg and F. Baker, 53-72. New York: Human Sciences Press.
2.
Attkisson, C.C., M. McIntyre, W.A. Hargreaves, M.R. Harris, and F.M. Ochberg.1974. A working model of mental health program evaluationAmerican Journal of Orthopsychiatry44:741-53
3.
. Cohen, J.1989. Statistical power analysis for the behavioral sciences2d ed. New York: Academic Press.
4.
Conner, R.F.1981. Measuring evaluation utilization: A critique of different techniques In Utilizing evaluation: Concepts and measurement techniques, edited by J. Ciarlo, 59-77. Beverly Hills, CA: Sage.
5.
Cook, T.D., and W.R. Shadish.1982. Metaevaluation: An assessment of the congressionally mandated evaluation system for community mental health centers In Innovative approaches to mental health evaluation , edited by G. J. Stahler and W. R. Tash, 221-53. New York: Academic Press.
6.
Cousins, J.B., and K.A. Leithwood.1986. Current empirical research on evaluation utilizationReview of Educational Research56 (3): 331-64
7.
. Cronbach, L.J., S.R. Ambron, S.M. Dombusch, R.D. Hess, R.C. Hornik, D.C. Phillips, D.F. Walker, and S.S. Weiner.1980. Toward reform of program evaluation: Aims, methods, and institutional arrangementsSan Francisco : Jossey-Bass.
8.
Cummings, O.W., J.R. Nowakowski, T.A. Schwandt, R.T. Eichelberger, K.C. Kleist, C.L. Larson, and T.D. Knott.1988. Business perspectives on internal/external evaluation In Evaluation utilization, edited by J. A. McLaughlin, L. J. Weber, R. W. Covert, and R. W. Ingle, 59-74. New Directions for Program Evaluation, no. 39. San Francisco: Jossey-Bass.
9.
Garagliano, C.1974. Mental health program evaluation: Culture and performance. Ph.D. diss, University of California, Los Angeles.
10.
Glaser, E.M., and K.E. Kirkhart.1982. Descriptive analysis of preassessment data. HIRI Technical ReportAustin, TX: Human Interaction Research Institute.
11.
Greene, J.C.1988. Communication of results and utilization in participatory program evaluationEvaluation and Program Planning11:341-51
12.
. Jerrell, J.M.1986. Evaluation in a down-loaded mental health systemEvaluation and Program Planning9:161-66
13.
. Kirkhart, K.E.1979. Program evaluation in community mental health centers. Ph.D. diss, University of Michigan.
14.
Kirkhart, K.E., and E.M. Glaser.1986. Improving the capability of community mental health centers for self-evaluation. Final Report, National Institute of Mental Health Grant No. R01 MH 33540. Los Angeles: Human Interaction Research Institute.
15.
Leviton, L.C., and E.F. Hughes.1981. Research on the utilization of evaluation: A review and synthesisEvaluation Review5:525-48
16.
. Majchrzak, A., and C. Windle.1980. Patterns of program evaluation in community mental health centersEvaluation Review4:677-91
17.
. McCullough, P.1975. Training for evaluators In Program evaluation: Alcohol, drug abuse, and mental health, edited by J. Zusman and C. Wurster, 247-66. Lexington, MA: D. C. Heath.
18.
Mowbray, C.T.1988. Getting the system to respond to evaluation findings In Evaluation utilization, edited by J. A. McLaughlin, L. J. Weber, R. W. Covert, and R. W. Ingle, 47-58. New Directions for Program Evaluation, no. 39. San Francisco: Jossey-Bass.
19.
Muscatello, D.B.1988. Developing an agenda that works: The right choice at the right time In Evaluation utilization, edited by J. A. McLaughlin, L. J. Weber, R. W. Covert, and R. W. Ingle, 21-31. New Directions for Program Evaluation, no. 39. San Francisco : Jossey-Bass.
20.
Patton, M.Q.1986. Utilization-focused evaluation2d ed. Beverly Hills, CA: Sage.
21.
—. 1988. Integrating evaluation into a program for increased utility and cost-effectiveness In Evaluation utilization , edited by J. A. McLaughlin, L. J. Weber, R. W. Covert, and R. W. Ingle, 85-94. New Directions for Program Evaluation, no. 39. San Francisco : Jossey-Bass.
22.
Philadelphia Health Management Corporation.1978. A review of community mental health centers' evaluation activities 3 vols. Final report on National Institute of Mental Health Contract No. 278-77-0067 MH. Mimeo.
23.
Rossi, P.H., and H.E. Freeman.1989. Evaluation: A systematic approach4th ed. Newbury Park, CA: Sage.
24.
SAS Institute, Inc.1987. SAS user's guide: Statistics Version 6.03 ed. Cary, NC: SAS Institute, Inc.
25.
Weiss, C.1981. Measuring the use of evaluation In Utilizing evaluation: Concepts and measurement techniques , edited by J. A. Ciarlo, 17-33. Beverly Hills, CA: Sage.
26.
Wildavsky, A.1972. The self-evaluating organizationPublic Administration Review32 (November/December): 509-20
27.
. Windle, C., and F.M. Ochberg.1975. Enhancing program evaluation in the community mental health centers programEvaluation2:31-36
28.
. Windle, C., and E. Volkman.1973. Evaluation in the centers programEvaluation1(2): 69-70.
