Evaluating Healthcare Reform: The Challenge of Evaluating Changing Policy Environments

Abstract

Evaluators in the health sector struggle to develop effective mechanisms with which to evaluate healthcare reform programmes. Evaluation of these programmes is difficult because they are entangled in complex, inherently political processes and often shift away from their original aims and objectives as a result of policy changes within the health system. This article describes the evaluation of a healthcare reform process occurring in South Australia and discusses some of the methodological challenges encountered in evaluating a case study of continuous change with shifting policy objectives. It describes how the evaluators have attempted to address these challenges through a flexible and dynamic action-research approach. The article concludes by discussing the role of the evaluator, as intelligent observer, in reflecting on and documenting change in the highly political and complex field of healthcare reform.

Keywords

action research evaluating change evaluation methodology healthcare reform health policy

Get full access to this article

View all access options for this article.

References

Abma, T. A. (1999) ‘Introduction: Narrative Perspectives on Program Evaluation’, in T. A. Abma (ed.) Telling Tales: On Evaluation and Narrative, Advances in Program Evaluation 6, pp. 1–27. Stamford, CT: JAI Press.

Boston, J. (2000) ‘The Challenge of Evaluating Systemic Change: The Case of Public Management Reform’, International Public Management Journal 3(1): 23–46.

Braithwaite, J. (1993) ‘Strategic Management and Organisational Structure: Transformational Processes at Work in Hospitals’, Australian Health Review 16(4): 383–404.

Eoyang, G. H. and T. H. Berkas (1999) ‘Evaluating Performance in a Complex Adaptive System’, in M. R. Lissack and H. P. Gunz (eds) Managing Complexity in Organisations – A View in Many Directions. Westport, CT: Quorum Books.

Figueras, J. , R. Saltman and E. Mossialos (1997) Challenges in Evaluating Health Sector Reform: An Overview. Discussion Paper 8, May 1997. London: London School of Economics and Political Science.

Ham, C. (1997) ‘Reforming the New Zealand Health Reforms – Big Bang Gives Way to Incrementalism as Competition is Abandoned’, British Medical Journal 314: 1844–1845.

Hillman, K. (1998) ‘Restructuring Hospital Services’, Medical Journal of Australia 169: 239 239.

Kemmis, S. and R. McTaggart (1988) The Action Research Planner. Geelong: Deakin University.

Klein, R. (1998) ‘How Can Health Care Reform Be Evaluated?’, Journal of Health Services Research and Policy 3(3): 185–186.

10.

Morgan, D. L. (1988) Focus Groups as Qualitative Research. Portland, OR: Sage.

11.

Patton, M. Q. (1990) Qualitative Evaluation and Research Methods, 2nd edn. London: Sage.

12.

Pawson, R. and N. Tilley (1994) ‘What Works in Evaluation Research?’, British Journal of Criminology 34(3): 291–306.

13.

Pawson, R. and N. Tilley (1997) Realistic Evaluation. London: Sage.

14.

Pollitt, C. (1995) ‘Justification by Works or by Faith? – Evaluating the New Public Management’, Evaluation 1(2): 133–154.

15.

Rein, M. (1983) From Policy to Practice. New York: M.E Sharpe, Inc.

16.

Senghe, P. M. (1990) The Fifth Discipline – The Art and Practice of the Learning Organisation. London: Century Business.

17.

Yin, R. (1984) Case Study Research: Design and Methods. London: Sage.