A Dialogical,Story-Based Evaluation Tool: The Most Significant Change Technique

Abstract

The Most Significant Change (MSC) technique is a dialogical, story-based technique. Its primary purpose is to facilitate program improvement by focusing the direction of work towards explicitly valued directions and away from less valued directions. MSC can also make a contribution to summative evaluation through both its process and its outputs. The technique involves a form of continuous values inquiry whereby designated groups of stakeholders search for significant program outcomes and then deliberate on the value of these outcomes in a systematic and transparent manner. To date, MSC has largely been used for the evaluation of international development programs, after having been initially developed for the evaluation of a social development program in Bangladesh (Davies, 1996). This article provides an introduction to MSC and discusses its potential to add to the basket of choices for evaluating programs in developed economies. We provide an Australian case study and outline some of the strengths and weaknesses of the technique. We conclude that MSC can make an important contribution to evaluation practice. Its unusual methodology and outcomes make it ideal for use in combination with other techniques and approaches.

Get full access to this article

View all access options for this article.

References

Bennett, C. F. (1975, March/April). Up the hierarchy. Journal of Extension, 6-12.

Bennett, C. , & Rockwell, K. (1995). Targeting outcomes of program (TOP): An integrated approach to planning and evaluation. Washington: United States Department of Agriculture (NOTE: 2003. The most recent information is on the website: http://deal.unl.edu/TOP/).

Boje, D. M. (1991). The storytelling organization: A study of story performance in an office-supply firm. Administrative Science Quarterly, 36, 106-126.

Campbell, D. T. (1969). Variation, selection and retention in socio-cultural evolution. General Systems, 14, 69-85.

Costantino, T. E. , & Greene, J. C. (2003). Reflections on the use of narrative in evaluation. American Journal of Evaluation, 24, xx-yy.

Dart, J. J. (1999a). A story approach for monitoring change in an agricultural extension project. In Proceedings of the Association for qualitative research (AQR) international conference. Melbourne: AQR [online]: http://www.latrobe.edu.au/www/aqr/offer/papers/JDart.htm.

Dart, J. J. (1999b). The tale behind the performance story approach. Evaluation News and Comment, 8, 12-13.

Dart, J. J. (2000). Stories for change: A new model of evaluation for agricultural extension projects in Australia. Melbourne: PhD thesis, Institute of Land and Food Resources, University of Melbourne.

and in Impact assessment and project Appraisal, 16. No. 3, September 1998, 243-250.

10.

Davies, R. J. (1998). Order and diversity: Representing and assisting organisational learning in non-government aid organisations. PhD thesis. Swansea. UK: Centre for Development Studies [online]: www.swan.ac.uk/cds/rd/thesis.htm.

11.

Delbecq, A. , Van de Ven, A. , & Gustafson., D. (1975). Group techniques for program planning: A guide to nominal group and Delphi processes. WI: Briar Press Middleton.

12.

Guba, E. G. , & Lincoln, Y. S. (1989). Fourth generation evaluation. Newbury Park: Sage.

13.

Guijt, I. , Arevalo, M. , & Salsdores, K. (1998). Tracking change together. PLA Notes: Participatory Learning and Action, 31, 28-36.

14.

Henry, G. T. (2002). Choosing criteria to judge program success: A values inquiry. Evaluation, 8, 182-204.

15.

House, E. R. , & Howe, K. R. (1999). Values in evaluation and social research. Newbury Park: Sage.

16.

Jacobs, A. M. , Fivars, G. , Edwards, D. S. , & Fitzpatrick, R. (1978). Critical requirements for safe-effective nursing practice. Kansas City, MO: American Nurses’ Association.

17.

Kibel, B. M. (1999). Success stories as hard data: An introduction to results mapping. New York: Plenum.

18.

Mark, M. M. , Henry, G. T. , & Julnes, G. (1999). Toward an integrative framework for evaluation practice. American Journal of Evaluation, 20, 177-198.

19.

Mark, M. M. , Henry, G. T. , & Julnes, G. (2000). Evaluation: An integrated framework for understanding, guiding, and improving public and nonprofit policies and programs. San Francisco: Jossey-bass.

20.

McDonald, B. , Rogers, P. , & Kefford, B. (2003). Teaching people to fish? Building the evaluation capability of public sector organisations. Evaluation (in press).

21.

Nelson, R. R. , & Winter, S. G. (1982). An evolutionary theory of economic change. Cambridge, MA: Harvard University Press.

22.

Nichols, K. , & Kelly, S. (2002). Targeting outcomes of programs. [Review of the web site http://deal.unl.edu/TOP/, edited by Dr. Kay Robckwell of the University of Nebraska, Lincoln and Dr. Claude Bennett of the Cooperative State Research, Educatrion and Extension Service, USDA.] American Journal of Evaluation, 23, 505-509.

23.

Patton, M. Q. (1990). Qualitative evaluation and research methods. Newbury Park: Sage.

24.

Patton, M. Q. (1997). Utilization focused evaluation. Newbury Park: Sage.

25.

Riessman, C. K. (1993). Narrative analysis. Newbury Park: Sage

26.

Shaw, G. , Brown, R. , & Bromiley, P. (1998, May-June). Strategic stories. Harvard Business Review, 76, 41-50.

27.

Starr-Schneidkraut, N. , Cooper, M. D. , & Wilson S. R. (1997). Use of the Critical Incident Technique to Evaluate the Impact of MEDLINE. United States National Library of Medicine [online]: http://www.nlm.nih.gov/od/ope/cit.html [Accessed November 13, 2000].