Closing the Gap through Respect,Relevance,Reciprocity and Responsibility: Issues in the Evaluation of Programs for Indigenous Communities in Australia

Abstract

This article outlines key considerations and issues inherent in undertaking effective evaluations of programs developed for Indigenous Australians, written from the perspective of a non-Indigenous evaluator.

The considerations identified include a number of professional practice areas, namely the need for evaluators to: operate with an understanding and appreciation of the historical and systemic context experienced by Indigenous Australians; work from the basis of a solid code of ethics and practice standards; and operate with high levels of cultural sensitivity and the ability to appreciate Indigenous world views and differences. Another central consideration identified in the article is that evaluators need to hold a commitment to produce useful and useable evaluation findings that can inform future program design and social policy in relation to improving the health and wellbeing of Indigenous Australians.

The article develops a number of principles for evaluators to follow in evaluating programs developed for Indigenous Australians. These include: having respect for the importance of historical, socioeconomic and psychological context; commitment to ensuring relevance in methodologies and approaches used; reciprocity in considering the benefits for participating Indigenous communities; and responsibility in undertaking effective communication and consultation.

While most evaluators would agree with the above principles and values, the latter can be more challenging to apply in practice and examples of their application to evaluation, and the inherent challenges in the Indigenous context, are described. Such challenges include: operating within the restrictions of government silos with difficulties portraying the complex interrelationships that reflect the lives of Indigenous Australians; reaching consensus on evaluation design and implementation; working within restricted time frames with lack of opportunity to appreciate and reflect the unique differences among Indigenous communities across Australia; limitations inherent in engaging a broad and representative group of Indigenous stakeholders; and lack of control by the evaluator over the dissemination and influence of evaluation findings and results that link with subsequent decision-making processes.

Get full access to this article

View all access options for this article.

References

ABS 2006, Population distribution, Aboriginal and Torres Strait Islander Australians, 2006, cat no. 4705.0, Australian Bureau of Statistics, Canberra, viewed 16 March 2012, <http://www.abs.gov.au/ausstats/abs@.nsf/mf/4705.0>.

AIATSIS 2002, Guidelines for ethical research in Indigenous studies, Australian Institute of Aboriginal and Torres Strait Islander Studies, Canberra.

Berends

, & Roberts

2003, ‘Evaluation standards and their application to Indigenous programs in Victoria, Australia’, Evaluation Journal of Australasia, vol. 3, no. 2, pp. 54–59.

Closing the Gap Clearinghouse (AIHW, AIFS) 2012, What works to overcome Indigenous disadvantage: key learnings and gaps in the evidence 2010-11, produced for the Closing the Gap Clearinghouse, Australian Institute of Health and Welfare, Canberra and Australian Institute of Family Studies, Melbourne.

Evans

, Hole

, Berg

, Hutchinson

, & Sookraj

2009, ‘Common insights, differing methodologies: towards a fusion of Indigenous methodologies, participatory action research, and White studies, in an urban Aboriginal research agenda’, Qualitative Inquiry, vol. 15, no. 5, pp. 893–910.

Frierson

, Hood

, & Hughes

G.B.

2002, ‘Strategies that address culturally responsive evaluations’, in Frechtling

, The 2002 user-friendly handbook for project evaluation, National Science Foundation, Arlington, Virginia.

Kusek

J.Z.

, & Rist

R.C.

2004, Ten steps to a results-based monitoring and evaluation system, The World Bank, Washington, DC.

McKay

2007, How to build M&E systems to support better government, World Bank, Washington, DC.

NHMRC 2003a, Values and ethics: guidelines for ethical conduct in Aboriginal and Torres Strait Islander health research, National Health and Medical Research Council, Canberra.

10.

NHMRC 2003b, Keeping research on track: a guide for Aboriginal and Torres Strait Islander peoples about heath research ethics, National Health and Medical Research Council, Canberra.

11.

Office of Evaluation and Audit (Indigenous Programs) 2007, Evaluation of the FaCSIA family violence programs: Family Violence Regional Activities Program – Family Violence Partnership Program (FVRAP–FVPP, Office of Evaluation and Audit (Indigenous Programs), Department of Finance and Deregulation, Canberra.

12.

Pawson

, & Tilley

1997, Realistic evaluation, Sage, London.

13.

Pyett

, Waples-Crowe

, & van der Sterren

2009, ‘Engaging with Aboriginal communities in an urban context: some practical suggestions for public health researchers’, Australian and New Zealand Journal of Public Health, vol. 33, no. 1, pp. 51–54.

14.

Richard

, & LaFrance

2006, ‘Indigenous evaluation: respecting and empowering Indigenous knowledge’, Tribal College Journal, vol. 19, no. 2. pp. 32–35.

15.

Scougall

2008, ‘Lessons learned about strengthening indigenous families and communities’, occasional paper no. 19, Stronger Families and Communities Strategy 2000-2004, Department of Families, Housing, Community Services and Indigenous Affairs, Canberra.

16.

Taylor

2003, ‘An Indigenous perspective on evaluations in the intercultural context: how far can one throw a Moree boomerang?’, Evaluation Journal of Australasia, vol. 3, no. 2, pp. 44–52.

17.

Wehipeihana

2008, ‘Indigenous evaluation: a strategic objective of the Australasian Evaluation Society’. Evaluation Journal of Australasia, vol. 8, no. 1, pp. 40–44.