Integrating recovery-oriented and realistic evaluation principles into an evaluation of a Partners in Recovery programme

Abstract

Partners in Recovery (PIR) was an Australian government initiative designed to provide support and service linkage for individuals with complex needs living with severe and persistent mental illness. This article reports the external evaluation process and approach that was undertaken of the Gold Coast Partners in Recovery initiative between September and December 2015 regarding the achievement of PIR outcomes. The evaluation of this consortia-based initiative was framed using principles of realistic evaluation and recovery-oriented practice. Numerous evaluations of similar initiatives have recently been undertaken, each adopting different approaches and methods in accordance with local needs and expectations. The incorporation of realistic evaluation with recovery-oriented principles in this mixed methods research design, however, offers a unique perspective. This can be used to inform future developments in evaluative practice particularly in the area of recovery-oriented services and/or partnership-focused, capacity-building initiatives.

Keywords

consortium mental health partnership realist evaluation recovery

Get full access to this article

View all access options for this article.

References

Andresen

Oades

Caputi

(2003). The experience of recovery from schizophrenia: Towards an empirically validated stage model. Australian and New Zealand Journal of Psychiatry, 37(5), 586–594. https://doi.org/10.1046/j.1440-1614.2003.01234x

Astbury

(2013). Some reflections on Pawson’s Science of evaluation: A realist manifesto. Evaluation, 19(4), 383–401. https://doi.org/10.1177/1356389013505039

Asthana

Richardson

Halliday

(2002). Partnership working in public policy provision: A framework for evaluation. Social Policy and Administration, 36(7), 780–795. https://doi.org/10.1111/1467-9515.00317

Australasian Evaluation Society. (2019). Strategic priorities: 2019 to 2022. https://www.aes.asn.au/images/aes_strategicPlan2019_22_finalWeb.pdf

Australian Government. (2010). Australia’s National Standards for Mental Health Services. https://www1.health.gov.au/internet/main/publishing.nsf/Content/CFA833CB8C1AA178CA257BF0001E7520/$File/servst10v2.pdf

Australian Government. (2013). Partners in Recovery (PIR) coordinated support and flexible funding for people with severe, persistent mental illness and complex needs initiative: PIR evaluation framework. Department of Health.

Australian Health Ministers’ Advisory Council. (2013). National framework for recovery-oriented mental health services: Guide for practitioners and providers. http://www.health.gov.au/internet/main/publishing.nsf/content/67D17065514CF8E8CA257C1D00017A90/$File/recovgde.pdf

Babbie

(2013). The practice of social research. Wadsworth Cengage Learning.

Bailey

Koney

(2000). Strategic alliances among health and human services organizations: From affiliations to consolidations. SAGE.

10.

Burgess

Pirkis

Coombs

Rosen

(2010). Australian mental health outcomes and classification network: Review of recovery measures. http://www.mentalhealth.va.gov/communityproviders/docs/review_recovery_measures.pdf

11.

Cheverton

Janamian

(2016). The Partners in Recovery program: Mental health commissioning using value co-creation. Medical Journal of Australia, 204(7), S38. https://doi.org/10.5694/mja16.00124

12.

Comfort

Hoggarth

(2010). Practical guide to outcome evaluation. Jessica Kingsley Publishers.

13.

Commonwealth of Australia. (2013). A national framework for recovery-oriented mental health services. Guide for Practitioners and Providers. https://www1.health.gov.au/internet/main/publishing.nsf/Content/mental-pubs-n-recovgde

14.

Connecticut Department of Mental Health and Addiction Services. (2006). Practice guidelines for recovery-oriented behavioral health care. http://www.ct.gov/dmhas/lib/dmhas/publications/practiceguidelines.pdf

15.

Cooperrider

D. L.

Whitney

D. K.

(2005). Appreciative inquiry: A positive revolution in change (1st ed.). Berrett-Koehler.

16.

Council on Federal Financial Relations. (2011). Intergovernmental agreement on federal financial relations: Standing council on financial relations. http://www.federalfinancialrelations.gov.au/content/intergovernmental_agreements.aspx

17.

Creswell

J. W.

Plano Clark

V. L.

(2007). Designing and conducting mixed methods research. SAGE.

18.

Davidson

Rakfeldt

Strauss

J. S.

(2010). The roots of the recovery movement in psychiatry: Lessons learned. Wiley-Blackwell.

19.

Davidson

Roe

(2007). Recovery from versus recovery in serious mental illness: One strategy for lessening confusion plaguing recovery. Journal of Mental Health, 16(4), 459–470. https://doi.org/10.1080/09638230701482394

20.

Department of Health and Ageing. (2012a). Partners in recovery initiative information paper. Australian Government.

21.

Department of Health and Ageing. (2012b). Partners in Recovery (PIR) coordinated support and flexible funding for people with severe, persistent mental illness and complex needs initiative. Program guidelines for the engagement of PIR organisation 2012-13 to 2015-16. Australian Government.

22.

Dickinson

(2006). The evaluation of health and social care partnerships: An analysis of approaches and synthesis for the future. Health & Social Care in the Community, 14(5), 375–383. https://doi.org/10.1111/j.1365-2524.2006.00650.x

23.

Doi

Jepson

Cheyne

(2015). A realist evaluation of an antenatal programme to change drinking behaviour of pregnant women. Midwifery, 31(10), 965–972. https://doi.org/10.1016/j.midw.2015.06.007

24.

Dowling

Powell

Glendinning

(2004). Conceptualising successful partnerships. Health and Social Care in the Community, 12(4), 309–317. https://doi.org/10.111/j.1365-2524.2004.00500.x

25.

Flood

Minkler

Hennessey Lavery

Estrada

Falbe

(2015). The collective impact model and its potential for health promotion: Overview and case study of a healthy retail initiative in San Francisco. Health Education & Behavior, 42(5), 654–668. https://doi.org/10.1177/1090198115577372

26.

Frey

B. B.

Lohmeier

J. H.

Lee

S. W.

Tollefson

(2006). Measuring collaboration among grant partners. American Journal of Evaluation, 27(3), 383–392. https://doi.org/10.1177/1098214006290356

27.

Gajda

(2004). Utilizing collaboration theory to evaluate strategic alliances. American Journal of Evaluation, 25(1), 65–77. https://doi.org/10.1177/1098214004020500105

28.

Gold Coast Partners in Recovery. (2014a). Completion of Department of Health and Aging template for six month report [Unpublished paper].

29.

Gold Coast Partners in Recovery. (2014b). Gold Coast Mental Health: Mapping service needs for regional flexible funding investment considerations.

30.

Gold Coast Partners in Recovery. (2015). Reporting to Department of Health and Aging through completion of schedule 38 [Unpublished paper].

31.

Gilburt

Slade

Bird

Oduola

Craig

T. K. J.

(2013). Promoting recovery-oriented practice in mental health services: A quasi-experimental mixed-methods study. BMC Psychiatry, 13(1), 167–167. https://doi.org/10.1186/1471-244X-13-167

32.

Gold Coast Medical Local. (2012). Tendering document to Department of Health and Aging [Unpublished paper].

33.

Gomez-Bonnet

Thomas

(2015). A three-way approach to evaluating partnerships: Partnership survey, integration measure and social network analysis. Evaluation Journal of Australasia, 15(1), 28–37. http://vocational-rehab.com/wp-content/uploads/SuRGE-6_Evaluating-Partnerships.pdf

34.

Greenhalgh

Robert

MacFarlane

Bate

Kyriakidou

(2004). Diffusion of innovations in service organizations: Systematic review and recommendations. The Milbank Quarterly, 82(4), 581–629. https://doi.org/10.1111/j.0887-378X

35.

Gulliver

Morse

A. R.

Wilson

Sargent

Banfield

(2018). An Evaluation of a tailored care program for complex and persistent mental health problems: Partners In Recovery program. Evaluation and Program Planning, 68, 99–107.

36.

Halliday

Asthana

S. N. M.

Richardson

(2004). Evaluating partnership: The role of formal assessment tools. Evaluation, 10(3), 285–303. https://doi.org/10.1177/1356389004048279

37.

Hancock

Scanlan

J. N.

Gillespie

J. A.

Smith-Merry

Yen

(2018). Partners in Recovery program evaluation: Changes in unmet needs and recovery. Australian Health Review, 42(4), 445–452. https://doi.org/10.1071/AH17004

38.

Kania

Kramer

(2011). Collective impact. Stanford Social Innovation Review, 9(1), 36. https://ssir.org/articles/entry/collective_impact

39.

Kazi

M. A. F.

(2007). A reply to Eric Weir’s review of realistic evaluation in practice. Evaluation Journal of Australasia, 7(1), 51–54.

40.

Keast

Mandell

Brown

Woolcock

(2004). Network structures: Working differently and changing expectations. Public Administration Review, 64, 363–371. https://doi.org/10.1111/j.1540-6210.2004.00380.x

41.

Lakoff

Johnson

(1980). Metaphors we live by. University of Chicago Press.

42.

Lawson

(1985). Problems of modern European thought series: Reflexivity: The post-modern predicament. https://www.qualityresearchinternational.com/Harvey%20papers/Harvey%201985%20Review%20of%20Lawson.pdf

43.

Le Boutillier

Chevalier

Lawrence

Leamy

Bird

V. J.

Macpherson

. . . Slade

(2015). Staff understanding of recovery-orientated mental health practice: A systematic review and narrative synthesis. Implement Science, 10(1), 87. https://doi.org/10.1186/s13012-015-0275-4

44.

Legard

Keegan

Ward

(2003). In-depth interviews. In Ritchie

Lewis

(Eds.), Qualitative research practice: A guide for social science students and researchers (pp. 138–169). SAGE.

45.

Lofland

(1995). Analyzing social settings: A guide to qualitative observation and analysis. Wadsworth.

46.

Mason

(2002). Qualitative researching. SAGE.

47.

Mellish

(1999). Appreciative inquiry. Training Journal, (Nov), 12–14.

48.

Mental Health Association of Queensland. (2015). Participant Feedback Survey GCPIR (Unpublished).

49.

Miller

Crabtree

(2004). Depth interviewing. In Hesse-Biber

Leavy

(Eds.), Approaches to qualitative research: A reader on theory and practice (pp. 185–202). Oxford University Press.

50.

Mukova-Moses

(2017). A case for the advancement of utilisation of realist evaluation in international development: An Oxfam focused illustration. Evaluation Journal of Australasia, 17(2), 37–41.

51.

Palinkas

L. A.

Aarons

G. A.

Horwitz

Chamberlain

Hurlburt

Landsverk

(2011). Mixed method designs in implementation research. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 44–53. https://doi.org/10.1007/s10488-010-0314-z

52.

Pawson

Tilley

(2004). Realist evaluation. British Cabinet Office.

53.

Porter

O’Halloran

(2012). The use and limitation of realistic evaluation as a tool for evidence-based practice: A critical realist perspective. Nursing Inquiry, 19(1), 18–28. https://doi.org/10.111/j.1440-18800.2011.00551.x

54.

Punton

Vogel

Lloyd

(2016). Reflections from a realist evaluation in progress: Scaling ladders and stitching theory. Centre for Development Impact. http://www.ids.ac.uk/publication/reflectionsfrom-a-realist-evaluation-in-progress-scaling-ladders-andstitching-theory

55.

Rhodes

Brown

A. D.

(2005). Narrative, organizations and research. International Journal of Management Reviews, 7(3), 167–188. https://doi.org/10.1111/j.1468-2370.2005.00112.x

56.

Rickwood

(2006). Pathways of recovery: Preventing further episodes of mental illness. https://www.health.gov.au/internet/main/publishing.nsf/content/85A27F4558113287CA257BF00021207D/$File/mono.pdf

57.

Riggs

Block

Warr

Gibbs

(2014). Working better together: New approaches for understanding the value and challenges of organizational partnerships. Health Promotion International, 29(4), 780–793. https://doi.org/10.1093/heapro/dat022

58.

Rodgers

K. C.

Akintobi

Thompson

W. W.

Evans

Escoffery

Kegler

M. C.

(2014). A model for strengthening collaborative research capacity: Illustrations from the Atlanta Clinical Translational Science Institute. Health Education & Behavior, 41(3), 267–274. https://doi.org/10.1177/1090198113511815

59.

Slade

(2009). Rethink recovery series: 100 ways to support recovery. http://recoverylibrary.unimelb.edu.au/__data/assets/pdf_file/0005/1391270/100_ways_to_support_recovery.pdf

60.

Slade

Amering

Farkas

Hamilton

O’Hagan

Panther

Perkins

. . .Whitley

(2014). Uses and abuses of recovery: Implementing recovery-oriented practices in mental health systems. World Psychiatry, 13(1), 12–20. https://doi.org/10.1002/wps.20084

61.

Soh

D. S. B.

Martinov-Bennie

(2011). The internal audit function: Perceptions of internal audit roles, effectiveness and evaluation. Managerial Auditing Journal, 26(7), 605–622. https://doi.org/10.1108/02686901111151332

62.

Trankle

S. A.

Reath

(2019). Partners in Recovery: An early phase evaluation of an Australian mental health initiative using program logic and thematic analysis. BMC Health Services Research, 19, Article 524. https://doi.org/10.1186/s12913-019-4360-2

63.

Tsui

M. S.

(1998). Organizations as elephants: A Chinese metaphor or a metatheory? International Journal of Management, 15(2), 151–160.

64.

URBIS. (2014). Partners in recovery evaluation and monitoring project (Final annual report 2013-14). Department of Health. https://www1.health.gov.au/internet/main/publishing.nsf/Content/1FD43EB98AA86C7BCA25802800127C1E/$File/annual14.pdf

65.

VicHealth. (2011). The partnership analysis tool. https://www.vichealth.vic.gov.au/search/the-partnerships-analysis-tool

66.

Weir

(2007). Realist evaluation in practice: Health and social work [Book Review]. Evaluation Journal of Australasia, 7(1), 48–51.