Sage Journals: Discover world-class research

Abstract

Evaluability assessment focuses on the readiness of organisations to carry out evaluations. Scholars argue that evaluability assessment needs to focus on internal evaluation systems and tools and their capability to measure programmes and services reliably and credibly. Even so, literature on best practice guidelines on evaluability assessment within the context of the not-for-profit sector appears to be rare. We seek to begin to fill this gap by presenting lessons learned from Ngala, Western Australia, when we reviewed the organisation’s evaluation practice and culture in 2018/2019. The Service Model and Outcomes Measurement Audit project assessed the extent to which service models within Ngala aligned with the organisation’s standardised service model and individual service contracts, as well as consistency of outcomes, data collection and reporting practices. Insights obtained from the project and their implications for evaluability assessment practice are discussed.

Keywords

evaluability assessment evaluation audits evaluation reviews evaluation systems

Get full access to this article

View all access options for this article.

References

APB Consultants. (n.d.). Supplier audit checklist. http://isoconsultantpune.com/wp-content/uploads/2014/08/Supplier_Audit_Checklist-.pdf

Australian Government, Department of Social Services. (2014). Communities for children facilitating partner – Operational guidelines. https://www.dss.gov.au/our-responsibilities/families-and-children/programs-services/family-support-program/communities-for-children-facilitating-partner-operational-guidelines

Australian National Audit Office. (2018). The audit process. https://www.anao.gov.au/about/audit-process

Bennett

Hauck

Bindahneem

Banham

Owens

Priddis

L. E.

Wells

Sinclair

Shields

(2012). The development of an interdisciplinary research agenda at Ngala: An innovative case study. Journal Neonatal Paediatric & Child Health Nursing, 15, 20–25.

Boone

Minore

Katt

Kinch

(2009). Strength through sharing: Interdisciplinary teamwork in providing health and social services to northern native communities. Canadian Journal of Community Mental Health, 16(2), 15–28.

Bourgeois

Cousins

J. B.

(2013). Understanding dimensions of organizational evaluation capacity. American Journal of Evaluation, 34(3), 299–319.

Brunner

Craig

Watson

(2019). Evaluability assessment: An application in a complex community improvement setting. Evaluation, 25(3), 349–365. https://doi.org/10.1177/1356389019852126

Conley-Tyler

(2005). A fundamental choice: Internal or external evaluation? Evaluation Journal of Australasia, 4(1–2), 3–11.

Dahler-Larsen

(2012). Evaluation as a situational or a universal good? Scandinavian Journal of Public Administration, 16(3), 29–46.

10.

Davies

(2013). Planning evaluability assessments: A synthesis of the literature with recommendations (Working Paper 40). Department for International Development. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/248656/wp40-planning-eval-assessments.pdf

11.

Davies

Payne

(2015). Evaluability assessment: Reflections on a review of the literature. Evaluation, 21(2), 216–231. https://doi.org/10.1177/1356389015577465

12.

Demuijnck

Fasterling

(2016). The social license to operate. Journal of Business Ethics, 136(4), 675–685. https://doi.org/10.1007/s10551-015-2976-7

13.

Fitzpatrick

J. L.

Sanders

J. R.

Worthen

B. R.

(2011). Program evaluation: Alternative approaches and practical guidelines. Pearson.

14.

Greve

Broder

Erdfelder

(2013). Result-blind peer reviews and editorial decisions: A missing pillar of scientific culture. European Psychologist, 18(4), 286–294. https://doi.org/10.1027/1016-9040/a000144

15.

Hare

Guetterman

(2014). Evaluability assessment: Clarifying organizational support and data availability. Journal of Multidisciplinary Evaluation, 10(23), 9–25. https://journals.sfu.ca/jmde/index.php/jmde_1/article/view/395

16.

HIPPY Australia. (n.d.). Benefits for children. https://hippyaustralia.bsl.org.au/benefits/children/

17.

Holvoet

Esbroeck

Inberg

Popelier

Peeters

Verhofstadt

(2018). To evaluate or not: Evaluability study of 40 interventions of Belgian development cooperation. Evaluation and Program Planning, 67, 189–199. https://doi.org/10.1016/j.evalprogplan.2017.12.005

18.

International Organization for Standardization. (2015). Quality management systems: Fundamentals and vocabulary. https://www.iso.org/standard/45481.html

19.

King

J. A.

(2007). Developing evaluation capacity through process use. New Directions for Evaluation, 2007(116), 45–59.

20.

Maxwell

J. C.

(2007). The 21 irrefutable laws of leadership: Follow them and people will follow you. HarperCollins Leadership.

21.

Mayne

(2010). Building an evaluative culture: The key to effective evaluation and results management. The Canadian Journal of Program Evaluation, 24(2), 1–30.

22.

McCoy

Rose

Connolly

(2013). Developing evaluation cultures in human service organisations. Evaluation Journal of Australasia, 13(1), 15–20.

23.

McDavid

J. C.

Huse

Hawthorn

L. R.

(2018). Program evaluation and performance measurement: An introduction to practice. SAGE.

24.

Owen

J. M.

(2006). Program evaluation: Forms and approaches (3rd ed.). SAGE.

25.

Patton

M. Q.

(2008). Utilization-focused evaluation. SAGE.

26.

Preskill

Boyle

(2008). A multidisciplinary model of evaluation capacity building. American Journal of Evaluation, 29(4), 443–459. https://doi.org/10.1177/1098214008324182

27.

Schwarzman

Bauman

Gabbe

B. J.

Rissel

Shilton

Smith

B. J.

(2019). Understanding the factors that influence health promotion evaluation: The development and validation of the evaluation practice analysis survey. Evaluation and Program Planning, 74, 76–83. https://doi.org/10.1016/j.evalprogplan.2019.03.002

28.

Smith

M. F.

(2013). Evaluability assessment: A practical approach (Vol. 26). Springer Science & Business Media.

29.

Stewart

(2014). Developing a culture of evaluation and research (CFCA Paper No. 28). Australian Institute of Family Studies. https://aifs.gov.au/cfca/publications/developing-culture-evaluation-and-research

30.

Strange

Bennett

Tait

Hauck

(2019). A qualitative evaluation of a Young Parents Program (YPP)—Parent and facilitator perspectives. Health Promotion Journal of Australia, 30(3), 402–412. https://doi.org/10.1002/hpja.228

31.

Trevisan

M. S.

Huang

Y. M.

(2003). Evaluability assessment: A primer. Practical Assessment, Research & Evaluation, 8, 20. https://doi.org/10.7275/30ww-jr45

32.

Trevisan

M. S.

Walser

T. M.

(2014). Evaluability assessment: Improving evaluation quality and use. SAGE.

33.

Volkov

King

J. A.

(2007). A checklist for building organizational evaluation capacity. The Evaluation Center, Western Michigan University.

34.

Watts

B. R.

Washington

H. M.

(2016). Adaptation and use of a five-task model for evaluability assessment. Journal of Multidisciplinary Evaluation, 12(27), 46–63. https://journals.sfu.ca/jmde/index.php/jmde_1/article/view/460

35.

Wilburn

K. M.

Wilburn

(2011). Achieving social license to operate using stakeholder theory. Journal of International Business Ethics, 4(2), 3–16. https://pdfs.semanticscholar.org/95cd/57fb1129d38a57de09b9b96a132854d8df11.pdf?_ga=2.164219135.1341600239.1589354667-293731396.1589354667

Evaluating our evaluability: Lessons from Ngala,Western Australia

Abstract

Keywords

Get full access to this article

References