Quality Assessment Frameworks (QAFs) are standardized templates that include pre-defined criteria, open questions, or a combination to assess the quality of evaluations. They support organizations to limit risks associated with poor-quality evaluation. Qualitative research focused on four QAFs shows that five dimensions of quality are assessed within the frameworks that include pre-defined criteria. These are aligned with the criteria that audiences of evaluation identify as important to assessing quality: substantive findings, robust and appropriate methodology, accessibility, inclusive processes and analysis of wider systems. Consequently, these QAFs are likely to support critical assessment of the main dimensions of quality that are relevant to audiences of evaluation. Audiences also note synergies between these five dimensions of quality and appropriate contextualization as crucial to assessing quality. Consequently, the structure and categorization of quality within QAFs which only include pre-defined criteria risks limiting reviewers’ ability to reflect on these synergies and contextualize assessments.
CooksyL. J.CaracelliV. J. (2009). Meta-evaluation in practice: Selection and application of criteria. Journal of Multi-Disciplinary Evaluation, 6(11), 1–15.
5.
Dahler-LarsenP. (2014). Global inscription devices and the politics of education. Knowledge Cultures, 2(4), 90–109.
6.
Dahler-LarsenP. (2019). Quality: From Plato to performance. Cham: Palgrave MacMillan. ISB 978-3-030-10392-7
7.
DonaldsonS.PattonM. Q.FettermanD. M.ScrivenM. (2010). The 2009 Claremont debates: The promise and pitfalls of utilization-focused and empowerment evaluation. Journal of Multi-Disciplinary Evaluation, 6(13), 15–57.
GreeneJ. C. (1992). A case study of evaluation auditing as meta-evaluation. Evaluation and Program Planning, 15(1), 71–74.
10.
LincolnY.GubaE. G. (1985). Naturalistic inquiry. Newbury Park, CA: Sage Publications.
11.
McCoyM.HargieO. D. W. (2001). Evaluating evaluation: Implications for assessing quality. International Journal of Health Care Quality Assurance, 14(17), 317–327.
NowellL. S.NorrisJ. M.WhiteD. E.MoulesN. J. (2017). Thematic analysis: Striving to meet the trustworthiness criteria. International Journal of Qualitative Method, 16(1), 13.
PerrinB. (2009). Review of the quality of DFID evaluations: A delicate balancing act. London: Independent Advisory Committee on Development Impact (IACDI).
19.
PirsigR. (1991). Zen and the art of motorcycle maintenance: An inquiry into values. London: Vintage Classics.
20.
RaifmanJ. G.LamF.KellerJ. M.RadunskyA.SavedoffW. (2017). Evaluating evaluations: Assessing the quality of aid agency evaluations in global health working. Paper 461. Washington DC: Centre for Global Development.
21.
SchwandtT. A. (2002). Evaluation practice reconsidered. New York: Peter Lang.
ShermanL. W.GottfredsonD. C.MacKenzieD. L.EckJ.ReuterP.BushwayS. D. (1997). Preventing crime: What works, what doesn’t, and what’s promising. Washington, DC: National Institute of Justice, US Department of Justice.
27.
StufflebeamD. L. (2001). The meta-evaluation imperative. American Journal of Evaluation, 22(2), 183–209.
WampoldB.AhnH.KimD. (2012). Meta-analysis in the social sciences. Asia Pacific Education Review, 1, 67–74.
34.
YarboroughD.ShulhaL. M.HopsonR. K.CaruthersF.A. (2010). The program evaluation standards: A guide for evaluators and evaluation users (3rd ed.). Thousand Oaks, CA: Cornwin Press.
