Selecting and justifying relevant criteria is critical to defensible evaluative reasoning, yet there is little advice in the literature to guide practitioners in criteria development. In this article, we introduce a framework that draws on normative ethical perspectives to systematically identify and justify relevant dimensions of value for a public sector–funded program. We illustrate, through an example taken from a recent research project, how evaluators might use the framework to be more thoughtful about identifying and selecting the values against which judgements are made. The potential strengths and limitations of the framework are then discussed.
ChelimskyE. (1995). The political environment of evaluation and what it means for the development of the field: Evaluation for a new century: A global perspective. American Journal of Evaluation, 16, 215–225.
DavidsonJ. E. (2005). Evaluation methodology basics. SAGE.
8.
FootP.MontefioreA. R. (1961). Goodness and choice. Proceedings of the Aristotelian Society, 35, 45–80.
9.
FournierD. M. (1993). Reasoning in evaluation: A distinction between general and working logic. Syracuse University.
10.
FournierD. M. (1995). Establishing evaluative conclusions: A distinction between general and working logic. New Directions for Evaluation, 68, 15–32.
11.
GreeneJ. C.DestefanoL.BurgonH.HallJ. (2006). An educative, values-engaged approach to evaluating STEM educational programs. New Directions for Evaluation, 2006, 53–71.
12.
GreeneJ. C. (2011). The Construct(Ion) of validity as argument. In ChenH. T.DonaldsonS. I.MarkM. M. (Eds.), Advancing validity in outcome evaluation: Theory and practice (Vol. 130, pp. 81–91). New Directions for Evaluation.
13.
GullicksonA. M.HannumK. M. (2019). Making values explicit in evaluation practice. Evaluation Journal of Australasia, 19, 1–17.
14.
HareR. (1952). The language of morals. Oxford University Press.
15.
HarperT. L.SteinS. M. (1992). The centrality of normative ethical theory to contemporary planning theory. Journal of Planning Education and Research, 11(2), 105–116.
16.
HouseE. R. (1980). Evaluating with validity. SAGE.
17.
HouseE. R. (1995). Putting things together coherently: Logic and justice. New Directions for Evaluation, 1995(68), 33–48.
18.
HouseE. R.HoweK. R. (1999). Values in evaluation and social research. SAGE.
19.
JulnesG. (2012). Developing policies to support valuing in the public interest. New Directions for Evaluation, 133, 109–129.
20.
KristjanssonK. (2006). Emulation and the use of role models in moral education. Journal of Moral Education, 35(1), 37–49.
21.
MartensK. S. R. (2018). How program evaluators use and learn to use rubrics to make evaluative reasoning explicit. Evaluation and Program Planning, 69(14), 25–32.
NewmanD. L.BrownR. D. (1996). Applied ethics for program evaluation. SAGE.
24.
NoddingsN. (1984). Caring: A feminine approach to ethics & moral education. University of California Press.
25.
NunnsH.PeaceR.WittenK. (2015). Evaluative reasoning in public-sector evaluation in Aotearoa New Zealand: How are we doing?Evaluation Matters – He Take Tō Te Aromatawai, 1, 137–163.
PopkinR. H.StrollA. (1993). Philosophy made simple. Three Rivers Press.
28.
RawlsJ. (1971). A theory of justice. Harvard University Press
29.
RescherN. (1969). Introduction to value theory. Prentice Hall.
30.
RoordaM. (in press). Developing defensible criteria for public sector evaluations. University of Melbourne.
31.
RyanB. (2003). ‘Death by evaluation?’ Reflections on monitoring and evaluation in Australia and New Zealand. Evaluation Journal of Australasia, 3(1), 6–16.
32.
SchwandtT. A. (2008). Educating for intelligent belief in evaluation. American Journal of Evaluation, 29, 139–150.
33.
ScrivenM. (1982). The logic of evaluation. Edgepress.
34.
ScrivenM. (1993). Hard-won lessons in program evaluation. Jossey-Bass.
35.
ScrivenM. (1995). The logic of evaluation and evaluation practice. New Directions for Evaluation, 1995(68), 49–70.
36.
ScrivenM. (1996). The theory behind practical evaluation. Evaluation, 2, 393–404.
37.
ScrivenM. (2005). Logic of evaluation: Types of value claim. In MathisonS. (Ed.), Encyclopedia of evaluation (pp. 235–238). SAGE.
38.
ScrivenM. (2012). The logic of valuing. In JulnesG. (Ed.), Promoting valuation in the public interest: Informing policies for judging value in evaluation (Vol. 133, pp. 17–28). New Directions for Evaluation.
