The technical and the social: Assessing evaluation tools’ multiple functions and constraints

Abstract

What do tools do for evaluators? An affordances framework sheds light on the multiple ways that evaluators interact with tools. “Affordances” are ways that tools enable and constrain action, but the ways that tools do so depend on underlying social conditions. I apply this framework to the case of the Organization for Economic Cooperation and Development Assistance Committee criteria, which are widely used in international development evaluation. Through interviews and document analysis, I find that the DAC criteria offer technical and social affordances—they enable and constrain ways that evaluators navigate evaluation’s technical tasks, but they also enable and constrain ways that evaluators interact with others and draw symbolic boundaries around evaluation expertise. I also find these affordances are shaped by social conditions, like an evaluator’s perceptions or position in the organization. These findings suggest the importance of assessing evaluation tools’ multiple dimensions in their social contexts to understand tools’ role in evaluation work.