This article presents a discourse analytic study of how the concept of impartiality is socially constructed by members of the development aid community through an examination of linguistic traits and patterns within (a) inter- and intraorganizational interactions and (b) relevant aid evaluation policy documents. A qualitative analysis of unstructured and semistructured interviews with development professionals in Japan and a content analysis of relevant evaluation policies and documents have revealed observable evidence of distinct institutional practices of impartiality via project evaluations. Most notably, in contrast to a notion of evaluator impartiality that is strengthened through the principle of independence, development professionals in Japan perceive evaluations (‘hyouka’) to be a fundamentally hierarchical construct in which evaluator impartiality can be strengthened or legitimized through authoritative or hierarchical means.
CamfieldL.DuvendackM. (2014). Impact evaluation—Are we ‘off the gold standard?’European Journal of Development Research, 26, 1–11.
8.
ChelimskyE. (1995). Where we stand today in the practice of evaluation: Some reflections. Knowledge and Policy, 8, 8–19.
9.
ChelimskyE. (1997). The political environment of evaluation and what it means for the development of the field. In ChelimskyE.ShadishW. (Eds.), Evaluation for the 21st century: A handbook (pp. 53–68). Thousand Oaks, CA: Sage.
10.
Conley-TylerM. (2005). A fundamental choice: Internal or external evaluation?Evaluation Journal of Australasia, 4, 3.
11.
CousinsJ. B.EarlL. M. (1992). The case for participatory evaluation. Educational Evaluation and Policy Analysis, 14, 397–418.
12.
CullenA. E.CorynC. L.RughJ. (2011). The politics and consequences of including stakeholders in international development evaluation. American Journal of Evaluation, 32, 345–361.
13.
DattaL. E. (1999). The ethics of evaluation neutrality and advocacy. New Directions for Evaluation, 1999, 77–88.
14.
DattaL. E. (2000). Seriously seeking fairness: Strategies for crafting non-partisan evaluations in a partisan world. American Journal of Evaluation, 21, 1–14.
FillenbaumS.RapoportA. (1974). Verbs of judging, judged: A case study. Journal of Verbal Learning and Verbal Behavior, 13, 54–62.
17.
FillmoreC. J. (1969). Verbs of judging: An exercise in semantic description. Research on Language & Social Interaction, 1, 91–117.
18.
FishmanD. B. (1992). Postmodernism comes to program evaluation: A critical review of Guba and Lincoln’s fourth generation evaluation. Evaluation and Program Planning, 15, 263–270.
19.
ForestiM.ArcherC.O’NeilT.LonghurstR. (2007). A comparative study of evaluation policies and practices in development agencies. Report for AFD Evaluation Department. London, England: Overseas Development Institute and Agence Française de Développement.
20.
FriedmanM. (1989). The impracticality of impartiality (Eighty-Sixth Annual Meeting American Philosophical Association, Eastern Division). The Journal of Philosophy, 86, 645–656.
21.
GeeJ. P. (2004). Discourse analysis: What makes it critical? In RogersR. (Ed.), An introduction to critical discourse analysis in education (pp. 19–50). Mahwah, NJ: Lawrence Erlbaum.
22.
GlaserB.StraussA. (1967). The discovery of grounded theory. New York, NY: Aldin.
23.
GreeneJ. C. (1997). Evaluation as advocacy. American Journal of Evaluation, 18, 25–35.
24.
GubaE. G.LincolnY. S. (1989). Fourth generation evaluation. Beverly Hills: Sage.
25.
GuijtI.RocheC. (2014). Does impact evaluation in development matter? Well, it depends what it’s for!European Journal of Development Research, 26, 46–54.
O’NeillT. (1995). Implementation frailties of Guba and Lincoln’s fourth generation evaluation theory. Studies in Educational Evaluation, 21, 5–21.
37.
Organisation for Economic Co-operation and Development. (1991). OECD DAC principles for the evaluation of development assistance. Paris, France: Author. Retrieved fromhttp://www.oecd.org/development/evaluation/2755284.pdf
Organisation for Economic Co-operation and Development. (2010b, 6). Evaluating development co-operation: Summary of key norms and standards. Paris, France: Author. Retrieved fromhttp://www.oecd.org/dac/evaluation/dcdndep/41612905.pdf
42.
Organisation for Economic Co-operation and Development. (2010c, 12). Evaluation in development agencies—Better aid. Paris, France: Author. Retrieved fromhttp://dx.doi.org/10.1787/9789264094857-en
PattonM. Q.GornickJ. (2011). Evaluation of the Phase 2 evaluation of the Paris declaration: An independent review of strengths, weaknesses, and lessons. Saint Paul, MN: Utilization-Focused Evaluation. Retrieved fromhttp://www.oecd.org/dac/evaluation/dcdndep/48620425.pdf
45.
PhillipsN.HardyC.LawrenceT. (2004). Discourse and institutions. Academy of Management Review, 29, 635–652.
46.
PicciottoR. (2005). The evaluation of policy coherence for development. Evaluation, 11, 311–330.
47.
PicciottoR. (2007). The new environment for development evaluation. American Journal of Evaluation, 28, 509–521.
48.
PotterJ. (2012). Discursive psychology and discourse analysis. In GeeJ. P.HandfordM. (Eds.), The Routledge handbook of discourse analysis (pp. 104–119). New York, NY: Routledge.
49.
RoschE. (1978). Principles of categorization. In RoachE.LloydB. (Eds.), Cognition and categorization (pp. 27–48). Hillsdale, NJ: Lawrence Erlbaum.
ScrivenM. (1997). Truth and objectivity in evaluation. In ChelimskyE.ShadishW. (Eds.), Evaluation for the 21st Century: A handbook (pp. 477–500). Thousand Oaks, CA: Sage.
52.
SeminG. R.FiedlerK. (1988). The cognitive functions of linguistic categories in describing persons: Social cognition and language. Journal of Personality and Social Psychology, 54, 558–568.
53.
SenA. (2002). Open and closed impartiality. The Journal of Philosophy, 99, 445–469.
TalmyL. (1983). How language structures space. In PickH. L.AcredeloL. P. (Eds.), Spatial orientation—Theory, research, and application (pp. 225–282). New York, NY: Plenum Press.
U.S. Agency for International Development. (2013). Evaluation at USAID. November 2013. Washington, DC: Office of Learning, Evaluation and Research, USAID Bureau for Policy, Planning and Learning.
58.
United Nations Office of the High Commissioner for Human Rights. (2003). Human Rights in the Administration of Justice: A Manual on Human Rights for Judges, Prosecutors and Lawyers. Professional Training Series No. 9, In Cooperation with the International Bar Association. New York, NY: United Nations Publication.
59.
WhiteH. (2013). An introduction to the use of randomised control trials to evaluate development interventions. Journal of Development Effectiveness, 5, 30–49.
