This study analyzes evaluation influence theories to understand their unified contributions to a conceptual framework for research on evaluation influence in non-western contexts. Specifically, these theories are analyzed according to their usefulness for interpreting the consequences of the Annual Status of Education Report (ASER)—a cyclical, large-scale educational monitoring and evaluation effort in India. An ethnographic case study of ASER facilitates an empirical investigation of the field's evaluation influence theories and uncovers a new source of influence—evaluation design. The study's findings challenge a common belief in the field: that evaluation process and results are the two fountainheads of evaluation impact. Design, as a distinct third source of influence, gives rise to a novel form of evaluation consequence—design diffusion. The phenomenon of design diffusion illuminates why ASER's model for evaluation has been adopted by several other Global South nations.
AlkinM. C.ChristieC. A. (2019). Theorists’ models in action: A second look. In ChristieC. A.AlkinM. C. (Eds.), Theorists’ models in action: A second Look. New directions for evaluation (Vol. 163, pp. 11–18). Thousand Oaks: Sage.
5.
AlkinM. C.KingJ. A. (2016). The historical development of evaluation use. American Journal of Evaluation, 37, 568–579.
6.
AlkinM. C.KingJ. A. (2017). Definitions and factors associated with evaluation use and misuse. American Journal of Evaluation, 38, 434–450.
7.
ASER Centre (2014). Quality control framework. New Delhi: ASER Centre.
8.
ASER Centre (2015). Annual Status of Education Report (Rural) 2014. New Delhi, India.
9.
AshleyS. R. (2009). Innovation diffusion: Implications for evaluation. In J. M. Ottoson & P. Hawe (Eds.), Knowledge utilization, diffusion, implementation, transfer, and translation: Implications for evaluation. New Directions for Evaluation, 124, 35–45.
10.
BarrettA. (2011). An education millennium development goal for quality: Complexity and democracy. Compare, 41(1), 145–148.
11.
BykerE.BanerjeeA. (2015). Evidence for action: Translating field research into a large scale assessment. Current Issues in Comparative Education Journal, 18, 42–53.
12.
ChilisaB.MajorT. E.GaotlhobogweM.MokgolodiH. (2016). Decolonizing and indigenizing evaluation practice in Africa: Toward african relational evaluation approaches. Canadian Journal of Program Evaluation, 30(3), 313–328.
13.
ChouinardJ. (2013). The case for participatory evaluation in an era of accountability. American Journal of Evaluation, 34(2), 237–253.
14.
ChouinardJ.CousinsJ. (2009). A review and synthesis of current research on cross-cultural evaluation. American Journal of Evaluation, 30(4), 457–494.
15.
ChristieC.AlkinM. (2013). An evaluation theory tree. In AlkinM. (Ed.), Evaluation roots: A wider perspective of theorists’ views and influences, second edition (pp. 11–57). Thousand Oaks: Sage.
16.
ClarkeA.DawsonR. (1999). Evaluation research: An introduction to principles, methods, and practice. SAGE.
17.
CorynC.OzekiS.WilsonL.GreenmanG.SchröterD.HobsonK., … VoA. (2016). Does research on evaluation matter? Findings from a survey of American evaluation association members and prominent evaluation theorists and scholars. American Journal of Evaluation, 37(2), 159–173.
18.
CronbachL. (1982). Designing evaluations of educational and social programs. San Francisco: Jossey-Bass.
19.
GildemynM. (2014). Understanding the influence of independent civil society monitoring and evaluation at the district level. American Journal of Evaluation, 35(4), 507–524.
20.
GoodnightM.BobdeS. (2018). Missing children in educational research: Investigating school-based versus household-based assessments in India. Comparative Education, 54(2), 225–249.
21.
GoodnightM. (2017b). Critical race theory in India: Theory translation and the analysis of social identities and discrimination in Indian schooling. Compare: A Journal of Comparative and International Education, 47(5), 665–683.
22.
GoodnightM. (2017a). Learning to “measure what affects their lives”: Ethnography of a citizen-engaged evaluation of primary education in India(10599943). [Doctoral Dissertation, University of California Los Angeles]. ProQuest.
23.
Government of India (2014). Chapter 13: Human development, economic survey 2013–2014 (pp. 230–254). New Delhi: Ministry of Finance.
24.
GreeneJ. C. (1988). Stakeholder participation and utilization in program evaluation. Evaluation Review, 12(2), 91–116.
25.
GreeneJ. C. (1997). Evaluation as advocacy. Evaluation Practice, 18(1), 25–35.
26.
HallM. (2004, November). Persuasive language, responsive design: Towards broader evaluation utilization. In K. Kirkhart (Chair), What is the nature of evaluation use/ influence? Panel at the American Evaluation Association conference, Atlanta, GA.
27.
HenryG.MarkM. (2003b). Toward an agenda for research on evaluation. In C. M. Christie (Ed.), The Practice-Theory Relationship in Evaluation. New Directions for Evaluation, 97, 69–80.
28.
HenryG.MarkM. (2003a). Beyond use: Understanding evaluation's Influence on attitudes and actions. The American Journal of Evaluation, 24(3), 293–314.
29.
HoodS. (2004). A journey to understand the role of culture in program evaluation: Snapshots and personal reflections of one african American evaluator. New Directions for Evaluation, 102, 21–37.
30.
HopsonR. (2009). Reclaiming knowledge at the margins: Culturally responsive evaluation in the current evaluation moment. In RyanK.CousinsJ. (Eds.), The SAGE international handbook of educational evaluation (pp. 429–446). Thousand Oaks: Sage.
31.
HouseE.HoweK. (2000). Deliberative democratic evaluation. New Directions for Evaluation, 85(3), 3–12.
32.
JohnsonR. B. (1998). Toward a theoretical model of evaluation utilization. Evaluation and Program Planning, 21(1), 93–110.
33.
KingJ. (2016). Taking what action for change? Dare evaluation build a new social order?Visitor Studies, 19(1), 3–11.
34.
KingJ. A.AlkinM. C. (2019). The centrality of use: Theories of evaluation use and influence and thoughts on the first 50 years of use research. American Journal of Evaluation, 40(3), 431–458.
35.
KirkhartK. (1995). Seeking multicultural validity: A postcard from the road. Evaluation Practice, 16(1), 1–12.
36.
KirkhartK. (2000). Reconceptualizing evaluation use: An integrated theory of influence. In V. J. Caracelli & H. Preskill (Eds.), The Expanding Scope of evaluation Use, New Directions for Evaluation, 88, 5–23.
37.
KirkhartK. (2010). Eyes on the prize: Multicultural validity and evaluation theory. American Journal of Evaluation, 31(3), 400–413.
38.
KirkhartK. E. (2011). Culture and influence in multisite evaluation. In J. A. King & F. Lawrenz (Eds.), Multisite evaluation practice: Lessons and reflections from four cases. New Directions for Evaluation, 129, 73–85.
39.
Learning Metrics Task Force (2013). Toward universal learning: Recommendations from the learning metrics task force UIS/2013/ED/TD/07. Washington, D.C.: UNESCO Institute for Statistics and Center for Universal Education at Brookings Institution.
40.
MarkM. (2011). Toward better research on—and thinking about—evaluation influence, especially in multisite evaluations. In KingJ. A.LawrenzF. (Eds.), Multisite evaluation practice: Lessons and reflections from four cases. New directions for evaluation (Vol. 129, pp. 107–119).
41.
MarkM.HenryG. (2004). The mechanisms and outcomes of evaluation influence. Evaluation, 10(1), 35–57.
NunneleyR.Jr.KingJ.JohnsonK.PejsaL. (2015). The value of clear thinking about evaluation theory: The example of use and influence. In ChristieC.VoA. (Eds.), Evaluation Use and decision making in society: A tribute to marvin C. Alkin (pp. 53–71). Charlotte: Information Age Publishing.
44.
OfirZ.KumarA. (2013). Evaluation in developing countries. In DonaldsonS. I.AzzamT.ConnerR. F. (Eds.), Emerging practices in international development evaluation (pp. 11–24). Charlotte, NC: Information Age Publishing.
45.
OttosonJ. M.HaweP. (2009). Knowledge utilization, diffusion, implementation, transfer, and translation: Implications for evaluation. New Directions for Evaluation, 124.
46.
PattonM. Q. (2020). Evaluation use theory, practice, and future research: Reflections on the alkin and king AJE series. American Journal of Evaluation, 41(4), 581–602.
47.
Project TeamT. O. H. (2006). The oral history of evaluation, part 4: The professional evolution of carol H. Weiss. American Journal of Evaluation, 27(4), 475–484.
48.
Results for Development Institute. (2015). Bringing Learning to Light: The Role of Citizen-led Assessments in Shifting the Education Agenda. Washington, D.C.
49.
RichR. F. (1991). Knowledge creation, diffusion, and utilization: Perspectives of the founding editor of knowledge. Knowledge, 12(3), 319–337.
50.
RogersE. (2003). Diffusion of innovations (5th ed.). New York: Free Press.
51.
SchwandtT.GatesE. (2016). What can evaluation do? An agenda for evaluation in service of an equitable society. In DonaldsonS.PicciottoR. (Eds.), Evaluation for an equitable society (pp. 67–82). Charlotte: Information Age Publishing.
52.
WeissC. (1977). Using social research in public policy making. Lexington, Massachusetts: Lexington Books.
53.
WeissC. (1999). The interface between evaluation and public policy. Evaluation, 5, 468–486.
54.
WinthropR.SimonsK. (2013). Can international large-scale assessments inform a global learning goal? Insights from the learning metrics task force. Research in Comparative and International Education, 8(3), 279–295.
