Extensive research has explored the uses and consequences of high-stakes assessment systems within accountability policies. However, the gap between the stated objectives of these systems and the perspectives of actors at the micro- and meso-levels of the school system has received limited attention. This qualitative study, based on individual and group interviews with local education administrators and school leaders, examines the theory of action attributed to Chile’s national teacher assessment system. The results highlight various interpretations and positions of actors toward the system. These are discussed in relation to the consequential validity of the assessment, as well as the implications of discrepancies between policy design and the ways in which beneficiaries interpret and enact it.
American Educational Research Association, American Psychological Association, and National Council on Measurement in Education (AERA, APA, & NCME). (2014). Standards for educational and psychological testing. American Educational Research Association.
2.
ArgyrisC.SchönD. (1974). Theory in practice increasing professional effectiveness. Jossey-Bass Publishers.
3.
Avalos-BevanB. (2018). Teacher evaluation in Chile: Highlights and complexities in 13 years of experience. Teachers and Teaching, 24(3), 297–311. https://doi.org/10.1080/13540602.2017.1388228
4.
BallS. J. (1993). What is policy? Texts, trajectories and toolboxes. Discourse: Studies in the Cultural Politics of Education, 13(2), 10–17. https://doi.org/10.1080/0159630930130203
5.
BallS. J. (2012). Politics and policy making in education: Explorations in policy sociology. Routledge.
6.
BallS. J. (2015). What is policy? 21 Years later: Reflections on the possibilities of policy research. Discourse: Studies in the Cultural Politics of Education, 36(3), 306–313. http://dx.doi.org/10.1080/01596306.2015.1015279
7.
BallS. J.MaguireM.BraunA.HoskinsK. (2011a). Policy actors: Doing policy work in schools. Discourse: Studies in the Cultural Politics of Education, 32(4), 625–639. http://dx.doi.org/10.1080/01596306.2011.601565
8.
BallS. J.MaguireM.BraunA.HoskinsK. (2011b). Policy subjects and policy actors in schools: Some necessary but insufficient analyses. Discourse: Studies in the Cultural Politics of Education, 32(4), 611–624. https://doi.org/10.1080/01596306.2011.601564
9.
BandalosD. L. (2018). Measurement theory and applications for the social sciences. Guilford Press.
10.
BasciaNCummingA.DatnowA.LeithwoodK.LivingstoneD. (2005). International handbook of educational policy. Springer.
11.
BellC.KaneM. (2022). Formative and summative teacher evaluation in social context. In ManziJ.SunY.GarcíaM. R. (Eds.), Teacher evaluation around the world. Experiences, dilemmas and future challenges (pp. 9–38). Springer. https://doi.org/10.1007/978-3-031-13639-9_2
BraunH.MarionS. (2022). Accountability and assessment in U.S. education: Let’s not allow another crisis go to waste!Assessment in Education: Principles, Policy & Practice, 29(5), 555–574. https://doi.org/10.1080/0969594X.2022.2068503
16.
BrinkmannS. (2018). The Interview. In DenzinN.LincolnI. (Eds.), The Sage handbook of qualitative research (5th ed., pp. 997–1038). Sage.
17.
Chalhoub-DevilleM. (2016). Validity theory: Reform policies, accountability testing, and consequences. Language Testing, 33(4), 453–472. https://doi.org/10.1177/0265532215593312
ContrerasJ.MartínC. S.MuñozS. (2025). Uses and consequences in the theory of action of high-stakes teacher evaluation in Chile. Educational Assessment, Evaluation and Accountability, 37, 273-295. https://doi.org/10.1007/s11092-024-09448-3
21.
CreswellJ. W.CreswellJ. D. (2023). Research design: Qualitative, quantitative, and mixed methods approaches (6th ed.). SAGE.
22.
DalyC.MiltonE.LangdonF. (2022). How do ecological perspectives help understand schools as sites for teacher learning? In SwaffieldS.PoekertP. (Eds.), Leadership for professional learning (pp. 192–203). Routledge.
23.
DatnowA.LocktonM.WeddleH. (2020). Redefining or reinforcing accountability? An examination of meeting routines in schools. Journal of Educational Change, 21, 109–134. https://doi.org/10.1007/s10833-019-09349-z
24.
DonaldsonM. L.FirestoneW. (2021). Rethinking teacher evaluation using human, social, and material capital. Journal of Educational Change, 22, 501–534. https://doi.org/10.1007/s10833-020-09405-z
25.
DubetF. (2006). Injustices: l’expérience des inégalités au travail [Injustices: The e xperience of i nequalities at work] . Seuil.
26.
EhrenM. C.JanssensF. J. G.BrownM.McNamaraG.O’HaraJ.ShevlinP. (2017). Evaluation and decentralised governance: Examples of inspections in polycentric education systems. Journal of Educational Change, 18(3), 365–383. https://doi.org/10.1007/s10833-017-9297-9
27.
ElwoodJ. (2013). Educational assessment policy and practice: A matter of ethics. Assessment in Education: Principles, Policy and Practice, 20(2), 205–220. https://doi.org/10.1080/0969594X.2013.765384
28.
EvansC. M.LyonsS. (2017). Comparability in balanced assessment systems for state accountability. Educational Measurement: Issues and Practice, 36(3), 24–34. https://doi.org/10.1111/emip.12152
29.
FalabellaA. (2021). The seduction of hyper-surveillance: Standards, testing, and accountability. Educational Administration Quarterly, 57(1), 113–142. https://doi.org/10.1177/0013161X20912299
30.
FirestoneW. A.DonaldsonM. L. (2019). Teacher evaluation as data use: What recent research suggests. Educational Assessment, Evaluation and Accountability, 31(3), 289–314. https://doi.org/10.1007/s11092-019-09300-z
31.
FlickU. (2018). Triangulation in data collection. In FlickU. (Ed.), The Sage handbook of qualitative data collection (pp. 527–544). Sage.
32.
FloresM. A.DerringtonM. L. (2015). School principals’ views of teacher evaluation policy: Lessons learned from two empirical studies. International Journal of Leadership in Education, 20(4), 416–431. https://doi.org/10.1080/13603124.2015.1094144
33.
FordT. G.HewittK. (2020). Better integrating summative and formative goals in the design of next generation teacher evaluation systems. Education Policy Analysis Archives, 28, 63. https://doi.org/10.14507/epaa.28.5024
34.
FunnellS. C.RogersP. J. (2011). Purposeful program theory: Effective use of theories of change and logic models. Jossey-Bass.
35.
GarretónM.SanfuentesM.ValenzuelaJ. P.NuñezI. M. (2022). Brechas y desafíos organizacionales en la implementación temprana de la Nueva Educación Pública en Chile [Organizational g aps and c hallenges in the e arly i mplementation of the n ew p ublic e ducation in Chile] . Pensamiento Educativo, 59(1), 1–18. https://doi.org/10.7764/PEL.59.1.2022.10
36.
GreanyT. (2022). Place-based governance and leadership in decentralised school systems: Evidence from England. Journal of Education Policy, 37(2), 247–268. https://doi.org/10.1080/02680939.2020.1792554
37.
GreanyT.CowhittT.NoyesA.GriptonC.HudsonG. (2024). Local learning landscapes: Conceptualising place-based professional learning by teachers and schools in decentralised education systems. Journal of Educational Change, 26, 1–28. https://doi.org/10.1007/s10833-024-09508-x
38.
HargreavesA.ShirleyD. (2020). Leading from the middle: Its nature, origins and importance. Journal of Professional Capital and Community, 5(1), 92–114. https://doi.org/10.1108/JPCC-06-2019-0013
39.
HassenteufelP. (2021). Sociologie politique de l’action publique [Political s ociology of p ublic action] (3e éd.). Armand Colin.
40.
HenninkM. (2014). Understanding focus group discussions. Oxford University Press.
41.
HitchcockJ. H.OnwuegbuzieA. J.KhoshaimH. B. (2015). Examining the consequential validity of standardized examinations via public perceptions: A review of mixed methods survey design considerations. International Journal of Multiple Research Approaches, 9(1), 24–39. https://doi.org/10.1080/18340806.2015.1076757
JephcoteM.DaviesB. (2004). Recontextualizing discourse: An exploration of the workings of the meso-level. Journal of Education Policy, 19(5), 547–564. https://doi.org/10.1080/0268093042000269135
KaneM. (2013b). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1–73. https://doi.org/10.1111/jedm.12000
KaneM.WoolsS. (2020). Perspectives on the validity of classroom assessments. In BrookhartS.McMillanJ. (Eds.), Classroom assessment and educational measurement (pp. 11–26). Routledge.
48.
KimballS.ArrigoniJ.MarlinD.GeraghtyE.HenemanH.CarlB.MilanowskiA.JonesC.RaineyK. (2023). Learning-centered teacher evaluation. Leadership and Policy in Schools, 22(2), 480–508. https://doi.org/10.1080/15700763.2021.1977965
49.
LaneS. (2014). Evidencia de validez basada en las consecuencias del uso del test [Validity evidence based on testing consequences]. Psicothema, 26(1), 127–135. https://doi.org/10.7334/psicothema2013.258
50.
LaneS. (2020). Test-based accountability systems: The importance of paying attention to consequences. ETS Research Report Series, 2020(1), 1–22. https://doi.org/10.1002/ets2.12283
51.
LeaderJ. A.PazeyB. L. (2023). More than a test score: Toward a more balanced school accountability system. School Leadership Review, 17(2), 4.
52.
Leiva-GuerreroM. V.Benavides-MenesesM. (2024). Servicios locales de educación pública: principales tensiones y desafíos para los directores escolares [Local p ublic e ducation s ervices: Key t ensions and c hallenges for s chool principals] . Calidad en la educación, (60), 222–248. https://dx.doi.org/10.31619/caledu.n60.1437
53.
LevatinoA.ParcerisaL.VergerA. (2023). Understanding the stakes: The influence of accountability policy options on teachers’ responses. Educational Policy, 38(1), 31–60. https://doi.org/10.1177/08959048221142048
54.
Ley 20.903: Crea el Sistema de Desarrollo Profesional Docente y modifica otras normas [Law 20,903: Creates the System for Professional Development of Teachers and modifies other norms]. 2016. Chile. https://www.bcn.cl/leychile/navegar?idNorma=1087343
55.
LoebS.ByunE. (2019). Testing, accountability, and school improvement. The ANNALS of the American Academy of Political and Social Science, 683(1), 94–109. https://doi.org/10.1177/0002716219839929
56.
MalenB.CroningerR.MunceyD.Redmond-JonesD. (2002). Reconstituting schools: “Testing” the “theory of action”. Educational Evaluation and Policy Analysis, 24(2), 113–132. https://doi.org/10.3102/01623737024002113
57.
MalenB.KnappM. (1997). Rethinking the multiple perspectives approach to education policy analysis: Implications for policy-practice connections. Journal of Education Policy, 12(5), 419–445. https://doi.org/10.1080/0268093970120509
58.
MandinachE.SchildkampK. (2021). Misconceptions about data-based decision making in education: An exploration of the literature. Studies in Educational Evaluation, 69, 1–10. https://doi.org/10.1016/j.stueduc.2020.100842
59.
ManziJ.SunY.GonzálezR. (2011). La Evaluación Docente en Chile [Teacher evaluation in Chile]. MIDE UC.
60.
ManziJ.SunY.GarcíaM. R. (Eds.). (2022). Teacher evaluation around the world: Experiences, dilemmas and future challenges. Springer Nature.
61.
MarionS.ThompsonJ.EvansC.MartineauJ.DadeyN. (2019). The challenges and opportunities of balanced systems of assessment: A policy brief. National Center for the Improvement of Educational Assessment.
62.
MaroyC.PonsX. (2019). Theoretical framework. In MaroyC.PonsX. (Eds.), Accountability policies in education (pp. 53–94). Springer.
63.
MathouC. (2018). Recontextualizing curriculum policies: A comparative perspective on the work of mid-level actors in France and Quebec, Journal of Curriculum Studies, 50(6), 789–804. https://doi.org/10.1080/00220272.2018.1513567
64.
McKimC. (2023). Meaningful member-checking: A structured approach to member-checking. American Journal of Qualitative Research, 7(2), 41–52.
65.
Mejía-RodríguezA. M.KyriakidesL. (2022). What matters for student learning outcomes? A systematic review of studies exploring system-level factors of educational effectiveness. Review of Education, 10(3), 1–35. https://doi.org/10.1002/rev3.3374
66.
MessickS. (1980). Test validity and the ethics of assessment. American Psychologist, 35(11), 1012–1027. http://dx.doi.org.pucdechile.idm.oclc.org/10.1037/0003-066X.35.11.1012
67.
MilesM. B.HubermanA. M.SaldañaJ. (2020). Qualitative data analysis: A methods sourcebook (4th ed.). SAGE.
68.
MizalaA.SchneiderB. (2020). Promoting quality education in Chile: The politics of reforming teacher careers. Journal of Education Policy, 35(4), 529–555. http://dx.doi.org/10.1080/02680939.2019.1585577
69.
MollaT. (2021). Critical policy scholarship in education: An overview. Education Policy Analysis Archives, 29(2), 1–30. https://doi.org/10.14507/epaa.29.5655
70.
MullerP. (2000). L’analyse cognitive des politiques publiques: Vers une sociologie politique de l’action publique. [Cognitive a nalysis of p ublic p olicies: Toward a p olitical s ociology of p ublic action].Revue Française de Science Politique, 50(2), 189–207. http://www.jstor.org/stable/43119726
71.
Muñoz-ChereauB.GonzálezA.MeyersC. (2022). How are the “losers” of the school accountability system constructed in Chile, the USA and England?Compare: A Journal of Comparative and International Education, 52(7), 1125–1144. https://doi.org/10.1080/03057925.2020.1851593
OrdofaB.AsgedomA. (2022). School accountability and its relationship with learning outcomes: A systematic literature review. Social Sciences & Humanities Open, 6(1), 100358. https://doi.org/10.1016/j.ssaho.2022.100358
74.
ParcerisaL.VergerA.BrowesN. (2022). Teacher a utonomy in the a ge of performance-based a ccountability: A review based on teaching profession regulatory models (2017-2020). Education Policy Analysis Archives, 30(100). https://doi.org/10.14507/epaa.30.6204
75.
PattonM. (2002). Qualitative research and evaluation methods (3rd ed.). Sage.
76.
PauflerN. A. (2018). Declining morale, diminishing autonomy, and decreasing value: Principal reflections on a high-stakes teacher evaluation system. International Journal of Education Policy & Leadership, 13(8), 1–15. https://doi.org/10.22230/ijepl.2018v13n8a813
77.
PauflerN. A.KingK. M.ZhuP. (2020). Promoting professional growth in new teacher evaluation systems: Practitioners’ lived experiences in changing policy contexts. Studies in Educational Evaluation, 65, 100873. https://doi.org/10.1016/j.stueduc.2020.100873
78.
PauflerN. A.SloatE. F. (2020). Using standards to evaluate accountability policy in context: School administrator and teacher perceptions of a teacher evaluation system. Studies in Educational Evaluation, 64, 1–15. https://doi.org/10.1016/j.stueduc.2019.07.007
79.
RodríguezS.WeinsteinJ. (2024). Who makes up the middle tier? A case study of school improvement organizations in Chile. International Journal of Educational Leadership and Management, 12(3), 167–192. http://dx.doi.org/10.17583/ijelm.14667
80.
RuffinelliA. (2016). Ley de desarrollo profesional docente en Chile: de la precarización sistemática a los logros, avances y desafíos pendientes para la profesionalización [Law on teacher professional development: From systematic precariousness to the achievements, the advances, and the remaining challenges for professionalization]. Estudios pedagógicos (Valdivia), 42(4), 261–279. http://dx.doi.org/10.4067/S0718-07052016000500015
ToloA.LillejordS.Flórez PetourM. T.HopfenbeckT. N. (2020). Intelligent accountability in schools: A study of how school leaders work with the implementation of assessment for learning. Journal of Educational Change, 21(1), 59–82. https://doi.org/10.1007/s10833-019-09359-x
86.
TorresR. (2021). Does test-based school accountability have an impact on student achievement and equity in education?: A panel approach using PISA (OECD Education Working Papers 250). https://doi.org/10.1787/0798600f-en
87.
TrinidadJ. E. (2023). Rethinking school improvement organizations: Understanding their variety, benefits, risks, and future directions. Educational Researcher, 52(6), 377–384. https://doi.org/10.3102/0013189X231179116
88.
TuytensM.DevosG.VanblaereB. (2020). An integral perspective on teacher evaluation: A review of empirical studies. Educational Assessment, Evaluation and Accountability, 32, 153–183. https://doi.org/10.1007/s11092-020-09321-z
89.
WylieE. C. (2017). Winsight™ Assessment System: Preliminary theory of action. ETS Research Report Series, 2017(1), 1–17. https://doi.org/10.1002/ets2.12155
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
