This study examines the application of the 2018 American Evaluation Association Evaluator Competencies in graduate-level program evaluation courses, specifically an introductory course and a practicum course, to teach novice evaluators how to plan and conduct evaluations in educational settings. Through self-assessments, journaling, and goal setting to acquire specific sets of competencies during the practicum, the novice evaluators reflect on their learning experiences, perceived levels of competencies, and key events that facilitated building their competencies in evaluation. The findings of this study demonstrate how novice evaluators developed their competencies in evaluation through these graduate educational courses, with implications for future research and educational development.
AndersonL. (2018). Competency-based education: Recent policy trends. The Journal of Competency-Based Education, 3(1), e01057. https://doi.org/10.1002/cbe2.1057
3.
Arévalo GrossC. J.Rodríguez-BilellaP.OlavarríaC. (2023). How to train better in evaluation: Teaching landscape and lessons learned from Latin America. American Journal of Evaluation, 44(2), 282–292. https://doi.org/10.1177/10982140211059373
4.
BettsM.SmithR. (1998). Developing the credit-based modular curriculum in higher education: Challenge, choice and change (1st ed.). London: Routledge.
5.
BourgeoisI.WildschutL.SteinbergM. (2021). How does teaching with cases support the development of evaluation competencies?New Directions for Evaluation, 2021(172), 37–51. https://doi.org/10.1002/ev.20486
6.
BoyceA. S.ChouinardJ. A. (2017). Moving beyond the buzzword: A framework for teaching culturally responsive approaches to evaluation. Canadian Journal of Program Evaluation, 32(2), 266–279. https://doi.org/10.3138/cjpe.31132
7.
BoyceA. S.McGowanB. L. (2018). An exploration of two novice evaluation educators’ experiences developing and implementing introduction to evaluation courses. American Journal of Evaluation, 40(1), 119–136. https://doi.org/10.1177/1098214018778812
8.
BoyerE. L. (1991). Scholarship reconsidered: Priorities of the professoriate. In Carnegie foundation for the advancement of teaching (3rd ed.). Lawrenceville, NJ: Princeton University Press.
9.
BoyteH. C. (2007). Public work: Civic populism versus technocracy in higher education. In BrownD.WhiteD. (Eds.), Agent of democracy: Higher education and the HEX journey (pp. 79–102). Washington, DC: Kettering Foundation.
CastroM. P.FragapaneS.RinaldiF. M. (2016). Professionalization and evaluation: A European analysis in the digital era. Evaluation, 22(4), 489–507. https://doi.org/10.1177/1356389016667887
12.
ChoM.CastlemanA. M.UmansH.MwirigiM. O. (2023). Measuring evaluator competencies: Developing and validating the evaluator competencies assessment tool. American Journal of Evaluation, 44(3), 474–494. https://doi.org/10.1177/10982140211056539
13.
ChristieC. A.QuiñonesP.FierroL. (2013). Informing the discussion on evaluator training: A look at evaluators’ course taking and professional practice. American Journal of Evaluation, 35(2), 274–290. https://doi.org/10.1177/1098214013503697
DaviesR.MacKayK. (2014). Evaluator training: Content and topic valuation in university evaluation courses. American Journal of Evaluation, 35(3), 419–429. https://doi.org/10.1177/1098214013520066
16.
DeweyJ. D.MontrosseB. E.SchröterD. C.SullinsC. D.MattoxJ. R. (2008). Evaluator competencies: What’s taught versus what’s sought. American Journal of Evaluation, 29(3), 268–287. https://doi.org/10.1177/1098214008321152
17.
DigheS.LaVelleJ. M.ChikateP.AcikgozM.KannanP.EspelienD.SarodeT. (2024). Values-engaged teaching for effective practice in applied disciplines. In Worldviews and values in higher education: Teaching, learning, curricula, and assessment (pp. 25–49). Emerald Publishing Limited. https://doi.org/10.1108/978-1-80262-897-520241003
18.
DillmanL. M. (2013). Evaluator skill acquisition: Linking educational experiences to competencies. American Journal of Evaluation, 34(2), 270–285. https://doi.org/10.1177/1098214012464512
19.
DweckC. S. (2006). Mindset: The new psychology of success. How we can learn to fulfill our potential. New York: Random House.
20.
FieldH. (1979). Competency based teacher education (CBTE): A review of the literature. British Journal of In-Service Education, 6(1), 39–42. https://doi.org/10.1080/0305763790060109
21.
FierroL. A.ChristieC. A. (2011). Understanding evaluation training in schools and programs of public health. American Journal of Evaluation, 32(3), 448–468. https://doi.org/10.1177/1098214010393721
22.
FitzpatrickJ. L. (1994). Alternative models for the structuring of professional preparation programs. New Directions for Program Evaluation, 1994(62), 41–50. https://doi.org/10.1002/ev.1675
23.
FinkL. D. (2013). Creating significant learning experiences: An integrated approach to designing college courses (2nd ed.). San Francisco, CA: Jossey-Bass.
24.
FrankJ. R.SnellL. S.CateO. T.HolmboeE. S.CarraccioC.SwingS. R.GlasgowN. J.CampbellC.DathD.HardenR. M.IobstW.LongD. M.MungrooR.RichardsonD. L.SherbinoJ.SilverI.TaberS.TalbotM.HarrisK. A. (2010). Competency-based medical education: Theory to practice. Medical Teacher, 32(8), 638–645. https://doi.org/10.3109/0142159X.2010.501190
GalportN.AzzamT. (2017). Evaluator training needs and competencies: A gap analysis. American Journal of Evaluation, 38(1), 80–100. https://doi.org/10.1177/1098214016643183
27.
GhereG.KingJ. A.StevahnL.MinnemaJ. (2006). A professional development unit for reflecting on program evaluator competencies. American Journal of Evaluation, 27(1), 108–123. https://doi.org/10.1177/1098214005284974
28.
GullicksonA. M.KingJ. A.LaVelleJ. M.ClintonJ. M. (2019). The current state of evaluator education: A situation analysis and call to action. Evaluation and Program Planning, 75, 20–30. https://doi.org/10.1016/j.evalprogplan.2019.02.012
29.
GurungR. A. R.WilsonJ. H. (2013). Advancing scholarly research on teaching and learning. New Directions for Teaching and Learning, 2013(136), 1–6. https://doi.org/10.1002/tl.20071
30.
HarnarM. A.PrieurA. M.Ross NelsonK. (2024). Using cognitive complexity to understand role-play as a pedagogical tool in graduate evaluation education. American Journal of Evaluation, 46(3), 338–352. https://doi.org/10.1177/10982140241242128
31.
HatcherR. L.FouadN. A.CampbellL. F.McCutcheonS. R.GrusC. L.LeahyK. L. (2013). Competency-based education for professional psychology: Moving from concept to practice. Training and Education in Professional Psychology, 7(4), 225–234. https://doi.org/10.1037/a0033765
32.
HobsonK. A.CorynC. L. S.FierroL. A.Sherwood-LaughlinC. M. (2019). Instruction of evaluation competencies in council on education for public health (CEPH)-accredited master of public health (MPH) degree programs. American Journal of Evaluation, 40(4), 590–606. https://doi.org/10.1177/1098214019845510
33.
KallemeynL. M.RoseveareC.SteinbergM.ToveyT. L. S. (2021). What does implementing case-centered teaching of evaluation look like in practice?New Directions for Evaluation, 2021(172), 69–84. https://doi.org/10.1002/ev.20482
34.
KingJ. A.AyooS. (2020). What do we know about evaluator education? A review of peer-reviewed publications (1978–2018). Evaluation and Program Planning, 79, 101785. https://doi.org/10.1016/j.evalprogplan.2020.101785
35.
KingJ. A.StevahnL. (2020). Presenting the 2018 AEA evaluator competencies. New Directions for Evaluation, 2020(168), 49–61. https://doi.org/10.1002/ev.20435
36.
LaVelleJ. M. (2014). An examination of evaluation education programs and evaluator skills across the world (Doctoral dissertation, The Claremont Graduate University), Claremont, CA.
37.
LaVelleJ.JohnsonD. (2022). Redesigning a university-based evaluator education program for scholarship and practice. Canadian Journal of Program Evaluation, 37(1), 117–134. https://doi.org/10.3138/cjpe.71430
38.
LaVelleJ. M.DonaldsonS. I. (2021). Opportunities and challenges ahead for university-based evaluator education programs, faculty, and students. American Journal of Evaluation, 42(3), 428–438. https://doi.org/10.1177/1098214020937808
39.
LaVelleJ. M.GalportN. (2020). Using the 2018 AEA evaluator competencies for evaluator education and professional development. New Directions for Evaluation, 2020(168), 99–116. https://doi.org/10.1002/ev.20437
LaVelleJ. M.NeubauerL. C.BoyceA. S.ArchibaldT. (2023). Setting the stage for critically defined and responsive evaluator education and training. New Directions for Evaluation, 2023(177), 13–22. https://doi.org/10.1002/ev.20542
42.
LaVelleJ. M.YangZ. (2021). In their own words: Student key learning experiences in an introductory evaluation course. Canadian Journal of Program Evaluation, 35(3), 450–465. https://doi.org/10.3138/cjpe.69601
43.
LinfieldK. J.ToveyT. L. S. (2021). What is the case for teaching with cases in evaluation?New Directions for Evaluation, 2021(172), 11–18. https://doi.org/10.1002/ev.20480
44.
LittleT. D.ChangR.GorrallB. K.WaggenspackL.FukudaE.AllenP. J.NoamG. G. (2020). The retrospective pretest–posttest design redux: On its validity as an alternative to traditional pretest–posttest measurement. International Journal of Behavioral Development, 44(2), 175–183. https://doi.org/10.1177/0165025419877973
45.
MilesM. B.HubermanA. M.SaldañaJ. (2018). Qualitative data analysis (4th ed.). Thousand Oaks, CA: Sage Publications, Inc.
46.
Montrosse-MoorheadB.EnsmingerD. C.RoseveareC. (2021). How do we teach and learn with cases?New Directions for Evaluation, 2021(172), 53–67. https://doi.org/10.1002/ev.20483
47.
NeubauerL. C.LaVelleJ.BoyceA. S.ArchibaldT. (2025). Research on evaluator education and training: Conceptual and practical bounds for a scholarship of teaching and learning (SoTL). New Directions for Evaluation, 2025(187), 59–66. https://doi.org/10.1002/ev.70011
48.
NewmanD. L.BrownR. D. (1996). Applied ethics for program evaluation. Thousand Oaks, CA: Sage Publications, Inc.
49.
PothC.SearleM. (2021). Competency-based evaluation education: Four essential things to know and do. Canadian Journal of Program Evaluation, 35(3), 296–309. https://doi.org/10.3138/cjpe.69577
50.
PothC. N.SearleM.AquilinaA. M.GeJ.ElderA. (2020). Assessing competency-based evaluation course impacts: A mixed methods case study. Evaluation and Program Planning, 79, 101789. https://doi.org/10.1016/j.evalprogplan.2020.101789
51.
QuadrelliC.StaheliJ.Cook-WatkinsC.AldermanL. (2022). Scoping recent investment in evaluation education in Australian universities. Evaluation Journal of Australasia, 22(4), 254–281. https://doi.org/10.1177/1035719X221120292
52.
ReidA. M.BoyceA. S.ChuquirunaJ. A.Somo-AinaO.MalloyC.StephensonS. (2023). A values-engaged, educative training model for novice evaluators. New Directions for Evaluation, 2023(177), 135–143. https://doi.org/10.1002/ev.20544
53.
RyanR. M.DeciE. L. (2009). Promoting self-determined school engagement: Motivation, learning, and well-being. In WenzelK. R.WigfieldA. (Eds.), Educational psychology handbook series. Handbook of motivation at school (pp. 171–195). New York, NY: Routledge/Taylor & Francis Group.
54.
SandersJ. R. (1986). The teaching of evaluation in education. New Directions for Program Evaluation, 29, 15–27. https://doi.org/10.1002/ev.1416
55.
SchwandtT. A. (2017). Professionalization, ethics, and fidelity to an evaluation ethos. American Journal of Evaluation, 38(4), 546–553. https://doi.org/10.1177/1098214017728578
56.
SchwartzS. H.CieciuchJ.VecchioneM.DavidovE.FischerR.BeierleinC.KontyM. (2012). Refining the theory of basic individual values. Journal of Personality and Social Psychology, 103(4), 663–688. https://doi.org/10.1037/a0029393
57.
SchweigertF. J. (2007). The priority of justice: A framework approach to ethics in program evaluation. Evaluation and Program Planning, 30(4), 394–399. https://doi.org/10.1016/j.evalprogplan.2007.06.007
58.
SowlS.Amrein-BeardsleyA.CollinsC. (2022). Teaching program evaluation: How blending theory and practice enhance student-evaluator competencies in an education policy graduate program. Evaluation and Program Planning, 94, 102139. https://doi.org/10.1016/j.evalprogplan.2022.102139
59.
StevahnL.KingJ. A.GhereG.MinnemaJ. (2005). Establishing essential competencies for program evaluators. American Journal of Evaluation, 26(1), 43–59. https://doi.org/10.1177/1098214004273180
60.
TeasdaleR. M.PittsR. T.GatesE. F.ShimC. (2023). Teaching specification of evaluative criteria: A guide for evaluation education. New Directions for Evaluation, 2023(177), 31–37. https://doi.org/10.1002/ev.20546
61.
ThomasV. G.MadisonA. (2010). Integration of social justice into the teaching of evaluation. American Journal of Evaluation, 31(4), 570–583. https://doi.org/10.1177/1098214010368426
62.
TrevisanM. S. (2004). Practical training in evaluation: A review of the literature. American Journal of Evaluation, 25(2), 255–272. https://doi.org/10.1177/109821400402500212
63.
VillalobosJ.BledsoeK. L.DonaldsonS. I. (2023). Integrating theory and research on structural racism and social justice into evaluator preparation. New Directions for Evaluation, 2023(177), 49–55. https://doi.org/10.1002/ev.20534
64.
VoorheesR. A. (2001). Competency-based learning models: A necessary future. New Directions for Institutional Research, 2001(110), 5–13. https://doi.org/10.1002/ir.7
YarbroughD. B.ShulhaL. M.HopsonR. K.CaruthersF. A. (2010). The program evaluation standards: A guide for evaluators and evaluation users. Thousand Oaks, CA: Sage Publications, Inc.
