Narrative case studies have shown that, when people are involved in an evaluation of a program they are part of, it can change how they experience the program. This study used a quasi-experiment to test this proposition empirically in the context a participatory action research curriculum called Youth as Researchers. Half of all Youth as Researcher groups engaged in a participatory evaluation (PE) of their program experience through writing reflective essays, creating their own evaluation questions, and conducting peer interviews. The other half served as control groups and did not engage in the PE activities. Pre-/posttest surveys and focus group data were used to assess differences among the experimental and control groups. Study results show that participants in the experiment had important differences in their experiences in the program as a result of participation in the evaluation. Implications for future practice and research are also explored.
Abo-ZenaM. M.PavalowM. (2016). Being in-between: Participatory action research as a tool to partner with and learn about emerging adults. Emerging Adulthood, 4(1), 19–29.
2.
ArnoldM. E.DolencB.WellsE. E. (2008). Youth community engagement: A recipe for success. Journal of Community Engagement and Scholarship, 1(1), 56.
3.
BautistaM. A.BertrandM.MorrellE.ScorzaD.MatthewsC. (2013). Participatory action research and city youth: Methodological insights from the Council of Youth Research. Teachers College Record, 115(10), 1–18.
4.
BoruchR.SolomonP.DraineJ.DeMoyaD.WickermanR. (1998). Design-based evaluation: Process studies, experiments, and quasi-experiments. Scandinavian Journal of Social Welfare, 7, 126–131.
5.
BrandonP. R.FukunagaL. L. (2014). The state of the empirical research literature on stakeholder involvement in program evaluation. American Journal of Evaluation, 35, 26–44. http://doi.org/10.1177/1098214013503699
6.
BreauB. L.BrookD. (2007). “Mock” mock juries: A field experiment on the ecological validity of jury simulations. Law & Psychology Review, 31, 77–92.
7.
BrisolaraS. (1998). The history of participatory evaluation and current debates in the field. In WhitmoreE. (Ed.), Understanding and practicing participatory evaluation: New directions in evaluation, No. 80 (pp. 25–41). Jossey-Bass.
8.
BurkeK.GreeneS. (2015). Participatory action research, youth voices, and civic engagement. Language Arts, 92(6), 387–400.
9.
CabreraN. L.MilemJ. F.JaquetteO.MarxR. W. (2014). Missing the (student achievement) for all the (political) trees: Empiricism and the Mexican American studies controversy in Tucson. American Educational Research Journal, 51(6), 1084–1118.
10.
CammarotaJ.FineM. (2008). Revolutionizing education: Youth participatory action research in motion. Routledge.
11.
CammarotaJ.RomeroA. (2006). A critically compassionate intellectualism for Latino/a students: Raising voices above the silencing in our schools. Multiculturalism Education, 14(2), 16–23.
12.
CheckowayB.Richards-SchusterK. (2003). Youth participation in community evaluation research. American Journal of Evaluation, 24(1), 21–33.
13.
ChenP.WeissF. L.NicholsonH. J., Girls Inc, & Girls Incorporated. (2010). Girls Study Girls Inc.: Engaging girls in evaluation through participatory action research. American Journal of Community Psychology, 46(1), 228–237.
14.
ChouinardJ. A. (2013). The case for participatory evaluation in an era of accountability. American Journal of Evaluation, 34(2), 237–253.
15.
ChristensB.PerkinsD. D. (2008). Transdisciplinary, multilevel action research to enhance ecological and psychopolitical validity. Journal of Community Psychology, 36(2), 214–231.
16.
CookA. L.CookB. G.LandrumT. J.TankersleyM. (2008). Examining the role of group experimental research in establishing evidence-based practices. Intervention in School and Clinic, 44(2), 76–82.
17.
CookA. L.Krueger-HenneyP. (2017). Group work that examines systems of power with young people: Youth participatory action research. The Journal for Specialists in Group Work, 42(2), 176–193.
18.
CooperS. (2014). Transformative evaluation: Organisational learning through participative practice. The Learning Organization, 21(2), 146–157.
19.
CooperS. (2018). Participatory evaluation in youth and community work: Theory and practice. Routledge.
20.
CousinsB.ChouinardJ. A. (2012). Participatory evaluation up close: An integration of research-based knowledge. Information Age.
21.
CousinsJ. B.WhitmoreE. (1998). Framing participatory evaluation. In WhitmoreE. (Ed.), Understanding and practicing participatory evaluation: New directions in evaluation, No. 80 (pp. 3–23). Jossey-Bass.
22.
CullenA. E.CorynC. L. S.RughJ. (2011). The politics and consequences of including stakeholders in international development evaluation. American Journal of Evaluation, 32, 345–361. http://doi.org/10.1177/1098214010396076
23.
CutronaC. E.RussellD. (1987). The provisions of social relationships and adaptation to stress. In JonesW. H.PerlmanD. (Eds.), Advances in personal relationships (Vol. 1, pp. 37–67). JAI Press.
24.
DavisM. H. (1983). Measuring individual differences in empathy: Evidence for a multidimensional approach. Journal of Personality and Social Psychology, 44(1), 113–126.
25.
DolanT.ChristensB. D.LinC. (2015). Combining youth organizing and youth participatory action research to strengthen student voice in education reform. National Society for the Study of Education, 114(1), 153–170.
26.
DuttaU. (2017). Creating inclusive identity narratives through participatory action research. Journal of Community & Applied Social Psychology, 27, 476–488.
27.
FlanaganC. A.SyvertsenA. K.StoutM. D. (2007). Civic measurement models: Tapping adolescents’ civic engagement[Circle Working Paper 55]. Center for Information and Research on Civic Learning and Engagement.
28.
FleischerD. N.ChristieC. A. (2009). Evaluation use: Results from a survey of U.S. American evaluation association members. American Journal of Evaluation, 30, 158–175. http://doi.org/10.1177/1098214008331009
29.
FlickerS.MaleyO.RidgleyA.BiscopeS.LombardoC.SkinnerH. (2008). E-PAR: Using technology and participatory action research to engage youth in health promotion. Action Research, 6(3), 285–303.
30.
Foster-FishmanP. G.LawK. M.LichtyL. F.AounC. (2010). Youth ReACT for social change: A method for youth participatory action research. American Journal of Community Psychology, 46(1–2), 67–83.
31.
GehrkeP. J. (2014). Ecological validity and the study of publics: The case for organic public engagement methods. Public Understanding of Science, 23(1), 77–91.
32.
GildinB. (2003). The All Stars Talent Show Network: Grassroots funding, community building, and participatory evaluation. New Directions for Evaluation, 98, 77–85.
33.
GomezR. J.RyanT. N. (2016). Speaking out: Youth led research as a methodology used with homeless youth. Child Adolescent Social Work, 33, 185–193.
34.
GreenoC. G. (2002). Major alternatives to the classic experimental design. Family Process, 41(4), 733–736.
35.
GregoryA. (2000). Problematizing participation: A critical review of approaches to participation in evaluation theory. Evaluation, 6(2), 179–199.
36.
HenryG. T.MarkM. M. (2003). Beyond use: Understanding evaluation’s influence on attitudes and actions. American Journal of Evaluation, 24, 293–314. http://doi.org/10.1177/109821400302400302
37.
Hesse-BiberS. N.LeavyP. (2006). The practice of qualitative research. Sage.
38.
JacobS.OuvrardL.BélangerJ. F. (2011). Participatory evaluation and process use within a social aid organization for at-risk families and youth. Evaluation and Program Planning, 34, 113–123. http://doi.org/10.1016/j.evalprogplan.2010.08.002
39.
JohnsonD. W.GreenseidL. O.ToalS. A.KingJ. A.LawrenzF.VolkovB. (2009). Research on evaluation use. A review of the empirical literature from 1986 to 2005. American Journal of Evaluation, 30, 377–410. http://doi.org/10.1177/1098214009341660
40.
KingJ.CousinsB.WhitmoreE. (2007). Making sense of participatory evaluation: Framing participatory evaluation. New Directions for Evaluation, 114, 83–105.
41.
KirshnerB.PozzoboniK.JonesH. (2011). Learning how to manage bias: A case study of youth participatory action research. Applied Developmental Science, 15(3), 140–155.
42.
LauG.NetherlandN. H.HaywoodM. L. (2003). Collaborating on evaluation for youth development. New Directions for Evaluation, 98, 47–59.
43.
LivingstoneA. M.CelemenckiJ.CalixteM. (2014). Youth participatory action research and school improvement: The missing voices of black youth in Montreal. Canadian Journal of Education, 37(1), 283–307.
44.
LondonJ.ZimmermanK. I.ErbsteinN. (2003). Youth-led research and evaluation: Tools for youth, organizational, and community development. New Directions for Evaluation, 98, 33–45.
45.
MathieA.CunninghamG. (2003). From clients to citizens: Asset-based community development as a strategy for community-driven development. Development in Practice, 5, 474–486.
46.
MertensD. M. (2009). Transformative research and evaluation. Guilford Press.
47.
MitraD. L. (2005). Adults advising youth: Leading while getting out of the way. Educational Administration Quarterly, 41(3), 520–553.
48.
MorsilloJ.PrilleltenskyI. (2007). Social action with youth: Interventions, evaluation, and psychopolitical validity. Journal of Community Psychology, 35(6), 725–740.
OrrS. (2010). Exploring stakeholder values and interests in evaluation. American Journal of Evaluation, 31(4), 557–569.
52.
OzerE. J. (2017). Youth-led participatory action research: Overview and potential for enhancing adolescent development. Child Development Perspectives, 11(3), 173–177.
53.
OzerE. J.NewlanS.DouglasL.HubbardE. (2013). Bounded empowerment: Analyzing tensions in the practice of youth-led participatory research in urban public schools. American Journal of Community Psychology, 52, 13–26.
54.
OzerE. J.RittermanM. L.WanisM. G. (2010). Participatory action research (PAR) in middle school: Opportunities, constraints, and key processes. American Journal of Community Psychology, 46(1–2), 156–166.
55.
OzerE. J.WrightD. (2012). Beyond school spirit: The effects of youth-led participatory action research in two urban high schools. Journal of Research on Adolescence, 22(2), 267–283.
PowersC. B.AllamanE. (2012). How participatory action research can promote social change and help youth development. The Kinder & Braver World Project: Research Series, No 2013-10. Burkman Center Research Publication.
58.
PowersJ. L.TiffanyJ. S. (2006). Engaging youth in participatory research and evaluation. Public Health Management Practice, 12(6), S79–S87.
59.
PriceP. G.MenckeP. D. (2013). Critical pedagogy and praxis with Native American youth: Cultivating change through participatory action research. The Journal of Educational Foundations, 27(3/4), 85–102.
60.
Quijada CerecerD. A.CahillC.BradleyM. (2013). Toward a critical youth policy praxis: Critical youth studies and participatory action research. Theory into Practice, 52, 216–223.
61.
RodriguezL. F.BrownT. M. (2009). From voices to agency: Guiding principles for participatory action research with youth. New Directions for Youth Development, 123, 19–34.
62.
RogersJ.MorrellE.EnyedyN. (2007). Studying the struggle: Contexts for learning and identity development for urban youth. American Behavioral Scientist, 51, 419–443.
63.
RomeroA.ArceS.CammarotaJ. (2009). A barrio in pedagogy: Identity, intellectualism, activism, and academic achievement through the evolution of critically compassionate intellectualism. Race Ethnicity and Education, 12(2), 217–233.
64.
RoselandD.LawrenzF.ThaoM. (2015). The relationship between involvement in and use of evaluation in multi-site evaluations. Evaluation and Program Planning, 48, 75–82. http://doi.org/10.1016/j.evalprogplan.2014.10.003
65.
RubinA.BabbieE. (1997). Research methods for social work (3rd ed.). Brooks/Cole.
66.
RubinB.JonesM. (2007). Student action research: Reaping the benefits for students and school leaders. National Association of Secondary School Principals Bulletin, 91, 363–378.
67.
SaboK. (2003). A Vygotskian perspective on youth participatory evaluation. New Directions for Evaluation, 2003(98), 13–24. http://doi.org/10.1002/ev.81
68.
Sabo FloresK. (2007). Youth participatory evaluation: Strategies for engaging young people (1st ed.). Jossey-Bass.
69.
SaldañaJ. (2013). The coding manual for qualitative researchers (2nd ed.). Sage.
70.
SchensulJ. J.BergM. (2004). Youth participatory action research: A transformative approach to service-learning. Michigan Journal of Community Service Learning, 10, 76–88.
ShamrovaD. P.CummingsC. E. (2017). Participatory action research (PAR) with children and youth: An integrative review of methodology and PAR outcomes for participants, organizations, and communities. Children and Youth Services Review, 81, 400–412.
73.
ShawJ.CampbellR. (2014). The “process” of process use: Methods of longitudinal assessment in a multisite evaluation. American Journal of Evaluation, 35(2), 250–260.
74.
StewartD. W.ShamdasaniP. N.RookD. W. (2007). Focus groups: Theory and practice (2nd ed.). Sage.
75.
StrobelK.OsbergJ.McLaughlinM. (2006). Participation in social change: Shifting adolescents’ developmental pathways. In GinwrightS.NogueraP.CammarotaJ. (Eds.), Beyond resistance! Youth activism and community change: New democratic possibilities for practice and policy for America’s youth (pp. 21–35). Routledge.
76.
TheodoriG. L. (2004). Community attachment, satisfaction, and action. Journal of the Community Development Society, 35, 73–86.
77.
ThomasA. J.BarrieR.BrunnerJ.ClawsonA.HewittA.Jeremie-BrinkA.Rowe-JohnsonM. (2014). Assessing critical consciousness in youth and young adults. Journal of Research on Adolescence, 24(3), 485–496.
78.
TorreM. E. (2009). Participatory action research and critical race theory: Fueling Spaces for Nos-otras to Research. Urban Review, 41, 106–120.
79.
TorreM. E.FineM.AlexanderN.BillupsA. B.BlandingY.GenaoE.MarboeE.SalahT.UrdangK. (2008). Participatory action research in the contact zone. In CammarotaJ.FineM. (Eds.), Revolutionizing education: Youth participatory action research in motion (pp. 23–44). Routledge.
80.
TorresR. T.PreskillH. (2001). Evaluation and organizational learning: Past, present, and future. American Journal of Evaluation, 22(3), 387–395.
81.
TuckE.AllenJ.BachaM.MoralesA.QuinterS.ThompsonJ.TuckM. (2008). PAR praxes for now and future change: The collective of researchers on educational disappointment and desire. In CammarotaJ.FineM. (Eds.), Revolutionizing education: Youth participatory action research in motion (pp. 49–83). Routledge.
82.
UcarX.PlanasA.NovellaA.MoricheM. P. R. (2017). Participatory evaluation of youth empowerment in youth groups. Cases analysis. Pedagogia Social, 30, 63–75.
83.
VaughanC. (2014). Participatory research with youth: Idealising safe social spaces or building transformative links in difficult environments?Journal of Health Psychology, 19(1), 184–192.
84.
WarenweilerD.MansukhaniR. (2016). Participatory action research with Filipino street youth: Their voice and action against corporal punishment. Child Abuse Review, 25, 410–423.
85.
WattsR.GriffithD.Abdul-AdilJ. (1999). Sociopolitical development as an antidote for oppression-theory and action. American Journal of Community Psychology, 27(2), 255–271.
86.
WattsR. J.FlanaganC. (2007). Pushing the envelope on youth civic engagement: A developmental and liberation psychology perspective. Journal of Community Psychology, 35(6), 779–792.
87.
WattsR. J.WilliamsN. C.JagersR. J. (2003). Sociopolitical development. American Journal of Community Psychology, 31(1–2), 185–194.
88.
WeaverL.CousinsJ. B. (2004). Unpacking the participatory process. Journal of MultiDisciplinary Evaluation, 1, 19–40.
89.
ZeldinS.PetrokubiJ.MacNeilC. (2008). Youth–adult partnerships in decision making: Disseminating and implementing an innovative idea into established organizations and communities. American Journal of Community Psychology, 41(3–4), 262–277.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
