Considerable empirical research, along with a growing body of conceptual and theoretical literature, exists on the role of culture and context in evaluation. Less scholarship has examined culturally responsive surveys in the context of international evaluation. In this article, the authors present lessons learned from the development, implementation, and analysis of surveys used to evaluate the struggle of Brazil’s quilombos, or former fugitive slave communities, for land rights and livelihood. The authors begin with a brief review of the culturally responsive evaluation literature, making the case for culturally responsive surveys. Following a brief introduction to quilombo communities, they discuss their survey work, including their efforts to be culturally responsive and the challenges they confronted. The article concludes with the lessons learned for culturally responsive survey inquirers.
AdlerS. M. (2004). Multiple layers of a researcher’s identity: Uncovering Asian American voices. In MutuaK.SwadenerB. B. (Eds.), Decolonizing research in cross-cultural contexts: Critical personal narratives (pp. 107–121). Albany, NY: SUNY Press.
American Indian Higher Education Association. (2009). Indigenous evaluation framework: Telling our story in our place and time. Alexandria, VA: AIHEC.
5.
ArrutiJ. M. A. (2006). Mocambo: Antropologia e história do processo de formação quilombola. Bauru, São Paulo, Brazil: EDUSC.
6.
BowenM. L. (2010). The struggle for black land rights in Brazil: An insider’s view on quilombos and the quilombo land movement. African and Black Diaspora: An International Journal, 3, 147–168.
7.
BowenM. L.BowserW. (2010, July). Beyond subsistence production: Paths to economic development in Brazil’s quilombo communities. Paper presented at the Tenth International Congress of the Brazilian Studies Association, Brasilia, Brazil.
8.
BowenM. L.MathyG. (2007, September). Beyond images: Gaps in the research on Brazil’s contemporary quilombo communities. Paper presented at the Latin American Studies Association Annual Meeting, Montreal, Canada.
9.
BowenM. L.TillmanA. S. (2013, April). Ser culturalmente sensíveis: Conceptualization, data collection, and analysis in Brazil’s African descent communities. Paper presented at the Center for Culturally Responsive Evaluation and Assessment Inaugural Conference, Chicago, IL.
10.
CarneyJ. A.RosomoffR. N. (2009). In the shadow of slavery: Africa’s botanical legacy in the Atlantic world. Berkeley: University of California.
11.
CarvalhoJ. J.DoriaS. Z.OliveiraA. N. (Eds.). (1996). Quilombo do rio das rãs: Histórias, tradições, lutas. Salvador, Bahia: Edufba.
12.
ChouinardJ.CousinsB. (2009). A review and synthesis of current research on cross-cultural evaluation. American Journal of Evaluation, 30, 457–494.
13.
Comissão Pró-Índio de São Paulo. (2012). Terras quilombolas: Balanço 2011. Retrieved fromhttp://www.cpisp.org.br
14.
CousinsJ. B.WhitmoreE. (1998). Framing participatory evaluation. New Directions for Evaluation, 80, 5–23.
15.
DenzinN. K.LincolnY. S. (Eds.). (2005). The sage handbook of qualitative research (3rd ed.). Thousand Oaks, CA: Sage.
16.
De SotoH. (2001). The mystery of capital. Why capitalism triumphs in the west and fails everywhere else. London, England: Black Swan.
17.
DwyerS. C.BuckleJ. L. (2009). The space between: On being an insider-outsider in qualitative research. International Journal of Qualitative Methods, 8, 54–63.
18.
EdelmanM. (2013). Messy hectares: Questions about the epistemology of land grabbing data. Journal of Peasant Studies, 40, 485–501.
19.
FiabaniA. (2005). Mato, palhoça e pilão: O quilombo, da escravidão as comunidades remanescentes, 1532-2004. São Paulo, Brazil: Expressão Popular.
20.
FontanaA.FreyJ. H. (2005). The interview: From neutral stance to political involvement. In DenzinN. K.LincolnY. S. (Eds.), The Sage Handbook of qualitative research (3rd ed., pp. 695–727). Thousand Oaks, CA: Sage.
21.
FrenchJ. H. (2009). Legalizing identities: Becoming Black or Indian in Brazil’s northeast. Chapel Hill, NC: University of North Carolina Press.
22.
FriersonH. T.HoodS.HughesG. (2002). Strategies that address culturally responsive evaluation. In WestatJ. F. (Ed.), The 2002 user friendly handbook for project evaluation (pp. 63–73). Arlington, VA: National Science Foundation.
23.
FriersonT.HoodS.HughesG. B.ThomasV. G. (2010). A guide to conducting culturally responsive evaluations. In FrechtlingJ. (Ed.), The 2010 user-friendly handbook for project evaluation (pp. 75–96). Arlington, VA: National Science Foundation.
24.
Fundação Cultural Palmares. (2013). Ministério da cultura. Brasília, DF, Brazil. Retrieved fromhttp://www.palmares.gov.br/
25.
GreeneJ. C. (2007). Mixed methods in social inquiry. San Francisco, CA: Jossey-Bass.
26.
HopsonR. (2001). Global and local conversations on culture, diversity, and social justice in evaluation: Issues to consider in a 9/11 era. American Journal of Evaluation, 22, 375–380.
27.
HopsonR. (2009). Reclaiming knowledge at the margins: Culturally responsive evaluation in the current evaluation moment. In RyanK. E.CousinsJ. B. (Eds.), The sage international handbook of educational evaluation (pp. 429–446). Thousand Oaks, CA: Sage.
28.
HouseE. R.HoweK. R. (2000). Deliberative democratic evaluation. In RyanK. E.DeStefanoL. (Eds.), Evaluation as a democratic process: Promoting inclusion, dialogue and deliberation. New directions for evaluation, (Vol. 85., pp. 3–12). San Francisco, CA: Jossey-Bass.
29.
JankieD. (2004). “Tell me who you are:” Problematizing the construction and positionalities of “insider”/“insider of a “Native” ethnographer in a postcolonial context. In MutuaK.SwadenerB. B. (Eds.), Decolonizing research in cross-cultural contexts: Critical personal narratives (pp. 87–105). Albany, NY: SUNY Press.
30.
JimenezM. B. (2012). An examination of the practice of culturally responsive evaluation: A case study of the girls adventures in mathematics, engineering, and science (G.A.M.E.S.) summer camp at the University of Illinois(Doctoral dissertation). Retrieved from ProQuest Dissertations and Theses database. (Accession Order No. 3600687).
31.
KamberelisG.DimitriadisG. (2005). Focus groups: Strategic articulations of pedagogy, politics, and inquiry. In DenzinN. K.LincolnY. S. (Eds.), The sage handbook of qualitative research (3rd ed., pp. 887–907). Thousand Oaks, CA: Sage.
32.
KanuhaV. K. (2000). “Being” native versus “going native”: Conducting social work research as an insider. Social Work, 45, 439–447.
33.
KingJ. A. (1998). Making sense of participatory evaluation. New Directions for Evaluation, 108, 57–67.
34.
KingJ. A. (2007). Making sense of participatory evaluation. New Directions for Evaluation, 114, 83–104.
35.
KingJ. A.NielsenJ. E.ColbyJ. (2004). Lessons for culturally competent evaluation from the study of a multicultural initiative. In Thompson-RobinsonM.HopsonR.SenGuptaS. (Eds.), In search of cultural competence in evaluation: Toward principles and practices, New Directions for Evaluation, (Vol. 102., pp. 67–80). San Francisco, CA: Jossey-Bass.
36.
KirkhartK. E. (2005). Through a cultural lens: Reflections on validity and theory in evaluation. In HoodS.HopsonR.FriersonH. (Eds.), The role of culture and cultural context: A mandate for inclusion, the discovery of truth, and understanding in evaluative theory and practice (pp. 21–39). Greenwich, CT: Information Age Publishing.
37.
KirkhartK. E. (2010). Eyes on the prize: Multicultural validity and evaluation theory. American Journal of Evaluation, 31, 400–413.
38.
LinharesL. F. (2004). Kilombos of Brazil: Identity and land entitlement. Journal of Black Studies, 34, 817–834.
39.
Manswell-ButtyJ. L.ReidM. D.LaPointV. (2004). A culturally responsive evaluation approach applied to the talent development model school-to-career intervention program. In ThomasV.StevensF. (Eds.), Co-constructing a contextually responsive evaluation framework: The talent development model of school reform. New directions for evaluation (pp. 37–49). San Francisco, CA: Jossey Bass.
40.
MutuaK.SwadenerB. B. (Eds.). (2004). Introduction. In Decolonizing research in cross-cultural contexts: Critical personal narratives (pp. 1–23). Albany, NY: SUNY Press.
41.
NdimandeB. S. (2012). Decolonizing research in postapartheid South Africa: The politics of methodology. Qualitative Inquiry, 18, 215–226.
42.
NdimandeB. S. (2013). The role of indigenous languages and focus groups in qualitative inquiry: Experiences from the global south. In DenzinN. K.GiardinaM. D. (Eds.), Global dimensions of qualitative inquiry (pp. 85–102). Walnut Creek, CA: Left Coast Press, Inc.
43.
Ngugi wa Thiong’o (1986). Decolonizing the mind: The politics of language in African literature. Portsmouth, NH: Heinemann.
44.
O’DwyerE. C. (Ed.). (2002). Quilombos: Identidade étnica e territorialidade. Rio de Janeiro, Brazil: Editora FGV.
45.
OyaC. (2013). Rural wage employment in Africa: Methodological issues and emerging evidence. Review of African Political Economy, 40, 251–273.
46.
ParkerL. (2004). Can critical theories of or on race be used in evaluation research in education?New Directions for Evaluation, 101, 85–93.
47.
Rapoport Center for Human Rights and Justice. (2008). Between the law and their land: Afro-Brazilian quilombo communities’ struggle for land rights. Austin, TX: University of Texas.
48.
ReisJ. J.GomesF. S. (Eds.). (1996). Liberdade por um fio: História dos quilombos no Brasil. São Paulo, Brazil: Companhia das Letras.
49.
ReisJ. J.GomesF. S. (2002). Quilombos: Brazilian maroons during slavery. Cultural Survival Quarterly, 24, 16–24.
50.
RyanK. E.ChandlerM.SamuelsM. (2007). What should school-based evaluation look like?Studies in Educational Evaluation, 33, 197–212.
51.
RyanK. E.Gannon-SlaterN.CilbertsonM. (2012). Improving survey methods with cognitive interviews in small-and medium-scale evaluation. American Journal of Evaluation, 33, 414–430.
52.
SamuelsM.RyanK. (2011). Grounding evaluations in culture. American Journal of Evaluations, 32, 83–98.
53.
SantosK. M.TattoN. (2008). Agenda socioambiental de comunidades quilombolas do Vale do Ribeira. São Paulo, Brazil: Instituto Socioambiental.
54.
Secretaria de Políticas de Promoção da Igualdade Racial (SEPPIR). (2013). Programa Brasil quilombola: Relatório de gestão. Brasília, Brazil: SEPPIR.
55.
SenGuptaS.HopsonR.Thompson-RobinsonM. (2004). Cultural competence in evaluation: An overview. New Directions for Evaluation, 102, 5–19.
56.
SilberlingL. S. (2003). Displacement and quilombos in Alcântara, Brazil: Modernity, identity and place. International Social Science Journal, 175, 145–157.
57.
SmithL. T. (1999). Decolonizing methodologies: Research and indigenous peoples. Dunedin, New Zealand: University of Otago Press.
58.
StakeR. E. (1973). Program evaluation, particularly responsive evaluation. Keynote address at the conference “New Trends in Evaluation,” Institute of Education, University of Goteborg, Sweden, October 1973. In MadausG. F.ScrivenM. S.StufflebeamD. L. (Eds.), Evaluation models: View points on education and human services evaluation. Boston, MA: Kluwer-Nijhoff, 1987.
59.
StakeR. E. (1975). (Ed.). Evaluating the arts in education. Columbus, OH: Merrill.
60.
StakeR. E. (2003). Standards-based and responsive evaluation. Thousand Oaks, CA: Sage.
61.
ThomasV. G. (2004). Building a contextually responsive evaluation framework: Lessons from working with urban school interventions. New Directions for Evaluation, 101, 3–24.
62.
ThomasV. G.StevensF. I. (2004). Editor’s notes. New Directions for Evaluation, 101, 1–2.
63.
Universidade de Brasilia. (2004). Comunidades quilombolas: Levantamento socio-econômico-cultural. Relatório estatística qualitative e quantitative das oficinas. Brasilia, Brazil: MDS/MDA/SEPPIR/FCP.
