There has recently been a tremendous growth in systematic literature review methods in social science, which may create confusion as to the role of researchers and reviewers. I argue that the tale of the Blind men and the elephants is a metaphor that illuminates the elusive nature of reality and the work of both primary researchers and various types of reviewers. The value of the metaphor is illustrated with a case drawn from the field of policy and program evaluation. Despite its limitations, this metaphor may lead social scientists to reflect upon the rigor of their literature review practices.
ArkseyH., and O'MalleyL. (2005) ‘Scoping Studies: Towards a Methodological Framework’, International Journal of Social Research Methodology, 8(1) 19–32.
BossonJ.K.SwannW.B.JrPennebakerJ.W. (2000) ‘Stalking the Perfect Measure of Implicit Self-Esteem: The Blind Men and the Elephant Revisited?’, Journal of Personality and Social Psychology, 79(4) 631–643.
4.
BrisolaraS. (1998) ‘The History of Participatory Evaluation and Current Debates in the Field’, New Directions for Evaluation, (80) 25–41.
5.
BrownT.L. (2003) Making Truth: Metaphor in Science. Urbana, IL: University of Illinois Press.
6.
CoupalF.P., and SimoneauM. (1998) ‘A Case Study of Participatory Evaluation in Haiti’, New Directions for Evaluation (80) 69–79.
7.
CousinsJ.B. (2003) ‘Utilization Effects of Participatory Evaluation’, in KellaghanT. and StufflebeamD.L. (eds.) International Handbook of Educational Evaluation (vol. 9). Dordrecht: Kluwer Academic, 245–265.
8.
CousinsJ. B., and ChouinardJ. A. (2012) Participatory Evaluation Up Close: An Integration of Research-Based Knowledge. Charlotte, NC: Information Age Publishing.
9.
DaigneaultP.-M., and JacobS. (2009) ‘Toward Accurate Measurement of Participation: Rethinking the Conceptualization and Operationalization of Participatory Evaluation’, American Journal of Evaluation, 30(3) 330–348.
10.
DaigneaultP.-M., and JacobS. (2013) ‘Unexpected but Most Welcome: Mixed Methods for the Validation and Revision of the Participatory Evaluation Measurement Instrument’, Journal of Mixed Methods Research. [Published online before print on May 7. Available online from: http://mmr.sagepub.com/content/early/2013/05/07/1558689813486190.abstract — Accessed July 16, 2013. Doi: 10.1177/1558689813486190].
11.
DaigneaultP.-M.JacobS., and OuimetM. (2012) ‘Using Systematic Review Methods Within a Ph.D. Dissertation in Political Science: Challenges and Lessons Learned from Practice’, International Journal of Social Research Methodology. [Published online before print on October 9. Available online from: http://www.tandfonline.com/doi/abs/10.1080/13645579.2012.730704?journalCode=tsrm20#.UeanwIcVGSo — Accessed April 6, 2013. Doi:10.1038/news.2011.115]
12.
DaigneaultP.-M.JacobS., and TremblayJ. (2012) ‘Measuring Stakeholder Participation in Evaluation: An Empirical Validation of the Participatory Evaluation Measurement Instrument’, Evaluation Review, 36(4) 243–270.
13.
DaigneaultP.-M.JacobS., and TererahoM. (2012) ‘Understanding and Improving the Take-up of Public Programs: Lessons Learned from the Canadian and International Experience in Human Services’, International Journal of Business and Social Science, 3(1) 39–50.
14.
GanannR.CiliskaD., and ThomasH. (2010) ‘Expediting Systematic Reviews: Methods and Implications of Rapid Reviews’, Implementation Science, 5(1) 56. [Available online from: http://www.implementationscience.com/content/5/1/56 —Accessed April 6, 2013. Doi:10.1186/1748-5908-5-56]
GrantM.J., and BoothA. (2009) ‘A Typology of Reviews: An Analysis of 14 Review Types and Associated Methodologies’, Health Information & Libraries Journal, 26(2) 91–108.
17.
GreeneJ.G. (1988) ‘Stakeholder Participation and Utilization in Program Evaluation’, Evaluation Review, 12(2) 91–116.
18.
GregoryA. (2000) ‘Problematizing Participation: A Critical Review of Approaches to Participation in Evaluation Theory’, Evaluation, 6(2) 179–199.
19.
HammersleyM. (2001) ‘On “Systematic” Reviews of Research Literatures: A “Narrative” Response to Evans & Benefield’, British Educational Research Journal, 27(5) 543–554.
20.
HammersleyM. (2008) ‘Paradigm War Revived? On the Diagnosis of Resistance to Randomized Controlled Trials and Systematic Review in Education’, International Journal of Research & Method in Education, 31(1), 3–10.
21.
HannesK., and HardenA. (2011) ‘Multi-Context Versus Context-Specific Qualitative Evidence Syntheses: Combining the Best of Both’, Research Synthesis Methods, 2(4), 271–278.
22.
HansenH.F., and RieperO. (2009) 'The Evidence Movement: The Development and Consequences of Methodologies in Review Practices, Evaluation, 15(2) 141–163.
23.
HimmelfarbJ.StenvinkelP., and IkizlerT.A., and HakimR.M. (2002) ‘The Elephant in Uremia: Oxidant Stress as a Unifying Concept of Cardiovascular Disease in Uremia’, Kidney Int, 62(5) 1524–1538.
24.
KingG.KeohaneR.O., and VerbaS. (1994) Designing Social Inquiry: Scientific Inference in Qualitative Research. Princeton: Princeton University Press.
25.
LevacD.ColquhounH., and O'BrienK. (2010). ‘Scoping Studies: Advancing the Methodology’, Implementation Science, 5(1): 69. [Available online from: http://www.implementationscience.com/content/5/1/69 — Accessed April 2, 2013].
26.
Loffler-LaurianA.M. (1994) ‘Réflexions sur la métaphore dans les discours scientifiques de vulgarisation [Reflections on the Metapor in Scientific Discourses of Vulgarization]’, Langue française, (101) 72–79.
27.
LorencT.PearsonM.JamalF.CooperC., and GarsideR. (2012) ‘The Role of Systematic Reviews of Qualitative Evidence in Evaluating Interventions: A Case Study’, Research Synthesis Methods, 3(1) 1–10.
28.
MacGregorS. (2011) ‘The Impact of Research on Policy in the Drugs Field’, Methodological Innovations Online, 6(1) 41–57.
29.
OakleyA. (2002) ‘Social Science and Evidence-Based Everything: The Case of Education’, Educational Review, 54(3) 277–286.
30.
PawsonR.GreenhalghT.HarveyG., and WalsheK. (2005) ‘Realist Review – A New Method of Systematic Review Designed for Complex Policy Interventions’, Journal of Health Service Research & Policy, 10(suppl_1) 21–34.
31.
PetticrewM., and RobertsH. (2006) Systematic Reviews in the Social Sciences: A Practical Guide. Malden, MA: Blackwell.
32.
RubinH.J., and RubinI.S. (2011) Qualitative Interviewing: The Art of Hearing Data (3rd ed.). Thousand Oaks, CA: Sage.
33.
SandelowskiM., and BarrosoJ. (2007) Handbook for Synthesizing Qualitative Research. New York: Springer.
34.
StanleyL. (2012) ‘Rethinking the Definition and Role of Ontology in Political Science’, Politics, 32(2):93–99.
