Transcending the Trade-off Between External and Internal Validity: A Critical Realist Perspective

Abstract

This article contributes to the mixed methods literature by proposing a critical realist philosophy of science to resolve the apparent, and ultimately unfruitful, trade-off between internal and external validity. It is argued that this trade-off is a false dichotomy rooted in contestable philosophical assumptions about causality. Thanks to its stratified ontology and epistemological focus on causal mechanisms, critical realism allows for reconceptualizing internal and external validity as interdependent components of explanatory adequacy. Identifying causal mechanisms is a prerequisite for understanding the scope conditions under which those mechanisms operate. Testing these scope conditions, in turn, strengthens confidence in the initial explanation. The article thus provides mixed methods research with a philosophical foundation for an integrated understanding of validity.

Keywords

qualitative methods validity critical realism causality scope conditions

Get full access to this article

View all access options for this article.

References

Allison

(1971). Essence of decision: Explaining the Cuban missile crisis. Little, Brown and Company.

Archer

M. S.

(1995). Realist social theory: The morphogenetic approach. Cambridge University Press.

Beach

Pedersen

R. B.

(2013). Process-tracing methods: Foundations and guidelines. University of Michigan Press.

Bean

(1973). War and the birth of the nation state. Journal of Economic History, 33(1), 203–221. https://doi.org/10.1017/s0022050700076531

Bhaskar

(1975). A realist theory of science. Leeds Books.

Bhaskar

(1979). The possibility of naturalism: A philosophical critique of the contemporary human sciences. Harvester Press.

Blumer

(1969). Symbolic interactionism. Prentice-Hall.

Brady

H. E.

Collier

(2004). Rethinking social inquiry: Diverse tools, shared standards. Rowman & Littlefield.

Byrne

(2002). Complexity theory and the social sciences: An introduction. Routledge.

10.

Campbell

D. T.

(1974). Evolutionary epistemology. In Schilpp

P. A.

(Ed.), The philosophy of Karl Popper (pp. 412–463). Open Court.

11.

Collier

(2011). Understanding process tracing. PS: Political Science & Politics, 44(4), 823–830. https://doi.org/10.1017/S1049096511001429

12.

Collier

Mahoney

(1996). Insights and pitfalls: Selection bias in qualitative research. World Politics, 49(1), 56–91. https://doi.org/10.1353/wp.1996.0023

13.

Danermark

Ekström

Karlsson

J. C.

(2019). Explaining Society: Critical realism in the social sciences (2nd ed.). Routledge.

14.

Demiryol

(2023). Economic crisis, elite conflict and institutional change in empires. Peter Lang GmbH, Internationaler Verlag der Wissenschaften.

15.

Ertman

(1997). Birth of the leviathan: Building states and regimes in medieval and early modern Europe. Cambridge University Press.

16.

Evans

B. C.

Coon

D. W.

Ume

(2011). Use of theoretical frameworks as a pragmatic guide for mixed methods studies: A methodological necessity? Journal of Mixed Methods Research, 5(4), 276–292. https://doi.org/10.1177/1558689811412972

17.

Fearon

J. D.

(1994). Domestic political audiences and the escalation of international disputes. American Political Science Review, 88(3), 577–592. https://doi.org/10.2307/2944796

18.

Flyvbjerg

(2006). Five misunderstandings about case-study research. Qualitative Inquiry, 12(2), 219–245. https://doi.org/10.1177/1077800405284363

19.

George

A. L.

Bennett

(2005). Case studies and theory development in the social sciences. MIT Press.

20.

Gerring

(2007). Is there a (Viable) crucial-case method? Comparative Political Studies, 40(3), 231–253. https://doi.org/10.1177/0010414006290784

21.

Gerring

(2017). Case study research. Cambridge University Press.

22.

Glynos

Howarth

(2007). Logics of critical explanation in social and political theory. Routledge.

23.

Goertz

Haggard

(2023). Large-N qualitative analysis (LNQA): Causal generalization in case study and multimethod research. Perspectives on Politics, 21(4), 1221–1239. https://doi.org/10.1017/S1537592723002037

24.

Gorski

P. S.

(2013). What is critical realism? And why should you care? Contemporary Sociology, 42(5), 658–670. https://doi.org/10.1177/0094306113499533

25.

Hedström

Swedberg

(1998). Social mechanisms: An analytical approach to social theory. Cambridge University Press.

26.

Hempel

(2014). The function of general laws in history. In Philosophy, science, and history Editor: Lydia Patton (pp. 48–60). Routledge.

27.

Hicken

(2011). Clientelism. Annual Review of Political Science, 14(1), 289–310. https://doi.org/10.1146/annurev.polisci.031908.220508

28.

Hume

(2007). An enquiry concerning human understanding. Oxford University Press.

29.

King

Keohane

R. O.

Verba

(1994). Designing social inquiry: Scientific inference in qualitative research. Princeton University Press.

30.

Kitschelt

Wilkinson

S. I.

(Eds.), (2007). Patrons, clients and policies: Patterns of democratic accountability and political competition. Cambridge University Press. https://doi.org/10.1017/CBO9780511585869

31.

Lawson

(2004). Economics and reality. Routledge.

32.

Levy

J. S.

(2008). Case studies: Types, designs, and logics of inference. Conflict Management and Peace Science, 25(1), 1–18. https://doi.org/10.1080/07388940701860318

33.

Little

(1998). Microfoundations, method and causation: On the philosophy of the social sciences. Transaction Publishers.

34.

Mahoney

Goertz

(2006). A tale of two cultures: Contrasting quantitative and qualitative research. Political Analysis, 14(3), 227–249. https://doi.org/10.1093/pan/mpj017

35.

Massimi

(2004). Non-defensible middle ground for experimental realism: Why we are justified to believe in colored quarks. Philosophy of Science, 71(1), 36–60. https://doi.org/10.1086/381412

36.

Maxwell

J. A.

(1992). Understanding and validity in qualitative research. Harvard Educational Review, 62(3), 279–300. https://doi.org/10.17763/haer.62.3.8323320856251826

37.

Maxwell

J. A.

(2004). Causal explanation, qualitative research, and scientific inquiry in education. Educational Researcher, 33(2), 3–11. https://doi.org/10.3102/0013189X033002003

38.

Maxwell

J. A.

(2012a). A realist approach for qualitative research. Sage.

39.

Maxwell

J. A.

(2012b). The importance of qualitative research for causal explanation in education. Qualitative Inquiry, 18(8), 655–661. https://doi.org/10.1177/1077800412452856

40.

Maxwell

J. A.

(2017). The validity and reliability of research: A realist perspective. In Wyse

Selwyn

Smith

Suter

L. E.

(Eds.), The BERA/SAGE handbook of educational research (pp. 116–140). Sage Publications Ltd.

41.

Maxwell

J. A.

(2020). Why qualitative methods are necessary for generalization. Qualitative Psychology, 8(1), 111–118. https://doi.org/10.1037/qup0000173

42.

Maxwell

J. A.

(2022). Critical realism as a stance for designing qualitative research. In Flick

(Ed.), The SAGE handbook of qualitative research design (pp. 142–154). Sage Publications Ltd.

43.

Miles

M. B.

Huberman

A. M.

(1994). Qualitative data analysis: An expanded sourcebook. Sage.

44.

Mitchell

Massoud

(2009). Anatomy of failure: Bush’s decision‐making process and the Iraq war. Foreign Policy Analysis, 5(3), 265–286. https://doi.org/10.1111/j.1743-8594.2009.00093.x

45.

Mukumbang

F. C.

(2021). Retroductive theorizing: A contribution of critical realism to mixed methods research. Journal of Mixed Methods Research, 17(1), 93–114. https://doi.org/10.1177/15586898211049847

46.

Oneal

J. R.

Ray

J. L.

(1997). New tests of the democratic peace: Controlling for economic interdependence, 1950-85. Political Research Quarterly, 50(4), 751–775. https://doi.org/10.1177/106591299705000402

47.

Pawson

Tilley

(1997). Realistic evaluation. Sage Publications Ltd.

48.

Popper

(2005). The logic of scientific discovery. Routledge.

49.

Putnam

R. D.

(1994). Making democracy work: Civic traditions in modern Italy. Princeton University Press.

50.

Putnam

R. D.

(2001). Bowling alone: The collapse and revival of American community. Simon & Schuster.

51.

Ragin

C. C.

(2000). Fuzzy-set social science. University of Chicago Press.

52.

Ragin

C. C.

(2009). Redesigning social inquiry: Fuzzy sets and beyond. University of Chicago Press.

53.

Ronkainen

N. J.

Wiltshire

(2021). Rethinking validity in qualitative sport and exercise psychology research: A realist perspective. International Journal of Sport and Exercise Psychology, 19(1), 13–28. https://doi.org/10.1080/1612197X.2019.1637363

54.

Russett

Layne

Spiro

D. E.

Doyle

M. W.

(1995). The democratic peace. International Security, 19(4), 164–184.

55.

Sayer

(2000). Realism and social science. Sage.

56.

Schneider

C. Q.

Wagemann

(2012). Set-Theoretic methods for the social sciences: A guide to qualitative comparative analysis. Cambridge University Press. https://doi.org/10.1017/CBO9781139004244

57.

Seawright

Gerring

(2008). Case selection techniques in case Study research: A menu of qualitative and quantitative options. Political Research Quarterly, 61(2), 294–308. https://doi.org/10.1177/1065912907313077

58.

Shadish

W. R.

Cook

T. D.

Campbell

D. T.

(2001). Experimental and quasi-experimental designs for generalized causal inference. Cengage Learning.

59.

Spruyt

(1994). The sovereign state and its competitors: An analysis of systems change. Princeton University Press.

60.

Stokes

S. C.

(2005). Perverse accountability: A formal model of machine politics with evidence from Argentina. American Political Science Review, 99(3), 315–325. https://doi.org/10.1017/s0003055405051683

61.

Stokes

S. C.

Dunning

Nazareno

Brusco

(2013). Brokers, voters, and clientelism. Cambridge University Press. https://doi.org/10.1017/CBO9781107324909

62.

Zachariadis

Scott

Barrett

(2013). Methodological implications of critical realism for mixed methods research. MIS Quarterly, 37(3), 855–879. https://doi.org/10.25300/misq/2013/37.3.09