Strange Bedfellows: Exploring Methodological Intersections Between Realist Inquiry and Structural Equation Modeling

Abstract

Realist inquiry, based on the philosophy of critical realism, focuses on exploring the underlying mechanisms that drive social phenomena. Structural equation modeling is a collection of quantitative analytical methods that take a theory-based, confirmatory approach to examining statistical relationships between measured (observable) and underlying (latent) variables. Despite originating from different scientific traditions, the apparent similarities between these two approaches hold promise for their combination in mixed methods research. This article contributes to the field of mixed methods research by exploring their potential synergies, how each approach could contribute to the other, and proposing a framework for their combinations in mixed methods research, which has implications in terms of the implied and explicit ontological and epistemological positionings of these two approaches.

Keywords

context realist inquiry structural equation modeling mixed methods research medical education

Get full access to this article

View all access options for this article.

References

Archer

M. S.

(1995). Realist social theory: The morphogenetic approach. Cambridge University Press. https://doi.org/10.1017/cbo9780511557675

Astbury

Leeuw

F. L.

(2010). Unpacking black boxes: Mechanisms and theory building in evaluation. American Journal of Evaluation, 31(3), 363-381. https://doi.org/10.1177/1098214010371972

Bates

Ellaway

R. H.

(2016). Mapping the dark matter of context: A conceptual scoping review. Medical Education, 50(8), 807-816. https://doi.org/10.1111/medu.13034

Bhaskar

(1975). A realist theory of science. Leeds Books.

Bhaskar

(1979). The possibility of naturalism: A philosophical critique of the contemporary human sciences. Humanities Press.

Bhaskar

(1989). Reclaiming reality: A critical introduction to contemporary philosophy. Taylor & Francis. https://doi.org/10.4324/9780203843314

Bickman

(1987). The functions of program theory. New Directions for Program Evaluation, 1987(33), 5-18. https://doi.org/10.1002/ev.1443

Blalock

H. M.

(1963). Making causal inferences for unmeasured variables from correlations among indicators. American Journal of Sociology, 69(1), 53-62. https://doi.org/10.1086/223510

Bok

H. G. J.

de Jong

L. H.

O’Neill

Maxey

Hecker

K. G.

(2018). Validity evidence for programmatic assessment in competency-based education. Perspectives on Medical Education, 7(6), 362-372. https://doi.org/10.1007/s40037-018-0481-2

10.

Bollen

K. A.

(1989). Structural equations with latent variables. John Wiley & Sons. https://doi.org/10.1002/9781118619179

11.

Bollen

K. A.

Long

J. S.

(1993). Testing structural equation models. Sage.

12.

Brennan

R. L.

(2006). Educational measurement (American Council on Education/Praeger series on higher education). Praeger.

13.

Bullock

H. E.

Harlow

L. L.

Mulaik

S. A.

(1994). Causation issues in structural equation modeling research. Structural Equation Modeling: A Multidisciplinary Journal, 1(3), 253-267. https://doi.org/10.1080/10705519409539977

14.

Cartwright

(1999). The dappled world: A study of the boundaries of science. Routledge. https://doi.org/10.1017/CBO9781139167093

15.

Cartwright

(2000). Against the completability of science. In Wolff

Stone

(Eds.), The proper ambition of science (pp. 209-222). Routledge. https://doi.org/10.4324/9780203446263

16.

Cartwright

Hardie

(2012). Evidence-based policy: A practical guide to doing it better. Oxford University Press.

17.

Cole

(1996). Cultural psychology: A once and future discipline. Harvard University Press.

18.

Creswell

J. W.

(2011). A concise introduction to mixed methods research. Sage.

19.

Creswell

J. W.

Plano Clark

V. L.

(2010). Designing and conducting mixed methods research (2nd ed.). Sage.

20.

Creswell

J. W.

Plano Clark

V. L.

Gutmann

Hanson

W. E.

(2003). Advanced mixed methods research designs. In Tashakkori

Teddlie

(Eds.), Handbook of mixed methods in social and behavioural research (pp. 209-240). Sage.

21.

Crocker

Algina

(1986). Introduction to classical and modern test theory. Wadsworth. https://doi.org/10.1006/geno.1997.4967

22.

Douven

(2017). Abduction (The Stanford Encyclopedia of philosophy). https://doi.org/10.5860/choice.43sup-0193

23.

Elger

(2012). Critical realism. In Encyclopedia of case study research (pp. 254-257). Sage. https://doi.org/10.4135/9781412957397

24.

Ellaway

R. H.

Kehoe

Illing

(2020). Critical realism and realist inquiry in medical education. Academic Medicine, 95(7), 984-988. https://doi.org/10.1097/acm.0000000000003232

25.

Ellaway

R. H.

Mackay

M. P.

Lee

Hofmeister

Malin

Archibald

Lawrence

Dogba

Côté

Ross

(2018). The impact of a national competency-based medical education initiative in family medicine. Academic Medicine, 93(12), 1850-1857. https://doi.org/10.1097/ACM.0000000000002387

26.

Ellaway

R. H.

Ogorman

Strasser

Marsh

D. C.

Graves

Fink

Cervin

(2016). A critical hybrid realist-outcomes systematic review of relationships between medical education programmes and communities: BEME Guide No. 35. Medical Teacher, 38(3), 229-245. https://doi.org/10.3109/0142159X.2015.1112894

27.

Emmel

Greenhalgh

Manzano

Monaghan

Dalkin

(2018). Doing realist research. Sage. https://doi.org/https://dx.doi.org/10.4135/9781526451729

28.

Fetters

M. D.

Molina-Azorin

J. F.

(2020). Utilizing a mixed methods approach for conducting interventional evaluations. Journal of Mixed Methods Research, 14(2), 131-144. https://doi.org/10.1177/1558689820912856

29.

Fletcher

A. J.

(2017). Applying critical realism in qualitative research: Methodology meets method. International Journal of Social Research Methodology, 20(2), 181-194. https://doi.org/10.1080/13645579.2016.1144401

30.

Flyvbjerg

(2001). Making social science matter: Why social inquiry fails and how it can succeed again. Cambridge University Press.

31.

Ghiara

(2020). Disambiguating the role of paradigms in mixed methods research. Journal of Mixed Methods Research, 14(1), 11-25. https://doi.org/10.1177/1558689818819928

32.

Gibbons

(1999). Science’s new social contract with society. Nature, 402(6761), C81-C84. https://doi.org/10.1038/35011576

33.

Greene

J. C.

Caracelli

V. J.

Graham

W. F.

(1989). Toward a conceptual framework for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 11(3), 255-274. https://doi.org/10.2307/1163620

34.

Greenhalgh

Wong

Westhorp

Pawson

(2011). Protocol-realist and meta-narrative evidence synthesis: Evolving standards (RAMESES). BMC Medical Research Methodology, 11(1), 115. https://doi.org/10.1186/1471-2288-11-115

35.

Hathcoat

J. D.

Meixner

(2017). Pragmatism, factor analysis, and the conditional incompatibility thesis in mixed methods research. Journal of Mixed Methods Research, 11(4), 433-449. https://doi.org/10.1177/1558689815622114

36.

Hauser

R. M.

Goldberger

A. S.

(1971). The treatment of unobservable variables in path analysis. Sociological Methodology, 3, 81-117. https://doi.org/10.2307/270819

37.

Hecker

Violato

(2009). Medical school curricula: Do curricular approaches affect competence in medicine? Family Medicine, 41(6), 420-426.

38.

Howe

K. R.

(1988). Against the quantitative-qualitative incompatibility thesis or dogmas die hard. Educational Researcher, 17(8), 10-16. https://doi.org/10.3102/0013189x017008010

39.

Jackson

D. L.

(2003). Revisiting sample size and number of parameter estimates: Some support for the N:q hypothesis. Structural Equation Modeling: A Multidisciplinary Journal, 10(1), 128-141. https://doi.org/10.1207/S15328007SEM1001

40.

Jöreskog

K. G.

(1970). A general method for estimating a linear structural equation system. ETS Research Bulletin Series, 1970(2), i-41. https://doi.org/10.1002/j.2333-8504.1970.tb00783.x

41.

Kline

R. B.

(2016). Principles and practice of structural equation modeling. Guilford Press.

42.

Lacouture

Breton

Guichard

Ridde

(2015). The concept of mechanism from a realist approach: A scoping review to facilitate its operationalization in public health program evaluation. Implementation Science, 10(1), 1-10. https://doi.org/10.1186/s13012-015-0345-7

43.

Lawson

(1994). A realist theory for economics (New Directions in Economic Methodology). Routledge.

44.

C. C.

(1975). Path analysis: A primer. The Boxwood Press.

45.

Linnan

Steckler

(2002). Process evaluation for public health interventions and research: An overview. Jossey-Bass.

46.

Lowry

P. B.

Gaskin

(2014). Partial least squares (PLS) structural equation modeling (SEM) for building and testing behavioral causal theory: When to choose it and how to use it. IEEE Transactions on Professional Communication, 57(2), 123-146. https://doi.org/10.1109/TPC.2014.2312452

47.

Maxwell

J. A.

(2012). A realist approach for qualitative research. Sage.

48.

Maxwell

J. A.

(2019). Distinguishing between quantitative and qualitative research: A response to Morgan. Journal of Mixed Methods Research, 13(2), 132-137. https://doi.org/10.1177/1558689819828255

49.

Merton

R. K.

(1949). On sociological theories of the middle range. Free Press.

50.

Morgan

D. L.

(2014). Pragmatism as a paradigm for social research. Qualitative Inquiry, 20(8), 1045-1053. https://doi.org/10.1177/1077800413513733

51.

Nastasi

B. K.

Hitchcock

J. H.

Brown

L. M.

(2010). An inclusive framework for conceptualizing mixed methods design typologies: Moving toward fully integrated synergistic research models. In SAGE handbook of mixed methods in social and behavioral research (pp. 305-338). Sage. https://doi.org/10.4135/9781506335193.n13

52.

Norman

(2014). Context, curriculum and competence. Advances in Health Sciences Education, 19(5), 281-289. https://doi.org/10.1007/s10459-014-9565-x

53.

Norris

J. M.

Hecker

K. G.

Rabatach

Noseworthy

T. W.

White

D. E.

(2017). Development and psychometric testing of the clinical networks engagement tool. PLOS ONE, 12(3), Article e0174056. https://doi.org/10.1371/journal.pone.0174056

54.

Nunnally

Bernstein

(1994). Psychometric theory. McGraw Hill.

55.

O’Cathain

(2018). A practical guide to using qualitative research with randomized controlled trials. Oxford University Press. https://doi.org/10.1093/med/9780198802082.001.0001

56.

Oliver

Hecker

Hausdorf

P. A.

Conlon

(2014). Validating MMI scores: Are we measuring multiple attributes? Advances in Health Sciences Education, 19(3), 379-392. https://doi.org/10.1007/s10459-013-9480-6

57.

Pawson

(1978). Empiricist explanatory strategies: The case of causal modelling. Sociological Review, 26(3), 613-646. https://doi.org/10.1111/j.1467-954X.1978.tb00148.x

58.

Pawson

(1989). A measure for measures: a manifesto for empirical sociology. Routledge.

59.

Pawson

(2006). Evidence-based policy: A realist perspective. Sage.

60.

Pawson

(2008). Causality for beginners. National Centre for Research Methods Festival.

61.

Pawson

(2013). The science of evaluation: a realist manifesto. Sage.

62.

Pawson

Tilley

(1997). Realistic evaluation. Sage.

63.

Pearl

(2012). The causal foundations of structural equation modeling. In Handbook of structural equation modelling (pp. 68-91). Guildford Press.

64.

Regehr

(2006). The persistent myth of stability: On the chronic underestimation of the role of context in behavior. Journal of General Internal Medicine, 21, 544-545. https://doi.org/10.1111/j.1525-1497.2006.00447.x

65.

Rogosa

(1987). Casual models do not support scientific conclusions: A comment in support of Freedman. Journal of Educational Statistics, 12(2), 185-195. https://doi.org/10.3102/10769986012002185

66.

Rogosa

(1988). Myths about longitudinal research. In Methodological issues in aging research (pp. 171-209). Springer.

67.

Sandelowski

Voils

C. I.

Leeman

Crandell

J. L.

(2012). Mapping the mixed methods–mixed research synthesis terrain. Journal of Mixed Methods Research, 6(4), 317-331. https://doi.org/10.1177/1558689811427913

68.

Sommer Harrits

. (2011). More than method? A discussion of paradigm differences within mixed methods research. Journal of Mixed Methods Research, 5(2), 150-166. https://doi.org/10.1177/1558689811402506

69.

Spearman

(1904). “General intelligence,” objectively determined and measured. American Journal of Psychology, 15(2), 201-292. https://doi.org/10.2307/1412107

70.

Tarka

(2018). An overview of structural equation modeling: Its beginnings, historical development, usefulness and controversies in the social sciences. Quality & Quantity, 52(1), 313-354. https://doi.org/10.1007/s11135-017-0469-8

71.

Tashakkori

Teddlie

(2003). Handbook of mixed methods in social and behavioral research. Sage.

72.

Tesson

Strasser

Pong

R. W.

Curran

(2005). Advances in rural medical education in three countries: Canada, the United States and Australia. Rural and Remote Health, 18(3), 405-415. https://doi.org/10.1080/13576280500289728

73.

Thurstone

L. L.

(1931). The measurement of social attitudes. Journal of Abnormal and Social Psychology, 26(3), 249-269. https://doi.org/10.1037/h0070363

74.

Tyler

R. W.

(1983). A rationale for program evaluation. In Evaluation models (pp. 67-78). Springer. https://doi.org/10.1007/978-94-009-6669-7_4

75.

Violato

Hecker

K. G.

(2007). How to use structural equation modeling in medical education research: A brief guide. Teaching and Learning in Medicine, 19(4), 362-371. https://doi.org/10.1080/10401330701542685

76.

Walters

Greenhill

Richards

Ward

Campbell

Ash

Schuwirth

L. W. T.

(2012). Outcomes of longitudinal integrated clinical placements for students, clinicians and society. Medical Education, 46(11), 1028-1041. https://doi.org/10.1111/j.1365-2923.2012.04331.x

77.

Westhorp

(2018). Understanding mechanisms in realist evaluation and research. In Doing realist research (pp. 41-58). Sage. https://doi.org/10.4135/9781526451729.n4

78.

Wholey

J. S.

(1987). Evaluability assessment: Developing program theory. New Directions for Program Evaluation, 1987(33), 77-92. https://doi.org/10.1002/ev.1447

79.

Williams

G. H.

(2003). The determinants of health: Structure, context and agency. Sociology of Health & Illness, 25(3), 131-154. https://doi.org/10.1111/1467-9566.00344

80.

Wong

Greenhalgh

Westhorp

Pawson

(2012). Realist methods in medical education research: What are they and what can they contribute? Medical Education, 46(1), 89-96. https://doi.org/10.1111/j.1365-2923.2011.04045.x

81.

Wong

Westhorp

Pawson

Greenhalgh

(2013). Realist synthesis: RAMESES training materials. RAMESES Project.

82.

Wright

(1934). The method of path coefficients. Annals of Mathematical Statistics, 5(3), 161-215. https://doi.org/10.1214/aoms/1177732676