Pawson and Tilley’s acknowledgment of programs embedded in multiple social systems has gained little traction in realist synthesis and evaluation practice. A practice focusing on fairly closed systems—explaining how programs work and do not work—has emerged. This article negotiates the boundaries of knowledge pertinent to have in program design and evaluation from a realist perspective. It highlights critical realism as another possible response to systems thinking in evaluation. Moving one level up a program, it theorizes about social structures, mechanisms, and causes operating in a complex system within which an education-to-work program is nested. Three implications of the approach are highlighted: it foregrounds the relational nature of social, psychological, and programmatic structures and mechanisms; enables policymakers to develop a broader understanding of structures needed to support a program; and enables program architects to ascertain how a planned program might assimilate and adapt to social structures and mechanisms already established in a context.
Al-AmoudiIWillmottH (2011) Where constructionism and critical realism converge: Interrogating the domain of epistemological relativism. Organization Studies32(1): 27–46.
2.
AlqazlanSAlallawiBTotsikaV (2019) Post-secondary education for young people with intellectual disabilities: A systematic review of stakeholders’ experiences. Educational Research Review28: 100295.
3.
ArcherMS (1995) Realist Social Theory: The Morphogenetic Approach. Cambridge: Cambridge University Press.
4.
ArcherMS (1996) Culture and Agency: The Place of Culture in Social Theory, revised edn. Cambridge, MA: Cambridge University Press.
5.
ArcherMS (1998) Social theory and the analysis of society. In: MayTWilliamsM (eds) Knowing the Social World. Buckingham; Philadelphia, PA: Open University Press, 69–85.
6.
ArcherMS (2014) Social Origins of Educational Systems. London: Routledge.
7.
ArcherMS (2020) The morphogenetic approach; critical realism’s explanatory framework approach. In: RónaPZsolnaiL (eds) Agency and Causal Explanation in Economics, vol. 5. Cham: Springer, 137–50. DOI: 10.1007/978-3-030-26114-6_9.
8.
ArcherMSElder-VassD (2012) Cultural system or norm circles? An exchange. European Journal of Social Theory15: 93–115.
BarnettP (2015) Transition to Employment for Those with Disabilities: Policies, Funding and Planning in the USA. Carlton, VIC: International Specialised Skills Institute.
14.
BelrhitiZVan DammeWBelaliaA, et al. (2020) Unravelling the role of leadership in motivation of health workers in a Moroccan public hospital: A realist evaluation. BMJ Open10(1): e031160.
15.
BhaskarR (1975/2008) A Realist Theory of Science. London and New York: Routledge.
16.
BhaskarR (1998) The Possibility of Naturalism: A Philosophical Critique of the Contemporary Human Sciences. Florence, KY: Routledge.
17.
BhaskarR (2005) How to change reality: Story v. structure—A debate between Rom Harre and Roy Bhaskar. In: LopezJPotterG (eds) After Postmodernism: An Introduction to Critical Realism. London: The Athlone Press, 22–39.
18.
Buch-HansenH (2005) Critical realism in the social sciences: An interview with Roy Bhaskar. Distinktion: Scandinavian Journal of Social Theory6(2): 59–69.
19.
CareyGDickinsonHMalbonE, et al. (2018) The vexed question of market stewardship in the public sector: Examining equity and the social contract through the Australian National Disability Insurance Scheme. Social Policy & Administration52(1): 387–407.
20.
ConsidineMPNguyenPO’SullivanS (2018) New public management and the rule of economic incentives: Australian welfare-to-work from job market signalling perspective. Public Management Review20(8): 1186–204.
21.
DanermarkBEkstromMJakobsenL, et al. (2002) Explaining Society: Critical Realism in the Social Sciences. London: Routledge.
DarcySTaylorTGreenJ (2016) “But I can do the job”: Examining disability employment practice through human rights complaint cases. Disability & Society31(9): 1242–74.
24.
De SouzaDE (2013) Elaborating the context-mechanism-outcome configuration (CMOc) in realist evaluation: A critical realist perspective. Evaluation19(2): 141–54.
25.
De SouzaDEVongalis-MacrowA (2021) Evaluating a pilot education-to-work program for adults with Down syndrome. Studies in Educational Evaluation70: 1–10.
26.
DevineADickinsonHBrophyL, et al. (2021) “I don’t think they trust the choices I will make.”—Narrative analysis of choice and control for people with psychosocial disability within reform of the Australian Disability Employment Services program. Public Management Review23(1): 10–30.
FisherKREdwardsRGleesonR, et al. (2010) Effectiveness of individual funding approaches for disability support. FaHCSIA Occasional Paper No. 29. Available at: https://ssrn.com/abstract=1700847
29.
FlynnRRotterTHartfieldD, et al. (2019) A realist evaluation to identify contexts and mechanisms that enabled and hindered implementation and had an effect on sustainability of a lean intervention in paediatric healthcare. BMC Health Serv Res19: 912.
30.
GorskiPS (2015) Causal mechanisms: Lessons from the life sciences. In: ArcherM (ed.) Generative Mechanisms Transforming the Social Order. Cham: Springer, 27–48.
31.
HargreavesMBPodemsD (2012) Advancing systems thinking in evaluation: A review of four publications. American Journal of Evaluation33(3): 462–70.
32.
ImamILaGoyAWilliamsB (2007) Introduction. In: WilliamsBImamI (eds) Systems Concepts in Evaluation: An Expert Anthology. Point Reyes, CA: Edge Press of Inverness, 3–10.
33.
JacksonMC (1982) The nature of soft systems thinking: The work of Churchman, Ackoff and Checkland. Journal of Applied Systems Analysis9(1): 17–29.
34.
JagoshJ (2020) Retroductive theorizing in Pawson and Tilley’s applied scientific realism. Journal of Critical Realism19(2): 121–30.
35.
LemireSKwakoANielsenSB, et al. (2020) What is this thing called a mechanism? Findings from a review of realist evaluations. Special Issue: Causal Mechanisms in Program Evaluation2020(167): 73–86.
36.
MacdonaldFCharlesworthS (2016) Cash for care under the NDIS: Shaping care workers’ working conditions?Journal of Industrial Relations58(5): 627–46.
37.
McCauslandR (2019) ‘I’m sorry but I can’t take a photo of someone’s capacity being built’: Reflections on evaluation of Indigenous policy and programmes. Evaluation Journal of Australasia19(2): 64–78.
38.
MidgleyG (2006) Systems thinking for evaluation. In: WilliamsBImamI (eds) Systems Concepts in Evaluation: An Expert Anthology. Point Reyes, CA: Edge Press of Inverness, 11–34.
39.
MingersJ (2014) Systems Thinking, Critical Realism and Philosophy: A Confluence of Ideas. London: Routledge.
40.
MoskosMIsherwoodL (2019) Individualised funding and its implications for the skills and competencies required by disability support workers in Australia. Labour & Industry: A Journal of the Social and Economic Relations of Work29(1): 34–51.
41.
MukumbangFCMarchalBVan BelleS, et al. (2020) Using the realist interview approach to maintain theoretical awareness in realist studies. Qualitative Research20(4): 485–515.
42.
MutchA (2017) Practices and morphogenesis. Journal of Critical Realism16(5): 499–513.
43.
NDIS Act (2013) National Disability Insurance Scheme Act 2013. Canberra: Australian Government.
44.
NeubertDAMoonMSGrigalM, et al. (2001) Post-secondary educational practices for individuals with mental retardation and other significant disabilities: A review of the literature. Journal of Vocational Rehabilitation16(3): 155–68.
45.
OECD (2017) Connecting People with Jobs: Key Issues for Raising Labour Market Participation in Australia. DOI: 10.1787/9789264269637-en
46.
PawsonR (2002) Evidence-based policy: The promise of realist synthesis. Evaluation8(3): 340–58.
47.
PawsonR (2013) The Science of Evaluation: A Realist Manifesto. London: SAGE.
PorporaDV (1998) Four concepts of social structure. In: ArcherMSBhaskarRCollierA, et al. (eds) Critical Realism: Essential Readings. London and New York: Routledge.
52.
QuintonA (1975) The presidential address: Social objects. Proceedings of the Aristotelian Society, New Series76(1975–1976): 1–27+viii.
53.
RamageMShippK (2009) Systems Thinkers. London: Springer.
SayerRA (1992) Method in Social Science : A Realist Approach, 2nd edn.London: Routledge.
56.
ShawJGrayCSBakerGR, et al. (2018) Mechanisms, contexts and points of contention: Operationalizing realist-informed research for complex health interventions. BMC Medical Research Methodology18(1): 1–12.
57.
SpibyHGreenJMDarwinZ, et al. (2015) Multisite implementation of trained volunteer doula support for disadvantaged childbearing women: A mixed-methods evaluation. Health Services and Delivery Research3(8): 1–366.
ThomaCALakinKCCarlsonD, et al. (2011) Participation in postsecondary education for students with intellectual disabilities: A review of the literature 2001–2010. Journal of Postsecondary Education and Disability24(3): 175–91.
Van BelleSWongGWesthorpG, et al. (2016) Can “realist” randomised controlled trials be genuinely realist?Trials17(1): 1–6.
62.
WesthorpG (2018) Understanding mechanisms in realist evaluation and research. In: EmmelNGreenhalghJManzanoA, et al. (eds) Doing Realist Research. London: SAGE, 41–58.
63.
WightC (2015) Mechanisms and models: Some examples from international relations. In: ArcherM (ed.) Generative Mechanisms Transforming the Social Order. Cham: Springer, 49–64.
64.
WilliamsBImamI (2007) Systems Concepts in Evaluation: An Expert Anthology. Point Reyes, CA: Edge Press of Inverness.
65.
YoungKAshbyDBoazA, et al. (2002) Social science and the evidence-based policy movement. Social Policy and Society1(3): 215–24.
