Abstract
Our comment on Archer’s enticing article focuses on his extension of Bhaskar’s philosophy to psychology, and on direct replication, which the author says is of very limited use in psychology. We only deal with critical realism indirectly. Our arguments boil down to two points: experiments can usefully be seen as performative (and the same goes for research in general), and replication is not problematic in psychology because of the variability of results, but useful for precisely that reason. Replication studies may even inspire psychologists to accept a wider range of epistemic approaches, although there are reasons to be pessimistic about the likelihood of fundamental change in the discipline.
Experimentalism, Archer (2024) tells us, is inherently problematic in the natural sciences because the natural world is an open system in which causal mechanisms causally depend on other causal mechanisms, and therefore operate as tendencies rather than as strictly determining factors. As a result, the artificial closure of the laboratory situation introduces “inexactness,” because “what happens in concrete real-world settings cannot be mirrored in controlled experiments” (Archer, 2024, p. 574).
But perhaps the experiment does not have to “mirror” what happens in the real world. Perhaps the laboratory can also be a place for the production of realities, rather than for the representation of reality. Science and technology do not mirror the world, they change it. Laboratories are indeed very artificial places. That is precisely what makes them so useful. There is little nature in a laboratory: researchers work with purified materials and specially bred animals, the processes they create are reduced in scale and force, and many of the effects they produce have no analogue in the “real world.” They are artificial products made in an unnatural environment in which the researchers are at an advantage. The only way to keep these products stable when they are put to use is to move the lab with them, by shielding them from the environment or modifying that environment to mimic the lab, and/or by precisely scripting the interaction with them (Latour, 1984/1988).
Thus, science produces realities, it is not a mirror of nature (Rorty, 1979; see also, e.g., Law, 2004; MacKenzie, 2006). That, at least, is one way of looking at it. Archer (2024) seems to adopt this perspective when he writes that “it is not the regularity of events . . . that underwrites replication, but rather the artificial creation of such regularity of events through human manipulation” (p. 575). Note that such regularity is highly prized. The author indefatigably stresses the instability of the natural world and the cultural and historical variability of the social world, due to their character of open systems, and the author will go no further than acknowledging the existence of “demi-regularities.” We trust no one needs to be reminded of the many unexpected events (often cruel, sometimes felicitous) that can affect our lives, nor of the fact that society today is different from what it was in earlier times and what it is elsewhere. Like the author, we believe that that social and psychological variability is unjustly neglected in psychology, with its obsession with universals. But precisely because the natural and social world are changeable and often present us with unwelcome surprises do we also seek to create regularity and stability. With science and technology, we have created a world to live in that is an increasingly closed system, relatively shielded from the open system of the natural world. It is a life world that incorporates nature in the form of, for example, specially bred cattle (the Holstein) that produce predictable amounts of milk (34 litres daily on average) when fed with the standardised food that is available at reasonably predictable prices and usually will be delivered at the agreed date, to fill a silo via a tube with an opening of standardised size and shape that fits the mouth of the silo well enough not to have any spillage (Holstein Association USA, 2017). We are so used to this kind of standardisation, stability, and predictability that we hardly see what an immense achievement it is. Of course it is true that in this world, perfect regularity does not exist, but to say that any stability created in the artificially established systemic closure of the laboratory is inexact because it does not mirror the messiness of nature out there misses the productive character of science. In an enactive view of science, the focus shifts away from the question of how well science mirrors nature, to the ethical and political issue of the desirability of the products of technoscience (which is questionable in the case of the cattle).
The limitations in stability, exactitude, and desirability when moving laboratory productions outdoors are more significant in psychology given the dynamic complexity of its human objects. The author’s hypothetical example of the indeterminant complexity of student/teacher relations is supported in applications of empirical psychology findings. Behaviour-control technologies offer a case: operant conditioning was realised first in laboratories with strict controls on subjects’ behaviour choice and a precisely structured reward system in an otherwise pristine chamber. Moved outside the lab, precisely because operant conditioning yielded stability and control of human behaviour, it functioned most successfully in highly controlled environments like total institutions where persons have severely restricted choices for action and no other access to rewards. Though moderately successful in psychiatric hospitals and prisons, many of these behaviour-change technologies encountered insurmountable ethical, legal, and cultural resistance (Rutherford, 2009). The implementation of “nudge” theory provides a contemporary case: Tests of nudge applications have been found to be acutely troubled by the heterogeneity of instruments, measures, and effects as well as methodological problems and publication bias (Maier et al., 2022; Szaszi et al., 2018, 2022).
Informed by critical realism, the author recommends an alternative ontology for psychology’s primary object of analysis, psychological processes. That ontology is as generous as it is generative: these processes are to be understood as having essential powers, yet dependent on other mechanisms, an “innate ‘plasticity’” (Archer, 2024, p. 577), “sociocultural sensitivity” (p. 563), variable, emergent, unconscious as well as conscious, and occurring in an “open system” (p. 565) though with dependencies on other mechanisms. Given these ontological properties of psychological processes, Archer (2024) underscores “the fact that human beings are intrinsically open and irreducibly complex psycho-biological systems whose emergent psychological properties are ontologically distinct from the physical causal mechanisms studied by the natural sciences” (p. 577). In affirming a core distinction between objects in the natural sciences and those in the human sciences, he rehearses a long-lived, recurrently posited claim about the distinctiveness of human science.
Contrary to the author, it seems to us that direct replications can be informative in such a psychology, even if they fail to reproduce the original results or if there is reason to think they will fail. Firstly, not all replications do fail. Psychological mechanisms are inherently connected to many other mechanisms, but some may be less connected than others and thus allow fairly stable effects. Secondly, it has also become clear in the replication crisis, if it wasn’t already, that some results do not replicate because they were the product of bad research: poorly designed studies, measurement instruments with little validity, small samples, inappropriate statistical analyses, and so forth. A direct replication can never yield a decisive falsification (nor a decisive confirmation), but it can certainly alert the field to results that need further scrutiny. Appeals to the contextual variability of psychological phenomena as an explanation for failed replications risk papering over real problems in psychological research. Thirdly, as we have argued before, direct replication is an important kind of research precisely for those researchers interested in the variability of psychology’s subject matter (Derksen, 2019; Derksen & Morawski, 2022; see also Leonelli, 2018; Penders & Janssens, 2018). The fact that a direct replication fails can point to the messiness of the world that Archer is at pains to impress on the reader, and can be a first step in exploring that messiness. The variability in the phenomenon of interest can show itself against the background of variables that were kept the same as much as possible between experiments. One effect of the replication crisis has been that heterogeneity has become a topic of concern, particularly in the context of the use of behavioural science in practice. Hallsworth (2023), for example, sees in the replication crisis a reason to “have a hard conversation about heterogeneity in results” (p. 314). Bryan et al. (2021) have called for a “heterogeneity revolution.” “Interventions” must be adapted to the contextual variability of behaviour. Finally, direct replication highlights the importance of procedural details in conducting experiments. Direct replication puts the spotlight on the intricacies of producing results in psychology, and thus gives a richer understanding of the process of psychological science. In an ethnographic study of replication practices that one of us conducts with Stephanie Meirmans, Jonna Brenninkmeijer, and Jeannette Pols (Derksen et al., in press), researchers mentioned this as a definite benefit of conducting direct replications.
Perhaps this view of the potential transformative effects of direct replications is too optimistic. The history of proposals for alternative epistemologies and ontologies for psychology does not bode well in this regard, and that also raises questions regarding the author’s Bhaskarian alternative. There exist numerous theories and models that embrace most of the author’s tenets about the ontology of psychological processes. These ontologically informed models include phenomenology, constructivism, humanism, and various forms of dynamic nominalism (sometimes called historical psychology). Developed and advanced by committed and astute researchers, these models and the alternative ontology being advocated have gained little traction beyond the vibrant if modest communities in which they are forged. The same unappreciation extends to critical psychology studies which have elucidated in detail trenchant problems with psychology’s dominant scientific epistemology (e.g., Slaney & Garcia, 2015; Stam, 2015; Teo, 2011).Thus, it is difficult to envision how much attention can be drawn to critical realism, admitted even by a proponent to be a “Bhaskarian forest of realism, dialectic, and meta-Reality” (Seo, 2009, p. ii).
It seems wise to be similarly cautious regarding the transformative potential of the current developments in psychology. Efforts to “clean up” methods, to realise a more “rigorous” science, have fuelled ongoing, often heated debates/disputes about technical issues such as p-values and effect sizes, preregistration of studies, frequentist versus Bayesian statistics, and the justification for conceptual replications and exploratory studies. Noticeable among some of the participants in these debates is a form of epistemological retrenchment that signals an enduring commitment to the dominant perspective on science, one grounded in Popper’s philosophy of science, including some practices of falsification and null-hypothesis significance testing (NHST) and tenets of stable and singular objects. Some proponents even quote Popper on, for instance, the essential role of reproducibility (Zwaan et al., 2018, p. 2) or entreat fellow psychologists to become acquainted with Popper’s writing. One imagined futuristic portrayal of the Popper-informed reform campaign for direct replication is a scientific system that is semi-automated (through Open Science or similar oversight entities) wherein scientists submit to highly regulated, ever scrutinisable practices of compliance, thus deemphasising the scientist whose actions are calibrated and revised through algorithmic feedback networks that coordinate the actions of all stakeholders—funders, editors, institutions, and reviewers as well as researchers (Morawski, 2022). Futuristic thinking aside, there is a risk that the reforms might be tightening the grip of an aged philosophy of science through the micromethods’ controversies and researchers’ resistance to thinking otherwise about philosophy of science.
The reforms might possibly intensify the artificial, closed system of experiments, thereby manifesting the author’s observation of “the methodologist tendency for mainstream psychologists to avoid or gloss over crucial ontological questions” (Archer, 2024, p. 561). In a discipline perpetually concerned with being a genuine science, psychologists might well take the reformed methods as achieving an objectivity that is encoded not in some articulated epistemology but, rather, in the reforms’ “new procedural bureaucracies” (Penders, 2023, p. 112). We have argued for the potential of replication studies to disrupt such epistemological retrenchment. Apart from their role in the corroboration of findings and in illuminating poor research methods that need attention, replication studies can trace the impact of procedural choices on the findings, highlighting the extent to which experimental effects are the product of specific circumstances. Importantly, replications can show the variability and heterogeneity of phenomena, the messiness of the world. Replication studies may help to convince psychologists of the irregularity (or imperfect regularity) of psychological phenomena that Archer is at pains to emphasise. As we have acknowledged before, perhaps we are too optimistic, but replication seems to us a useful methodological strategy in psychology. Rather than being “illusory” or “inherently problematic,” replication has shown great disruptive potential. Innovations of scientific practices have the potential to breach epistemologial divides like those illuminated by Archer: enhanced replication practices might corroborate Herrnstein-Smith’s (1997) observation that incommensurable worldviews sometimes meet in practical ways that might be conflictual yet, “in the long run, mutually transformative” (p. 212). Recognising that potential requires leaving the representational perspective on scientific research and adopting a performative outlook. If we see science, including psychology, as producing a world rather than picturing it, the variability of results is no longer a problem for the discovery of the essence of psychological mechanisms. Rather, heterogeneity becomes a much-needed product of psychology in a society that increasingly pushes people to live up to norms.
Footnotes
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
