Abstract
Scholars and practitioners who develop evidence-based crime policy debate on how best to translate criminological knowledge into better criminal justice practices. These debates highlight the counterpoised problems of over-selling the contribution of scientific evidence; or, alternately, overemphasizing the limitations of science. This challenge attends any attempt to translate research findings into practice; however, and problematically, in criminology this challenge is rarely approached in a theoretically coherent fashion. This article therefore seeks to theorize uncertainty in criminology by examining insights on communicating scientific uncertainty in other fields, and applying these insights specifically to the field of Evidence-Based Policing (EBP). Taking the position that all science is inherently uncertain, we examine the following four aspects of the field: the particular uncertainties of criminology, variance in receptivity to research, the lack of evidence regarding effective communication, and the boundaries of evidence. Building on this analysis, we set out the normative challenge of how researchers should characterize and balance the implications and limits of scientific findings in the decision-making process. Looking ahead, we argue for the need to invest in an empirical project for determining meaningful strategies to express research evidence to decision-makers.
Introduction
The purpose of this article is to draw insights from the wider field of science communication and apply these to the expression of criminological knowledge – and uncertainty about that knowledge – in the spheres of policing and justice policy-making. There is extensive debate in the wider scientific community on how to properly convey science that is inherently uncertain within policy and political decision processes. This challenge has led to attempts to find balance between over-selling the work that scientific evidence can do on one hand (e.g. suggesting that social science can prove ‘what works’, and by extension, what will work); or, on the other hand, overemphasizing the limitations of science (e.g. focusing on gaps in knowledge over what is known), thus providing practitioners and decision-makers with the justification they need to revert to professional experience or political or personal preference rather than scientific evidence as the basis for decision-making.
We argue that uncertainty as a concept is an important entry point for unpacking a better form of science communication in criminology. We reflect on the field of Evidence-Based Policing (EBP) as an opportunity to better develop an accessible theoretical understanding of the nature of criminological evidence vis-à-vis the translation of evidence into practice. EBP is, broadly speaking, an effort to use ‘the best available [research] evidence to inform and challenge policies, practices and decisions’ (College of Policing, 2021) in policing. EBP was developed based on principles from Evidence-Based Medicine (Sherman, 1998) and has since expanded into several international associations of researchers and affiliated police practitioners. The current field of EBP is replete with (mostly) good-faith attempts by criminologists and other social scientists to apply academic research to practical problems faced by police officers and leaders. However, most working in the field of EBP recognize that there remains a ‘disconnect between science and policing’ (Weisburd and Neyroud, 2011) and that new strategies are required to overcome historic antipathies between evidence-led interventions and craft-based practices, and between expert and experiential knowledge (Neyroud and Weisburd, 2014; Willis, 2013). Moreover, we know very little about how, and how well, different kinds of messages about criminological evidence are digested by practitioners (see Litmanovitz et al., in press; Lum and Koper, 2017: 267; Sherman, 2015).
This article is based on a review and synthesis of two main strands of literature. The article first examines insights from recent reports and articles detailing the state-of-the-art in science communication generally (see, for example, Fischhoff, 2012, 2019; National Academies of Sciences, Engineering, and Medicine, 2017), or commentaries by leading thinkers in scientific communication and knowledge translation in other fields (e.g. Greenhalgh and Papoutsi, 2018; Lewis and Wai, 2021; Nutley et al., 2007). The second part of the article reviews foundational empirical and theoretical works related to understanding and implementing EBP, especially focusing on key practitioner-oriented textbooks (e.g. Lum and Koper, 2017), literature assessing the state of the evidence in EBP (such as systematic reviews and ‘reviews of reviews’, for example, Weisburd et al., 2017), and empirical evidence on research ‘receptivity’ in police organizations (e.g. Blaskovits et al., 2018; Jonathan-Zamir et al., 2019).
The article then applies the insights from wider literature on scientific communication to the EBP field. The article discusses the normative and empirical problems that these insights raise for EBP and justice policy going forward, and sets out a future research agenda. The article contributes to the literature on EBP by synthesizing several strands of thought aimed at solving the puzzle of how to best entrench research evidence within policing, while also elaborating key considerations regarding the process by which this entrenchment may and should (or should not) happen. The article also contributes more generally to the development of theories supporting knowledge translation in criminology. Utilizing EBP as a mechanism to conceptually frame criminologists’ attempts at scientific communication, the considerations developed in this article can be applied to other criminological domains where researchers are facing similar challenges to those working in EBP.
Prior to delving further into these issues, there are several provisos that seem important to emphasize, given long-standing debates in criminology regarding what criminologists should do vis-à-vis the production of ‘public’ and/or policy-relevant knowledge (see, for example, Bosworth and Hoyle, 2012; Loader and Sparks, 2011; Uggen and Inderbitzin, 2010). Criminology as a field writ large is not only (or primarily) focused on developing applied knowledge for justice practitioners, and there are many ways criminologists can engage in the public sphere that have little to do with knowledge translation. Engaging with science ‘for’ the police (or any other criminal justice institution) is inherently conditioned by the political willingness of funders and police agencies to support a research agenda that unavoidably serves to reproduce certain versions of the institution, while often narrowing the scope of scientific inquiry (Manning, 2005). This is a fundamental bargain inherent in any engagement between academic research and government or institutional practices and agendas, and nothing in this article should be construed to suggest that this form of engagement is lionized above other approaches to criminological work. Moreover, we expressly accept that criminology is not synonymous with policing or EBP and there are aspects of uncertainty related to criminological evidence (e.g. related to studies of prison and probation practice, social services and community-led crime prevention, and early childhood development, to name a few) that go unexamined here. These areas of criminology remain generally outside the scope of this article.
All science is uncertain: Insights from the fields of science communication
In this section, we examine literature from fields that have a more developed evidence base regarding the effectiveness and impacts of different kinds of science communication. The discussion of ‘effectiveness’ in the context of science communication needs to be prefaced by a consideration of effectiveness ‘for what’ (or for whom) – traditionally formulated as Lasswell’s (1948) 5W model of communication:
It is within this context of science communication that this article aims to unpack the challenge of dealing with scientific uncertainty. Turning to a recently published synthesis of the forms of uncertainty in science communication, Van der Bles et al. (2019) examine the concept of epistemic uncertainty, which ‘concerns past or present phenomena that we currently don’t know but could, at least in theory, know or establish’ (p. 2). They contrast this with aleatory uncertainty, which relates to things that are inherently unknowable, such as luck or chance. Regarding epistemic uncertainty, in many cases, the people actually assessing epistemic uncertainty (e.g. the people conducting scientific studies) may differ from the people doing the communication (such as scientific advisors, health professionals or policy analysts). In addition, the type of information being expressed (e.g. facts, numbers, theories) may require specific characteristics to be present within the intended audience (such as numeracy), if the communicator wants the information to be digested properly.
Especially where science communication involves an attempt to influence decisions or behaviours, this raises several ethical questions about where the communication of science stops and its interpretation towards specific ends begins. The willingness to advocate a policy or value position based on empirical science does not require the communicator to overcome uncertainty before communicating, but rather to measure both the weight of evidence and expectations about the intended audience’s actions upon receiving new information about how a certain part of the world may work. The vitality of this issue stems from the nature of empirical science: that the creation of scientific evidence is inherently revisionist and therefore never a ‘settled’ matter.
Determining how the expression of certainty and uncertainty are balanced is often in the hands of those doing the communication (who may or may not be scientists). Fischhoff (2012: 65–67) suggests that experts are reluctant to express uncertainty to decision-makers because experts ‘see uncertainty as misplaced imprecision’, ‘do not expect uncertainties to be understood’, ‘anticipate being criticized for communicating uncertainty’ and ‘do not know how to express their uncertainties’. Still, Fischhoff argues, experts have an underlying duty to express uncertainty where it exists, and decision-makers have a responsibility to create an environment that allows a healthy expression of uncertainty without discounting scientific advice.
Several leading scholars in public health have argued that this issue of interpreting and applying scientific findings in novel contexts is an essential part of doing science. Greenhalgh and Papoutsi (2018), for example, highlight the limits of evidence and the importance of the context into which evidence is introduced: The world moves quickly; baselines shift; technologies crash; actions are (variously) constrained; and certainty is elusive. The gap between the evidence-based ideal and the political and material realities of the here-and-now may be wide. Decisions must be made on the basis of incomplete or contested data. People use their creativity and generate adaptive solutions that make sense locally. The articulations, workarounds and muddling-through that keep the show on the road are not footnotes in the story, but its central plot. They should be carefully studied and represented in all their richness. (p. 2)
Even where the evidence is well-comprehended by the intended audience, science communicators need to be cognizant of the context in which different actors make decisions and how this will affect the interpretation and application of research evidence. Fischhoff (2019) has identified several ways in which decision-making in science is different from decision-making in political contexts. He suggests that while researchers are able to isolate aspects of their problem by ignoring wider system considerations (what he refers to as ‘bounded rationality’), policymakers and practitioners have to make decisions with explicit knowledge of the wider context or system; they recognize that only some aspects of that wider system can be addressed by a proposed action, intervention or programme (referred to as ‘satisficing’).
The field of psychology also holds important lessons for our theoretical exploration. Scientific uncertainty as a dilemma has occupied a pertinent space in recent years in psychology, which is experiencing a ‘credibility revolution’. Lewis and Wai (2021) suggest that we need to both communicate ‘what we know, and what isn’t so’. As they point out, a small number of popularized findings in varied fields including educational psychology, media psychology, vaccine science and climate science have had the dual effect of demonstrably altering public attitudes for the worse on issues of broad public importance, while also damaging public perceptions about the attendant scientific fields. Considering this crisis, Lewis and Wai advocate for more transparency about what we (actually) know and how (methodologically and epistemologically) we know it. They further argue for more humility about what we do not know and accepting the limits to our knowledge, and that this sort of honesty is essential to retain the credibility of the field. This line of thinking, we suggest, translates meaningfully to the criminological field, to which we now turn.
Criminological uncertainty: Four aspects of the EBP field
In this section, we set out what we see as the central aspects of the challenge of communicating uncertainty regarding criminological evidence, particularly in relation to the policing field. As we indicated earlier, all science is inherently uncertain and subject to revision over time, and the social sciences, including criminology, are no exception. Yet, there are aspects of the use of evidence within the policing field, and the way the field consumes criminological knowledge, which bear specific attention. The field-specific aspects of criminological uncertainty, as they relate to EBP, are grounded in four interrelated aspects of uncertainty, which we also see reflected in the wider literature on communicating scientific uncertainty, but which take on a particular character in this field.
The first aspect that characterizes criminological uncertainty in EBP is a narrow and underdeveloped (and thus particularly uncertain) evidence base. The kinds of data that are available to inform policing are notably uncertain, and this has led many researchers to doubt even the possibility or desirability of a truly ‘evidence-based’ policing profession (Sparrow, 2011). The second aspect relates to research receptivity; several noteworthy efforts to understand the character of research receptivity have been undertaken in recent years and these yield some insight into the terrain on which scientific messages may land, both at the senior and front-line levels of police work. The third aspect is that we know very little about how criminological research is received by decision-makers, even when it is written conscientiously for a receptive audience – in other words, we know little about ‘what works’ in communicating ‘what works’. Fourth and finally, given the highly context-specific and localized nature of policing, the limits to evidence when confronted with police officer experience and local contextual factors is the last aspect of the field that we consider.
Criminology is particularly uncertain
The core challenge of criminological uncertainty is that the evidence base that practitioners can draw on to support decision-making is relatively narrow and underdeveloped. Criminology is a comparatively young discipline, and there is substantial disagreement within the field about both the most appropriate objects for, and methods of, inquiry (see, for example, Bosworth and Hoyle, 2012; Loader and Sparks, 2011). Of course, as one of many sub-disciplines within criminology, policing studies, and particularly EBP, tend to draw primarily on applied research, so may be to some degree insulated from these wider epistemological or empirical issues, taking a tacitly or explicitly positivist and meso-level approach and thus limiting its gaze to those areas where inquiry based on scientific measurement of specific problems has been de rigeur. In EBP, the evidence is grounded primarily in programme evaluation research (Lum and Koper, 2017: 23–30), which substantially limits the kinds of police actions that can be – or at least have been – inspected through ‘the evidence’.
Moreover – and this is a point that has been made extensively in the programme evaluation literature (see, for example, Pawson and Tilley, 1997 and derivative works in the Realist(ic) Evaluation tradition) – programme evaluations also struggle immensely with understanding things that are not in themselves ‘programmes’, and may struggle to tease apart the many factors that amount to ‘business as usual’. Even where a programme can be identified and could be evaluated, the results of that evaluation will be expressly place- and object-specific. This consideration matters not only for the identification of relevant outcome measures (e.g. did an intervention improve crime rates but have a negative impact on public trust?), but also for the transferability of findings and predictions of scalability. Such considerations have led to several methodological advancements in recent years, for example, the EMMIE (Effect size, Mechanism, Moderator (or context), Implementation and Economic impact) evaluation framework intended to answer the needs of policymakers in determining ‘what works’ (Tilley, 2016); the Realist Randomized Trials evaluation framework, which draws on both positivist and realist assumptions to capture the importance of the ecological and organizational setting and their interactions with the intervention (Bonell et al., 2012); and efforts by major scholars to identify active ingredients within policing interventions, thereby moving away from the problem of ‘black boxed’ interventions that cannot teach us about why a programme succeeded or failed (e.g. Famega et al., 2017; Weisburd et al., 2015). Despite these advancements, we still know as little about these new methodological tools’ roles as communication devices for improving decision-making as we do about other strands of scientific communication in criminology.
We acknowledge the policing field’s evidence base has grown significantly in the past decade and is continuing to grow (Wilson et al., 2021). Still, and while the relevant evidence base is too voluminous to summarize here, it seems fair to say that we are largely unable to predict how an intervention implemented in one place will fare in another – even similar – situation. Part of this has to do with the historical inattention to replication in criminology for those studies where replication would be appropriate (Huey and Bennell, 2017; McNeeley and Warner, 2015), and another part has to do with the context-specific issues outlined earlier. There are some notable exceptions; we seem to have decent evidence that hot-spots policing, problem-oriented policing and focused deterrence tend to have consistently positive outcomes (Wilson et al., 2021). Nonetheless, considering the interventions being grouped together for meta-analysis even in these well-established bodies of evidence, uncertainty remains regarding the common ingredients across evaluation sites (Braga et al., 2018; Sparrow, 2011). Other largely ‘settled’ perspectives in criminology, such as the consistency of the importance of procedural justice in determining police legitimacy, have also recently been challenged and shown to be both limited (Nagin and Telep, 2017) and notably context-specific (Roché and Oberwittler, 2017). The issue of transferability should be addressed as the evidence base is ‘sold’ around the world despite the overwhelming majority of experiments having been conducted in the United States, with a few in the United Kingdom or other countries (see, for example, Hinkle et al., 2020). In turn, the particular uncertainty of criminological evidence – even in the constrained sphere that is the focus of the EBP field – has to be factored into any theory about the ways in which communications of criminological evidence work.
Receptivity differs among listeners
Several researchers (e.g. Blaskovits et al., 2018; Telep and Lum, 2014) have examined the concept of research ‘receptivity’ in police organizations, which is often treated as a necessary condition for the utilization of research knowledge. Receptivity research has highlighted many positive findings, but also significant challenges in advancing the use of research evidence in policing practice. For example, officers surveyed by Blaskovits et al. (2018) and Telep and Lum (2014) generally felt that collaboration with researchers was necessary for their agencies to improve their ability to effectively address and reduce crime, but these same officers typically valued experience more than research in guiding police practices. Similarly, officers in these studies appeared willing to evaluate, or support evaluations of, policing strategies, but they tended to prefer less rigorous evaluation methodologies for this purpose (e.g. pre-post tests vs randomized trials), suggesting more receptivity to research methods that limit disruption of ongoing police practices (Blaskovits et al., 2018; Telep and Lum, 2014).
Sufficient receptivity, at the appropriate places in police organizations, is often expressed as an important factor a priori to the development of evidence-based policy (Khanizadeh et al., 2017; Telep, 2017). Equally, ‘resistance’ to research is seen as a barrier to more evidence-based practice in policing (see, for example, Koziarski and Kalyal, 2021; Sherman, 2015). Certainly, at a level, this must be true – it seems highly unlikely (except perhaps by lucky coincidence) that police organizations that reject a role for research would then implement policies that align with the best available evidence. Yet, the converse is not necessarily true – being receptive to a role for research does not guarantee that decisions taken will reflect the best available evidence. Moreover, given the relative nascence of the EBP concept and operationalizations of ‘resistance’ and ‘receptivity’, we know very little at this stage about whether self-reported attitudes towards EBP act as behavioural indicators at the organizational level – for example, a greater proportion of policies and procedures based in research evidence.
Sherman (2015) argues that the main roadblock to receptivity is ‘fear of the unknown’, which also accords with Sparrow’s (2008) account of why policing and regulatory organizations prefer traditional modes of operating over those, like EBP, that break down organizational boundaries by focusing on problems instead of process. Jonathan-Zamir et al. (2019) help explain this phenomenon. Studying senior police officers in Israel, they found that officers’ own experience and expertise were always placed ahead of research in decision-making, not only among those who had low receptivity to research, but also among those who were receptive to it.
The explanations for this ‘proclivity’ of personal professional experience over evidence are rooted in cognitive and social psychology (Jonathan-Zamir et al., 2019). First, our experiences enjoy the benefit of being ‘emotional memories’ which are easier to access (Ruscio, 2010) and therefore perceived as more trustworthy (Tversky and Kahneman, 1973). Second, because they come in ‘story format’, our experiences again are more trustworthy than evidence that comes in data format (Giluk and Rynes-Weller, 2012). Third, and perhaps most relevant, people have ‘a need to feel optimistic about the outcomes of a decision’ (Highhouse, 2008). We prefer operating in an ambiguous state in which we do not fully recognize that success is an uncertain or even unlikely result.
These tendencies are juxtaposed in policing due to the centrality of discretion both in the ethos of the profession and in practice (Rumbaut and Bittner, 1979). Using evidence as part of a professional decision-making process demands police officers to operate within a calculated ‘chance of success’ mode. It is not possible to pry apart this characteristic from evidence-based practice. ‘Giving evidence a seat at the table’ (Lum and Koper, 2017) may present a challenge for officers who wish to see themselves as having wide discretionary powers; police work is in many ways about being in the ‘driver’s seat’ emotionally (Bayley and Bittner, 1984; Willis and Mastrofski, 2018), and research evidence interferes with this position.
‘Effective’ communication is not evidence-based
Ultimately, we know very little about why or under what circumstances police leaders, or criminal justice policymakers more broadly, base their decisions on research evidence, especially where evidence points away from traditional or status quo strategies. Existing models of ‘translational criminology’ tend to focus on the structure and nature of partnerships that have worked (see, for example, Laub and Frisch, 2016; Nichols et al., 2019) rather than an empirical examination of what works best, under what conditions. Certainly, partnerships are important and have resulted in many innovations driven by research. For example, several published accounts of embedded researchers offer noteworthy stories of police reformers and researchers teaming up to reduce harms and improve police practices (see, for example, Beal and Kerlikowske, 2010; Braga and Davis, 2014; Engel and Whalen, 2010). However, most of these stories are inherently anecdotal rather than systematic forms of knowledge about what worked in these instances to put evidence at the fore in developing interventions (see Weisburd et al., 2020 and Litmanovitz et al., in press, for an attempt to identify mechanisms supporting EBP).
This issue has been highlighted by Lum and Koper (2017), who note that ‘much of the effort to increase the use of research outputs in the evidence-based crime policy arena . . . may not be evidence-based’ (p. 267). In turn, building on Nutley and colleagues’ work (2007, cited in Lum and Koper, 2017), they set out several ‘strategies and mechanisms’ through which EBP research may be successfully translated to practice. However, they also recognize a dearth of evidence on how, or whether, these strategies and mechanisms actually work, and encourage researchers to conduct more systematic study on the translational research project in criminal justice policy and practice. As Sherman (2015) puts it, ‘The obstacle all professions have faced is a lack of systematic evidence about how professions or organizations become evidence based’ (p. 12). Notably, both Sherman, and Lum and Koper, call for expanded research in this area, a point we return to in the discussion.
One key question that remains, where evidence is successfully incorporated into practice, is what brought these police leaders from their past practices to a belief that something needed to be changed and, moreover, that research evidence should guide that change. Without suggesting anything disingenuous in their decisions, we do not know whether these leaders’ intentions would have been different if the evidence had run counter to their intuition. The broader literature on science communication regularly highlights the possibility of research to be co-opted by political actors, to political ends (Fischhoff, 2012, 2019; Lupia, 2013). Furthermore, it is unclear whether the ‘recipe’ that worked to change minds in these instances can be ‘scaled up’ or cascaded outward to the wider world of policing that is traditionally resistant to innovation (Duxbury et al., 2018). Overall, the litmus test for whether scientific evidence is guiding police decision-making at the organizational or operational levels will be whether evidence is able to override other interests – in other words, when police leaders and political decision-makers are willing to follow the research evidence when it runs counter to established practice, their own personal preferences and the desires of at least some stakeholders.
There is some hope on this front. For example, until recently, the idea that police would engage in de facto decriminalization of drug possession was controversial in all but a few jurisdictions. Now, based at least in part on extensive research demonstrating the harm-reduction value of decriminalization, this has become a much less controversial position and has even become a mainstream perspective in some jurisdictions (see, for example, Canadian Association of Chiefs of Police, 2020). Certainly, research evidence played a role in this result; yet, it remains hard to pinpoint the role of research evidence (independent of professional experience, community advocates’ work, wider attitude change, other changes in government policy and so on) in convincing police leaders of this new direction.
Evidence has limits and is limited
As with Fischhoff’s (2019) conceptualization, policing is a satisficing context, where decision processes partially or wholly disregard certain perspectives or forms of evidence due to practical constraints. In turn, most conceptualizations of the EBP model therefore do not set research evidence above other inputs into decisions in police organizations. Rather, according to Lum and Koper (2017), an evidence-based approach desires that researchers and scientific evidence have a ‘seat at the table’, but this does not mean that research evidence holds the decision-making reins. While some perspectives hope to situate research evidence at the apex of the police decision-making structure (see, for example, Sherman, 2015), the mainstream of EBP thought does not generally displace the role of other forms of organizational knowledge (such as experience and political context; Lum et al., 2012; Telep and Lum, 2014; Willis and Mastrofski, 2018).
Some of the roadblocks between attitudes towards evidence and subsequent organizational behaviours have to do with the political context and the ways in which police policies and priorities are set. If, for example, elected or appointed political officials set the policies for a police organization, then what the organization can do with the evidence may be limited and evidence-based decisions may become impossible. Moreover, where criminal law or government crime-reduction programmes (and attendant funding) are misaligned with research evidence, the police are again constrained by their duty to enforce laws and the available resources they can use to reduce harms. As noted at the outset of the article, engaging in research ‘for’ police involves working within a research agenda that has been politically mediated and in which only certain kinds of ‘evidence’ may be accepted (Manning, 2005). This sets a challenge for injecting research evidence into police organizations, as well as into political processes.
Moreover, the ability to make an evidence-based decision must, at least in some cases, overcome the ‘institutional isomorphism’ of police organizations (see, for example, Giacomantonio and Litmanovitz, 2017). In other words, the available forms of police work are inherently limited within a narrow band of institutional structures; too radical a departure from these forms would be viewed by too many stakeholders as illegitimate, and thus, certain initiatives could not be pursued even if supported by the evidence. Furthermore, there are many paths for criminal justice practice that may appear potentially effective (e.g. at reducing crime) but would nonetheless be inappropriate if they run counter to community expectations (Loader, 2010).
Several scholars have recently been drawn towards an ‘engineering’ analogy for the development of police science, and away from the medical analogy from which the evidence-based practice concept has been drawn. These scholars (e.g. Tilley, 2016; Tilley and Laycock, 2016) recognize that, like engineering, policing is primarily concerned with the application of broader theories to solving specific problems (and, whether practitioners recognize the underlying theory that they are applying may be beside the point; as Pawson and Tilley (1997) point out, all programmes are ‘theories incarnate’). In turn, the use of ‘gold standard’ evidence from randomized controlled trials may have very limited applicability to a wide range of policing problems; instead, as Sparrow (2011) argues, a much wider set of ways of knowing will be appropriate to the field, particularly those that focus on observation and ‘pattern recognition’.
Considering the first three key aspects of uncertainty in the EBP field that we set out above, even where available evidence is reasonably robust, organizations are receptive to it, and the limitations to the evidence have been conscientiously communicated, limiting the role of research evidence within decision-making does not, to us, seem entirely problematic. This is not least because (a) as indicated earlier, the evidence base remains relatively narrow, with robust evidence relating to only a handful of policing interventions, and even these areas remain subject to revision; (b) even the best evidence is only suggestive (rather than predictive) in terms of the likely success of an intervention in a new area, with a new population or at a different time; and (c) local context and experience are inherently a form of relevant evidence that needs to be incorporated alongside the broader scientific consensus (see also Fleming and Rhodes, 2018; Sparrow, 2011).
Moreover, returning to the insights of Lewis and Wai (2021), such an approach re-centres EBP in humility about ‘what we know, and what isn’t so’. While such an approach may lead to a slower bench-to-bedside adoption of research evidence, it may also be more sustainable for the reputation of the research enterprise. This does not, of course, absolve police organizations of incorporating the best available research evidence into their decision-making process; this remains an ethical obligation connected to police responsibilities to both reduce harms and remain accountable as a public institution (Maxim et al., 2018). Rather, encouraging humility about the evidence base allows communicators of scientific evidence to set expectations among stakeholder audiences, and hopefully sets the stage for a more mutual understanding of what research evidence can do.
Discussion
As indicated at the outset of this article, we believe that the current state of criminological knowledge translation in the policing field creates both normative and empirical challenges regarding the viability of ‘evidence-based’ strategies, at what can be seen as a critical juncture in criminal justice policy-making around the world. In the context of widespread calls for police reform up to and including calls for outright abolition of the police, it is important for those involved in evidence-based policy and practice to consider how we encounter and evaluate proposed reforms and innovations based on existing – and potential – forms of evidence. While the below discussion focuses mainly on policing, the framing of both the normative and empirical challenges should remain relevant for anyone interested in translating criminological research into practice and policy within criminal justice institutions.
A normative challenge
The job of scientific communication involves a normative question for the communicator. Researchers who understand the evidence understand that it is uncertain. We cannot, generally, confidently predict the outcome of an intervention to cut crime or improve police work. As with social interventions more generally, we are always faced with the possibilities that a new intervention will work as intended, work (but in unintended ways), not work (but not do bad things either), or fail and create unintended negative consequences (i.e. a ‘backfire effect’). Yet, this uncertainty about outcomes cannot bind us to the status quo; we recognize the limitations to current practice and seek to improve it, as systematically as possible. Moreover, to focus only or primarily on uncertainty and unknowns appears more likely to reinforce the status quo (about which there is also, usually, much uncertainty and many unknowns, even where these go unacknowledged).
We therefore must, under some circumstances, express to practitioners and policymakers that the available evidence justifies one policy over another, and that the possibility for doing better represents a risk worth taking. However, equally, we must recognize when the evidence – or the theory that binds that evidence together – is not strong enough to warrant a change in direction, even though the current direction has its challenges (an important recent example is found in the work of Nagin and Telep (2017, 2020) in examining the available evidence on procedurally just policing). In both cases, we do this recognizing (based on the wider literature on science communication) that those practitioners and policymakers might only consume the main message (guiding towards or away from a certain action) without full or even meaningful comprehension of the underlying scientific evidence or process. They may therefore over-weight the high-level messages regardless of any cautionary or caveat messages we append therein.
We – researchers and allied communicators – are the ones making the decision about who communicates what, to whom, through which channel, and to what effect. The normative challenge of science communication requires us to be honest with ourselves that we are making these decisions, being reflexive about their impacts (Lewis and Wai, 2021) and retaining a ‘duty to inform’ our stakeholders where uncertainty exists (Fischhoff, 2012). The literature on science communication reviewed here indicates that attempts at science communication raise the risk of introducing misunderstanding – whether about the scope and scale of the problem, the likely success of a certain course of action, or the easiness in interpreting the relevant scientific information. It appears that we are unlikely to avoid this; misunderstanding is simply a risk inherent in science communication, though there may be strategies to limit misinterpretation.
This is therefore a challenge about parsimony in expressing evidence to those who need to incorporate it into their decisions. Presuming that there are a limited number of words in which to relay the core implications of one’s research (or some wider body of evidence) to a policy or practitioner audience, what messages should we choose to focus on? We treat this as a normative choice first, with some empirical considerations that we may then attach to it.
We present this normative issue not to come to a resolution on how to, with each attempt, achieve the appropriate ‘balance’ between objective expression of scientific data and its limits on one hand, and persuasive interpretation of the implications of that data on the other hand. The decision on this matter will often be personal, at least to a degree, and depend on one’s assessment of the likely outcome of different courses of action. As Loader and Sparks (2011) suggest, the choices researchers take may be influenced by ‘the issue that they are addressing, which audiences they are seeking to reach and the aspirations they are pursuing’ (p. 143), and choices may vary over a researcher’s career.
Researchers will also get it wrong, sometimes. While efforts can be made to ensure that the communication of scientific uncertainty will be systematic (weighing scientifically, to the degree possible, potential outcomes from different messages), available data usually support several potential policy options (and criminology is historically generally bad at prediction; see e.g. Farrell, Tilley and Tseloni, 2014). Equally, available data may relatively clearly reject at least some policy options – and so we suggest that researchers must clearly advocate for the elimination of those policies that are known to be harmful or wasteful. We can likely be most emphatic about the areas where we know ‘what isn’t so’ (Lewis and Wai, 2021).
But, we do not in turn suggest that a lack of evidence on effectiveness should be a barrier to experimentation with new or even radical changes to police practice. Recalling Greenhalgh and Papoutsi’s (2018) quote earlier, local experimentation and adaptation are in fact the main plot in the crime- and harm-reduction story, not a footnote in a wider body of evidence. Indeed, relying only on existing evidence or requiring high levels of implementation fidelity makes progress effectively impossible, since it narrows the possibilities to only progressive iterations of what has been tried before.
Past evaluations of crime prevention and reduction interventions do not really contain clear rules for future interventions. Rather, when examined collectively, they offer a set of broader principles on the ways in which patterns of crime and harm may be changed (or not). Indeed, distillation of such principles can be seen as one of the motivations behind the push towards systematic and scoping reviews in criminology (see, for example, Weisburd et al., 2017). It requires a relatively small step to move from these principles to onward-testable theoretical statements which can then be applied to novel contexts.
This suggests a role for a grounded-theory approach – broadly speaking, the iterative development of theoretical statements on an object of interest as diverse sources of systematic evidence are collected over time (Glaser and Strauss, 1967) – to interpreting the evidence base in EBP. Such a strategy is not without precedent in the literature (e.g. Neyroud, 2017) but it is worth emphasizing this valuable prospective aspect of retrospective research reviews. This is particularly important in the speculative assessment of the viability of innovative practices as the evidence base grows. The only requirement is that experimentation in new forms of police (or other crime prevention or reduction) practice must be attached to an empirical project in order to continue to measure and understand the impacts (broadly defined) of these new practices.
In turn, the normative challenge is first about accepting that there is, in any evidence-based decision, both uncertainty and demonstrably better and worse choices. The researcher’s role in this context is to push decisions towards that range of choices that are (on the basis of the best available evidence) better, or at least away from those that are worse. It is also, second, about ensuring that evidence – and particularly, the lack of evidence in a field that is often evidence-poor and highly uncertain – is not a barrier to creativity or innovation. This is especially important since most prior successful attempts at addressing ‘specific concentrations of harms’ – a core goal of EBP – were attempted before there was evidence of effectiveness (see also Sparrow, 2008, 2011). The current state of the evidence does provide us with directions towards these ‘better’ options: proactivity (Weisburd and Majmundar, 2018), problem solving (Hinkle et al., 2020), focus (Lum and Koper, 2017) and fairness (Nagin and Telep, 2020). These principles can be the foundation of decision-making that embraces uncertainty.
An empirical project
Researchers in the EBP field operate on a sort of folk wisdom about how to craft messages to enhance persuasiveness, change minds, and effect evidence-based policy development. Where these efforts have been successful (or unsuccessful) these narrative accounts shape the experiential craft knowledge of doing applied academic work. However, we need to be aware that this knowledge is no substitute for systematic study of the topic. There is a need for collective examination of the impacts of communication efforts, to understand if they are having the desired effects, and if any unexpected or negative impacts have been experienced. As noted above, if the evidence from other science fields is indicative, many good-faith attempts at translational research have resulted not only in ineffective transmission of key messages, but also potentially in harms to public understanding or the reputation of the criminological field.
The empirical challenge calls for a research agenda to guide this issue forward. There are examples of research and laboratory studies into the impacts of science communication in fields outside of criminology, which could be replicated between criminologists and criminal justice practitioners and policymakers. For example, different forms of the same message – each containing a different expression of uncertainty – could be delivered to randomly assigned police leaders to determine which was the most persuasive. Returning to Van der Bles et al.’s (2019) insights, respondents could be grouped based on underlying competencies, such as numeracy or research literacy, to determine what skills are needed when judging epistemic uncertainty. To understand the contextual and institutional roadblocks to incorporating research evidence into police practice, focus groups could be held with decision-makers to determine how and under what contexts they are able to use evidence in their decision-making. And, we as researchers could more systematically chart our own efforts to persuade, and systematically reflect on them.
In whatever way we collectively or interdependently move forward in developing the evidence base for communicating criminological uncertainty, we also need to remain cognizant of the inherent uncertainty that will attend this research. We should not expect research into the communication of criminological uncertainty to provide stable and consistent findings on ‘what works’ in communicating ‘what works’ to a broad range of audiences. This will be context-specific and evolve with the field, such that different factors may be important now compared to those that are relevant in a decade. For example, to the degree that EBP researchers and allied practitioners achieve positive outcomes in the application of social science research to crime and policing problems, police leaders may become more receptive. Conversely, where researchers miss the mark in their predictions or proscriptions (whether or not they are blame-worthy for doing so, and respecting the role of failure in science), trust in research findings – and ownership of science – may be adversely impacted (and, again, looking at the wider history of science communication and relevant behavioural science, we may expect negative experiences with research to be given more weight than positive experiences; see McCord, 2003). Factors outside of the field – such as wider discourses on (anti-)expert and elite knowledge – may also impact opportunities for evidence-based practice. The goal of this empirical project is therefore not to develop a complete set of rules for effective communication, but rather to follow our own advice: to continue to practice communication strategies that follow the best available evidence at the time (recognizing that the evidence will be revised sooner or later), to help us address the greatest concentrations of harm.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship and/or publication of this article.
