Abstract
This article describes realist evaluation research combining data signatures and theories of causal mechanism as a means of shedding light on why crime has declined in recent years. A data signature is an empirical indicator of how or why something has occurred. The use of multiple signatures – a ‘dish’ – from different angles and contexts can, if they point in the same direction, result in a form of triangulation that reduces the chance of interpretive error. The signatures identified strongly suggest that more and better security played a key role in the global ‘crime drop’, and in so doing, they rebut rival hypotheses.
Introduction
Historians of statistics will likely always debate the provenance of the Thomas Mann–Whitney Houston collaboration that led to the eponymous U-test. Hence, it is vitally important to document the origins of innovation that undergirds research, as it often involves more than simply the application of existing methodology. This article tells a story about crime and a problem-solving methodological approach. It tells of analytic hoops we jumped through in our research examining why many types of crime have been falling in England and Wales (Figure 1) and elsewhere over the last quarter century.
Crime survey for England and Wales 1981–2013 (incidence rates per 1000 households or adults).
The purpose of the story is to show how and why we focused on ‘data signatures’ to analyse the source of the puzzling and pervasive crime drops that began in many countries from the early 1990s (and a little earlier for some property crimes in the United States). Our focus on the data signatures found in the crime drop has been useful in eliminating many of the reasons that have been advanced for the crime drop as well as for corroborating one of them, at least for now (Farrell, 2013; Farrell et al., 2014).
The story begins in 2006 when we wrote our first research grant proposal on this topic, and at the time of writing in mid-2015, our third sequential project is underway. It is beginning to look like a research programme – almost as if we planned it (see, for example, Farrell, 2013, in press; Farrell and Brown, in press; Farrell et al., 2008, 2010, 2011a, 2011b, 2014; Farrell and Tilley, in press; Tilley, 2012; Tilley et al., 2011, 2015a, 2015b; Tseloni et al., 2010, 2012, 2014; Tseloni and Thompson, 2015; Van Dijk and Tseloni, 2012). Our overarching research question throughout has been: did security cause some or all of the massive drops in crime? To cut to the chase, our conclusion so far is as follows:
Security played a key role in the major falls in vehicle crime and domestic burglary, and probably other crimes including violence, over the last two decades.
The policy implications are huge: crime policy should more vigorously encourage the development of security for those other crimes that are currently increasing. These include cyber-crime, theft of smartphones and similar high-value portable electronics, and the growing prominence of organised crimes such as illicit trafficking of different types.
The realist deal
Key to the research has been a problem-solving approach, which is consistent with a realist orientation to evaluation (Pawson and Tilley, 1997). Pawson and Tilley stressed the development of Context Mechanism Outcome Pattern Configurations (CMOCs). CMOCs refer to the causal mechanisms that are activated or deactivated when a programme or policy is introduced in a range of contexts leading to changed regularities, producing an outcome pattern. The more precisely the changes in regularities can be specified through the CMOC conjecture, the more confidence can be had in the conjecture. The problem-solving counterpart to CMOCs is the development of interventions where problems are identified (‘Scanning’), casual mechanisms generating them analysed (‘Analysis’), measures developed that with undermine relevant mechanisms generating the problems (‘Response’) and evaluations undertaken to determine whether the problem has been eliminated or ameliorated (‘Assessment’). These terms form the acronym SARA, which was devised by Eck and Spelman (1987), and has been widely used to capture this process. In our work, we began with a puzzling outcome – steep, widespread and unexpected drops in crime rates – rather than a programme to assess. However, the approach we have adopted accords well with Pawson and Tilley’s realist evaluation and with Eck and Spelman’s SARA process.
In particular, the identification of a series of data signatures has been critical to our work. Data signatures per se are not new (cf. Bowers and Johnson, 2004; Garwood et al., 2000; Pawson and Tilley, 1997), but their formal demarcation as a methodological tool for crime prevention research has been spurred on by Eck and Madensen (2009). They find that ‘The analysis of crime signature change, as part of crime prevention evaluations, can improve the internal validity of evaluation findings’. (Eck and Madensen, 2009: 59). They highlight the distinctiveness of the mechanisms associated with different situational crime prevention interventions and the associated outcome patterns that would follow from the activation of these mechanisms. Our triangulation work is consistent with three important points on signatures made by Eck and Madensen (2009: 69):
Signature changes consistent with expected intervention mechanisms eliminate rival explanations;
Signature changes inconsistent with the expected intervention mechanism undermine the validity of the conclusion that the intervention produced the crime change;
The more specific and prominent the change in signatures, the fewer viable candidate explanations there can be.
The more precisely an observed pattern accords with the expected signature, the more confident we can be in attributing causality to it, as also stressed by Pawson and Tilley.
A proto-realist evaluation, to which Pawson and Tilley refer as one of their exemplars, which also accords well with Eck and Madensen’s emphasis on crime signatures, is Laycock’s study of property marking in South Wales villages as a means to prevent domestic burglary (Laycock, 1985, 1991). Overall falls in domestic burglary in the villages followed introduction of the initiative. Laycock, however, ‘directed her outcome measurements towards precise theoretically expected effects’ (Pawson and Tilley, 1997: 91). One mechanism might involve retrieving stolen property and another detecting offenders with stolen goods. Both were measured but neither found. Laycock conjectured that intensive and persuasive publicity leading to increased perceived risk to local burglars (remembering that the contexts was isolated villages in a Welsh Valley), deterring them from offending, might comprise an important mechanism to generate falls in burglary (the initiative had been highly publicised when first introduced to encourage uptake in the villages). Cleverly, Laycock also tracked offending patterns after her first report was published and received positive publicity, and immediately subsequent renewed falls in burglary appeared to corroborate her conjecture. Her conclusion was that rather than the property marking per se producing the crime drop, it was the intensive and persuasive publicity that risks to local offenders were being increased. The outcome signature was consistent with one but not the other explanation.
The specifics of the data signature approach as it relates to the historical crime drop on which our research focused should become apparent as the remainder of the article unfolds.
International relations
In parallel with the analysis described below, we examined the global crime drop. Back in 2007, it was not as well-known that crime had declined in many countries, and we confirmed the general finding of others (Van Dijk et al., 2007 in particular) and added to the knowledge base. We used the International Crime Victims Survey (ICVS) which is a unique resource as the only cross-national standardised general victim survey (Mayhew and van Dijk, 2014). It has been conducted every 3–5 years since 1989 across many countries, with small but representative national or city samples and an identical questionnaire. This means its findings are widely comparable, unlike police statistics which have definitional reporting and recording differences, and national crime surveys, which have definitional and survey methodology differences (Van Dijk and Tseloni, 2012).
We examined five volume crime types – domestic burglary, theft from the person, theft from car, car theft and assault – using multivariate multilevel regression modelling in an innovative manner: treating the different time periods of country-level data as repeated measures within each country to test for a linear or non-linear trend. Three of the insights this provided will be highlighted here (see Tseloni et al., 2010 for the full story). First, cross-nationally, domestic burglary and car theft were the first offences to decline, and they did so from the first ICVS sweep in 1988. Theft from the person and theft from cars began their decline with the 1995 sweep with assault declining from the 2000 sweep. Second, volume crime drops were roughly similar across countries except for burglary which, to the extent that could be determined with the more fragmented available data, fell faster outside Europe, North America and Australasia. Third, in addition to displaying common cross-national trends, the crimes of burglary, theft from the person and car theft were highly correlated cross-nationally, while assault appeared strongly related to burglary and car theft.
The international analysis is an important counterpart to the analysis that follows. The likelihood that crime declined rapidly by chance in many countries at a similar time is vanishingly small. For us, this implies the possibility of a common cause which first impeded car thefts and burglaries then, perhaps via a spill-over effect, other crime types including violence. In addition, it means that we suspect our England and Wales-oriented work, described below, has broader geo-political implications.
Our argument is, thus, that the data signatures we examine, although relating largely to one country, suggest more general patterns and hence a way of addressing a major puzzle in criminology. Of course to corroborate this, what we have done would need to be replicated by research in other countries to determine whether similar distinctive signatures are found there also.
The other logistic regression
We used the Crime Survey for England and Wales (CSEW, formerly the British Crime Survey) as our main data source. It provides data for crimes occurring in 1981 onwards, was conducted infrequently in the 1980s, biennially in the 1990s and continually since 2001/2002. A key feature is that it gathers much additional information including some on security measures, albeit with varying consistency over time.
We will skip past most survey method issues here, but experienced readers will know that the task of analysing separate sequential data files such as those of the CSEW is an immensely time-consuming task which necessitates great attention to detail. Each annual survey sweep requires separate analysis with new code to be written. It requires significant skills such as repeated reorganisation of the files – or what is known to us as the other logistic regression. So we are surprised and thankful to have made it thus far, and have two exhausted research assistants – who ruthlessly abandoned the team once they got their PhDs – to show for it.
It’s a sign of the crimes!
We started by analysing car theft, in part because we believed its decline to be under-researched (see Farrell et al., 2008). The first three signatures we identified provide information on the causal role of security. If opportunity is driving car theft, then we would expect to see some specific changes where security measures are introduced to reduce it.
1. The average age of stolen vehicles should increase because newer vehicles have better security.
2. Door lock-forcing should decline much more than other means of entering vehicles, due to central dead-locking systems.
3. Temporary theft for joyriding and transportation should decline sooner and faster than permanent thefts for chopping and re-sale – because younger less experienced offenders were more easily deterred.
We find that all these expectations are met. The change in means of entry to vehicles is shown as Figure 2 by means of example.
Means of entry to vehicles 1995–2010 (CSEW).
We were able to replicate the results with Australian data and added a fourth signature to the repertoire when we happened upon a natural experiment:
4. In Australia, electronic vehicle immobilisers were introduced in one state (Western Australia) in advance of elsewhere. Car theft fell sooner in that state and then fell in the rest of the country, both times tracking the spread of security (Farrell et al., 2011b).
Note that the statistical signatures only make sense when informed by the theoretical mechanisms by which different types of security device ought to work: we have information on central locking, electronic immobilisers (but also mechanical immobilisers – often the awkward old steering wheel clamps known as clubs or crook-locks), car alarms and tracking devices – and this aspect of the work is described next.
Competing explanations for the drop in car theft would, of course, have also to be consistent with these specific patterns.
Auto crime and autocorrelation
After releasing an early version of part of the work in a newsletter (Farrell et al., 2008), we were astonished at the speed with which teams in other countries replicated and extended it (though some may have evolved in parallel). By now, with some variation in method and data, the finding that security caused the fall in car theft appears true for Australia, England and Wales, the Netherlands and the United States (Fujita and Maxfield, 2012; Van Ours and Vollaard, 2013), with supporting evidence on the effectiveness of electronic immobilisers from Germany and elsewhere (Bassman, 2011; Brown, 2013). Not only does the replication bode well, but since automobile security was improved (often by legislation) at different times in different countries, together the studies provide the fifth signature:
5. The timing and trajectory of the spread of security in different countries appears to track the introduction and spread of electronic vehicle immobilisers.
Although it was not part of the research by the present authors, some confirmation of the likely importance of security was identified by interviews with offenders in Australia (Brown, 2015). Offenders were asked why they thought property crime had declined and the most popular response was that it was due to security.
But which security devices work best? And which work best for different types of car crime? Here, we included theft from cars as well as vandalism, initially comparing crime rates for cars without security to those with particular types. This evolved into what we termed the Security Impact Assessment Tool (SIAT). The SIAT uses odds ratios of the likelihood of crime when particular security devices, or combinations, are present, to tease out their relative crime prevention effects (Farrell et al., 2011a). The term we adopted for the measure this generates is the Security Protection Factor, or SPF, which is the same acronym as Sun Protection Factor – and both measure the units of time, relative to the absence of protection, after which the user is burnt. An example of SIAT output is shown later for burglary. Conceptually though, what it does is quantify the effectiveness of different combinations of devices. The SPFs are our sixth statistical signature and told us that:
6. Specific security devices and their combinations vary greatly in their security effectiveness, which varies by crime type, and these patterns fit with theoretical expectation.
We found tracking devices to be very effective against car theft but they were not around much in the 1990s when crime plummeted. Of devices that were, we found electronic immobilizers to be the most effective, mechanical immobilisers and central locking systems to be also quite effective, alarms only modestly effective, and other devices (window etching, parts marking) of little use. Theft from vehicles is more likely to be interrupted by central locking than electronic immobilisers, which makes sense since a car with only the latter remains easy to break into but is harder to steal. Car alarms only ever had a modest effect, but it was larger against theft from cars, which also makes sense (i.e. conforms with theoretical expectation of how the preventive mechanism works) as the less experienced offenders committing these crimes are more easily deterred.
An important finding is that devices work much better when used in combination. That is, combinations of good devices produced non-linear benefits. Electronic immobilisers and central locking were dominant but cars with those plus an alarm and a tracker were found to have a car theft SPF of 25 against theft – that is, to be 25 times safer than a vehicle without security. The importance of combinations of devices was confirmed in subsequent analysis of household security devices and burglary which is detailed next.
Alarming implications
The focus shifted to domestic burglary for the second main strand of the research. Building on the car theft work, we applied the SIAT to household security devices. This was more challenging because of the larger number of burglary devices recorded in the CSEW. There were nine devices in total which in principle produce hundreds of combinations. We identified the most prevalent devices and combinations and focused our analysis on them. We found that burglary security devices, individually but especially in certain combinations, are much more effective than car security. As expected, some work better than others and combinations are particularly effective, sometimes exponentially so. You can rest easy if you have the combination of:
Security protection factors (SPFs) of selected burglary security home features based on the 2008/2009–2011/2012 CSEW.
An unanticipated finding from our research – but one that in some ways gives it that real-world feel – is the finding that burglar alarms appear to be largely ineffective (Tilley et al., 2015). Although there has been previous research into household security devices, most of it tends to group security devices together, so it is possible this has been missed. While it has long been known that most alarm activations are false (Litton and Pease, 1984; Sampson, 2011) and therefore may not mobilise much formal or informal surveillance, it was a largely unanticipated finding that warrants further research.
Sneaky measures
Two things were particularly troublesome in the early stages of the research into household burglary – so much so that they potentially falsified the security hypothesis. The first was that, even before burglary plummeted in 1993, a fair proportion of households already had some security of different types. The second was that while there was a significant increase in security levels when burglary fell, it was too modest to account for the fall. So how, we pondered, could security be responsible if there was already a fair bit of it and if it failed to increase hugely?
The answers appear to be quality and combinations. We strongly suspect the quality of security devices dramatically improved from the early 1990s – but that this is masked in the simple counts of the prevalence of individual devices. Finding a signature to test this conjecture was challenging. To try to do so, we developed a sneaky measure that drew on how we had examined car theft, and looked at how offenders gained entry (with inspiration for sneaky measures from Garwood et al., 2000). We grouped entry methods into two broad types:
Forced entries – where security was overcome, including lock-forcing on doors or windows, the removal or breaking of door panels or windows;
Unforced entries – where no security was overcome including unlocked doors, entry with a key, pushing past the occupant or use of deception.
If it was security that improved, then the signature we would expect is that burglaries where security was involved would fall sooner and faster than the others. This is what occurred, shown in Figure 4. The issue is most evident when means of entry via doors or windows are examined separately. It is clear that a reduction in forced entries accounted for the bulk of the decline in household burglary whereas other types of entry declined less and declined later (Figures 5 and 6). While this does not directly show a change in the quality of devices, it does show that it was forced rather than unforced entries that declined, for which the best explanation we can come up with is more and better security.
Household burglary – means of entry, 1992–2011 (CSEW).
Forced and unforced door entry, 1991–2011.
Forced and unforced window entry, 1991–2011.
While the data were imperfect, we were also able to gain some insight into the use of combinations of devices. Unfortunately, the best data were only available from 1998 onwards but we extrapolated the trends backwards. Rather uncannily, the extrapolated trends identify a dramatic increase in the use of combinations of the WIDEr more effective devices in the early 1990s at the time when burglary began to fall. The same data imperfections meant the proportion of households without any security devices appeared to have fallen dramatically in the early 1990s due to different definition of ‘no security’ prior to the 1998 CSEW. These signatures dovetail well with the analysis of between-device variation in effectiveness and previous analysis of the role of combinations of devices against car theft.
So, there is mounting evidence that security may have induced the fall in household burglary. But how and why would security have spread and improved? We anticipate that the improvement in security occurred via the spread of home improvements, particularly insulation and double-glazing (sometimes termed storm windows). Double-glazed windows have two tougher panes of glass, and have stronger frames such that breaking them is more difficult. Further, double-glazed windows and doors tend to have built-in rather than add-on locks which means they are more likely to be used (the awkward nut on the bolt is not left lying around or lost). Figure 7 shows the spread of fully double-glazed households in the United Kingdom and the drop in burglary in the 1990s, and while the data are incomplete across the timespan of the decline in burglary, they are at least somewhat indicative.
Increasing double-glazing and declining burglary, 1996–2008.
This burglary analysis is detailed in a study that is under review for publication at the time of writing but versions of the findings were presented at the conference to mark the end of our funded research on burglary and an international conference (Farrell et al., 2015b; Tilley et al., 2015). Independent support appears to derive from the recent work of Brown (2015) in Australia. Offenders were asked why they thought crime had declined and the most popular response was that they believed it was due to security.
Looming violence
What about violence? This is probably the biggest question remaining and the topic of our more recently awarded research grant. We suspect that security played a key role both directly and indirectly. For some crimes such as robbery, the effect is likely to have been direct. Banks and commercial stores, buses and taxis have all introduced an array of improved security measures to reduce robberies. A recent US study demonstrated that taxi-driver homicides fell in the 1990s in those cabs with security cameras installed, relative to a control group without (Chaumont Menendez et al., 2014). Older readers will remember the days when bus drivers carried a large bag of fare monies – the days before correct-payment by machine and a secure drop-box for cash. Late-night shops now tend to have limited cash on the premises and an array of design improvements. We anticipate that these and other measures reduced violence in and around many public and private premises and spaces. Robbery disproportionately results in homicide, and other homicides are assaults-gone-wrong, so their decline may have reduced homicide as a natural statistical consequence.
Alongside the adoption of security technology to protect their cars and homes, individuals may have become more crime risk-averse, adopting or modifying their routine activities to reduce their exposure to stranger and acquaintance violence, especially in night-time economy (Ganpat et al., 2015; Garius, 2015). Here, the distinct signature would include reduced levels of exposure to risks of stranger violence in public places and hence lower levels of actual violence among previously highly victimised groups showing enhanced precautionary behaviour. However if ‘street’ crime is more difficult due to security it is possible that this also nudges adolescents to stay indoors.
We are also exploring the indirect effects, that is, how falling property crime may have reduced violence. It may surprise some generalist readers to learn that most ‘career criminals’ are not specialists. That is, they tend to commit different types of crime across the spectrum of property and violence – including sexual victimisation and domestic violence (Farrington, 1998; Piquero et al., 2014; Richards et al., 2013; Smallbone and Wortley, 2004). Since most crime is property crime, they commit mainly that plus a small amount of violence. Perhaps much of that violence is committed while undertaking, following-through on, or in the wake of completed property crimes, which could mean the reduction in property crime has triggered some of the broader reductions in violence. So, for example, acquisitive property crime is a precursor of fencing, and car theft may be a precursor to crime requiring a getaway vehicle such as suburban burglary. We term this the keystone hypothesis because the removal of the volume property crimes may, like the removal of the keystone in an arch, have caused all the other crime types to tumble.
The second main indirect route we are examining is the debut crime hypothesis. This involves examining the extent to which early-career offenders who are stifled if their debut crimes – mainly shoplifting, car theft and other property crimes – do not progress to commit the broader array of crimes including violence. The signatures for this might include falling levels of acquisitive criminal activity among younger offenders followed by falls in violence as they reach later adolescence, while levels of violent behaviour are maintained in other cohorts whose onset of criminality has not been inhibited by increases in security relating to typical debut crimes. Promising evidence on changing criminal career patterns speaks to this hypothesis (Farrell et al., 2015b). Hence, we suggest that while there is a great need for further research into security and the violent crime drop, there are some promising avenues to pursue.
Conclusion: of signatures and significance
Triangulation from a variety of data signatures and sources is, statistically speaking, an oddity. If one indicator has, say, a 5% chance of error, then two independent indicators, for example, based on different data sources, imply a (0.05 × 0.05 = 0.0025) 0.25% chance and three, a 0.0125% chance, and so on. With the broad range of data signatures identified to different data, plus some near-replications from other countries in relation to the car theft and burglary work, and since these findings fit with theoretical expectation at different levels plus there is reason to anticipate some broader application of these findings, then what, statistically speaking, is the chance of error? Well do not look at us because we do not know. But we now feel fairly safe suggesting that there is good evidence to support the security hypothesis while recognising that a long and winding research road lies ahead.
And we hope for responsible criticism and attempts at falsification, and challenge researchers to identify suitable data sources that facilitate such efforts. But we would also ask readers to identify plausible alternate hypotheses that could account for the various data signatures described herein, for we cannot. For example, when considering a rival hypothesis, first consider why it might reduce forced but not unforced entry to households. Then run it past some of the other signatures because a plausible rival hypothesis should be consistent with each one.
Crime drop research falls increasingly into the domain of modern history. It will require archival research and imaginative efforts to pursue. Trawling through the records of security manufacturers, insurers, construction companies, the market research industry, business groups and others, could shed light on where, when and what security developments occurred and were implemented. In-depth interviews with experts including ageing offenders or ex-offenders (see Brown, 2015) and experts involved in security at and around the time, are required. The clock is ticking on the last aspect because the likelihood of retirement of domain-specific experts increases daily. Other sources of information should emerge as the work progresses, consistent with the nature of the investigative triangulation approach.
The most important policy conclusion suggested by the research is that society should put time and effort into encouraging elegant security solutions. The process can, we feel, be accelerated. Government should assist the corporate sector in reducing its crime emissions – products and practices that generate volume crimes: it took decades to get secure cars, and we should have had secure phones long ago, while there is scope for Internet service providers to secure key components of the Internet much more immediately. And society has a significant comparative advantage over even the most adaptive of offenders. This story of our use and abuse of statistical data signatures and theories of causal mechanisms has been a key component of how we reached this conclusion. Rather than arrest for the wicked, we advocate elegant design and secure solutions that remove crime opportunities.
What we have described here is a research programme that embodies iterative progression and the development of multiple data signatures to test context-mechanisms outcome pattern conjectures relating to security improvements and the crime drop. This investigative triangulation approach sits within the realist evaluation framework: confidence in a conjecture can be multiplied where successive signatures corroborate one another and where alternative explanations can be compared for their consistency with signatures. The idea of multiple ‘signatures’ specifically comes from situational crime prevention and was initially formulated for evaluating small-scale interventions to address specific crime problems. We suspect that the method has more general application and its use in this article is intended to illustrate this and to encourage others working in different fields. There are many sources of data collected using many different methods. To mix metaphors, just as we need to select ingredients from a well-stocked larder to cook a bespoke dish, so too we need to look to data source larders to select the ingredients to empirically test our hypotheses. In referring to a dish, we hope the culinary allusion is not too abstract, for we regularly taste each doughy signature as it … proves.
Footnotes
Acknowledgements
We thank Jen Mailley, Rebecca Thompson, Emily Evans, Louise Grove and Emmanuel Nimo for their various endeavours as part of this research, and the members of our three advisory teams.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: We thank the Economic and Social Research Council for grants RES-000-22-2386, ES/K003771/1, ES/K003771/2 and ES/L014971/1.