We evaluate the Violent Offender Identification Directive (VOID) tool, a risk prediction instrument implemented within a police department to identify offenders likely to be involved with future gun violence. VOID uses a variety of static measures of prior criminal history that are available in police records management systems. The VOID tool is assessed for predictive accuracy by taking a historical sample and calculating scores for over 200,000 individuals known to the police at the end of 2012, and predicting 103 individuals involved with gun violence (either as a shooter or a victim) during 2013. Despite weights for the instrument being determined in an ad hoc manner, the VOID tool does very well in predicting involvement with gun violence compared with an optimized logistic regression and generalized boosted models. We discuss theoretical reasons why such ad hoc instruments are likely to perform well in identifying chronic offenders for all police departments.
AbrahamseA. F.EbenerP. A.GreenwoodP. W.FitzgeraldN.KosinT. E. (1991). An experimental evaluation of the Phoenix repeat offender program. Justice Quarterly, 8, 141-168.
2.
AndrewsD. A.BontaJ. (1995). The Level of Service Inventory—Revised. Toronto, Ontario, Canada: Multi-Health Systems.
3.
AndrewsD. A.BontaJ. (2006). The psychology of criminal conduct (4th ed.). Cincinnati, OH: Anderson.
4.
AndrewsD. A.BontaJ.WormithS. J. (2006). The recent past and near future of risk and/or need assessment. Crime & Delinquency, 52, 7-27.
5.
BerkR. (2011). Asymmetric loss functions for forecasting in criminal justice settings. Journal of Quantitative Criminology, 27, 107-123.
6.
BerkR. (2016, May). A primer on fairness in criminal justice risk assessments. The Criminologist, 41(6), 6-9.
7.
BerkR.ShermanL.BarnesG.KurtzE.AhlmanL. (2009). Forecasting murder within a population of probationers and parolees: A high stakes application of statistical learning. Journal of the Royal Statistical Society: Series A (Statistics in Society), 172, 191-211.
8.
BorumR.BartelP.ForthA. (2003). SAVRY: Structured Assessment of Violence Risk in Youth (Consultation version). Tampa: Florida Mental Health Institute, University of South Florida.
9.
BragaA.PapachristosA. V.HureauD. M. (2010). The concentration and stability of gun violence at micro places in Boston, 1980-2008. Journal of Quantitative Criminology, 26, 33-53.
10.
CampbellM. A.FrenchS.GendreauP. (2009). The prediction of violence in adult offenders: A meta-analytic comparison of instruments and methods of assessment. Criminal Justice and Behavior, 36, 567-590.
11.
ChoiratC.GandrudG.HonakerJ.ImaiK.KingG.LauO. (2018). Zelig: Everyone’s statistical software. Available from http://zeligproject.org/
12.
Corbett-DaviesS.PiersonE.FellerA.GoelS.HuqA. (2017). Algorithmic decision making and the cost of fairness. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 797-806). Halifax, Nova Scotia, Canada: Association for Computing Machinery.
13.
DavisJ.GoadrichM. (2006). The relationship between precision-recall and ROC curves. In Proceedings of the 23rd International Conference on Machine Learning (pp. 233-240). Halifax, Nova Scotia, Canada: Association for Computing Machinery.
14.
DawesR. M. (1979). The robust beauty of improper linear models in decision making. American Psychologist, 34, 571-582.
15.
DolanM.DoyleM. (2000). Violence risk prediction: Clinical and actuarial measures and the role of the Psychopathy Checklist. The British Journal of Psychiatry, 177, 303-311.
16.
DouglasK. S.YeomansM.BoerD. P. (2005). Comparative validity analysis of multiple measures of violence risk in a sample of criminal offenders. Criminal Justice and Behavior, 32, 479-510.
17.
DresselJ.FaridH. (2018). The accuracy, fairness, and limits of predicting recidivism. Science Advances, 4(1), eaa05580.
18.
FazelS.SinghJ. P.DollH.GrannM. (2012). Use of risk assessment instruments to predict violence and antisocial behaviour in 73 samples involving 24 827 people: Systematic review and meta-analysis. British Medical Journal, 345, Article e4692.
19.
FergusonA. G. (2017). The rise of big data policing: Surveillance, race, and the future of law enforcement. New York: New York University Press.
20.
FowlerK. A.DahlbergL. L.HaileyesusT.AnnestJ. L. (2015). Firearm injuries in the United States. Preventive Medicine, 79, 5-14.
21.
GloverA. J.NicholsonD. E.HemmatiT.BernfeldG. A.QuinseyV. L. (2002). A comparison of predictors of general and violent recidivism among high-risk federal offenders. Criminal Justice and Behavior, 29, 235-249.
22.
GoelS.RaoJ. M.ShroffR. (2016). Precinct or prejudice? Understanding racial disparities in New York City’s stop-and-frisk policy. The Annals of Applied Statistics, 10, 365-394.
23.
GroveW. M.ZaldD. H.LebowB. S.SnitzB. E.NelsonC. (2000). Clinical versus mechanical prediction: A meta-analysis. Psychological Assessment, 12, 19-30.
24.
HansonR. K. (2009). The psychological assessment of risk for crime and violence. Canadian Psychology/Psychologie Canadienne, 50, 172-182.
25.
HarcourtB. (2007). Against prediction: Profiling, policing, and punishing in an actuarial age. Chicago, IL: The University of Chicago Press.
26.
HiltonN. Z.HarrisG. T.RiceM. E.HoughtonR. E.EkeA. W. (2008). An indepth actuarial assessment for wife assault recidivism: The domestic violence risk appraisal guide. Law and Human Behavior, 32, 150-163.
27.
HiltonN. Z.HarrisG. T.RiceM. E.LangC.CormierC. A.LinesK. J. (2004). A brief actuarial assessment for the prediction of wife assault recidivism: The Ontario Domestic Assault Risk Assessment. Psychological Assessment, 16, 267-275.
28.
JenningsW. G. (2006). Revisiting prediction models in policing: Identifying high-risk offenders. American Journal of Criminal Justice, 31, 35-50.
29.
JenningsW. G.PiqueroA. R.ReingleJ. M. (2012). On the overlap between victimization and offending: A review of the literature. Aggression and Violent Behavior, 17, 16-26.
30.
JungJ.ConcannonC.ShroffR.GoelS.GoldsteinD. G. (2017). Simple rules for complex decisions. Social Science Research Network. doi:10.2139/ssrn.2919024
31.
KahnemanD. (2011). Thinking, fast and slow. New York, NY: Farrar, Straus and Giroux.
32.
KingG.ZengL. (2001). Logistic regression in rare events data. Political Analysis, 9, 137-163.
33.
KronerD. G.MillsJ. F.ReddonJ. R. (2005). A coffee can, factor analysis, and prediction of antisocial behavior: The structure of criminal risk. International Journal of Law and Psychiatry, 28, 360-374.
KrzanowskiW. J.HandD. J. (2009). ROC curves for continuous data. London, England: Chapman and Hall.
36.
LiawA.WienerM. (2002). Classification and regression by randomForest. R News, 2(3), 18-22.
37.
LumK.IsaacW. (2016). To predict and serve?Significance, 13(5), 14-19.
38.
MartinS. E.ShermanL. W. (1986). Catching career criminals: Proactive policing and selective apprehension. Justice Quarterly, 3, 171-192.
39.
MeehlP. (1954). Clinical versus statistical prediction: A theoretical analysis and a review of the evidence. Minneapolis: University of Minnesota Press.
40.
MessingJ. T.CampbellJ.WilsonS. J.BrownS.PatchellB. (2017). The lethality screen: The predictive validity of an intimate partner violence risk assessment for use by first responders. Journal of Interpersonal Violence, 32, 205-226.
41.
MessingJ. T.ThallerJ. (2013). The average predictive validity of intimate partner violence risk assessment instruments. Journal of Interpersonal Violence, 28, 1537-1558.
42.
MillerJ.MaloneyC. (2013). Practitioner compliance with risk/needs assessment tools: A theoretical and empirical assessment. Criminal Justice and Behavior, 40, 716-736.
43.
MillsJ. F.KronerD. G.HemmatiT. (2003). Predicting violent behavior through a static-stable variable lens. Journal of Interpersonal Violence, 18, 891-904.
44.
MonahanJ.SkeemJ. L. (2016). Risk assessment in criminal sentencing. Annual Review of Clinical Psychology, 12, 489-513.
45.
NeuillyM. A.ZgobaK. M.TitaG. E.LeeS. S. (2011). Predicting recidivism in homicide offenders using classification tree analysis. Homicide Studies, 15, 154-176.
46.
NofzigerS.KurtzD. (2005). Violent lives: A lifestyle model linking exposure to violence to juvenile violent offending. Journal of Research in Crime and Delinquency, 42, 3-26.
PapachristosA. V.BragaA. A.HureauD. M. (2012). Social networks and the risk of gunshot injury. Journal of Urban Health, 89, 992-1003.
49.
PapachristosA. V.KirkD. S. (2015). Changing the street dynamic: Evaluating Chicago’s group violence reduction strategy. Criminology & Public Policy, 14, 525-558.
50.
PearV. A.Castillo-CarnigliaA.KagawaR. M.CerdáM.WintemuteG. J. (2018). Firearm mortality in California, 2000-2015: The epidemiologic importance of within-state variation. Annals of Epidemiology, 28, 309-315.e2.
51.
PiqueroA.MacDonaldJ.DobrinA.DaigleL.CullenF. T. (2005). Self-control, violent offending, and homicide victimization: Assessing the general theory of crime. Journal of Quantitative Criminology, 21, 55-71.
52.
QuinseyV. L.HarrisG. T.RiceM. E.CormierC. A. (2006). The law and public policy: Violent offenders: Appraising and managing risk. Washington, DC: American Psychological Association.
53.
RidgewayG. (1999). The state of boosting. Computing Science and Statistics, 31, 172-181.
RozelJ. S.MulveyE. P. (2017). The link between mental illness and firearm violence: Implications for social policy and clinical practice. Annual Review of Clinical Psychology, 13, 445-469.
56.
SaundersJ.HuntP.HollywoodJ. S. (2016). Predictions put into practice: A quasi-experimental evaluation of Chicago’s predictive policing pilot. Journal of Experimental Criminology, 12, 347-371.
57.
SinghJ. P. (2013). Predictive validity performance indicators in violence risk assessment: A methodological primer. Behavioral Sciences & the Law, 31, 8-22.
58.
SinghJ. P.DesmaraisS. L.Van DornR. A. (2013). Measurement of predictive validity in violence risk assessment studies: A second-order systematic review. Behavioral Sciences & the Law, 31, 55-73.
59.
SpelmanW. (1990). Repeat offender programs for law enforcement. Washington, DC: Police Executive Research Forum.
60.
SpelmanW. (1995). Criminal careers of public places. Crime Prevention Studies, 4, 115-144.
61.
StoreyJ. E.KroppP. R.HartS. D.BelfrageH.StrandS. (2014). Assessment and management of risk for intimate partner violence by police officers using the Brief Spousal Assault Form for the Evaluation of Risk. Criminal Justice and Behavior, 41, 256-271.
62.
TetlockP. E.GardnerD. (2015). Superforecasting: The art and science of prediction. New York, NY: Crown Publishers.
63.
TollenaarN.van der HeijdenP. G. M. (2013). Which method predicts recidivism best? A comparison of statistical, machine learning and data mining predictive models. Journal of the Royal Statistical Society: Series A (Statistics in Society), 176, 565-584.
64.
WebsterC. D.EavesD.DouglasK. S.WintrupA. (1995). The HCR-20 scheme: The assessment of dangerousness and risk. Vancouver, British Columbia, Canada: Simon Fraser University, Forensic Psychiatric Services Commission of British Columbia.
65.
YangM.WongS. C.CoidJ. (2010). The efficacy of violence prediction: A meta-analytic comparison of nine risk assessment tools. Psychological Bulletin, 136, 740-767.
66.
ZengJ.UstunB.RudinC. (2017). Interpretable classification models for recidivism prediction. Journal of the Royal Statistical Society: Series A (Statistics in Society), 180, 689-722.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
