Addressing the Treatment-as-Assigned Assumption in Field Experiments: Lessons Learned From the Birmingham South Saturated Problem-Oriented Policing Hot Spots Experiment

Abstract

This study investigates the effectiveness of increasing police presence in hot spots typically managed with problem-oriented policing (POP) compared to using POP alone. Our block-randomized controlled trial reveals that overall, “saturated POP” does not significantly reduce crime compared to a POP-only approach. However, a more complex pattern emerged across different blocks, namely, an inverse relationship between the level of police presence and the level of recorded crime, regardless of the random assignment. Crime rates decreased in the experimental arm within two statistical blocks, where the assigned dosage of saturated POP was greater than in the control arm. However, in one block, the assigned control hot spots received a higher dosage of police presence than the assigned treatment hot spots, resulting in significantly lower crime rates in the control hot spots. These findings underscore the necessity of reporting field experiment results based on both treatment-as-assigned and some form of treatment-as-delivered models.

Keywords

experiment hot spots police community support officers saturated problem-oriented policing treatment fidelity intention to treat

Get full access to this article

View all access options for this article.

References

Altman, D. G., Schulz, K. F., Moher, D., Egger, M., Davidoff, F., Elbourne, D., Gøtzsche, P. C., Lang, T., & The Consort Group. (2001). The revised CONSORT statement for reporting randomised trials: Explanation and elaboration. Annals of Internal Medicine, 134(8), 663–694. https://doi.org/10.7326/0003-4819-134-8-200104170-00012

Angrist

J. D.

(2006). Instrumental variables methods in experimental criminological research: What, why and how. Journal of Experimental Criminology, 2(1), 23–44. https://doi.org/10.1007/s11292-005-5126-x

Anselin

(1995). Local indicators of spatial association—LISA. Geographical Analysis, 27(2), 93–115. https://doi.org/10.1111/j.1538-4632.1995.tb00338.x

Ariel

(2023). Implementation issues with hot spot policing. International Journal of Law, Crime and Justice, 75, 100629. https://doi.org/10.1016/j.ijlcj.2023.100629

Ariel

Bland

(2019). Is crime rising or falling? A comparison of police-recorded crime and victimization surveys. In Deflem

(Ed.), Methods of criminology and criminal justice research (pp. 7–31). Emerald Publishing Limited.

Ariel

Bland

Sutherland

(2022). Experimental designs. Sage.

Ariel

Farrington

D. P.

(2012). Randomised block designs. In Piquero

A. R.

Weisburd

(Eds.), Handbook of quantitative criminology (pp. 437–454). Springer.

Ariel

Sherman

L. W.

Newton

(2020). Testing hot-spots police patrols against no-treatment controls: Temporal and spatial deterrence effects in the London underground experiment. Criminology, 58(1), 101–128. https://doi.org/10.1111/1745-9125.12231

Ariel

Sutherland

Sherman

L. W.

(2019). Preventing treatment spillover contamination in criminological field experiments: The case of body-worn police cameras. Journal of Experimental Criminology, 15, 569–591. https://doi.org/10.1007/s11292-018-9344-4

10.

Ariel

Weinborn

Sherman

L. W.

(2016). “Soft” policing at hot spots–do police community support officers work? A randomized controlled trial. Journal of Experimental Criminology, 12(3), 277–317. https://doi.org/10.1007/s11292-016-9260-4

11.

Armijo-Olivo

Arienti

Stefano

(2019). Use of intention to treat and magnitude of treatment effects. International Journal of Technology Assessment in Health Care, 35(S1), 12–13. https://doi.org/10.1017/S0266462319001168

12.

Armitage

(1992). Some topics of current interest in clinical trials. The Canadian Journal of Statistics/La Revue Canadienne de Statistique, 20(1), 1–8. https://doi.org/10.2307/3315570

13.

Association of Chief Police Officers of England, Wales and Northern Ireland. (2008). Guidance on Police Community Support Officers. https://assets.publishing.service.gov.uk/media/5a7b0087ed915d429748c169/ACPO_guidance.pdf

14.

Baldassarri

Abascal

(2017). Field experiments across the social sciences. Annual Review of Sociology, 43(1), 41–73. https://doi.org/10.1146/annurev-soc-073014-112445

15.

Birmingham City Council. (2022). Birmingham South Locality Profile 2022. https://www.birmingham.gov.uk/download/downloads/id/23637/south_locality_profile_2022.pdf

16.

Bland

Ariel

Kumar

(2023). Criminal records versus rehabilitation and expungement: A randomised controlled trial. Journal of Experimental Criminology, 20(3), 717–741. https://doi.org/10.1007/s11292-023-09557-x

17.

Bland

Leggetter

Cestaro

Sebire

(2021). Fifteen minutes per day keeps the violence away: A crossover randomised controlled trial on the impact of foot patrols on serious violence in large hot spot areas. Cambridge Journal of Evidence-Based Policing, 5(3), 93–118. https://doi.org/10.1007/s41887-021-00066-3

18.

Blattman

Green

D. P.

Ortega

Tobón

(2021). Place-based interventions at scale: The direct and spillover effects of policing and city services on crime. Journal of the European Economic Association, 19(4), 2022–2051. https://doi.org/10.1093/jeea/jvab002

19.

Bloom

H. S.

(1984). Accounting for no-shows in experimental evaluation designs. Evaluation Review, 8(2), 225–246. https://doi.org/10.1177/0193841X8400800205

20.

Bloom

H. S.

(2008). The core analytics of randomized experiments for social research. In Alasuutari

Brannen

Bickman

(Eds.), The SAGE handbook of social research methods (pp. 115–133). Sage Publications.

21.

Braga

A. A.

Bond

B. J.

(2008). Policing crime and disorder hot spots: A randomized controlled trial. Criminology; An Interdisciplinary Journal, 46(3), 577–607. https://doi.org/10.1111/j.1745-9125.2008.00124.x

22.

Braga

A. A.

Turchan

Papachristos

A. V.

Hureau

D. M.

(2019). Hot spots policing of small geographic areas effects on crime. Campbell Systematic Reviews, 15(3), e1046. https://doi.org/10.1002/cl2.1046

23.

Braga

A. A.

Weisburd

D. L.

(2020). Does hot spots policing have meaningful impacts on crime? Findings from an alternative approach to estimating effect sizes from place-based program evaluations. Journal of Quantitative Criminology, 38(1), 1–22. https://doi.org/10.1007/s10940-020-09481-7

24.

Braga

A. A.

Weisburd

D. L.

Waring

E. J.

Mazerolle

L. G.

Spelman

Gajewski

(1999). Problem-oriented policing in violent crime places: A randomized controlled experiment. Criminology, 37(3), 541–580. https://doi.org/10.1111/j.1745-9125.1999.tb00496.x

25.

Braga

A. A.

Welsh

B. C.

Schnell

(2015). Can policing disorder reduce crime? A systematic review and meta-analysis. Journal of Research in Crime and Delinquency, 52(4), 567–588. https://doi.org/10.1177/0022427815576576

26.

Brantingham

P. J.

Brantingham

P. L.

(1995). Criminality of place: Crime generators and crime attractors. European Journal of Criminal Policy and Research, 3(3), 5–26. https://doi.org/10.1007/BF02242925

27.

Burke

R. H.

(2004). Hard cop, soft cop: Dilemmas and debates in contemporary policing. In Burke

R. H.

(Ed.), Hard cop, soft cop: Dilemmas and debates in contemporary policing (p. 310). Willan.

28.

Center for Problem-Oriented Policing. (2022). Step 5: Be true to POP. Arizona State University Center for Problem Oriented Policing. https://popcenter.asu.edu/content/step-5-be-true-pop

29.

Chainey

Ratcliffe

(2013). GIS and crime mapping. John Wiley & Sons.

30.

Cornish

D. B.

Clarke

R. V.

(1989). Crime specialisation, crime displacement and rational choice theory. In Criminal behavior and the justice system: Psychological perspectives (pp. 103–117). Springer Berlin Heidelberg.

31.

Cox

D. R.

Reid

(2000). The theory of the design of experiments. Chapman and Hall/CRC.

32.

Dau

P. M.

(2023). Disrupting everyday police presence: Mapping the geographies of policing and testing the crime reduction effects of police dosage [unpublished doctoral dissertation]. Ghent University.

33.

Dau

P. M.

Dewinter

Witlox

Beken

T. V.

Vandeviver

(2022). How concentrated are police on crime? A spatiotemporal analysis of the concentration of police presence and crime. Cambridge Journal of Evidence-Based Policing, 6(3), 109–133. https://doi.org/10.1007/s41887-022-00079-6

34.

Dau

P. M.

Vandeviver

Dewinter

Witlox

Vander Beken

(2021). Policing directions: A systematic review on the effectiveness of police presence. European Journal on Criminal Policy and Research, 29(2), 191–225. https://doi.org/10.1007/s10610-021-09500-8

35.

Eck

J. E.

(1994). Drug markets and drug places: A case-control study of the spatial structure of illicit drug dealing. University of Maryland.

36.

Eck

(1997). Preventing crime at places. In University of Maryland, Department of Criminology and Criminal Justice. (Ed.), Preventing crime: What works, what doesn’t, what’s promising (pp. 7‐1–7‐62). Office of Justice Programs, U.S. Department of Justice.

37.

Eck

(2002). Preventing crime at places. In Sherman

Farrington

Welsh

MacKenzie

D. L.

(Eds.), Evidence-based crime prevention (pp. 241–294). Routledge.

38.

Eck

J. E.

Clarke

R. V.

Guerette

R. T.

(2007). Risky facilities: Crime concentration in homogeneous sets of establishments and facilities. Crime Prevention Studies, 21, 225.

39.

Famega

Hinkle

J. C.

Weisburd

(2017). Why getting inside the “black box” is important: Examining treatment implementation and outputs in policing experiments. Police Quarterly, 20(1), 106–132. https://doi.org/10.1177/1098611116664336

40.

Fixsen

Blase

Metz

Van Dyke

(2013). Statewide implementation of evidence-based programs. Exceptional Children, 79(2), 213–230. https://doi.org/10.1177/0014402913079002071

41.

Friedman

L. M.

Furberg

C. D.

DeMets

D. L.

Reboussin

D. M.

Granger

C. B.

(2015). Fundamentals of clinical trials. Springer.

42.

Gates

Ariel

Assaraf

(2024). Responding to nonemergency calls for service via video: A randomized controlled trial. Criminology & Public Policy. https://doi.org/10.1111/1745-9133.12690

43.

Gill

Weisburd

Telep

C. W.

Vitter

Bennett

(2014). Community-oriented policing to reduce crime, disorder and fear and increase satisfaction and legitimacy among citizens: A systematic review. Journal of Experimental Criminology, 10, 399–428. https://doi.org/10.1007/s11292-014-9210-y

44.

Goldstein

(1990). Problem-oriented policing. McGraw-Hill.

45.

Groff

E. R.

Ratcliffe

J. H.

Haberman

C. P.

Sorg

E. T.

Joyce

N. M.

Taylor

R. B.

(2015). Does what police do at hot spots matter? The Philadelphia Policing Tactics Experiment. Criminology, 53(1), 23–53. https://doi.org/10.1111/1745-9125.12055

46.

Harachi

T. W.

Abbott

R. D.

Catalano

R. F.

Haggerty

K. P.

Fleming

C. B.

(1999). Opening the black box: Using process evaluation measures to assess implementation and theory building. American Journal of Community Psychology, 27(5), 711–731. https://doi.org/10.1023/A:1022194005511

47.

Hasisi

Itskovich

Khoury-Kassabri

(2022). A wall of treatments: An integrative problem-solving approach to the prevention of stone-throwing in East Jerusalem. Criminology & Public Policy, 22(2), 385–414. https://doi.org/10.1111/1745-9133.12607

48.

Hinkle

J. C.

Weisburd

Telep

C. W.

Petersen

(2020). Problem-oriented policing for reducing crime and disorder: An updated systematic review and meta-analysis. Campbell Systematic Reviews, 16(2), e1089. https://doi.org/10.1002/cl2.1089

49.

Home Office. (2020). Data tables for ‘Police workforce, England and Wales: 30 September 2020’. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/956740/police-workforce-sep20-tables.ods

50.

Hope

(1994). Problem-oriented policing and drug market locations: Three case studies. Crime Prevention Studies, 2(1), 5–32.

51.

Huey

Mitchell

R. J.

Kalyal

Pegram

(2021). Implementing evidence-based policing: A how-to guide for police organizations. Policy Press.

52.

Jagannathan

Camasso

M. J.

(2005). Beyond intention to treat analysis in welfare-to-work studies: The efficacy of labor force attachment, human capital investment and combined strategies for self-sufficiency. Journal of Social Service Research, 31(4), 43–60. https://doi.org/10.1300/J079v31n04_03

53.

Johnson

S. D.

Guerette

R. T.

Bowers

(2014). Crime displacement: What we know, what we don’t know, and what it means for crime reduction. Journal of Experimental Criminology, 10, 549–571. https://doi.org/10.1007/s11292-014-9209-4

54.

Jolliffe

Farrington

D. P.

(2007). A systematic review of the national and international evidence on the effectiveness of interventions with violent offenders. Ministry of Justice Research Series 16/07. https://www.crim.cam.ac.uk/sites/www.crim.cam.ac.uk/files/violmoj.pdf

55.

Kane

Evans

Shokraneh

(2018). Effectiveness of current policing-related mental health interventions: A systematic review. Criminal Behaviour and Mental Health, 28(2), 108–119. https://doi.org/10.1002/cbm.2058

56.

Kochel

T. R.

Burruss

G. W.

Weisburd

(2015). St. Louis County hot spots in residential areas (SCHIRA) final report: Assessing the effects of hot spots policing strategies on police legitimacy, crime, and collective efficacy. Southern Illinois University.

57.

Koper

C. S.

(1995). Just enough police presence: Reducing crime and disorderly behavior by optimizing patrol time in crime hot spots. Justice Quarterly, 12(4), 649–672. https://doi.org/10.1080/07418829500096231

58.

Lee

Eck

J. E.

Martinez

N. N.

(2017). How concentrated is crime at places? A systematic review from 1970 to 2015. Crime Science, 6(1), 1–16. https://doi.org/10.1186/s40163-017-0069-x

59.

Lipsey

M. W.

Cullen

F. T.

(2007). The effectiveness of correctional rehabilitation: A review of systematic reviews. Annual Review of Law and Social Science, 3(1), 297–320. https://doi.org/10.1146/annurev.lawsocsci.3.081806.112833

60.

Mills

L. G.

Barocas

Ariel

(2013). The next generation of court-mandated domestic violence treatment: A comparison study of batterer intervention and restorative justice programs. Journal of Experimental Criminology, 9(1), 65–90. https://doi.org/10.1007/s11292-012-9164-x

61.

Mills

L. G.

Barocas

Butters

R. P.

Ariel

(2019). A randomized controlled trial of restorative justice-informed treatment for domestic violence crimes. Nature Human Behaviour, 3(12), 1284–1294. https://doi.org/10.1038/s41562-019-0724-1

62.

Nagin

D. S.

(2013). Deterrence: A review of the evidence by a criminologist for economists. Annual Review of Economics, 5(1), 83–105. https://doi.org/10.1146/annurev-economics-072412-131310

63.

Norton

Ariel

Weinborn

O’Dwyer

(2018). Spatiotemporal patterns and distributions of harm within street segments: The story of the “harmspot”. Policing: An International Journal, 41(3), 352–371. https://doi.org/10.1108/PIJPSM-03-2017-0041

64.

Pandis

Chung

Scherer

R. W.

Elbourne

Altman

D. G.

(2017). CONSORT 2010 Statement: Extension checklist for reporting within-person randomised trials. British Medical Journal (Clinical Research Edition), 357, 2835. https://doi.org/10.1136/bmj.j2835

65.

Peck

L. R.

(2020). Experimental evaluation design for program improvement. Sage Publications.

66.

Pierce

Spaar

Briggs

(1988). The character of police work: Strategic and tactical implications. Center for Applied Social Research, Northeastern University.

67.

Pocock

S. J.

(2013). Clinical trials: A practical approach. John Wiley & Sons.

68.

Polit

D. F.

Gillespie

B. M.

(2010). Intention-to-treat in randomized controlled trials: Recommendations for a total trial strategy. Research in Nursing & Health, 33(4), 355–368. https://doi.org/10.1002/nur.20386

69.

Raudenbush

S. W.

(2007). Designing field trials of educational innovations. Scale Up in Education: Issues in Practice, 2, 1–15.

70.

Rosenfeld

Deckard

M. J.

Blackburn

(2014). The effects of directed patrol and self-initiated enforcement on firearm violence: A randomized controlled study of hot spot policing. Criminology; An Interdisciplinary Journal, 52(3), 428–449. https://doi.org/10.1111/1745-9125.12043

71.

Rubin

D. B.

(1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66(5), 688–701. https://doi.org/10.1037/h0037350

72.

Sagarin

B. J.

West

S. G.

Ratnikov

Homan

W. K.

Ritchie

T. D.

Hansen

E. J.

(2014). Treatment noncompliance in randomized experiments: Statistical approaches and design issues. Psychological Methods, 19(3), 317–333. https://doi.org/10.1037/met0000013

73.

Senn

(1994). Testing for baseline balance in clinical trials. Statistics in Medicine, 13(17), 1715–1726. https://doi.org/10.1002/sim.4780131703

74.

Sheiner

L. B.

(2002). Is intent-to-treat analysis always (ever) enough? British Journal of Clinical Pharmacology, 54(2), 203–211. https://doi.org/10.1046/j.1365-2125.2002.01628.x

75.

Sherman

L. W.

(1990). Police crackdowns: Initial and residual deterrence. In Tonry

Morris

(Eds.), Crime and justice: A review of research (12th ed., pp. 1–48). University of Chicago Press.

76.

Sherman

(1991). Book review: Problem-oriented policing. Journal of Criminal Law and Criminology, 82(3), 690–707. https://doi.org/10.2307/1143749

77.

Sherman

L. W.

(1995). Hot spots of crime and criminal careers of places. Crime and Place, 4, 35–52.

78.

Sherman

L. W.

(2013). The rise of evidence-based policing: Targeting, testing, and tracking. Crime and Justice, 42(1), 377–451. https://doi.org/10.1086/670819

79.

Sherman

L. W.

Eck

J. E.

(2003). Policing for crime prevention. In Evidence-based crime prevention (pp. 309–343) Routledge.

80.

Sherman

Gartin

Buerger

(1989). Hot spots of predatory crime: Routine activities and the criminology of place. Criminology, 27(1), 27–56. https://doi.org/10.1111/j.1745-9125.1989.tb00862.x

81.

Sherman

Weisburd

(1995). General deterrent effects of police patrol in crime hot spots: A randomized controlled trial. Justice Quarterly, 12(4), 625–648. https://doi.org/10.1080/07418829500096221

82.

Taylor

Koper

Woods

(2011). A randomized controlled trial of different policing strategies at hot spots of violent crime. Journal of Experimental Criminology, 7(2), 149–181. https://doi.org/10.1007/s11292-010-9120-6

83.

Telep

C. W.

(2017). Police officer receptivity to research and evidence-based policing: Examining variability within and across agencies. Crime & Delinquency, 63(8), 976–999. https://doi.org/10.1177/0011128716642253

84.

Telep

C. W.

Mitchell

R. J.

Weisburd

(2012). How much time should the police spend at crime hot spots? Answers from a police agency directed randomized field trial in Sacramento, California. Justice Quarterly, 31(5), 905–933. https://doi.org/10.1080/07418825.2012.710645

85.

Telep

C. W.

Weisburd

Gill

C. E.

Vitter

Teichman

(2014). Displacement of crime and diffusion of crime control benefits in large-scale geographic areas: A systematic review. Journal of Experimental Criminology, 10, 515–548. https://doi.org/10.1007/s11292-014-9208-5

86.

Townsley

Reid

Reynald

Rynne

Hutchins

(2014). Risky facilities: Analysis of crime concentration in high-rise buildings. Trends and Issues in Crime and Criminal Justice, 476, 1–7.

87.

Vickers

A. J.

Altman

D. G.

(2001). Analysing controlled trials with baseline and follow up measurements. BMJ, 323(7321), 1123–1124. https://doi.org/10.1136/bmj.323.7321.1123

88.

Wain

Ariel

(2014). Tracking of police patrol. Policing: A Journal of Policy and Practice, 8(3), 274–283. https://doi.org/10.1093/police/pau017

89.

Wain

Ariel

Tankebe

(2017). The collateral consequences of GPS-LED supervision in hot spots policing. Police Practice and Research, 18(4), 376–390. https://doi.org/10.1080/15614263.2016.1277146

90.

Weisburd, D., Eck, J., Braga, A., Telep, C., Cave, B., Bowers, K., Bruinsma, G., Gill, C., Groff, E., Hinkle, J., Hibdon, J., Johnson, S., Lawton, B., Lum, C., Ratcliffe, J., Rengert, G., Taniguchi, T. & Yang, S. (2016). Place matters. Cambridge University Press.

91.

Weisburd

(2015). The 2014 Sutherland address: The law of crime concentration and the criminology of place. Criminology; An Interdisciplinary Journal, 53(2), 133–157. https://doi.org/10.1111/1745-9125.12070

92.

Weisburd

Gill

(2014). Block randomized trials at places: Rethinking the limitations of small N experiments. Journal of Quantitative Criminology, 30(1), 97–112. https://doi.org/10.1007/s10940-013-9196-z

93.

Weisburd

Green

(1995). Policing drug hot spots: The Jersey City drug market analysis experiment. Justice Quarterly, 12(4), 711–735. https://doi.org/10.1080/07418829500096261

94.

Weisburd

Majmundar

M. K.

(eds) (2018). Proactive policing: Effects on crime and communities. The National Academies Press.

95.

Weisburd, D., Majmundar, M. K., Aden, H., Braga, A., Bueermann, J., Cook, P. J., Goff, P. A., Harmon, R. A., Haviland, A., Lum, C., Manski, C., Mastrofski, S., Meares, T., Nagin, D., Owens, E., Raphael, S., Ratcliffe, J., & Tyler, T. (2019). Proactive policing: A summary of the report of the National Academies of Sciences, Engineering, and Medicine. Asian Journal of Criminology, 14(2), 145–177. https://doi.org/10.1007/s11417-019-09284-1

96.

Weisburd

Telep

C. W.

(2014). Hot spots policing: What we know and what we need to know. Journal of Contemporary Criminal Justice, 30(2), 200–220. https://doi.org/10.1177/1043986214525083

97.

Weisburd, D., Telep, C. W., Hinkle, J. C., & Eck, J. E. (2010). Is problem-oriented policing effective in reducing crime and disorder? Findings from a Campbell systematic review. Criminology & Public Policy, 9(1), 139–172. https://doi.org/10.1111/j.1745-9133.2010.00617.x

98.

Weisburd

D. L.

Wyckoff

L. A.

Ready

Eck

J. E.

Hinkle

J. C.

Gajewski

(2006). Does crime just move around the corner? A controlled study of spatial displacement and diffusion of crime control benefits. Criminology; An Interdisciplinary Journal, 44(3), 549–592. https://doi.org/10.1111/j.1745-9125.2006.00057.x

99.

Williams

Coupe

(2017). Frequency vs. length of hot spots patrols: A randomised controlled trial. Cambridge Journal of Evidence-Based Policing, 1(1), 5–21. https://doi.org/10.1007/s41887-017-0003-1

100.

Wright

R. T.

Decker

S. H.

(1994). Burglars on the job: Streetlife and residential break-ins. Northeastern University Press.

101.

Zelen

(1979). A new design for randomised clinical trials. New England Journal of Medicine, 300(22), 1242–1245. https://doi.org/10.1056/NEJM197905313002203

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

1.32 MB