Restricted accessResearch articleFirst published online 2025-9
Addressing the Treatment-as-Assigned Assumption in Field Experiments: Lessons Learned From the Birmingham South Saturated Problem-Oriented Policing Hot Spots Experiment
This study investigates the effectiveness of increasing police presence in hot spots typically managed with problem-oriented policing (POP) compared to using POP alone. Our block-randomized controlled trial reveals that overall, “saturated POP” does not significantly reduce crime compared to a POP-only approach. However, a more complex pattern emerged across different blocks, namely, an inverse relationship between the level of police presence and the level of recorded crime, regardless of the random assignment. Crime rates decreased in the experimental arm within two statistical blocks, where the assigned dosage of saturated POP was greater than in the control arm. However, in one block, the assigned control hot spots received a higher dosage of police presence than the assigned treatment hot spots, resulting in significantly lower crime rates in the control hot spots. These findings underscore the necessity of reporting field experiment results based on both treatment-as-assigned and some form of treatment-as-delivered models.
Altman, D. G., Schulz, K. F., Moher, D., Egger, M., Davidoff, F., Elbourne, D., Gøtzsche, P. C., Lang, T., & The Consort Group. (2001). The revised CONSORT statement for reporting randomised trials: Explanation and elaboration. Annals of Internal Medicine, 134(8), 663–694. https://doi.org/10.7326/0003-4819-134-8-200104170-00012
2.
AngristJ. D. (2006). Instrumental variables methods in experimental criminological research: What, why and how. Journal of Experimental Criminology, 2(1), 23–44. https://doi.org/10.1007/s11292-005-5126-x
ArielB.BlandM. (2019). Is crime rising or falling? A comparison of police-recorded crime and victimization surveys. In DeflemM. (Ed.), Methods of criminology and criminal justice research (pp. 7–31). Emerald Publishing Limited.
ArielB.FarringtonD. P. (2012). Randomised block designs. In PiqueroA. R.WeisburdD. (Eds.), Handbook of quantitative criminology (pp. 437–454). Springer.
8.
ArielB.ShermanL. W.NewtonM. (2020). Testing hot-spots police patrols against no-treatment controls: Temporal and spatial deterrence effects in the London underground experiment. Criminology, 58(1), 101–128. https://doi.org/10.1111/1745-9125.12231
9.
ArielB.SutherlandA.ShermanL. W. (2019). Preventing treatment spillover contamination in criminological field experiments: The case of body-worn police cameras. Journal of Experimental Criminology, 15, 569–591. https://doi.org/10.1007/s11292-018-9344-4
10.
ArielB.WeinbornC.ShermanL. W. (2016). “Soft” policing at hot spots–do police community support officers work? A randomized controlled trial. Journal of Experimental Criminology, 12(3), 277–317. https://doi.org/10.1007/s11292-016-9260-4
11.
Armijo-OlivoS.ArientiC.StefanoN. (2019). Use of intention to treat and magnitude of treatment effects. International Journal of Technology Assessment in Health Care, 35(S1), 12–13. https://doi.org/10.1017/S0266462319001168
12.
ArmitageP. (1992). Some topics of current interest in clinical trials. The Canadian Journal of Statistics/La Revue Canadienne de Statistique, 20(1), 1–8. https://doi.org/10.2307/3315570
BlandM.ArielB.KumarS. (2023). Criminal records versus rehabilitation and expungement: A randomised controlled trial. Journal of Experimental Criminology, 20(3), 717–741. https://doi.org/10.1007/s11292-023-09557-x
17.
BlandM.LeggetterM.CestaroD.SebireJ. (2021). Fifteen minutes per day keeps the violence away: A crossover randomised controlled trial on the impact of foot patrols on serious violence in large hot spot areas. Cambridge Journal of Evidence-Based Policing, 5(3), 93–118. https://doi.org/10.1007/s41887-021-00066-3
18.
BlattmanC.GreenD. P.OrtegaD.TobónS. (2021). Place-based interventions at scale: The direct and spillover effects of policing and city services on crime. Journal of the European Economic Association, 19(4), 2022–2051. https://doi.org/10.1093/jeea/jvab002
BloomH. S. (2008). The core analytics of randomized experiments for social research. In AlasuutariP.BrannenJ.BickmanL. (Eds.), The SAGE handbook of social research methods (pp. 115–133). Sage Publications.
21.
BragaA. A.BondB. J. (2008). Policing crime and disorder hot spots: A randomized controlled trial. Criminology; An Interdisciplinary Journal, 46(3), 577–607. https://doi.org/10.1111/j.1745-9125.2008.00124.x
22.
BragaA. A.TurchanB.PapachristosA. V.HureauD. M. (2019). Hot spots policing of small geographic areas effects on crime. Campbell Systematic Reviews, 15(3), e1046. https://doi.org/10.1002/cl2.1046
23.
BragaA. A.WeisburdD. L. (2020). Does hot spots policing have meaningful impacts on crime? Findings from an alternative approach to estimating effect sizes from place-based program evaluations. Journal of Quantitative Criminology, 38(1), 1–22. https://doi.org/10.1007/s10940-020-09481-7
24.
BragaA. A.WeisburdD. L.WaringE. J.MazerolleL. G.SpelmanW.GajewskiF. (1999). Problem-oriented policing in violent crime places: A randomized controlled experiment. Criminology, 37(3), 541–580. https://doi.org/10.1111/j.1745-9125.1999.tb00496.x
25.
BragaA. A.WelshB. C.SchnellC. (2015). Can policing disorder reduce crime? A systematic review and meta-analysis. Journal of Research in Crime and Delinquency, 52(4), 567–588. https://doi.org/10.1177/0022427815576576
26.
BrantinghamP. J.BrantinghamP. L. (1995). Criminality of place: Crime generators and crime attractors. European Journal of Criminal Policy and Research, 3(3), 5–26. https://doi.org/10.1007/BF02242925
27.
BurkeR. H. (2004). Hard cop, soft cop: Dilemmas and debates in contemporary policing. In BurkeR. H. (Ed.), Hard cop, soft cop: Dilemmas and debates in contemporary policing (p. 310). Willan.
ChaineyS.RatcliffeJ. (2013). GIS and crime mapping. John Wiley & Sons.
30.
CornishD. B.ClarkeR. V. (1989). Crime specialisation, crime displacement and rational choice theory. In Criminal behavior and the justice system: Psychological perspectives (pp. 103–117). Springer Berlin Heidelberg.
31.
CoxD. R.ReidN. (2000). The theory of the design of experiments. Chapman and Hall/CRC.
32.
DauP. M. (2023). Disrupting everyday police presence: Mapping the geographies of policing and testing the crime reduction effects of police dosage [unpublished doctoral dissertation]. Ghent University.
33.
DauP. M.DewinterM.WitloxF.BekenT. V.VandeviverC. (2022). How concentrated are police on crime? A spatiotemporal analysis of the concentration of police presence and crime. Cambridge Journal of Evidence-Based Policing, 6(3), 109–133. https://doi.org/10.1007/s41887-022-00079-6
34.
DauP. M.VandeviverC.DewinterM.WitloxF.Vander BekenT. (2021). Policing directions: A systematic review on the effectiveness of police presence. European Journal on Criminal Policy and Research, 29(2), 191–225. https://doi.org/10.1007/s10610-021-09500-8
35.
EckJ. E. (1994). Drug markets and drug places: A case-control study of the spatial structure of illicit drug dealing. University of Maryland.
36.
EckJ. (1997). Preventing crime at places. In University of Maryland, Department of Criminology and Criminal Justice. (Ed.), Preventing crime: What works, what doesn’t, what’s promising (pp. 7‐1–7‐62). Office of Justice Programs, U.S. Department of Justice.
37.
EckJ. (2002). Preventing crime at places. In ShermanL.FarringtonD.WelshB.MacKenzieD. L. (Eds.), Evidence-based crime prevention (pp. 241–294). Routledge.
38.
EckJ. E.ClarkeR. V.GueretteR. T. (2007). Risky facilities: Crime concentration in homogeneous sets of establishments and facilities. Crime Prevention Studies, 21, 225.
39.
FamegaC.HinkleJ. C.WeisburdD. (2017). Why getting inside the “black box” is important: Examining treatment implementation and outputs in policing experiments. Police Quarterly, 20(1), 106–132. https://doi.org/10.1177/1098611116664336
FriedmanL. M.FurbergC. D.DeMetsD. L.ReboussinD. M.GrangerC. B. (2015). Fundamentals of clinical trials. Springer.
42.
GatesS.ArielB.AssarafN. (2024). Responding to nonemergency calls for service via video: A randomized controlled trial. Criminology & Public Policy. https://doi.org/10.1111/1745-9133.12690
43.
GillC.WeisburdD.TelepC. W.VitterZ.BennettT. (2014). Community-oriented policing to reduce crime, disorder and fear and increase satisfaction and legitimacy among citizens: A systematic review. Journal of Experimental Criminology, 10, 399–428. https://doi.org/10.1007/s11292-014-9210-y
GroffE. R.RatcliffeJ. H.HabermanC. P.SorgE. T.JoyceN. M.TaylorR. B. (2015). Does what police do at hot spots matter?The Philadelphia Policing Tactics Experiment. Criminology, 53(1), 23–53. https://doi.org/10.1111/1745-9125.12055
46.
HarachiT. W.AbbottR. D.CatalanoR. F.HaggertyK. P.FlemingC. B. (1999). Opening the black box: Using process evaluation measures to assess implementation and theory building. American Journal of Community Psychology, 27(5), 711–731. https://doi.org/10.1023/A:1022194005511
47.
HasisiB.ItskovichE.Khoury-KassabriM. (2022). A wall of treatments: An integrative problem-solving approach to the prevention of stone-throwing in East Jerusalem. Criminology & Public Policy, 22(2), 385–414. https://doi.org/10.1111/1745-9133.12607
48.
HinkleJ. C.WeisburdD.TelepC. W.PetersenK. (2020). Problem-oriented policing for reducing crime and disorder: An updated systematic review and meta-analysis. Campbell Systematic Reviews, 16(2), e1089. https://doi.org/10.1002/cl2.1089
HopeT. (1994). Problem-oriented policing and drug market locations: Three case studies. Crime Prevention Studies, 2(1), 5–32.
51.
HueyL.MitchellR. J.KalyalH.PegramR. (2021). Implementing evidence-based policing: A how-to guide for police organizations. Policy Press.
52.
JagannathanR.CamassoM. J. (2005). Beyond intention to treat analysis in welfare-to-work studies: The efficacy of labor force attachment, human capital investment and combined strategies for self-sufficiency. Journal of Social Service Research, 31(4), 43–60. https://doi.org/10.1300/J079v31n04_03
53.
JohnsonS. D.GueretteR. T.BowersK. (2014). Crime displacement: What we know, what we don’t know, and what it means for crime reduction. Journal of Experimental Criminology, 10, 549–571. https://doi.org/10.1007/s11292-014-9209-4
KaneE.EvansE.ShokranehF. (2018). Effectiveness of current policing-related mental health interventions: A systematic review. Criminal Behaviour and Mental Health, 28(2), 108–119. https://doi.org/10.1002/cbm.2058
56.
KochelT. R.BurrussG. W.WeisburdD. (2015). St. Louis County hot spots in residential areas (SCHIRA) final report: Assessing the effects of hot spots policing strategies on police legitimacy, crime, and collective efficacy. Southern Illinois University.
57.
KoperC. S. (1995). Just enough police presence: Reducing crime and disorderly behavior by optimizing patrol time in crime hot spots. Justice Quarterly, 12(4), 649–672. https://doi.org/10.1080/07418829500096231
58.
LeeY.EckJ. E.MartinezN. N. (2017). How concentrated is crime at places? A systematic review from 1970 to 2015. Crime Science, 6(1), 1–16. https://doi.org/10.1186/s40163-017-0069-x
MillsL. G.BarocasB.ArielB. (2013). The next generation of court-mandated domestic violence treatment: A comparison study of batterer intervention and restorative justice programs. Journal of Experimental Criminology, 9(1), 65–90. https://doi.org/10.1007/s11292-012-9164-x
61.
MillsL. G.BarocasB.ButtersR. P.ArielB. (2019). A randomized controlled trial of restorative justice-informed treatment for domestic violence crimes. Nature Human Behaviour, 3(12), 1284–1294. https://doi.org/10.1038/s41562-019-0724-1
NortonS.ArielB.WeinbornC.O’DwyerE. (2018). Spatiotemporal patterns and distributions of harm within street segments: The story of the “harmspot”. Policing: An International Journal, 41(3), 352–371. https://doi.org/10.1108/PIJPSM-03-2017-0041
64.
PandisN.ChungB.SchererR. W.ElbourneD.AltmanD. G. (2017). CONSORT 2010 Statement: Extension checklist for reporting within-person randomised trials. British Medical Journal (Clinical Research Edition), 357, 2835. https://doi.org/10.1136/bmj.j2835
65.
PeckL. R. (2020). Experimental evaluation design for program improvement. Sage Publications.
66.
PierceG.SpaarS.BriggsL. (1988). The character of police work: Strategic and tactical implications. Center for Applied Social Research, Northeastern University.
67.
PocockS. J. (2013). Clinical trials: A practical approach. John Wiley & Sons.
68.
PolitD. F.GillespieB. M. (2010). Intention-to-treat in randomized controlled trials: Recommendations for a total trial strategy. Research in Nursing & Health, 33(4), 355–368. https://doi.org/10.1002/nur.20386
69.
RaudenbushS. W. (2007). Designing field trials of educational innovations. Scale Up in Education: Issues in Practice, 2, 1–15.
70.
RosenfeldR.DeckardM. J.BlackburnE. (2014). The effects of directed patrol and self-initiated enforcement on firearm violence: A randomized controlled study of hot spot policing. Criminology; An Interdisciplinary Journal, 52(3), 428–449. https://doi.org/10.1111/1745-9125.12043
71.
RubinD. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66(5), 688–701. https://doi.org/10.1037/h0037350
72.
SagarinB. J.WestS. G.RatnikovA.HomanW. K.RitchieT. D.HansenE. J. (2014). Treatment noncompliance in randomized experiments: Statistical approaches and design issues. Psychological Methods, 19(3), 317–333. https://doi.org/10.1037/met0000013
ShermanL. W. (1990). Police crackdowns: Initial and residual deterrence. In TonryM.MorrisN. (Eds.), Crime and justice: A review of research (12th ed., pp. 1–48). University of Chicago Press.
76.
ShermanL. (1991). Book review: Problem-oriented policing. Journal of Criminal Law and Criminology, 82(3), 690–707. https://doi.org/10.2307/1143749
77.
ShermanL. W. (1995). Hot spots of crime and criminal careers of places. Crime and Place, 4, 35–52.
78.
ShermanL. W. (2013). The rise of evidence-based policing: Targeting, testing, and tracking. Crime and Justice, 42(1), 377–451. https://doi.org/10.1086/670819
79.
ShermanL. W.EckJ. E. (2003). Policing for crime prevention. In Evidence-based crime prevention (pp. 309–343) Routledge.
ShermanL.WeisburdD. (1995). General deterrent effects of police patrol in crime hot spots: A randomized controlled trial. Justice Quarterly, 12(4), 625–648. https://doi.org/10.1080/07418829500096221
82.
TaylorB.KoperC.WoodsD. (2011). A randomized controlled trial of different policing strategies at hot spots of violent crime. Journal of Experimental Criminology, 7(2), 149–181. https://doi.org/10.1007/s11292-010-9120-6
83.
TelepC. W. (2017). Police officer receptivity to research and evidence-based policing: Examining variability within and across agencies. Crime & Delinquency, 63(8), 976–999. https://doi.org/10.1177/0011128716642253
84.
TelepC. W.MitchellR. J.WeisburdD. (2012). How much time should the police spend at crime hot spots? Answers from a police agency directed randomized field trial in Sacramento, California. Justice Quarterly, 31(5), 905–933. https://doi.org/10.1080/07418825.2012.710645
85.
TelepC. W.WeisburdD.GillC. E.VitterZ.TeichmanD. (2014). Displacement of crime and diffusion of crime control benefits in large-scale geographic areas: A systematic review. Journal of Experimental Criminology, 10, 515–548. https://doi.org/10.1007/s11292-014-9208-5
86.
TownsleyM.ReidS.ReynaldD.RynneJ.HutchinsB. (2014). Risky facilities: Analysis of crime concentration in high-rise buildings. Trends and Issues in Crime and Criminal Justice, 476, 1–7.
87.
VickersA. J.AltmanD. G. (2001). Analysing controlled trials with baseline and follow up measurements. BMJ, 323(7321), 1123–1124. https://doi.org/10.1136/bmj.323.7321.1123
WainN.ArielB.TankebeJ. (2017). The collateral consequences of GPS-LED supervision in hot spots policing. Police Practice and Research, 18(4), 376–390. https://doi.org/10.1080/15614263.2016.1277146
90.
Weisburd, D., Eck, J., Braga, A., Telep, C., Cave, B., Bowers, K., Bruinsma, G., Gill, C., Groff, E., Hinkle, J., Hibdon, J., Johnson, S., Lawton, B., Lum, C., Ratcliffe, J., Rengert, G., Taniguchi, T. & Yang, S. (2016). Place matters. Cambridge University Press.
91.
WeisburdD. (2015). The 2014 Sutherland address: The law of crime concentration and the criminology of place. Criminology; An Interdisciplinary Journal, 53(2), 133–157. https://doi.org/10.1111/1745-9125.12070
92.
WeisburdD.GillC. (2014). Block randomized trials at places: Rethinking the limitations of small N experiments. Journal of Quantitative Criminology, 30(1), 97–112. https://doi.org/10.1007/s10940-013-9196-z
93.
WeisburdD.GreenL. (1995). Policing drug hot spots: The Jersey City drug market analysis experiment. Justice Quarterly, 12(4), 711–735. https://doi.org/10.1080/07418829500096261
94.
WeisburdD.MajmundarM. K. (eds) (2018). Proactive policing: Effects on crime and communities. The National Academies Press.
95.
Weisburd, D., Majmundar, M. K., Aden, H., Braga, A., Bueermann, J., Cook, P. J., Goff, P. A., Harmon, R. A., Haviland, A., Lum, C., Manski, C., Mastrofski, S., Meares, T., Nagin, D., Owens, E., Raphael, S., Ratcliffe, J., & Tyler, T. (2019). Proactive policing: A summary of the report of the National Academies of Sciences, Engineering, and Medicine. Asian Journal of Criminology, 14(2), 145–177. https://doi.org/10.1007/s11417-019-09284-1
96.
WeisburdD.TelepC. W. (2014). Hot spots policing: What we know and what we need to know. Journal of Contemporary Criminal Justice, 30(2), 200–220. https://doi.org/10.1177/1043986214525083
97.
Weisburd, D., Telep, C. W., Hinkle, J. C., & Eck, J. E. (2010). Is problem-oriented policing effective in reducing crime and disorder? Findings from a Campbell systematic review. Criminology & Public Policy, 9(1), 139–172. https://doi.org/10.1111/j.1745-9133.2010.00617.x
98.
WeisburdD. L.WyckoffL. A.ReadyJ.EckJ. E.HinkleJ. C.GajewskiF. (2006). Does crime just move around the corner? A controlled study of spatial displacement and diffusion of crime control benefits. Criminology; An Interdisciplinary Journal, 44(3), 549–592. https://doi.org/10.1111/j.1745-9125.2006.00057.x
99.
WilliamsS.CoupeT. (2017). Frequency vs. length of hot spots patrols: A randomised controlled trial. Cambridge Journal of Evidence-Based Policing, 1(1), 5–21. https://doi.org/10.1007/s41887-017-0003-1
100.
WrightR. T.DeckerS. H. (1994). Burglars on the job: Streetlife and residential break-ins. Northeastern University Press.
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
