Evaluating the Effectiveness of a Police Early Intervention System: From the Predictive Validity of Officer Identification to the Impact of Intervention

Abstract

As a non-punitive approach to addressing and minimizing officer misconduct, early intervention systems (EISs) have become a best practice in policing. These systems focus on identifying officers at risk of future problematic behavior and providing effective early interventions. Despite the widespread use of EISs, limited research has evaluated these systems, and the evaluations that do exist often concentrate solely on either the selection of officers for intervention or the effects of interventions, which restricts our understanding of the overall effectiveness of EISs. Instead of concentrating on evaluating only a part of an EIS, we explore both the implementation and effectiveness of the Phoenix Police Department (PPD) EIS at two different stages. Utilizing data from over 2000 officers employed by the PPD between 2016 and 2020, we use receiver operating characteristic (ROC) curves to analyze the accuracy of officer selection for EIS involvement and a population-averaged interrupted time series model to determine whether EIS implementation is associated with a reduction in problematic officer behavior. Our findings indicated that the indicators and thresholds used by PPD to identify at-risk officers resulted in low predictive validity (none greater than 4%). Furthermore, establishing the EIS was not significantly associated with a decrease in problematic officer behavior. We conclude with a discussion of research and policy implications and future directions.