Pretrial assessment instruments provide courtroom actors with information about the likelihood that someone who has been charged with a criminal offense would engage in criminal behavior if released prior to trial. Although prior research supports the ability of pretrial instruments to predict pretrial outcomes, there are concerns that pretrial instruments may inadvertently exacerbate racial-ethnic and sex disparities found in the larger criminal legal system. In the current study, we conduct multi-site predictive bias tests of a widely used pretrial instrument—the Public Safety Assessment (PSA)—across six racial-ethnic and sex groups (i.e., White males, White females, Black males, Black females, Hispanic males, and Hispanic females). Study results support the PSA as a valid and consistent predictor of failure to appear, new criminal activity, and new violent criminal activity across these six racial-ethnic and sex groups. These findings support the use of the PSA and identify several areas for future research.
American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. (2014). Standards for educational and psychological testing. American Psychological Association. https://www.apa.org/science/programs/testing/standards
AresC. E.RankinA.SturtzH. (1963). The Manhattan bail project: An interim report on the use of pretrial parole. New York University Law Review, 38, 67–95.
AshfordL.SpivakB.OgloffJ.ShepherdS. (2023). Statistical learning methods and cross-cultural fairness: Trade-offs and implications for risk assessment instruments. Psychological Assessment, 35(6), 484–496. https://doi.org/10.1037/pas0001228
7.
BiddleD. (2006). Adverse impact and test validation: A practitioner’s guide to valid and defensible employment testing. Ashgate.
8.
BrennanP. K.SpohnC. (2009). The joint effects of offender race/ethnicity and sex on sentence length decisions in federal courts. Race and Social Problems, 1(4), 200–217. https://psycnet.apa.org/doi/10.1007/s12552-009-9016-0
9.
BrittainB. J.GeorgesL.MartinJ. (2021). Examining the predictive validity of the Public Safety Assessment. Criminal Justice and Behavior, 48(10), 1431–1449. https://doi.org/10.1177/00938548211005836
10.
CampbellC. M.LabrecqueR. M.WeinermanM.SanchagrinK. (2020). Gauging detention dosage: Assessing the impact of pretrial detention on sentencing outcomes using propensity score modeling. Journal of Criminal Justice, 70, 101719. https://doi.org/10.1016/j.jcrimjus.2020.101719
11.
ChouldechovaA. (2017). Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big data, 5(2), 153–163. https://arxiv.org/pdf/1703.00056.pdf
12.
CohenJ. (1988). Statistical power analysis for the behavioural sciences. Lawrence Erlbaum Associates.
DeLongE. R.DeLongD. M.Clarke-PearsonD. L. (1988). Comparing the areas under two or more correlated receiver operating characteristic curves: A nonparametric approach. Biometrics, 44(3), 837–845.
17.
DeMicheleM.BaumgartnerP.WengerM.BarrickK.ComfortM. (2020). Public safety assessment: Predictive utility and differential prediction by race in Kentucky. Criminology & Public Policy, 19(2), 409–431. https://doi.org/10.1111/1745-9133.12481
18.
DeMicheleM.ComfortM.BarrickK.BaumgartnerP. (2021). The intuitive-override model: Nudging judges toward pretrial risk assessment instruments. Federal Probation, 85(2), 22–39.
19.
DemuthS. (2003). Racial and ethnic differences in pretrial release decisions and outcomes: A comparison of Hispanic, Black, and White felony arrestees. Criminology, 41(3), 873–908. https://doi.org/10.1111/j.1745-9125.2003.tb01007.x
DesmaraisS. L.ZottolaS. A.Duhart ClarkeS. E.LowderE. M. (2021). Predictive validity of pretrial risk assessments: A systematic review of the literature. Criminal Justice and Behavior, 48(4), 398–420. https://doi.org/10.1177/0093854820932959
22.
EckhouseL.LumK.Conti-CookC.CiccoliniJ. (2019). Layers of bias: A unified approach for understanding problems with risk assessment. Criminal Justice and Behavior, 46(2), 185–209. https://doi.org/10.1177/0093854818811379
FrankJ. (1930). Law and the modern mind. Coward-Mccann.
25.
FreiburgerT. L.BurkeA. S. (2011). Status offenders in the juvenile court: The effects of gender, race, and ethnicity on the adjudication decision. Youth Violence and Juvenile Justice, 9(4), 352–365. https://psycnet.apa.org/doi/10.1177/1541204011399933
26.
GouletteN.WooldredgeJ. (2018). Collateral consequences of pretrial detention. In HuebnerB. M.FrostN. A. (Eds.), Handbook on the consequences of sentencing and punishment decisions (pp. 271–289). Routledge.
27.
HochschildJ. L.WeaverV. (2007). The skin color paradox and the American racial order. Social Forces, 86(2), 643–670. https://doi.org/10.1093/sf/86.2.643
28.
HolmesB.FeldmeyerB.KuligT. C. (2020). Sentencing departures and focal concerns: The joint effect of race and gender on departures in United States district courts, 2014–2016. Journal of Crime and Justice, 43(5), 598–622. https://doi.org/10.1080/0735648X.2020.1730933
29.
HoodK.SchneiderD. (2019). Bail and pretrial detention: Contours and causes of temporal and county variation. RSF: The Russell Sage Foundation Journal of the Social Sciences, 5(1), 126–149.
30.
KahnemanD. (2011). Thinking, fast and slow. Macmillan.
31.
KangJ.BennettM.CarbadoD.CaseyP.LevinsonJ. (2011). Implicit bias in the courtroom. University of California Los Angeles Law Review, 59, 1124–1186.
32.
KimH. Y. (2017). Statistical notes for clinical researchers: Chi-squared test and Fisher's exact test. Restorative Dentistry and Endodontics, 42(2), 152–155. https://doi.org/10.5395%2Frde.2017.42.2.152
33.
KleinbergJ.LudwigJ.MullainathanS.SusteinC. R. (2019). Discrimination in the age of algorithms. Pre-Print. https://arxiv.org/abs/1902.03731
34.
LattimoreP. K.TuellerS.Levin-RectorA.WitwerA. (2020). The prevalence of local criminal justice practices. Federal Probation, 84, 28–37.
35.
LowderE. M.DiazC. L.GrommonE.RayB. R. (2022). Differential prediction and disparate impact of pretrial risk assessments in practice: A multi-site evaluation. Journal of Experimental Criminology, 19, 561–594. https://doi.org/10.1007/s11292-021-09492-9
36.
LowderE. M.LawsonS. G.GrommonE.RayB. R. (2020). Five-county validation of the Indiana Risk Assessment System–Pretrial Assessment Tool (IRAS-PAT) using a local validation approach. Justice Quarterly, 37(7), 1241–1260. https://doi.org/10.1080/07418825.2020.1829006
MaysonS. G. (2019). Detention by any other name. Duke Law Journal, 69, 1643–1680.
39.
McShaneB. B.GalD.GelmanA.RobertC.TackettJ. L. (2019). Abandon statistical significance. The American Statistician, 73(suppl. 1), 235–245. https://doi.org/10.1080/00031305.2018.1527253
40.
PetersenN. (2020). Do detainees plead guilty faster? A survival analysis of pretrial detention and the timing of guilty pleas. Criminal Justice Policy Review, 31(7), 1015–1035. https://doi.org/10.1177/0887403419838020
41.
RiceM. E.HarrisG. T. (2005). Comparing effect sizes in follow-up studies: ROC Area, Cohen’s d, and r. Law and human behavior, 29, 615–620. https://doi.org/10.1007/s10979-005-6832-7
42.
SackettP. R.BornemanM. J.ConnellyB. S. (2008). High stakes testing in higher education and employment: Appraising the evidence for validity and fairness. American Psychologist, 63(4), 215–227. https://doi.org/10.1037/0003-066x.63.4.215
43.
SackettP. R.LaczoR. M.LippeZ. P. (2003). Differential prediction and the use of multiple predictors: The omitted variables problem. Journal of Applied Psychology, 88(6), 1046–1056. https://doi.org/10.1037/0021-9010.88.6.1046
44.
SacksM.SainatoV. A.AckermanA. R. (2015). Sentenced to pretrial detention: A study of bail decisions and outcomes. American Journal of Criminal Justice, 40(3), 661–681. https://doi.org/10.1007/s12103-014-9268-0
45.
SchuckA. M. (2004). The masking of racial and ethnic disparity in police use of physical force: The effects of gender and custody status. Journal of Criminal Justice, 32(6), 557–564.
SkeemJ. L.LowenkampC. T. (2016). Risk, race, and recidivism: Predictive bias and disparate impact. Criminology, 54, 680–712. https://doi.org/10.1111/1745-9125.12123
48.
SkeemJ. L.MonahanJ.LowenkampC. (2016). Gender, risk assessment, and sanctioning: The cost of treating women like men. Law and Human Behavior, 40(5), 580–593. https://doi.org/10.1037/lhb0000206
49.
SpohnC. (2008). Race, sex, and pretrial detention in federal court: Indirect effects and cumulative disadvantage. University of Kansas Law Review, 57, 879–901.
50.
StanovichK. E.WestR. F. (2000). Individual differences in reasoning: Implications for the rationality debate?Behavioral and Brain Sciences, 23(5), 645–665. https://doi.org/10.1017/s0140525x00003435
51.
StevensonM. (2018). Assessing risk assessment in action. Minnesota Law Review, 103, 303–384.
TuellerS. J.ChewR.GodwinA. K.WilliamsJ. A.WeitzelK. (2022). psa2013: Validation tools for the Public Safety Assessment (PSA) (R package version 0.3.9). https://osf.io/3gsmf/
54.
Van BruntA.BowmanL. E. (2018). Toward a just model of pretrial release: A history of bail reform and a prescription for what’s next. Journal of Criminal Law & Criminology, 108, 701–774.
55.
VincentG. M.ViljoenJ. L. (2020). Racist algorithms or systemic problems? Risk assessments and racial disparities. Criminal Justice and Behavior, 47(12), 1576–1584. https://doi.org/10.1177/0093854820954501
56.
WoldgabrealY.DayA.TamateaA. (2020). Do risk assessments play a role in the enduring “color line..”Advancing Corrections, 10, 18–28.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
