Probation Officer Assessments of Risk when the Youth Look Different: Contributions of Structured Professional Judgment to Concerns About Racial Bias

Abstract

Various groups have expressed considerable concern about the potential for actuarial risk assessments to exacerbate racial disparities in justice settings. This study examined that potential when using a different approach to risk assessment, structured professional judgment (SPJ), by comparing risk decisions made by evaluators when the examinee’s race was different versus the same as theirs. A large sample of youth (N = 1,308) evaluated on the Structured Assessment of Violence Risk in Youth (SAVRY) by 137 juvenile probation officers (JPOs) in five states indicated the only moderation effect for the match or mismatch between JPOs’ and youths’ race/ethnicity was in the weight JPOs placed on five (out of 24) risk factors in their overall risk opinions. The match between JPOs' and youths' race had no bearing on JPOs’ final determination of youths’ risk levels. This study lends support for investigating the use of SPJ instruments as a method for minimizing racial bias.

Keywords

risk assessment structured professional judgment race/ethnicity juvenile probation officers

Get full access to this article

View all access options for this article.

References

Ægisdóttir

White

M. J.

Spengler

P. M.

Maugherman

A. S.

Anderson

L. A.

Cook

R. S.

Nichols

C. N.

Lampropoulos

G. K.

Walker

B. S.

Cohen

Rush

J. D.

(2006). The meta-analysis of clinical judgment project: Fifty-six years of accumulated research on clinical versus statistical prediction. The Counseling Psychologist, 34(3), 341–382. https://doi.org/10.1177/0011000005285875

American Educational Research Association, American Psychological Association, National Council on Measurement in Education, Joint Committee on Standards for Educational and Psychological Testing (U.S.). (2014). Standards for educational and psychological testing. AERA.

Barbaras

Benjamin

Bowers

Broussard

Buolamwini

Constanza-Chock

. (2019). Technical flaws of pretrial risk assessments raise grave concerns. https://www.lasttreelaws.com/uploads/1/1/8/5/118540061/technicalflawsofpretrial_ml_site.pdf

Bartel

Borum

Forth

(2002). Structured assessment for violence risk in youth (SAVRY). University of South Florida.

Betancourt

Regeser

(1993). The study of culture, ethnicity, and race in American psychology. https://dornsifecms.usc.edu/assets/sites/376/docs/1993_Betancourt__Lopez.pdf

Bonta

(1996). Risk-needs assessment and treatment. In Harland

A. T.

(Ed.), Choosing correctional options that work: Defining the demand and evaluating the supply (pp. 18–32). Sage.

Borum

Bartel

Forth

(2006). Manual for the structured assessment for violence risk in youth (SAVRY). Psychological Assessment Resources.

Bridges

G. S.

Steen

(1998). Racial disparities in official assessments of juvenile offenders: Attributional stereotypes as mediating mechanisms. American Sociological Review, 63(4), 554–570. https://doi.org/10.2307/2657267

Chapman

J. F.

Desai

R. A.

Falzer

P. R.

Borum

(2006). Violence risk and race in a sample of youth in juvenile detention: The potential to reduce disproportionate minority confinement. Youth Violence and Juvenile Justice, 4(2), 170–184. https://doi.org/10.1177/1541204006286316

10.

Charman

S. D.

Kavetski

Mueller

D. H.

(2017). Cognitive bias in the legal system: Police officers evaluate ambiguous evidence in a belief-consistent manner. Journal of Applied Research in Memory and Cognition, 6(2), 193–202. https://doi.org/10.1016/j.jarmac.2017.02.001

11.

Da Silva

D. J.

Barbosa

Normando

Abdo

C. C.

(2010). Race versus ethnicity: Differing for better application. https://www.researchgate.net/publication/228746811_Race_versus_ethnicity_Differing_for_better_application

12.

Degner

Wentura

(2010). Automatic prejudice in childhood and early adolescence. Journal of Personality and Social Psychology, 98(3), 356–374. https://doi.org/10.1037/a0017993

13.

Douglas

K. S.

Hart

S. D.

Webster

C. D.

Belfrage

(2013). HCR-20V3: Assessing risk of violence—User guide. Mental Health, Law, and Policy Institute, Simon Fraser University.

14.

Douglas

K. S.

Skeem

J. L.

(2005). Violence risk assessment: Getting specific about being dynamic. Psychology, Public Policy, and Law, 11(3), 347–383. https://doi.org/10.1037/1076-8971.11.3.347

15.

Ewert v Canada (2018). https://scc-csc.lexum.com/scc-csc/scc-csc/en/item/17133/index.do

16.

Fleiss

J. L.

(1986). Analysis of data from multiclinic trials. Controlled Clinical Trials, 7, 267–275. https://doi.org/10.1016/0197-2456(86)90034-6

17.

Fujii

D. E. M.

Tokioka

A. B.

Lichton

A. I.

Hishinuma

(2005). Ethnic differences in prediction of violence risk with the HCR-20 among psychiatric inpatients. Psychiatric Services, 56(6), 711–716.

18.

Gieco

E. M.

Cassidy

R. C.

(2001). Overview of race and hispanic origin. https://www.census.gov/prod/2001pubs/cenbr01-1.pdf

19.

Grove

W. M.

Meehl

P. E.

(1996). Comparative efficiency of informal (subjective, impressionistic) and formal (mechanical, algorithmic) prediction procedures: The clinical–statistical controversy. Psychology, Public Policy, and Law, 2(2), 293–323. https://doi.org/10.1037/1076-8971.2.2.293

20.

Grove

W. M.

Zald

D. H.

Lebow

B. S.

Snitz

B. E.

Nelson

(2000). Clinical versus mechanical prediction: A meta-analysis. Psychological Assessment, 12(1), 19–30. https://doi.org/10.1037//1040-3590.12.1.19

21.

Guy

L. S.

Douglas

K. S.

Hart

S. D.

(2015). Risk assessment and communication. In Cutler

B. L.

Zapf

P. A.

(Eds.), APA handbooks in psychology. APA handbook of forensic psychology, Vol. 1. Individual and situational influences in criminal and civil contexts (pp. 35–86). American Psychological Association. https://doi.org/10.1037/14461-003

22.

Guy

L. S.

Vincent

G. M.

Grisso

Perrault

(2015). Advancing use of risk assessment in juvenile probation. Technical report submitted to the Office of Juvenile Justice and Delinquency Prevention. www.ncjrs.gov/pdffiles1/ojjdp/grants/249155.pdf

23.

Hannah-Moffat

Maurutto

(2010). Re-contextualizing pre-sentence reports: Risk and race. Punishment & Society, 12(3), 262–286. https://doi.org/10.1177/1462474510369442

24.

Hanson

R. K.

Morton-Bourgon

K. E.

(2009). The accuracy of recidivism risk assessments for sexual offenders: A meta-analysis of 118 prediction studies. Psychological Assessment, 21(1), 1–21. https://doi.org/10.1037/a0014421

25.

Harcourt

B. E.

(2007). Against prediction: Profiling, policing, and punishing in an actuarial age. University of Chicago Press.

26.

Harcourt

B. E.

(2015). Risk as a proxy for race: The dangers of risk assessment. Federal Sentencing Reporter, 27(4), 237–243. https://scholarship.law.columbia.edu/faculty_scholarship/2564

27.

Hart

S. D.

(2016). Culture and violence risk assessment: The case of Ewert v. Canada. Journal of Threat Assessment and Management, 3(2), 76–96. https://doi.org/10.1037/tam0000068

28.

Hayes

A. F.

(2016). PROCESS macro for SPSS and SAS (Version 2.16) [Computer software]. https://www.processmacro.org/index.html

29.

Hewstone

(1990). The ‘ultimate attribution error’? A review of the literature on intergroup causal attribution. European Journal of Social Psychology, 20, 311–335. https://doi.org/10.1002/ejsp.2420200404

30.

Hockenberry

Puzzanchera

(2020, April). Juvenile Court Statistics 2018. National Center for Juvenile Justice.

31.

Holder

(2015). Attorney General Eric Holder Speaks at the National Association of Criminal Defense Lawyers 57th Annual Meeting and 13th State Criminal Justice Network Conference [Speech transcript]. The United States Department of Justice. Retrieved August 1, 2014, from http://www.justice.gov/opa/speech/attorney-general-eric-holder-speaks-national-association-criminal-defense-lawyers-57th

32.

Jessor

Van Den Bos

Vanderryn

Costa

F. M.

Turbin

M. S.

(1995). Protective factors in adolescent problem behavior: Moderator effects and developmental change. Developmental Psychology, 31(6), 923–933. https://doi.org/10.1037/0012-1649.31.6.923

33.

Keenan

S. J.

Rush

J. P.

(2019). Effect of extra-legal factors on juvenile probation officers’ sentencing recommendations, roles, and practices. Justice Policy Journal, 16, 1–25. www.cjcj.org/jpj

34.

Kerr

N. L.

Hymes

R. W.

Anderson

A. B.

Weathers

J. E.

(1995). Defendant-juror similarity and mock juror judgments. Law and Human Behavior, 19, 545–567. https://doi.org/10.1007/BF01499374

35.

Lason

Mattu

Kirchner

Angwin

(2016). How we analyzed the COMPAS recidivism algorithm. ProPublica. https://www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm

36.

Leiber

Peck

J. H.

Beaudry-Cyr

(2014). When does race and gender matter? The interrelationships between the gender of probation officers and juvenile court detention and intake outcomes. Academy of Criminal Justice Sciences, Justice Quarterly, 33, 614–641. https://doi.org/10.1080/07418825.2014.958185

37.

Leifker

Sample

L. L.

(2011). Probation recommendations and sentences received: The association between the two and the factors that affect recommendations. Criminal Justice Policy Review, 22(4), 494–517. https://doi.org/10.1177/0887403411388405

38.

Lowder

E. M.

Desmarais

S. L.

Rade

C. B.

Johnson

K. L.

Van Dorn

R. A.

(2019). Reliability and validity of START and LSI-R assessments in mental health jail diversion clients. Assessment, 26, 1347–1361. https://doi.org/10.1177/1073191117704505

39.

Marques

J. M.

Yzerbyt

V. Y.

Leyens

J. P.

(1988). The “black sheep effect”: Extremity of judgments towards ingroup members as a function of group identification. European Journal of Social Psychology, 18, 1–16. https://doi.org/10.1002/ejsp.2420180102

40.

McGrath

Thompson

A. P.

(2012). The relative predictive validity of the static and dynamic domain scores in risk-need assessment of juvenile offenders. Criminal Justice and Behavior, 39, 250–263. https://doi.org/10.1177/0093854811431917

41.

McGraw

K. O.

Wong

S. P.

(1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1, 30–46. https://doi.org/10.1037/1082-989X.1.1.30

42.

Monahan

(2008). Structured risk assessment of violence. In Simon

R. I.

Tardiff

(Eds.), Textbook of violence assessment and management (pp. 17–33). American Psychiatric Publishing.

43.

Muir

N. M.

Viljoen

J. L.

Jonnson

M. R.

Cochrane

D. M.

Rogers

B. J.

(2020). Predictive validity of the structured assessment of violence risk in youth (SAVRY) with indigenous and Caucasian female and male adolescents on probation. Psychological Assessment, 32, 594–607. https://doi.org/10.1037/pas0000816

44.

Murrie

D. C.

Boccaccini

M. T.

Turner

D. B.

Meeks

Woods

Tussey

(2009). Rater (dis) agreement on risk assessment measures in sexually violent predator proceedings: Evidence of adversarial allegiance in forensic evaluation? Psychology, Public Policy, and Law, 15, 19–53. https://doi.org/10.1037/a0014897

45.

Murrie

D. C.

Boccaccini

M. T.

Zapf

P. A.

Warren

J. I.

Henderson

C. E.

(2008). Clinician variation in findings of competence to stand trial. Psychology, Public Policy, and Law, 14, 177. https://doi.org/10.1037/a0013578

46.

National Research Council. (2013). Reforming juvenile justice: A developmental approach. The National Academies Press. https://doi.org/10.17226/14685

47.

O’Shea

L. E.

Picchioni

M. M.

Mason

F. L.

Sugarman

P. A.

Dickens

G. L.

(2014). Characteristics of aggressive behavior among male inpatients with schizophrenia. Psychiatry Research, 220, 669–678. http://dx.doi.org/10.1016/j.psychres.2014.07.080

48.

Olver

M. E.

Stockdale

K. C.

Wormith

J. S.

(2009). Risk assessment with young offenders: A meta-analysis of three assessment measures. Criminal Justice and Behavior, 36, 329–353. https://doi.org/10.1177/0093854809331457

49.

Perrault

R. T.

Vincent

G. M.

Guy

L. S.

(2017). Are risk assessments racially biased?: Field study of the SAVRY and YLS/CMI in probation. Psychological Assessment, 29, 664–678. https://doi.org/10.1037/pas0000445

50.

Peterson-Badali

Skilling

Hoqanee

(2015). Implementation of risk assessment in case management for youth in the justice system. Criminal Justice and Behavior, 42, 304–320. https://doi.org/10.1177/0093854814549595

51.

Picard

Watkins

Rempel

Kerodal

(2019). Beyond the algorithm: Pretrial reform, risk assessment, and racial fairness. ProPublica. https://www.courtinnovation.org/sites/default/files/media/document/2019/Beyond_The_Algorithm.pdf

52.

R v Gladue (1999). https://scc-csc.lexum.com/scc-csc/scc-csc/en/item/1695/index.do

53.

R. v. Ipeelee (2012). https://scc-csc.lexum.com/scc-csc/scc-csc/en/item/8000/index.do

54.

Robinson

D. G.

Koepke

(2019). Civil rights and pretrial risk assessment instruments. Safety and Justice Challenge. http://www.safetyandjusticechallenge.org/wp-content/uploads/2019/12/Robinson-Koepke-Civil-Rights-Critical-Issue-Brief.pdf

55.

Schmader

Lickel

(2006). Stigma and shame: Emotional responses to the stereotypic actions of one’s ethnic ingroup. In Van Laar

Levin

(Eds.), Stigma and group inequality: Social psychological perspectives. Lawrence Erlbaum Associates Publishers.

56.

Shepherd

S. M.

Anthony

(2018). Popping the cultural bubble of violence risk assessment tools. Journal of Forensic Psychiatry and Psychology, 29(2), 211–220. https://doi.org/10.1080/14789949.2017.1354055

57.

Shepherd

S. M.

Luebbers

Ferguson

Ogloff

J. R. P.

Dolan

(2014). The utility of the SAVRY across ethnicity in Australian young offenders. Psychology, Public Policy and Law, 20(1), 31–45. https://doi.org/10.1037/a0033972

58.

Shepherd

S. M.

Willis-Esqueda

(2018). Indigenous perspectives on violence risk assessment: A thematic analysis. Punishment and Society, 20(5), 599–627. https://doi.org/10.1177/1462474517721485

59.

Singh

J. P.

Desmarais

S. L.

Hurducas

Arbach-Lucioni

Condemarin

Dean

Doyle

Foline

J. O.

Godoy-Cevera

Grann

Mei Yee Ho

Large

M. M.

Hjort Nielsen

Pham

T. H.

Reboco

M. F.

Reeves

K. A.

Rettenberger

de Ruitier

Seewald

Otto

R. K.

(2014). International perspectives on the practical application of violence risk assessment: A global survey of 44 countries. Surveying Violence Risk Assessment, 13, 193–2016. https://doi.org/10.1080/14999013.2014.922141

60.

Singh

J. P.

Grann

Fazel

(2011). A comparative study of violence risk assessment tools: A systematic review and metaregression analysis of 68 studies involving 25,980 participants. Clinical Psychology Review, 31, 499–513. https://doi.org/10.1016/j.cpr.2010.11.009

61.

Skeem

J. L.

Lowenkamp

C. T.

(2016). Risk, race, and recidivism: Predictive bias and disparate impact. Criminology, 54, 680–712. https://doi.org/10.1111/1745-9125.12123

62.

Skeem

Lowenkamp

(2020). Using algorithms to address trade-offs inherent in predicting recidivism. Behavioral Sciences & the Law, 38(3), 259–278. https://doi-org.umasslowell.idm.oclc.org/10.1002/bsl.2465

63.

Snowden

R. J.

Gray

N. S.

Taylor

(2010). Risk assessment for future violence in individuals from an ethnic minority group. International Journal of Forensic Mental Health, 9, 118–123. https://doi.org/10.1080/14999013.2010.501845

64.

St. John

Murphy

Liberman

(2020). Recommendations for addressing racial bias in risk and needs assessment in the juvenile justice system. Child Trends. https://www.childtrends.org/wp-content/uploads/2020/01/Duke-Risk-Assessment-FAQ_ChildTrends_Jan2020-1.pdf

65.

Starr

S. B.

(2015). The risk assessment era: An overdue debate. Federal Sentencing Reporter, 27, 205–206. https://www.ned.uscourts.gov/internetDocs/jpar/RGK-FSR_Editor_Observations.pdf

66.

Tonry

(2014). Legal and ethical issues in the prediction of recidivism. Federal Sentencing Reporter, 26, 167–176. https://doi.org/10.2139/ssrn.2329849

67.

U.S. Census Bureau. (2000). Census of Population and Housing, Summary File 3: Technical Documentation, 2002. https://www.census.gov/prod/cen2000/doc/sf3.pdf

68.

Viljoen

J. L.

Jonnson

M. R.

Cochrane

D. M.

Vargen

L. M.

Vincent

G. M.

(2019). Impact of risk assessment instruments on rates of pretrial detention, postconviction placements, and release: A systematic review and meta-analysis. Law and Human Behavior, 43(5), 397–420. http://dx.doi.org/10.1037/lhb0000344

69.

Vincent

G. M.

Chapman

Cook

N. E.

(2011). Risk-needs assessment in juvenile justice: Predictive validity of the SAVRY, racial differences, and the contribution of needs factors. Criminal Justice and Behavior, 38, 42–62. https://doi.org/10.1177/0093854810386000

70.

Vincent

G. M.

Guy

L. S.

* Fusco

S. L.

* Gershenson

B. G.

(2012). Field reliability of the SAVRY with probation officers: Implications for training. Law and Human Behavior, 36, 225–236. https://doi.org/10.1007/s10979-011-9284-2

71.

Vincent

G. M.

Guy

L. S.

Perrault

R. T.

Gershenson

(2016). Risk assessment matters, but only when implemented well: A multisite study in juvenile probation. Law and Human Behavior, 40, 683–696. https://doi.org/10.1037/lhb0000214

72.

Vincent

G. M.

Perrault

(2018). Risk Assessment and Behavioral Health Screening (RABS) Project. Technical Report to the Office of Juvenile Justice and Delinquency Prevention. NCJ# 251912. https://ojjdp.ojp.gov/library/publications/risk-assessment-and-behavioral-health-screening-rabs-project-final-technical

73.

Vincent

G. M.

Perrault

Drawbridge

(2020). Long-term impact and cost-effectiveness of risk-needs assessment and risk-need-responsivity (RNR) reforms in juvenile probation: The long-term rnr impact study. Technical Report submitted to the Office of Juvenile Justice and Delinquency Prevention.

74.

Wachter

(2015). Statewide risk assessment in juvenile probation. JJGPS StateScan. National Center for Juvenile Justice.

75.

Webster

C. D.

Martin

M. L.

Brink

Nicholls

T. L.

Desmarais

S. L.

(2009). Short-term assessment of risk and treatability (START) (Version. 1.1). British Columbia Mental Health & Addiction Services.

76.

Wroblewski

(2014, July 29). [Letter to the Honorable Patti B. Saris]. U.S. Department of Justice, Criminal Division. https://www.justice.gov/sites/default/files/criminal/legacy/2014/08/01/2014annual-letter-final-072814.pdf

77.

Yang

Wong

S. C.

Coid

(2010). The efficacy of violence prediction: A meta-analytic comparison of nine risk assessment tools. Psychological Bulletin, 136, 740–767. https://doi.org/10.1037/a0020473

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.22 MB