The Council of State Governments Justice Center Approach to Increasing Risk-Level Consistency in the Application of Risk Assessment Instruments

Abstract

Correctional and forensic mental health settings potentially have multiple risk assessment instruments administered on a single client. Because of the various methods of determining risk categories, risk-level consistency can become an issue. The Council of State Governments Justice Center developed a Five-Level System that can be applied to most risk assessment instruments. Using the Level of Service Inventory–Revised and two created risk assessment instruments, the present study assessed if the Five-Level System (vs. normative percentile categories) demonstrated greater agreement between the two instruments, and, if so, the percentage of greater agreement. The Five-Level System demonstrated 4% to 5% greater agreement for both risk-level placement and recidivism rates. The implications of this greater consistency among risk assessment instruments is an increased fairness in making risk-level assignments.

Keywords

risk assessment risk communication standardized assessment

Get full access to this article

View all access options for this article.

References

Andrews

D. A.

Bonta

J. L.

(2001). The Level of Service Inventory–Revised: User’s manual. Multi Health Systems. (Original work published 1995)

Andrews

D. A.

Zinger

Hoge

R. D.

Bonta

Gendreau

Cullen

F. T.

(1990). Does correctional treatment work? A clinically relevant and psychologically informed meta-analysis. Criminology, 28(3), 369-404. https://doi.org/10.1111/j.1745-9125.1990.tb01330.x

Babchishin

K. M.

Kroner

D. G.

Hanson

R. K.

(2017). Standardized risk/need levels for corrections. Crime Scene, 24(1), 9-13.

Bakeman

(2000). Behavioral observation and coding. In Reis

H. T.

Judd

C. M.

(Eds.), Handbook of research methods in social and personality psychology (pp. 138-159). Cambridge University Press.

Barbaree

H. E.

Langton

C. M.

Peacock

E. J.

(2006). Different actuarial risk measures produce different risk rankings for sexual offenders. Sexual Abuse, 18(4), 423-440. https://doi.org/10.1007/s11194-006-9029-9

Blackburn

(2000). Classification and assessment of personality disorders in mentally disordered offenders: A psychological perspective. Criminal Behaviour and Mental Health, 10(S1), S8-S33. https://doi.org/10.1002/cbm.2000.10.s1.s8

Blackburn

(2007). Personality disorder and psychopathy: Conceptual and empirical integration. Psychology, Crime & Law, 13(1), 7-18. https://doi.org/10.1080/10683160600869585

Blais

(2015). Preventative detention decisions: Reliance on expert assessments and evidence of partisan allegiance within the Canadian context. Behavioral Sciences & the Law, 33(1), 74-91. https://doi.org/10.1002/bsl.2155

Bourgon

Armstrong

(2005). Transferring the principles of effective treatment into a “real world” prison setting. Criminal Justice and Behavior, 32(1), 3-25. https://doi.org/10.1177/0093854804270618

10.

Bourgon

Mugford

Hanson

R. K.

Coligado

(2018). Offender risk assessment practices vary across Canada. Canadian Journal of Criminology and Criminal Justice, 60(2), 167-205. https://doi.org/10.3138/cjccj.2016-0024

11.

Copas

Marshall

(1998). The Offender Group Reconviction Scale: A statistical reconviction score for use by probation officers. Journal of the Royal Statistical Society: Series C (Applied statistics), 47(1), 159-171. https://doi.org/10.1111/1467-9876.00104

12.

Council of State Governments Justice Center. (2014). A common language for risk assessment: Experts convene in Washington. http://csgjusticecenter.org/a-common-language-for-riskassessments-experts-convene-in-washington/

13.

Edens

J. F.

Boccaccini

M. T.

(2017). Taking forensic mental health assessment “out of the lab” and into “the real world”: Introduction to the special issue on the field utility of forensic assessment instruments and procedures. Psychological Assessment, 29(6), 599-610. https://doi.org/10.1037/pas0000475

14.

Elbogen

E. B.

Mercado

C. C.

Scalora

M. J.

Tomkins

A. J.

(2002). Perceived relevance of factors for violence risk assessment: A survey of clinicians. International Journal of Forensic Mental Health, 1(1), 37-47. https://doi.org/10.1080/14999013.2002.10471159

15.

Gwet

K. L.

(2019). irrCAC: Computing chance-corrected agreement coefficients (CAC) (Version 1.0) [Computer software]. https://CRAN.R-project.org/package=irrCAC

16.

Hanson

R. K.

Babchishin

K. M.

Helmus

L. M.

Thornton

Phenix

(2017). Communicating the results of criterion referenced prediction measures: Risk categories for the Static-99R and Static-2002R sexual offender risk assessment tools. Psychological Assessment, 29(5), 582-597. https://doi.org/10.1037/pas0000371

17.

Hanson

R. K.

Bourgon

Helmus

Hodgson

(2009). The principles of effective correctional treatment also apply to sexual offenders: A meta-analysis. Criminal Justice and Behavior, 36(9), 865-891. https://doi.org/10.1177/0093854809338545

18.

Hanson

R. K.

Bourgon

McGrath

R. J.

Kroner

D. G.

D’Amora

D. A.

Thomas

S. S.

Tavarez

L. P.

(2017). A five-level risk and needs system: Maximizing assessment results in corrections through the development of a common language. The Council of State Governments Justice Center.

19.

Hanson

R. K.

Morton-Bourgon

K. E.

(2009). The accuracy of recidivism risk assessments for sexual offenders: A meta-analysis of 118 prediction studies. Psychological Assessment, 21(1), 1-21. https://doi.org/10.1037/a0014421

20.

Heilbrun

O’Neill

M. L.

Stevens

T. N.

Strohman

L. K.

Bowman

Y.-W.

(2004). Assessing normative approaches to communicating violence risk: A national survey of psychologists. Behavioral Sciences & the Law, 22(2), 187-196. https://doi.org/10.1002/bsl.570

21.

Helmus

Hanson

R. K.

(2011). More fun with statistics! How to use logistic regression to predict criminal recidivism risk. Crime Scene, 18(2), 8-12.

22.

Hilterman

E. L. B.

Nicholls

T. L.

van Nieuwenhuizen

(2014). Predictive validity of risk assessments in juvenile offenders comparing the SAVRY, PCL:YV, and YLS/CMI with unstructured clinical assessments. Assessment, 21(3), 324-339. https://doi.org/10.1177/1073191113498113

23.

Jung

Pham

Ennis

(2013). Measuring the disparity of categorical risk among various sex offender risk assessment measures. Journal of Forensic Psychiatry & Psychology, 24(3), 353-370. https://doi.org/10.1080/14789949.2013.806567

24.

Kingston

D. A.

Yates

P. M.

Firestone

Babchishin

Bradford

J. M.

(2008). Long-term predictive validity of the Risk Matrix 2000: A comparison with the Static-99 and the sex offender risk appraisal guide. Sexual Abuse, 20(4), 466-484. https://doi.org/10.1177/1079063208325206

25.

Kroner

D. G.

Mills

J. F.

Reddon

J. R.

(2005). A Coffee Can, factor analysis, and prediction of antisocial behavior: The structure of criminal risk. International Journal of Law and Psychiatry, 28(4), 360-374. https://doi.org/10.1016/j.ijlp.2004.01.011

26.

Kroner

D. G.

Morrison

M. M.

Lowder

E. M.

(2020). A principled approach to the construction of risk assessment categories: The Council of State Governments Justice Center Five-Level System. International Journal of Offender Therapy and Comparative Criminology, 64(10-11), 1074-1090. https://doi.org/10.1177/0306624X19870374

27.

Link

B. G.

Cullen

F. T.

(1986). Contact with the mentally ill and perceptions of how dangerous they are. Journal of Health and Social Behavior, 27(4), 289-302. https://doi.org/10.2307/2136945

28.

Lowder

E. M.

Morrison

M. M.

Kroner

D. G.

Desmarais

S. L.

(2019). Racial bias and LSI-R assessments in probation sentencing and outcomes. Criminal Justice and Behavior, 46(2), 210-233. https://doi.org/10.1177/0093854818789977

29.

Luong

Wormith

J. S.

(2011). Applying risk/need assessment to probation practice and its impact on the recidivism of young offenders. Criminal Justice and Behavior, 38(12), 1177-1199. https://doi.org/10.1177/0093854811421596

30.

Makarios

Sperber

K. G.

Latessa

E. J.

(2014). Treatment dosage and the risk principle: A refinement and extension. Journal of Offender Rehabilitation, 53(5), 334-350. https://doi.org/10.1080/10509674.2014.922157

31.

Mills

J. F.

(2017). Violence risk assessment: A brief review, current issues, and future directions. Canadian Psychology, 58(1), 40-49. https://doi.org/10.1037/cap0000100

32.

Mills

J. F.

Jones

M. N.

Kroner

D. G.

(2005). An examination of the generalizability of the LSI-R and VRAG probability bins. Criminal Justice and Behavior, 32(5), 565-585. https://doi.org/10.1177/0093854805278417

33.

Mills

J. F.

Kroner

D. G.

(2006). The effect of discordance among violence and general recidivism risk estimates on predictive accuracy. Criminal Behaviour and Mental Health, 16(3), 155-166. https://doi.org/10.1002/cbm.623

34.

Monahan

Skeem

J. L.

(2015). Risk assessment in criminal sentencing (SSRN Scholarly Paper No. ID 2662082). Social Science Research Network. http://papers.ssrn.com/abstract=2662082

35.

Nunes

K. L.

Firestone

Bradford

J. M.

Greenberg

D. M.

Broom

(2002). A comparison of codified versions of the Static-99 and the sex offender risk appraisal guide. Sexual Abuse, 14(3), 253-269. https://doi.org/10.1177/107906320201400305

36.

Nunnally

J. C.

(1978). Psychometric theory (2nd ed.). McGraw-Hill.

37.

Oleson

J. C.

(2011). Risk in sentencing: Constitutionally suspect variables and evidence-based sentencing. SMU Law Review, 64, 1329-1402. https://herculesandtheumpire.files.wordpress.com/2014/08/64smulrev-4-oleson-1.pdf

38.

Olver

M. E.

Mundt

J. C.

Thornton

Beggs Christofferson

S. M.

Kingston

D. A.

Sowden

J. N.

Nicholaichuk

T. P.

Gordon

Wong

S. C. P.

(2018). Using the Violence Risk Scale-Sexual Offense version in sexual violence risk assessments: Updated risk categories and recidivism estimates from a multisite sample of treated sexual offenders. Psychological Assessment, 30(7), 941-955. https://doi.org/10.1037/pas0000538

39.

Olver

M. E.

Stockdale

K. C.

Wormith

J. S.

(2011). A meta-analysis of predictors of offender treatment attrition and its relationship to recidivism. Journal of Consulting and Clinical Psychology, 79(1), 6-21. https://doi.org/10.1037/a0022200

40.

Olver

M. E.

Stockdale

K. C.

Wormith

J. S.

(2014). Thirty years of research on the Level of Service Scales: A meta-analytic examination of predictive accuracy and sources of variability. Psychological Assessment, 26(1), 156-176. https://doi.org/10.1037/a0035080

41.

Ostermann

Salerno

L. M.

(2016). The validity of the Level of Service Inventory–Revised at the intersection of race and gender. Prison Journal, 96(4), 554-575. https://doi.org/10.1177/0032885516650878

42.

Polanin

J. R.

Snilstveit

(2016). Campbell methods policy note on converting between effect sizes (Version 1.1, updated December 2016). The Campbell Collaboration. https://doi.org/10.4073/cmpn.2016.3

43.

Quinsey

V. L.

Ambtman

(1979). Variables affecting psychiatrists’ and teachers’ assessments of the dangerousness of mentally ill offenders. Journal of Consulting and Clinical Psychology, 47(2), 353-362. https://doi.org/10.1037/0022-006X.47.2.353

44.

R Core Development Team. (2019). R: A language and environment for statistical computing. R Foundation for Statistical Computing.

45.

Schmidt

Sinclair

S. M.

Thomasdóttir

(2016). Predictive validity of the Youth Level of Service/Case Management Inventory with youth who have committed sexual and non-sexual offenses: The utility of professional override. Criminal Justice and Behavior, 43(3), 413-430. https://doi.org/10.1177/0093854815603389

46.

Silver

Chow-Martin

(2002). A multiple models approach to assessing recidivism risk implications for judicial decision making. Criminal Justice and Behavior, 29(5), 538-568. https://doi.org/10.1177/009385402236732

47.

Skeem

J. L.

Lowenkamp

C. T.

(2016). Risk, race, and recidivism: Predictive bias and disparate impact. Criminology, 54(4), 680-712. https://doi.org/10.1111/1745-9125.12123

48.

Snowden

R. J.

Gray

N. S.

Taylor

MacCulloch

M. J.

(2007). Actuarial prediction of violent recidivism in mentally disordered offenders. Psychological Medicine, 37(11), 1539-1549. https://doi.org/10.1017/S0033291707000876

49.

Viglione

Rudes

D. S.

Taxman

F. S.

(2015). Misalignment in supervision: Implementing risk/needs assessment instruments in probation. Criminal Justice and Behavior, 42(3), 263-285. https://doi.org/10.1177/0093854814548447

50.

Vose

Cullen

F. T.

Smith

(2008). The empirical status of the Level of Service Inventory. Federal Probation, 72(3), 22-29.

51.

Walters

G. D.

Kroner

D. G.

Dematteo

Locklair

B. R.

(2014). The impact of base rate utilization and clinical experience on the accuracy of judgments made with the HCR-20. Journal of Forensic Psychology Practice, 14(4), 288-301. https://doi.org/10.1080/15228932.2014.941726

52.

Wormith

J. S.

Hogg

S. M.

Guzzo

(2015). The predictive validity of the LS/CMI with aboriginal offenders in Canada. Criminal Justice and Behavior, 42(5), 481-508. https://doi.org/10.1177/0093854814552843